Looking For a Few Good SysAdmin Problem To Solve in Python

by Noah Gift

Jeremy and I are turning in the final draft of our book on "Python for Unix and Linux Systems Administration" on Monday, April 16th. But, if you are a sysadmin and have a really tricky problem you need solved shoot us an email ASAP and there is a chance we will solve it for you. Yes, this is free consulting, but you have to act very quickly!

Some examples could be:

1. Need to process 3TB's of data everyday.
2. Need a solution to manage virtual hosting.

Tell us the specifics and we will do our best to tackle it as one of our case studies.

22 Comments

bsergean
2008-04-16 17:47:55
I had a program working on one version of Solaris 10, but not on an older one (they have updates, like service pack). Solaris uses SVR4 packages, that are bundled into patches. I wrote a python script that told me the differences in patch revision and that helped me solve my problem.


http://www.opensolaris.org/jive/thread.jspa?threadID=55776&tstart=15

Rick
2008-04-16 18:03:46
I have several thousand files that I need to parse. All of the files exist on NFS. I'd like to see your take on creating a distributed system to process files. (Perhaps something using pyprocessing.)


I look forward to your book!

Simon Hibbs
2008-04-17 00:26:20
A re-implementation of Puppet in Python?


Only kidding (well, half kidding).

Noah Gift
2008-04-17 00:38:14
Thanks for all of the suggestions we will try to fit them in if we can.
Simon/I did a simple example of the beginnings of this actually. I agree this would be quite nice. You might take a look at Func, which is a Red Hat only version of something quite similar.
Mike MacCana
2008-04-17 05:20:35
Produce an OpenOffice report of a system, with serial number, vendor, list of all services started at boot, non-distro packages, build date, total amount of memory, total amount of swap, etc.


Currently I do this in Bash, using my own OpenOffice function library. It's a hack, but it works. I'd love to do it in Python.

Mike MacCana
2008-04-17 05:39:42
I should probably add: the OpenOffice report should have headers, proper tables and images. A bonus (although not done by the current bash based tool) would be generated graphs as inline images. In reward, I promise to buy your book and love you forever.
Spike
2008-04-17 06:23:47
* a MQ framework such as it would be easier to write distributed applications requiring asynchronous messaging.
* a load testing framework extendable via plugins and allowing the simulation of failure conditions


Examples:
* First one could be used to implement a monitoring solution, which I'm actually doing and finding quite a bit of a headache to get right
* Second one would be a way to unify the dozen of nice yet limited and quite inflexible tools that can be used for load testing. An example would be testing a 2 tiers web application in an independent way, being able to simulate a large amount of web requests and database queries both distributed against masters and slaves, with occasional simulated failures (for example hanging connections with no data exchange)


thank you

GS
2008-04-17 08:25:54
Writing an SNMP agent (pysnmp? snmpy? yapsnmp?). Writing a program using this agent to collect data from a remote host.
Jim
2008-04-17 09:12:58
Parse 100's of MB of Sun Directory server logs. Distinguish between external logins (external workstations using LDAP for authentication) and internal stuff (Sun One Mail, local portal logins, etc). Rebuild the login session (because of caching the access log writes, the details are spread throughout the file), provide stats on general usage, peak times... stuff like that.


I have a bash script hack that allows me to pull together the login session of a single user in a single file that I can wrap to create an ugly report (with details but no stats), but I'd like to see something more general that allows me to focus on a single user (with details and stats) or all users (with details and stats).


I suspect Python could do this quicker (my hack can take up to two minutes to parse 2GB of data)... looking for just one uid and filtering out everything but the external system connections.


Interpreting the data would be extra nice as well, as I usually have to hand this sort of stuff over to people that don't read the LDAP log format (http://docs.sun.com/source/816-6699-10/logfiles.html).

Ian Clark
2008-04-17 09:14:47
Don't forget to include how to build Python from source. I'm doing so and am stuck on a make...


# make
rm -f libpython2.5.a
ar cr libpython2.5.a Modules/getbuildinfo.o
make: ar: Command not found
make: *** [libpython2.5.a] Error 127



Anonymous
2008-04-17 11:26:24
Ian, ar is part of binutils packages. Make sure its installed, or found in your path, on whatever system your trying to build python on.
rodney
2008-04-17 13:26:38
monitor a flexlm license server for license usage. Collect
data on amount of licenses used vs time. Http request to the
license server gets page showing current license usage. Scrape
info from returned page and store it in db.
Mike Lowe
2008-04-17 14:09:29
We have a 284TB scratch parallel file system with millions of files and several hundred users that we want to purge files older than 60 days. We would also like to generate reports to track usage.
Mike MacCana
2008-04-17 20:43:11
I'd rather not pay for instruction on compiling python. Compiling generally implies using an unstable version which won't be relevant for a book.


Unnecessary instructions on compiling software are usually used to pad out Unix books or satisfy gentoo weenies - not system administrators.

fishingfanatic
2008-04-18 08:42:23

Hello fellows

Had a look around the forum and decided to introduce myself

I work on [url=http://costaricafishingtour.net/]costa rica sport fishing [/url]
to specific I work majorly on [url=http://costaricafishingtour.net/fishing-accesories.html]costa rica sport fishing[/url]


I am from Michigan

My favorite band is Black Sabbath and I also like electronica like FERRY CORSTEN

I am learning Spanish

I love to play field hockey

We will meet at the forums


jacobeachrealtor
2008-04-19 13:00:23


Hello pals.

Some details about me

I've fed sharks in Belize...no body parts missing

I'm too vain to resist writing this post and loving every second of it.

If I got 99 in an exam I would blame myself for not getting 100.

I am into real estate, mostly my deals are url=http://www.costaricajacobeachcondos.com/]condos for sale in costa rica[/url] and
[url=http://costaricajacobeachcondos.com/jaco-house/jaco-houses-list.html]jaco beach costa rica real estate[/url]


My favorite color is yellow.

Pisces

That's it

Cynic
2008-04-19 16:36:11
This isn't really in the same league as the examples you gave (but I'm going to post it anyway). The newer versions of VIM can supposedly be scripted in Python; thus a section on how to tweak a vim install to work as an IDE (as well as how to call scripts from inside Vim; use it as an email client; ect) would be nice to see; if not exactly ground breaking.
Cynic
2008-04-19 16:36:50
This isn't really in the same league as the examples you gave (but I'm going to post it anyway). The newer versions of VIM can supposedly be scripted in Python; thus a section on how to tweak a vim install to work as an IDE (as well as how to call scripts from inside Vim; use it as an email client; ect) would be nice to see; if not exactly ground breaking.
Qin Li
2008-04-22 07:31:30
Need to get active and backup mailing list server -- mailman 2.1.10 to run in HA mode or auto-failover when one server is down.
spycrasher
2008-04-23 03:09:34
Hi folks

Just trapped on a little tool to slice website images.
Quick in preparing ready html from simple single-image site design.
Program is rather crappy and cheap (especially comparing to similar function in PhotoShop).
But! Just look what books come as a bonus:

Search Engine Optimization: An Hour a Day
Web Design in a Nutshell, 3rd Edition
Webmaster in a Nutshell, 3rd Edition
etc...

I wonder how $270 priced books can be sold for just 20 dollars...
Anyway here's link: http://www.sliceimage.com/

Hope these books will help you as well.

PS: Don't forget to click on the links in my signature in reward ;)

Regards,
Mark
Danny
2008-04-25 09:52:09
My task,


1. Best way to process syslogs on a central server. (look for patterns and stuff)

Charles Dean
2008-04-27 17:34:58
Providing a framework that would support Test Driven Infrastructure. Allow for tests to be added to ensure that a piece of equipment, exists and is configured correctly. Allow for the insertion of newly developed tests.


Support regression testing of the system.