easy_install tip - setting up your own repository

by Jeremy Jones

I've been digging into setuptools and easy_install lately, so you can expect to see more of these tips in the coming weeks.

One of the really cool features of easy_install is that you can install packages from the Python Cheese Shop, which is a web-based repository of Python packages. For example, in order to install IPython using easy_install and the Cheese Shop, you would simply type `easy_install ipython`. It's all fine and dandy for open source projects to put their source code out there for the world to look at, but what if you work on (gasp) closed source code? Do you have to miss out on all this easy_install remote repository goodness? Not at all.

First, you have to setup a web server that will serve up your packages. I am using lighttpd on my laptop, which misses the point in practice since i could just as easily install the eggs locally rather than by pointing at http://localhost, but it works for example purposes. I have lighttp serving up a directory that contains a repository directory called "repo". So, http://localhost/repo will give a directory listing of all the packages I can serve up.

Second, you need to build your packages and put them in a directory that your webserver can serve up, the "repo" directory in my case. I have 2 packages, foo v0.1 and bar v0.1, that I made into eggs. The filenames are bar-0.1-py2.5.egg and foo-0.1-py2.5.egg. In this example, foo has a dependency on bar. Since easy_install handles dependencies so well, I would expect an easy_install of foo to resolve the bar dependency and install them both. Step three will prove this assumption either correct or incorrect.

Third, and finally, you simply tell easy_install to look at your local repository for packages. Here is the command and output from installing from my local repository:

root@lanik:/usr/lib/python2.5/site-packages# easy_install -f http://localhost/repo foo
Searching for foo
Reading http://localhost/repo
Best match: foo 0.1
Downloading http://localhost/repo/foo-0.1-py2.5.egg
Processing foo-0.1-py2.5.egg
Moving foo-0.1-py2.5.egg to /usr/lib/python2.5/site-packages
Adding foo 0.1 to easy-install.pth file

Installed /usr/lib/python2.5/site-packages/foo-0.1-py2.5.egg
Processing dependencies for foo
Searching for bar==0.1
Best match: bar 0.1
Downloading http://localhost/repo/bar-0.1-py2.5.egg
Processing bar-0.1-py2.5.egg
Moving bar-0.1-py2.5.egg to /usr/lib/python2.5/site-packages
Adding bar 0.1 to easy-install.pth file

Installed /usr/lib/python2.5/site-packages/bar-0.1-py2.5.egg
Finished processing dependencies for foo

The "-f" flag tells easy_install where else it can look for packages. That's it!


Noah Gift
2007-11-14 19:19:08
That IS sweet! I can see using something like this for host only network virtual machines to do QA testing. Somebody should put this in a book....
Phillip J. Eby
2007-11-16 14:51:42
You can do the same thing with "file:" URLs or directory paths, no web server needed. Just make sure you include a trailing '/' after the directory name.

This should also work with UNC paths on Windows, or NFS-mounted paths on *nix.

2008-01-16 10:08:31
This is nice. The turbogears project used this idea to provide a repo of all packages used in their project, which saved me more than once.

I've run into the problem on windows of a package not installing because it couln't compile impure modules.

Tip suggestion: I think there is an option on setuptools/easy_install to not compile. This is/would be handy when developing and wanting to install a package that works.

Getting a compiled/faster version can come later.

Of course I can't remember the option/how it works at the moment.