Problematic perl prerequisites

by Chris Josephes

I just installed XMLRPC::Lite on a Linux host. You may not have seen this module used too often, but it actually comes bundled in the SOAP::Lite distribution.

Installation time of one perl module in a 5.8.8 environment? One hour. Was it slow Internet bandwidth? Am I counting the meeting time where we discussed the need for this code? Neither. The install took one hour due to the need to satisfy all of the code dependencies.

On a host with a default perl installation, I also had to install the dependencies TimeDate, HTML-Tagset, HTML-Parser, MailTools, MIME-Types, Email-Date-Format, Test-Pod, Pod-Escapes, libwww-perl, Crypt-SSLeay, XML-Parser, Compress-Raw-Zlib, IO-stringy, File-Temp, IO-Compress-Zlib, IO-Compress-Base, Pod-Simple, MIME-Tools, FCGI, and Compress-Zlib. I also had to install the openssl-devel and expat-devel RPMs to satisfy the build requirements of some of the modules. And I didn't get one nice full list of 20 dependencies to be satisfied; I had a list that seemed to grow with each module distribution that I installed.

I didn't want to make the job too difficult, because this was a special case server outside of our normal support environment. No RPM packages were readily available, so going through a build process was still the quickest option.

I'm not angry, but I'll admit to some frustration and confusion. The first question that will come to my mind is why aren't some of these modules shipped by default yet? Or, out of all of these modules, how many of them are actually being used?

Is there anyone using Compress::Raw::Zlib that wouldn't also want to install Compress::Zlib? Why bundle these modules separately? Are the extra Test and Pod modules really necessary for the end user, or does it just make the developers job easy? And is Email::Date::Format the only module out there that outputs the time in the RFC 2822 time format?

Again, no harm's done; but I hope that perl authors and distributors keep this anecdote in mind. The less effort that I need to go through to install your code, the easier my job gets.

16 Comments

Claire
2008-01-15 13:33:18
I've had very similar experiences with a variety of Perl modules, sometimes made additionally ridiculous by new dependencies being reported only after some others have been dealt with. I have, on a number of occasions, spent some significant amount of time (generally 40 minutes to an hour) building packages with cpan2rpm before giving up in disgust.


However, for your particular need, there *are* some repositories that have all these modules packaged for you. I'm currently using RPMForge for a lot of secondary software, and expect to switch to rpmrepo when that comes on line. RPMForge includes a perl-SOAP-Lite package and, presumably, all its dependencies. (Scarily, I would only need to install five additional packages on my workstation to cover all the dependencies for SOAP::Lite.)


(Note that Fedora also seems to have a perl-SOAP-Lite package, and Debian, of course, does, too.)

Chris Josephes
2008-01-15 13:59:34
I never tried RPMForge before, but I'll be sure to give it an honest look.


My luck with perl RPMs (from distributors) has been mixed. For example, on a RHEL5 host, with perl 5.8.5, I have run into a couple of times where I could find module FooBar, but it was built for a different version of perl. I wouldn't notice the problem until I looked at the file location in the RPM.


$ rpm -l perl-FooBar
/usr/lib/perl5/site_perl/5.8.7/FooBar.pm


In the above case, I just modified my @INC variable to accomodate the new path. To package perl modules, the distribution of the OS, and the perl environment needs to match. For anyone that packages and distributes code, that process can generate a large number of files.


But cpan2rpm is pretty nice for internal use in IT departments.


Brandon Checketts
2008-01-15 17:34:16
I'd second that recommendation to try rpmforge. It has saved me lots of time not only on perl modules, but with a lot of other applications that aren't in the distro repositories (or where the distro repositories are kindof old). Stuff like rrdtool, trac, spamassassin, and others are all available.


I've run into an occasional issue, but for the most part the packages from rpmforge work really well.

Chris Josephes
2008-01-16 06:30:04
I looked over RPMforge, and it looks like it had a nice selection of modules, but I think this is still an issue that needs addressing by the perl community. Also, for other sysadmins in my situation, the RPMforge site won't help them unless they're also using an RPM based packaging system.


Since perl6 will use a virtual machine, maybe there's the potential to streamline the installation/distribution process as well.

chromatic
2008-01-16 11:20:36
If you don't use CPAN.pm to install these modules, you're at the mercy of your packaging system. I'm not sure what Perl developers can do to help; I don't use RPM-based systems and I certainly don't have time to build packages for every distribution out there.


In general, the recommendation is to use CPAN.pm to install modules and follow dependencies.

Stosh
2008-01-16 12:51:42
Handling module dependencies in some modern way is long overdue. But I also know a lot of people who use Perl but just realistically aren't going to install modules that aren't available from their distribution. So my questions are: who decides what modules are in the Perl core, and on what basis do they decide to add new modules to the core?
It might just be that what modules are in the core hasn't been addressed in a loooong time.
Chris Josephes
2008-01-16 12:56:07
If you don't use CPAN.pm to install these modules....


The best scenario for managing perl in an IT environment seems to be as follows:


1. On a build/development host (matching your production environment), build our perl RPMs by downloading them from CPAN.


2. Package the results of the build using your packager of choice. Make sure you catch dependencies on shared object libraries in your packaging specifications.


3. For each host that requires module FooBar, install your pre-built module Foobar.


I can agree with this. I just had a one off scenario where I had a server outside of the normal environment, that no pre-built RPM could handle.


My greater concern was that it seemed like developers were not considering the effort that it may take to actually install a ommon service, like SOAP.


And the MIME modules, the Compression modules don't change too often, why not make them part of the Perl standard; especially if a high number of sites will install them anyways?

chromatic
2008-01-16 13:23:07

Why not make any given module part of the Perl core? Easy: who decides what's necessary and what's not?


You use the SOAP and archival modules. I don't. I use SQLite and YAML frequently, while you may use Data::Dumper and DBD::Pg. What if there are competing modules? Do you include XML::Simple as well as XML::LibXML or XML::Twig? How about templating modules?


Who has time in p5p to babysit and maintain all of these new core modules and keep up with bugfixes and new features maintained by their own authors? Who releases a new minor version of Perl to address a critical security hole in a new core module, or a bugfix? Do patches go to p5p or to the author?


What happens when everyone decides that a core module just plain sucks (File::Find) and for which exist several good replacements, but we can't remove it for the purposes of backwards compatibility and we're stuck maintaining code that no one wants to work on and no one recommends for serious use on all of the platforms that we support?


These are not easy questions to answer.


Again, the CPAN community goes to great lengths to make it possible (and where possible, even easy) to install modules through CPAN.pm, because we can all use that across all of the platforms we use. You don't have to use that, and you can certainly install all of the same software by hand, if you're willing to give up CPAN's built-in dependency tracking and test reporting and such. (As well, CPAN as a repository is older than most other software repositories.)


I'm just not sure it's a reasonable expectation that installing modules outside of the recommended Perl approach and your package manager's approach is going to be easy.

Chris Josephes
2008-01-16 20:17:59
Why not make any given module part of the Perl core? Easy: who decides what's necessary and what's not?


Ah, but who makes that decision right now? I'm not saying it's an easy decision, or even a fair decision, but there has to be some kind of metric out there of determining what's really useful (and what isn't).


Maybe another solution is building more Bundles:: distributions. It'd be a great way for a lot of these contributors to collaborate more closely on these projects.


shixilun
2008-01-16 22:42:49
I didn't realize File::Find wasn't so hot -- probably because I found it in the core and thought, "Hey, it's in the core..." So I use it all the time.


Could someone please let me know what the good replacements are?

Dirk
2008-01-17 01:02:34
apt-get install libsoap-lite-perl
Robert
2008-01-18 06:45:49
@Chris


I think Bundle:: is being replaced by Task::

chromatic
2008-01-18 09:28:51

@Chris, there appears to be no general rule for Perl 5.


For Perl 6, the rule is "the only core modules are those necessary to download, configure, and install new modules". Thus someone will have to solve the module installation and dependency problem to get anything done with Perl 6.


(I won't mind slapping facist system administrators who refuse to install anything non-core. Their businesses should fail and we should salt the land where their cubicles used to be.)

chromatic
2008-01-18 09:30:54

@shixilun, see File::Finder and File::Find::Rule. They have much nicer interfaces.

Zak Elep
2008-01-23 03:47:32
The Compress:: and IO:: modules above could have been installed faster if you installed Bundle::CPAN first, like what older CPAN versions recommended.


On the other hand, SOAP::Lite could have made better perhaps by upgrading its install system, as it still uses ExtUtils::MakeMaker. While still useful, iirc EU::MM doesn't prompt for dependencies automatically like Module::Install.

fmerges
2008-02-16 20:35:49
Debian packages a lot of perl modules, so it makes it very easy, simple, and fast to install the required modules.


Then, also, as someone already said, if you install Bundle::CPAN first, you will have less problems later ;-)


In Bundle:: or Task:: modules, you have only the reference to the required modules, but it's not being included in it, so is just for helping you in not manually writing and checking the list of required modules.