Perl Success Story: Perl is Precious to Venezuelan ISP

by Todd Mezzulo

Well, its been three years since Betsy Waliszewski started promoting Perl's prowess through this extremely popular series of success stories. 25 stories and 3 volumes later we can honestly say that Perl has gained, and continues to gain widespread acceptance as an important programming component in mission-critical applications within many small, medium, and large enterprises.

At O'Reilly, we've believed in Perl from the beginning and feel proud that we've played a part (some would even say a large part) in its rise to acclaim, and its with that sense of pride and a good amount of joy that I'm honored to post this first tale in the next generation of Perl Success Stories.

Luis Muñoz, a Systems Manager at, sent this story which details how Perl enabled him and his staff of *zero* to streamline and automate provisioning systems at the largest ISP in Venezuela. ¡Viva Perl!

Perl: A Jewel for an ISP

As part of my job at the largest ISP in my country, I'm in charge of all the aspects of the logical service provisioning for ADSL customers. Our network is not simple, as it combines equipment from several vendors with the joys of sharing the administration of the underlying transport equipment with our parent company.

For the sake of this story, let's say that we inherited the ADSL network in a fairly unstable condition and my job has been, primarily, to stabilize it and get rid of a number of shortcomings in its initial design. At the beginning, this meant dealing with equipment located at several central offices. Piece of cake I hear you saying...

Once the network was up and running, lots of customers were brought in by our sales staff. Managing the network became more challenging. Service provisioning, manual in its beginnings, had to be automated. Worse, it had to support a number of different vendors' devices with little in common. And errors were no longer tolerated, as customers had grown used to a reliable network pretty quickly.

This is where Perl really helped us make all this a success. Perl allowed the creation of sophisticated tools that could act as an interface between our homebred service management systems and our DSL gear. Thanks to this, provisioning time went down from two working days to a record-setting thirty seconds. Errors were almost eliminated thanks to the full automation. In fact, when we roll out our next version of these tools in a couple of months, our customers will be able to switch service speeds on demand and in real time (say, to download the next release of Perl) and return to their original connection speed after they're done.

But this is only part of the story. There's much more in managing a network than making sure every customer is getting the correct service level. Another set of tools allowed us to automate the process of applying changes to the network gear in real time and in a consistent way. We developed "agents" that could get into one of our network devices and carry out specific instructions, often constructed automatically out of an interaction with external (and formerly incompatible) systems.

Earlier this year, one of the nightmares of a network administrator came true. A new network architecture was called for, and it required significant changes in the configuration of our access concentrators, DSL modems and customer databases. At about at the same time, we learned the hard way about how sensitive some of those modems were to common threats such as CodeRed.

We had to perform a firmware upgrade on a device installed on the customers' premises. Well, actually 29,000 of them. Had no staff to do it. Had no budget to do it. Had very little time to do it. Thank God we also had Perl to do it. Thanks to Perl, I managed to upgrade all of them, scattered through all of Venezuela while sitting in my desk. The customers didn't have to worry about this either. It happened automatically and swiftly.

Next we needed to change the ADSL modem's configuration. This involved altering the configuration in three different places at the same time, while making sure that after the customer's modem was told to reload, it would come back online. To make matters interesting, we did not want to depend on any help from the customer, so the process had to be bulletproof and automatic. And it was.

This was not our first experience discovering the joys of Perl for ISP management. Our RADIUS server, which happily serves more than 1.5 million dialup accounts (out of a few modest PC-class servers running Linux, mind you), is also written in Perl. It allows us to quickly write a module that can interface with almost anything in order to provide authentication services for unlimited, prepaid and postpaid access.

We also have a DNS server written in Perl, which we use to spare ourselves from having to deal with old-fashioned zone files. This DNS server lets us keep information in database tables and query it in real time. It provides up-to-the-second information without human intervention, as data about the names is extracted automatically from the network devices' configuration with tools written in Perl. Now if an administrator manages to break our DNS data by accident, we simply issue a rollback in one of our Oracle databases and move on to the next task.

This DNS server is so powerful in its ability to extract user information from various databases and external systems, that we use it to tell our mail system where each customer's mailbox is located. Because we needed some custom functionality, we wrote the interface for the local delivery agent for our mail system in Perl.

In short, thanks to Perl we're able to do many things in ways that are more scalable, easier and cheaper than our competition. And fun. Don't forget fun.

Luis Muñoz - Venezuela

Provisioning Systems Manager

To learn how large and small companies are using Perl to meet their goals, check out Perl Success Stories.

If you have a Perl success story of your own that you'd like to share, please let me know. You can reach me at: