The Year in Perl 2005
Related link: http://www.perl.org/
A year is a surprisingly long and short time in the life of a software project. It may go by quickly, but when you review what actually happened, you may find that many things happened.
The theme for Perl in 2005 is one of rebirth. Several projects that looked dormant started again — not just technical, but social. While Perl may have entered the year a little slowly, it's leaving 2005 much stronger.
Perl 5.8.x pumpking Nicholas Clark announced last year that he intended to change the quarterly stable release schedule to once every four months. Unfortunately, real life intervened. Fortunately, his excellent work in making the 5.8.x series stable and usable continues. He released Perl 5.8.7 at the end of May. Few people have tested release candidates for future releases with their code or their modules — feedback is very important.
Hopefully next year will see more regular releases.
Despite the fact that stable releases bring bugfixes, newer versions of the core modules, and, as always, ever-improving test coverage, some businesses still stick with Perl versions that have gone unmaintained this millennium. Perl 5.004 is ancient, Perl 5.005 is old, and Perl 5.6.1 is creaky. If you're deploying a new installation of Perl, please consider the latest stable 5.8.x release, or if you really need old software, at least Perl 5.6.2.
Perl 5.10 pumpking Hugo van der Sanden, perhaps the kindest man in Perl, passed on the full pumpking to Rafael Garcia-Suarez, another candidate for the title of "the kindest man in Perl". Unfortunately, this means that Hugo is unlikely to tackle the proposed and much-anticipated rewrite of the regular expression engine.
Perl 5.10 will be impressive, though. Proposed new features include:
- the defined-or operator, ported from Perl 6
- a smaller memory footprint, with much slimmer internal data structures
- an improved Switch module with smart matching and no source filtering
- a lexical
- variable names in uninitialized value warnings
- a new pragma,
feature, which allows you to use features from Perl 6
- more tests
- updated core modules
- continual improvements to the internals, thanks to Ponie
- regexp trie optimizations (especially thanks to Yves Orton)
- a somewhat cleaner and safer API (thanks to Andy Lester and cleanup patches)
New features planned for inclusion are:
- integrated Module::Build (and perhaps CPANPLUS)
- proper lexical pragmas
- state variables, as found in Perl 6
There's no timetable yet on the 5.10 release, but there's a sense that it's getting close to the time to think about such things. Pumpking Rafael Garcia-Suarez plans to release the 5.9.3 development version just after the start of the year.
Adam Kennedy continued the Australian domination of the weird on the CPAN,
releasing PPI 1.0 — an
independent Perl parser that can parse Perl without running it. PPI makes a lot
of things possible — better IDE integration, refactoring, automated style
guides, et cetera. Still, only
perl can still parse every Perl
file especially when you get into areas where the other crazy Australian hacker
Which other crazy Australian hacker? Damian Conway. He did not rest this
year either. In conjunction with the release of his new book, Perl Best
Practices, he released several modules to help you write code well. One of
the most interesting is
which itself touched off a few debates on Perl Monks about the use, abuse,
value, and implementation of inside-out objects. Though
itself may not be the ultimate and final way to write clean, maintainable, and
performant OO code in Perl 5, it does show why Damian's a master Perl hacker.
Would you have considered inside-out objects if he hadn't written it?
The web world has seen some work too. Simon Cozens's Maypole has new
maintainership and new evolution. The spinoff project Catalyst has a lot of
activity, excitement, and open job requests. Of course,
is still vital and valuable, and Randal Schwartz's
takes a new approach.
There's no standout Ruby on Rails killer yet, as much as Catalyst would like to claim that title. Perhaps the Jifty project from Jesse Vincent and Best Practical will be the right blend of database magic, cleanliness, and intelligent defaults. Certainly its use of continuations is very compelling.
Curtis "Ovid" Poe's
is also compelling, bringing the designish goodness of Perl 6 roles to Perl 5
in a very Perl 5 way.
The venerable object-relational mapper
has some healthy competition in the form of
extension is a worthwhile complement.
Tangram is still
around too... and
might possibly make Rails's ActiveRecord look verbose and clunky, at least when
it gets a little more documentation.
2005 was yet another year for Perl 6. "When will it be out?" you ask. "Sooner, if you help" the designers and implementers and true believers still answer. After struggling for a few months, sooner really is, well, sooner.
Allison Randal, president of the Perl Foundation and project manager of Perl 6, stepped back from both positions to devote more time to coding. Jesse Vincent replaced her as the day-to-day project manager of Perl 6 and now spends his time attempting to elicit progress reports from developers and asking both "What's blocking you?" and "Are you having fun?"
So far, it's working ever better.
Parrot hit a low point late last year, with several unanswered design decisions, a lack of direction, and arguments on the list replacing actual working code. Citing a lack of motivation and time, longtime designer Dan Sugalski stepped down and Perl 5.004 pumpking emeritus Chip Salzenberg surprised many people by volunteering as a replacement.
Though Chip has spent several months fighting a persistent (and, in the opinion of your editor, unfair) legal battle (see GeeksUnite), he's also reviewed several pending design decisions and started to meet the mini-milestones on the way to completing Parrot's design and implementation. For example, calling conventions have improved (yet again, but this time they fixed the sticky continuation issue) and the lexical implementation works.
Thanks and well-wishes go to Dan for several years of tireless and often thankless work. Thanks and well-wishes also go to Chip and Parrot Pumpking Leo Toetsch for their existing service and in anticipation of even more good work.
In semi-related news, the Dutch foundation NLnet generously sponsored two milestones of Parrot development, enabling both Chip and Leo to devote paid time to the project. These two milestones include nine critical subsystems. The developers finished two in 2005, leaving seven to go. (Thanks to Mark Overmeer for corrections here.)
No one could have known at this point last year that a working Perl 6 prototype implementation was only a couple of months away, for various "in one sense or another" definitions of the word "working". The Taiwanese world-traveler Audrey "Autrijus" Tang sparked a near-revolution in two ways. First, by making actual Perl 6 code run with a few weeks of hacking and second, by demonstrating a guided near-anarchy development style that brought new energy, new questions and ideas, and, more importantly, new contributors to the Perl 6 development process.
The project has been exceedingly liberal in granting commit bits (to the point of answering all bug reports and feature requests not with "patches welcome" but "if you check in a test, someone will add the feature"), even to the point of offering commit bits to Python designer Guido van Rossum (who kindly declined).
The roaring velocity of Pugs did slow somewhat after the normal conference season, but lately work has started again on the infrastructural features necessary not only to produce Parrot-compatible code but to support all of the amazingly powerful but conceptually complex features of the Perl 6 object system. Oh, and it looks very likely that the Pugs solution to connecting to Parrot will help the rest of the Parrot and Perl 6 compiler tools evolve and grow too.
In the beginning, there were Perl 6 mailing lists and Requests For Comments. Then Larry began synthesizing and producing long, weighty Apocalypses. Damian followed, extracting the practical wisdom from the Apocalypses to produce practical code examples in Exegeses.
When it's too much work to build thirty pretty great pyramids to show how to build a Really Great Pyramid, pass around blueprints instead.
Now the Apocalypses and Exegeses have become historical curiosities (for various comedic misspellings of the word "hysterical"). The Perl 6 Synopses are the new Apocalypses, except shorter, more accurate, and more up to date. (Pity the poor Perl.com editor who tries to keep Larry's hundreds of thousands of words current. That's a hypothetical editor, by the way, because the one you have now just doesn't.)
Fortunately for mere mortals who lack Larry's and Damian's metaphysical ability to hoist great blocks of metaphorical sandstone into place during the Nile delta's dry season, the Synopses are also easier to read and digest.
Patrick Michaud, Perl 6 pumpking, released the Parrot Grammar Engine early this year. This is (now) a set of Parrot libraries implementing rules (as described in Apocalypse, Exegesis, and Synopsis 5). PGE not only provides regular expression support to all languages hosted on Parrot but it forms the basis of the grammar engine used to build Perl 6.
This has lead to small cleanups in Parrot, namely removing the experimental and long underused regular expression operations.
Patrick's first cut at the grammar engine was a prototype written in C. The current version is a port of that to PIR, the "native language" of Parrot as far as Parrot has a native language that you might actually use to write programs. Perhaps it's a compliment to the design of PIR that Patrick considers this version more maintainable and cleaner than his prototype.
Of course, PIR is an object oriented assembly language with higher-order functions and aggregate data structures.
The next step in the process is a shift-reduce parser. As of the last report, Patrick's code can parse Perl 6 expressions. That leaves the rest of the compiler tool suite to transform the parse tree into code that Parrot (or any other compatible backend) can execute.
A Compiler Roadmap
Speaking of the compiler tools, Allison Randal published her ideas on how to build the full compiler suite for Perl 6 while making the same tools available to any other language hosted on Parrot. In particular, the approach seems to be to perform a series of tree transformations (see "attribute grammars") to turn the output from PGE and the language parser into syntax trees which various tools can analyze, annotate, optimize, and translate into Parrot's AST input format.
Allison's Punie project (which bears some relation to a failed project your editor started a couple of years ago) has succeeded in running a subset of Perl 1 code (you read that correctly) through the entire compiler suite. Perl 1 now runs on Parrot and passes a few tests. At least, part of Perl 1 does, with more on the way.
Though that sounds frivolous, Perl 1 is a real language, if somewhat simpler than Perl 6. If Parrot and the compiler tools work to run Perl 1 programs (and its test suite), it validates the design decisions for the process. It's also a small project with no particular pressure to succeed on its own merits, the nice poetic joy of working aside.
Larry recently revealed that he has almost completed a Perl 5 to Perl 5 translator. That may seem flippant and silly until you realize that he's modified the Perl 5.9.2 parser and lexer to save all of the information — including whitespace and comments — it normally throws away. He can reconstruct the original Perl 5 program from an annotated parse tree.
If he can emit working Perl 5 code, he can emit working Perl 6 code.
It may not be perfect (the latest estimate is that it handles 97% of the core Perl code correctly), but even if it only works for 95% of your code, that's still 95% of your code you don't have to translate by hand.
Audrey Tang and Ingy döt Net (you thought your editor had an odd name) might have a similar project up their collective sleeves, too.
Nicholas Clark, wearer of many hats, officially retired as the Ponie pumpking. He and Jesse Vincent have put out a call for a new Ponie pumpking to work with Nicholas to continue porting Perl 5.10-to-be to Parrot.
There's little point in participating in the Perl community without a community. There may be code out there, but unless people can work together, the code isn't as useful or as well-used as it could be. Fortunately, 2005 was a good year.
The Perl Foundation doesn't lead the Perl community. Its mission is to help things happen when it can, whether by organizing events and donations or to find the right people to fund for projects.
One common concern in recent years has been that TPF seems awfully quiet, apart from issuing a press release about Perl 6 once in a while. This illustrates a sort of truism in open source development. Ideas are cheap. Implementors are worth their weight in gold. After several years of overworking the same volunteers, TPF has recruited new volunteers and harnessed their energy and ideas to revamp the group.
In particular, Bill Odom (the man who knows practically everyone) is now the new TPF president. Jim Brandt, the primary organizer of 2004's successful YAPC::NA in Buffalo, is the new chairman of the YAPC committee. Andy Lester is in charge of PR and grant manager Curtis "Ovid" Poe is now the head of the Grants Committee. Richard Dice, who lead the organization of this year's YAPC::NA in Toronto, replaced Bill Odom as the chairman of the steering committee.
That doesn't mean that everything TPF could possibly do has someone ready, willing, and able to do it — if you'd like to make the Perl community a little better and have the time and energy and commitment to make it so, TPF could use your help!
TPF also launched The Perl Foundation group weblog for its volunteers to communicate with the broader Perl community about what the Foundation is doing, how it's doing it, and why.
Google's Summer of Code
Does anyone remember an Internet before Google? Even if you do, the 800-exabyte gorilla doled out a cool couple of million dollars to sponsor college and graduate students to increase the amount and quality of open source code in the world. Perl had eight projects participating this year.
Leon Brocard kept an eye on all of the projects and posted interviews with each Summer of Code Perl grant recipient.
In addition to Adam Kennedy's PPI project mentioned earlier, TPF funded Ivan Tubert-Brohman to build AnnoCPAN. Ivan's site mirrors the documentation of CPAN modules while allowing users to annotate sections that are confusing or unclear, or to add additional notes where possible. Novices often compare Perl's documentation to that of PHP.net; this is one place to improve and surpass that example.
Ivan launched AnnoCPAN just before YAPC::NA in Toronto. With all of the spare time and goodwill he earned, he immediately set to work making the Perl documentation better indexable.
A Conference for All Seasons
Several YAPCs took place this year, including those in Toronto, Israel, Portugal, and somewhere in Asia. There were also hackathons all over the globe (wherever Audrey Tang is, there's a hackathon), workshops, weekends, and various Perl monger meetings.
The Perl 5 list summaries restarted.
The Perl 6 list summaries did not stop, so they did not restart.
Everything Else That Didn't Fit Into a Parallel Top-Level Heading
CPAN celebrated its 10th anniversary. Take that, pretty much every other language. See you on Parrot.
Did I miss something? Do you want to try prognosticating? What was significant to you? Let us all know right here.
Thanks! and slight correction
That's CGI::Prototype (not -ed), built on Class::Prototyped. Yeah, I've made that mistake a few times myself, so maybe you've picked it up from one of my quick comments somewhere.
Thanks! and slight correction
Link fixed. Thanks for the correction, Randal.
Guido's commit bit
... is actually active, as he accepted the commit bit on OSCON when Larry invited him. We have seen several commits from Larry, but there's currently still zero commits from Guido. Maybe when we start interoperating with PyPy...
Fixed also. Thank you, Aristotle.
The Perl Foundation doesn't lead the Perl community. It's mission is to help things happen when it can, whether by organizing events and donations or to find the right people to fund for projects.
TPF is an organization composed entirely of volunteers. You could easily be a positive influence on the organization.
You mean volunteer.
"governance" of what exactly?
... and remains major concern
If stuff concerns you then volunteer and help fix it. At the very least be specific so other people can help fix whatever problems you are perceiving.
What I find controversial and of major concern is that you misquoted chromatic by adding an improper apstrophe to the word "its."
Actually, I saw the quote and the homophone error and corrected my text without marking it as a correction.
I mean TPF governance. TPF is a legal entity and existing TPF
Those are not new things. In general, some form of reliable,
Anyway, the quoting semantics is preserved by the very tag name. Don't mind, it's immaterial.
Why is copyright an issue?
Sorry. Still don't see what the problem is.
From the legal
TPF happens to hold copyrights for Perl
"It does? As far as I am aware the TPF holds no copyrights on any Perl code. References?"
"...Holds the copyright on Perl 6 and Parrot..." - on the very first page at http://www.perlfoundation.org/.
Also, when you visit there, it is worth reading legal staff published - it
I'll answer other questions a just bit later, sorry...
PS: It seems that this site badly lacks preview for comments:)
I will answer a bit later, sorry...
"What exactly is the concern? What problems have this governance caused?
The common mistake people make is messing governance with
The concern is that TPF, as it is now, is a privately-held legal entity, not
The TPF's governance model is actually scandalous. I was shocked when
I think you had better start by defining exactly what you mean by the "Perl community", because that seems like the place where our understanding starts to diverge.
I believe the agreements are still in the final stages of legal review. TPF's main counsel was on holiday at the end of 2005.
"Why is copyright an issue? The Perl "community" itself holds no
How copyright could possibly not be the issue? Perl and its copyright
"It's very difficult for me to believe that any other governance model
The TPF's current governance model has nothing in common with "meritocratic
"How does community representation work, when of the, say, million people
It is very simple. In our case there are two principal solutions for