Updating Seven Centuries of Scottish Land Rights
by Ed Stephenson
Siemens Business Services Chooses Perl for Its Landmark Application
By their very nature, databases are often indispensable; but rarely do they carry the tradition, symbolism, and historical importance equal to the Registers of Scotland. Originally founded seven centuries ago--not long after King John signed the Magna Carta--these registers hold millions of records and guarantee the property rights of individuals throughout the country. Bringing this database into the digital age posed serious problems for Siemens Business Services, until the development team discovered Perl.
The application Siemens developed is not a historical archive but the centerpiece of a very active system. By law, all heritable property in Scotland today must be registered in order to guarantee title. And with more than 95,000 applications for title transfer per year, Siemens' U.K. government customer, the Registers of Scotland Executive Agency (ROS), is a busy organization. Everyday, employees at more than 1,000 workstations in Edinburgh and Glasgow access and update files that define each property, its current owners, price, and any mortgages and conditions affecting it. This new electronic register replaces property deeds, with the state guaranteeing details against error or inaccuracy.
"I think this is the first use of 100 percent Perl in a large client-server application, especially for a government customer," remarks John Clutterbuck, Principal Consultant with Siemens' Geographical Information Services (GIS) Group and prime architect of the Land Register System (LRS) application. "LRS is a very text-centric application. A lot of the formatting for addresses and other components is quite complex. Perl is very easy to use for this because it is so flexible."
Yet, Clutterbuck hadn't considered Perl when his team began work on the LRS application in 1994. It wasn't until a critical juncture early in the development, when his team attempted to convert half a million ASCII text files from an old mainframe system, that they used Perl as a temporary fix. The language worked so well that when the time came to reengineer the entire application, they chose Perl/Tk over several other programming solutions.
A Tale of Two Registers
Scotland actually has two massive registers for property. The first, and still the larger, is the time-honored "General Register of Sasines" (from an old French word "to seize"), created by the Scottish Parliament in the 17th century to record property transactions in the form of writs and deeds.
In 1979, the ROS Agency introduced the new Land Register to gradually supersede this old system, recording the exact legal status of property and defining locations on a national mapping system. Rather than convert Sasines data wholesale, the Land Register recorded titles only when property changed hands, and in the 15 years preceding Siemen Business Services' involvement, ROS had made the new system active for only half of Scotland's 33 counties. The rest of the country--and most of the property--was still recorded as hard-copy documents in the Sasines Register.
The Land Register stored titles on a mainframe, but most activities were still performed with paper. To replace this system, Clutterbuck's team created a prototype with Informix Hyperscript, a fourth-generation programming language also known as HST/Wingz, for client workstations using Windows 3.1 and Solaris. It was the first truly computerized workflow and database system for Scottish land registration.
"The Land Register System (LRS) application was initially used for three 'new' Land Register counties," Clutterbuck recalls, referring to counties that still relied on the Sasine Register for deeds and transactions. The first version of the HST/Wingz-based application was limited in the size of its text fields, yet for two years "it performed well enough for ROS to want to transfer the mature counties over to it."
Perl to the Rescue
That's when Clutterbuck's team ran into trouble. "Mature counties" were those recorded on the Land Register's mainframe after 1979. Roughly 500,000 titles in 17 counties were stored in a Unisys machine running a product called Mapper. In 1996, the development team began to convert the unstructured ASCII text files from Mapper for use in the LRS Informix database, a task involving tedious data mining to extract names, addresses, and other components using patterns within the files.
"This was originally done using sed and awk, which worked moderately successfully for the first county attempted," Clutterbuck explains. "We then found that the data varied enormously for the next county." The conversion tools became unwieldy and slow, a disaster in the making. No one on Clutterbuck's team had actually used Perl, but "after a brief investigation, we could see that with its regular expressions and unlimited string lengths, Perl would enable us to selectively replace the conversion tool components. Early attempts proved successful, so more and more were replaced."
O'Reilly recently released (July 2001) Learning Perl, 3rd Edition, which has been updated to cover Perl Version 5.6 and rewritten to reflect the needs of programmers learning Perl today.
After the conversion, Perl 4.0 steadily became the team's language of choice for many server-based aspects of LRS, such as general administration and reporting scripts. So, when Clutterbuck considered alternatives for a major revision of the Land Register System (LRS) application in 1998 to meet Y2K requirements and ROS Agency's new Windows NT workstation environment, Perl 5.0 became a candidate.
"To upgrade to the new version of HST/Wingz would have meant a repurchase by ROS," Clutterbuck explains. "This created the opportunity to change products. LRS had to run on Solaris as well as Windows NT, so the Microsoft stable--Visual Basic and the rest--were ruled out. There was some pressure to use Java, but Perl was ideal for this and many of the routines had already been developed."
The team convinced ROS through a series of demonstrations that Perl/Tk was most appropriate for the Land Register System upgrade. They wanted to show the formatting in a WYSIWYG fashion as users input data, and having Perl within the GUI made sense. At least, it did at first. Among numerous obstacles the team encountered during the upgrade's development, the GUI design took the most time to resolve.
"We could have done with a good GUI builder," Clutterbuck admits. "There are a lot of IDEs (integrated development environments) appearing now, which, if we had known about then, might have made the task easier. Our experiences showed that using hybrid technology always adds problems. As with all first forays into unknown territory, one tends to live with the initial mistakes."
This foray worked quite well. Development on the upgrade started in early 1999, and by the end of the year, LRS was rolled out to 600 client workstations with 11 Irix servers. Perl/Tk enabled the team to reduce approximately 200 workflow screens, including numerous popup dialogs, down to about 50, and processing simple transfer applications has gone from weeks to days. Currently, LRS holds details of more than 600,000 titles in 22 counties, and by 2003, the computerized Land Register will be live in all 33 counties.
"This project has certainly added to the confidence of using open source products within Siemens," Clutterbuck says. His own crew plans to use Perl for a long time. Recently, they released ready-to-go Perl builds, with lots of modules, onto CPAN for Windows 95/98/NT, Solaris, and Linux. As Clutterbuck has experienced over the past seven years, both Scotland's Land Register and Perl have come a long way from their hoary beginnings.
Learn how large and small companies are putting Perl to work by reading other Perl Success Stories.