Irrespective of the language programmers choose for expressing solutions, their wants and needs are similar. They need to be productive and efficient, with technologies that do not get in the way but rather help them produce high-quality software. In this article, we share our top ten list of programmers' common wants and needs.
Many of us at SpikeSource are actively using new languages such as Ruby, Python, and PHP; have used class libraries from Java, C++, and .NET; have developed component-based models in MFC, STL, Rogue Wave, CORBA, COM/DCOM, OLE, and ActiveX; have programmed in scripting languages such as JSP/ASP/CFML/VB; and have built architectures based on design patterns and service-oriented architectures (SOA). Many of the concepts promoted as part of these technologies, such as reusability (class libraries), separation of interfaces from implementations (CORBA, COM), and structured and modular programming (design patterns and SOA), are powerful and very useful. At the same time, however, how much productivity improvement have we have seen? It's clear that these library and languages features aren't enough to be useful by themselves.
This leads us to ask what really makes us productive. We thought about the systems we'd used, and the things we'd wished for at 2 a.m. on a death march. In general, our collective wants are:
Here are the top ten items that we believe programmers want to see.
Programmers don't like convoluted designs or overengineered modules. Programmers want simple, predictable, and consistent interfaces that require minimal effort on design, development, deployment, and runtime requirements. We call that the "what you need is what you use" (WYNIWYU) approach. Apache httpd with its mod_* architecture is a great example of simple design that supports multiple language runtimes. Typically, open source components--due to their very nature of peer-level meritocracy and continuous design--tend to provide simple and consistent interfaces. PHP, Python, and Ruby have gained adoption because of their simplicity and intuitiveness. Consistency and predictability of interfaces are important for making programmers productive; for example, Michael Daconta, in his book Java Pitfalls, discusses various issues that confuse programmers in Java.
One key to a short learning curve is documentation: in addition to quick-start guides and reference materials, the ability to bounce ideas off others and get technical product reviews helps a programmer be productive. The documentation for PHP and MySQL is good, but they're even more useful with the Architecture of Participation philosophy, which allows users to create notes in the documentation itself. Often users' fresh eyes can provide valuable insight that the technology provider overlooks. Community sites and a clear and recognizable forum--be it IRC, a mailing list, a newsgroup, or a web site--make it easy to climb the learning curve.
Programmers like well-optimized, atomic subroutines that are ready to use.
Data structure algorithms are a good example of this. How many of us have
qsort algorithm? Very few, I imagine. We either find the
algorithm in a textbook or online. Programmers want to leverage deep libraries
of prebuilt components complete with clean APIs, documentation of best
practices, and code samples. Class libraries are reusable, but with the caveat
that they be bounded by the language runtimes. The Standard Template
Libraries (STL) for C++ that uses templates was a very good idea--it would
have been more successful if it had support for the other languages. One of
the keys to Perl's success was its huge library of prebuilt code, CPAN, and
similarly the PEAR for PHP.
Often good ideas sprout ports in different languages. For example, consider Log4J, Log4php, and Log4rss for logging, and Junit, HTTPUnit, CPPUnit, Nunit, and PHPUnit for unit testing. Rather than doing the porting as an afterthought, an algorithmic approach (as in data structures or design patterns) to implement various common routines defined as "application service modules" across languages improves consistency and shortens learning curves. We need a repository of common services, such as order status notification, forgotten password service, and email template generation, available as pseudo code-based algorithms along with language-specific implementations. Combining design patterns with service-oriented architectures (SOA) applied to application-specific constructs is a step in this direction.
If there existed an exhaustive collection of reusable, extensible application service modules implemented using SOA, with interfaces for different languages (Java, C, PHP, Python), why would a programmer need to implement a new module? Even if there were such a need, he could contribute it back to the community for other programmers to use.
Programmers typically overlook migration tools and upgrade mechanisms. A simple example is LDAP schema mismatches between different components that use LDAP services. Similarly, anyone who has developed with SOAP and web services will appreciate how migration issues can eat up valuable development time. Specification version mismatches and vendor implementations add up to a headache for programmers, let alone users.
Tools that let programmers leverage standard data export and import mechanisms are very useful. An example might be the ability to convert spreadsheet data into database schema values or XMLSchema instances. There often exists an expectation mismatch between a programmer who tends to view things at a lower level (be it data stored in database rows, with SQL batch-loader scripts for loading data and web- or CLI-based query/update/insert/delete operations) and an administrator who has maintained data using local spreadsheets and wants to upload the data with a single click.
Similarly, migrating configuration data through version upgrades is a task that deserves abstraction to a level that does not require hand-tuning.
A comprehensive functional module is an end-to-end, integrated, self-contained horizontal framework (such as a portal or content management system) instantiated with the right configurations and settings for immediate usage. Application developers like to have usage-specific (for instance, a portal framework targeted for an employee portal versus a supplier portal), comprehensive functional modules that are ready to use within their given ecosystems. For example, a programmer developing an employee portal wants to have adaptors and connectors for LDAP integration, personalization capabilities, and groupware systems. Typically, developers of software components predicate everything on that component being the center of the universe, while in a real-world scenario that specific component becomes one of the peripherals. Our employee portal at SpikeSource becomes one of the components we integrate with our help desk (OTRS), SFA/CRM systems (SugarCRM), federated search (Lucene/Nutch), and email systems.
Typical development projects use only one language or toolkit, which makes it difficult for programmers to use features and services available in other languages. Service-oriented architectures solve this problem. A loosely coupled federation of modules with interfaces separated from implementation is useful for implementations that spans across systems and language boundaries. As one of my colleagues says, "An easy answer to accessibility might be SOA--drop it behind a web service. A web service is the only way for shared libraries. Web services have already defined all the data-type translations and support objects." This quote occurred within the context of using a <CFMAIL> tag for generating dynamic email within CFML, while another programmer reimplemented the same feature in PHP. If this behavior existed as a web service, there would be less duplication of code, and we'd have saved the time of implementing and testing the new implementation. This brings out the importance of application service modules, either as SOA services or simple subroutines.
Not only does SOA solve the problem of language interoperability, but it also allows programmers to be creative. As Tim O'Reilly put it, SOA and web services allow the rip, mix, and burn of the web sites.
Another good example for plug-and-play architecture is again Apache's mod_* architecture. Apache httpd is the only server module that caters to the "academic definition" of an application server. (I define an app server as a managed runtime environment in which to develop, deploy, and monitor applications independent of language runtimes.) The so-called application servers that exist today are Java servlet servers (Tomcat) and J2EE servers (Geronimo, JBOSS). PHP/Python app servers (Active Grid), while not broad "generic application servers," do have well-defined parameters in their own vertical space. However, programmers like runtime environments that can deal with multiple language runtimes--hence the success of Apache.
Performance-optimized blueprints, with associated physical and logical architectures targeted for specific usages and language runtimes, are essential for programmers to guarantee or provide guidelines for runtime SLAs of their implementation. Programmers need development time instrumentation (instead of post-development benchmarking) that can help calibrate the factors that impact performance--including latency, throughput, and the concept of concurrent versus named users--and think time between application accesses. Compile-time switches to identify performance bottlenecks and profiling tools associated with unit testing will be very helpful. Programmers need tools and online performance calculators to validate whether an application that works great on a desktop makes sense to deploy on an 8-way cluster, a 32-way server, or a 64-bit OS. Many vendors provide blueprints for infrastructure components such as databases and commercial application servers, but many applications still have very little tuning information available, let alone design blueprints.
Wouldn't it be nice if programmers had a place to visit to calibrate performance requirements based on the libraries and modules they use?
Programmers need simple tools to convert and validate proprietary (but popular) formats to standards-based formats. For example, tools that help in the conversion from ad hoc XML or XML DTD to XMLSchema are very useful for interoperability and for complying with open standards.
Another need is to compare and contrast feature sets based on the standards environment (see MySQL's crash-me).
Integrated and extensible build-debug-deploy tools that make programmers productive through rapid development and deployment help to improve productivity during development time. Examples include Emacs, Eclipse, and Ant. The extensibility and plugins of both Eclipse and Emacs let users streamline time-consuming tasks (such as refactoring or reformatting) that computers can do better than humans. Ant gives Java developers the ability to build and deploy their applications quickly. The time invested in learning such tools as these quickly pays off in the large amount of time they save.
A common repository, along with a quick-reference glossary of configuration parameters with optimized name-value pairs, error codes, resource bundles for localized versions, expected behaviors, and reported exceptions, would be very useful.
Oracle users have a good example of this in action. A simple error-reporting tool called oraerror lets Oracle programmers quickly locate the meaning behind error codes.
Having debugging tools, profiling tools, probes, and heart-ping-monitoring messages as an integral part of the libraries is handy. For example, it's useful to have a cron-based script that works with Log4J to monitor the disk space and prune or clean the logs if the disk space is low. Similarly useful are simple interfaces that can dump the session and runtime state of the systems along with corresponding libraries.
To summarize, we have seen great changes happening to make programmers more productive. Many of these have appeared within the last five years. Open source projects such as Ant, Maven, and Junit; source code repositories such as SourceForge, Tigris, Berlios, and Apache; and search sites such as Google, Freshmeat, CodeZoo, and Koders.com have helped programmers become more productive. Further, we believe an open source development approach applied to new programming models like SOA along with extensible tools like Eclipse will empower programmers to achieve the true goals of modular programming, interoperability, and runtime independence.
Murugan Pal is the founder and CTO of SpikeSource, Inc.
Return to ONLamp.com.
Copyright © 2009 O'Reilly Media, Inc.