It's very much true that significant performance improvements in the application layer can be obtained by the application of proper software engineering principles. Some of the biggest performance improvements come about by algorithmic change; after that, just writing clean code helps immensely. The second edition of System Performance Tuning touches on this a bit in Chapter 8 -- of course, a full discussion takes a book in and of itself (I am partial to Fowler's "Refactoring," the application of which becomes somewhat obvious.)
However, we don't always have the ability to change the source code -- certainly the average software development team is not going to have the time to attempt to effect significant speedups in the compiler (as would be necessary in the case study illustrated here).
Part of systems engineering is often having to work with what you have, even if it is inelegant and poorly written.