ONJava.com -- The Independent Source for Enterprise Java
oreilly.comSafari Books Online.Conferences.


AddThis Social Bookmark Button

Java Performance Tuning Strategy

by Jack Shirazi

This article summarizes tuning experiences I've had on development teams with various levels of tuning experience. These summaries may help provide you with a broad outline for creating your own tuning strategy. I've categorized the team experiences into five levels, from no experience (level 1) to fully competent (level 5). Note that these are summaries, not actual reports, so no summary corresponds to any one development team I have worked with.

Level 1

Characterized by:
No tuning experience or strategy.
Major tuning target:
Design and architecture.
Tuning rationale:
The biggest performance improvement in an application is typically achieved by eliminating badly performing designs and architectures. Consequently, the design and architecture is the first aspect that needs to be visited.

Given the team's lack of tuning experience and strategy, they need to define performance targets; address design and architecture-performance issues, including hardware and operating system configuration; and ensure that the application performance is measureable for the given targets. The team must develop a test environment, which represents the running system. Chapters 1, 2, 10, 12, and 13 of Java Performance Tuning provide further detail and cover everything from setting targets and measuring performance to hardware and operating system issues.

Level 1 Summary

We start with a half- to full-day presentation of the overall architecture of the system. The focus is entirely on performance, analyzing the architecture for potential bottlenecks. (The obvious bottlenecks are easy to spot: they are the connecting lines on the diagrams, the single threaded components, the components with many connecting lines attached, and so on.) Refer the team to Chapter 13 of Java Performance Tuning, which discusses many generic architecture and design issues that affect performance, with an emphasis on shared resources. In addition, Chapter 10 on multithreading and Chapter 12 on distributed computing may be applicable.

Inform the team that they need to define performance targets. These targets should be detailed and sufficient to cover all identified bottlenecks and performance requirements. Initially, these targets can take the form of selecting the major user activities and specifying expected response times for these activities. Refer the team to Chapter 1 for more on setting targets and measuring performance.

Finally, ensure that the team will address the identified design and architecture issues, including any hardware or operating system configuration changes that may be identified. Ensure also that application performance is measurable for the given performance targets. The team must develop a test environment, which represents the running system. This test-bed should support testing the application at different loads, including a low load and a fully scaled load representing maximum expected usage. Refer to Chapters 1, 2, and 13, which discuss performance measurement techniques at the Java and the operating system level.

Level 2

Characterized by:
Team has performance targets and test-bed, but no code-tuning experience.
Major tuning target:
VM (virtual machine) and compiler.
Tuning rationale:
After targeting design and architecture, the biggest bang for your buck in terms of improving performance is choosing a better VM, and then choosing a better compiler.

The team has a benchmark harness on the application or on a good simulation of the application. However, the team has no code-tuning experience. The team should target the gross tuning aspects of compilers and VMs, and also needs to develop tuning expertise. Chapter 2 on tuning strategies and profiling and measuring performance, Chapter 3 on VMs and compilers, and the checklists in Chapters 4 through 12 all provide important information.

Level 2 Summary

The team has achieved resolution of the issues raised at level 1. As a consequence of now having well-defined performance targets and also being able to accurately measure performance across application boundaries, the actual application performance shortfall can be determined. Having done so, the design and architecture should be revisited because some aspects not previously noticed may now be highlighted by the results of the performance measurements. Refer the team to Chapter 13 of Java Performance Tuning, which discusses many generic architecture and design issues that affect performance.

The team should try out various compilers and, if applicable, various VMs to determine whether a different configuration will improve the performance of the application. Because these changes should not affect the code in any significant way, this should be a low-cost, high-impact performance tuning measure. (Sometimes different compilers and VMs require changes to the code, which may rule them out.) Refer the team to Chapter 3, which covers the differences between VMs and compilers.

The team also needs to develop code-tuning expertise. A prerequisite to developing tuning expertise is that the test-bed is ready to be used and that the developers are able to run tests, make changes quickly, and easily rerun tests. Assuming this ability is present, code profiling and tuning (sometimes called microtuning) can be started. The first tuning activity should be single-user tuning, and introduces the team to how to tune application code. Refer to Chapter 2, which discusses performance measurement techniques and profiling for Java code.

As the developers become more comfortable with the tuning procedures, each subsequent tuning activity speeds up. At this stage, the team's initial proof-of-concept tuning takes half an hour to half a day per activity. The time difference depends on how difficult it is to track down bottlenecks, and how easy it is to identify and remove the bottleneck. Refer the team to the performance checklists in Chapters 4 to 12, which give summary lists of techniques for speeding up Java code.

At level 2, proof of concept on removing the bottleneck is preferred over completely fixing and releasing the changes, so that the team gains expertise quickly in identifying and fixing bottlenecks, without spending too much time on associated standard development activities such as integration, documentation, testing, and so on. Proof-of-concept bottleneck removal consists of using profilers to identify bottlenecks, then making simplified changes, which may only improve the performance at the bottleneck for a specialized set of activities, then proceeding to the next bottleneck.

Multiuser testing can be difficult to set up, and the team needs to be aware that incorrectly simulated data or user-activity patterns can invalidate the testing completely. All tuning time could be targeted on the wrong parts of the system and would be completely wasted.

Multiuser tests can typically take a full day to run and analyze. Furthermore, the analysis often identifies several factors that may need to be varied, and many of these factors may need to be varied independently. For this reason, even simple multiuser performance tuning can take several weeks. My usual mode of operation is to analyze and identify possible problems, target the factors most likely to provide an immediate performance boost, and attempt to retest once with those factors altered appropriately. Refer the team to Chapters 1 and 13, which both discuss multiuser tuning.

Level 3

Characterized by:
Has code tuning expertise, but not yet tuned code.
Major tuning target:
Java code (using standard tuning techniques).
Tuning rationale:
Having tuned the design and architecture, and having selected improved VMs and compilers, the major bottlenecks remaining are probably due to the Java code implementation used.

The team has addressed level 2 issues, so the remaining obvious target is the Java code. The team now has the expertise to tune this. Relevant chapters from Java Performance Tuning are Chapters 1 and 2, on tuning strategies; and 4 through 12, on tuning techniques.

Level 3 Summary

Because the architecture and configuration is now deemed adequate, the team should now target bottlenecks in the Java code using their code-tuning expertise. The tuning strategy is straightforward. Individual improvements are likely to be one of a standard set, listed in the performance checklists of Chapters 4 to 12. In addition, the main body of text in Chapters 4 to 12 provide many examples of performance tuning Java code, and it is probable that one or more of the examples will be directly applicable.

Level 4

Characterized by:
Tuned code is still not fast enough.
Major tuning target:
Java code (using all tuning techniques).
Tuning rationale:
The Java code has been tuned to level 3, but the application is still not fast enough. Now you have to pull out all the stops.

The team has tuned to level 3, but the application is still not fast enough. Now every technique available to the team must be used. This may make some parts of the code difficult to maintain. Chapters 4 to 12 of Java Performance Tuning provide tuning techniques, with emphasis on examples in Chapters 7 and 11.

Level 4 Summary

It is unusual to come across an application requiring level 4 tuning. Sometimes these are specialist applications involving graphical games or intensive numerical calculations. More often, there are isolated subsections of applications that require more intensive tuning. Tuning according to level 3 still applies. However, profiling is often of little use at level 4. Instead loops, structures, and algorithms must be targeted more intensely, and performance improvements must be gradually gained after many successive changes. Level 4 tuning can take a lot of time, and it also leaves the code looking much more complex and difficult to maintain.

Chapters 4 to 12 provide examples using many of these techiques. And these chapters also provide examples of lesser-used performance tuning techniques. Chapters 7 and 11 specifically consider the detailed changes needed for level 4 tuning applied to loops, structures, and algorithms. In addition, Chapter 14 covers tuning the operating system. At level 4, tuning the operating system begins to be more important.

Level 5

Characterized by:
Application currently has adequate performance, and needs to be monitored.
Major tuning target:
Tuning rationale:
The application has adequate performance and its performance needs to be monitored to ensure that it remains adequate.
Tuning rationale:
The application has adequate performance, and its performance needs to be continually monitored to ensure that performance degradations are dealt with quickly.

Level 5 Summary

The application currently has adequate performance. If this is a shrink-wrapped application, feedback must be examined to identify any parts of the application that have inadequate performance, and these should be addressed in future versions. In enterprise systems, performance should be continually monitored to ensure that any performance degradation can be promptly identified and addressed. Chapters 13 and 14 of Java Performance Tuningprovide details on monitoring and maintaining performance of running systems.

Jack Shirazi is an independent consultant. He was an early adopter of Java, and for the last few years has consulted mainly for the financial sector, focusing on Java performance. He has even set up his own Java Performance Tuning Web site. Before using Java, Jack spent many years tuning Smalltalk apps, and early in his career was involved in research in theoretical physics and bioinformatics. Jack has published in the field of protein structure and is proud to have contributed to some of the core Perl5 modules. He can be reached at jack@JavaPerformanceTuning.com.

O'Reilly & Associates recently released (September 2000) Java Performance Tuning.