Skip to content

Narrow screen resolution Wide screen resolution Auto adjust screen size Increase font size Decrease font size Default font size default color brick color green color
You are here:Home arrow Analyst4j arrow Case Studies
Analyst4j - Case Study Print E-mail

Measure! To Control.

Case Study on using Software Metrics for evaluating code quality and maintainability in Java systems using Analyst4j.

Purpose

To evaluate the use of software metrics in ensuring quality and maintainability in Java systems.

Background

A system under development or maintenance undergoes continuous changes for various purposes. While changes are necessary to meet changing customer needs, ensuring the quality and maintainability of the system prepares the system for further changes without too much effort. Also, on an investment perspective IT systems are expensive to develop and maintain, so ensuring maintainability of the systems gives more longevity and best returns. There is no doubt that people who develop and maintain an IT systems are most important than the actual elements (source code) of a system, but people move on and the only thing that always remains as an asset to the organization is the source code. Therefore ensuring quality of source code also ensures that our best assets have good maintainability.

Software metrics collected directly from source code (Internal Metrics) are being put to use to measure various quality attributes of a system. One among the first software metrics is LOC[Source Lines of Code], which is used to measure the effort and subsequently cost of development of a computer program. Research in software measurement grew and now there are more and better choices of metrics to measure effort, cost, complexity, design quality, testability and other quality aspects.

However software metrics takes lot of effort to measure, if done manually. Automated software metrics involves analyzing the source code to collect the required measurement, thus giving a jump start to quality analysts with consistent measurement. With measurement process automated we can now focus on understanding the implications of data.
 

Introduction

In this case study we shall evaluate how metrics can be useful to ensure quality and maintainability of Java systems. The study shall use Analyst4j, a tool which automates most of the industry standard software metrics and provides visualization features to understand the implications of such measurements.

Applicable Scenarios 

1. In house Java systems/products development.
When your company is developing Java based products and you as a Quality Analyst or Project Manager would like to measure the quality of code and demonstrate or ensure maintainability of the product being developed.

2. Outsourced Java maintenance.
When you have outsourced or subcontracted maintenance/development and you would like to measure the changes, ensure that code entropy is under control and that there is no regression in quality of code.

3. Adopting open source products.
When you are evaluating open source products and you want to evaluate whether the system is being well maintained and progressive.

4. In General
When you have two or more versions of a Java system and would like understand the changes amongst the versions with the help of software metrics.

Questions

Some of the questions that arises from the above scenario and in quality control initiatives are
 

Case Study Candidate

JFreechart is an open source charting solution / component, a highly preferred solution when it comes to charting solution in Java. More details at http://www.jfree.org/jfreechart/

We shall take four versions of JFreechart source code for our analysis.
1. jfreechart-0.9.15
2. jfreechart-1.0.1-pre1
3. jfreechart-1.0.1
4. jfreechart-1.0.5

Scope

We shall analyze the JFreechart source code with Analyst4j and derive answers for our questions with reference to the scenarios mentioned above.

Analysis

  • Preparing the workspace.
  • Analyzing versions of JFreechart source code one by one.
  • Answering the questions.
Preparing the workspace
  • 1. Download the source code of Jfreechart versions. 
  • 2. Install Analyst4j - Plugin of Eclipse 3.1 [or above] or Analyst4j Standalone. 
  • 3. Open Eclipse and enter a new workspace path (for e.g., C:\MetricsCaseStudy).
  • 4. Create a Java Project using ant build scripts or manually for each version of JFreechart. The workspace should resemble as follows.
  • Workspace

Analyzing versions of JFreechart source code one by one

  • 1. Right click on a project node (for e.g., jfreechart-0.9.15) in the "Package Explorer" and choose "Analyst4j->Analyze" in the context menu.
  • Analyze
  • 2. In the dialog box that follows the above action, Select/Un Select packages/files which are not to be included in the analysis. In our case, we have excluded all "*.junit" packages from the analysis.
  • Selective Analysis
  • 3. Click "Finish" to kick start the source code analysis.
  • A4j Static Code Analysis Progress
  • 4. When the analysis complete, the following analysis report is displayed.
  • A4j Analysis Report
  • 5. Perform the steps 1 - 3 for all the versions in our case study one by one.

Quality Analysis Results

The following analysis reports are possible once we complete analyzing all the four version of our case study.

General  Project Analysis Report Comparison

Comparing some of the metrics in the analysis reports of the case study gives us a higher level view of the changes that has gone through the versions. The following table (Table1) represent the changes in our case study.

 Metricjfreechart-0.9.15jfreechart-1.0.1-pre1jfreechart-1.0.1jfreechart-1.0.5
Analyzed Files 538454 464 503 
N.of Packages
22
33
33
35
N.of Classes
480
388
397
436
N.of Interfaces
75
82
82
85
MI / File
111.83
110.80
108.31
107.56
HE / File
237608.58
298603.55
308292.38
325631.26
NLOC
62559
61238
68658
74429
POC / Method
51.33
52.57
52.11
53.82
DIT 7
7
7
7
 ..more    

MI = SEI Maintainability Index, HE = Halstead Effort, NLOC = Number of Lines of Code, POC = Percentage of Comments, DIT = Max Depth of Inheritance.

Download Complete Report(rtf)  

It can observed that effort (HE) increases with new versions and maintainability(MI) is slightly reducing. However maintainability is way ahead of the guideline threshold of 65, and that maintainability is tolerant considering the change in effort(HE) and NLOC across versions.

Measuring Design Changes / Regression

We shall attempt to use one ore more automated software metric to evaluate and understand the design changes at different level of code. The levels being Project , Package , Class & Method . 

Project Level: Amongst other measurements, higher level design changes can be measured by measuring increase / decrease in number of Packages, Interfaces and Max Depth of Inheritance. The following table (Table2) represents these measurements.

 Metricjfreechart-0.9.15jfreechart-1.0.1-pre1jfreechart-1.0.1jfreechart-1.0.5
N.of Packages
22
33
33
35
N.of Interfaces
75
82
82
85
DIT 7
7
7
7

DIT = Max Depth of Inheritance.

It can be observed that

  1. DIT i.e maximum depth of inheritance tree of the project has remained same across all the versions and this is a good indicator that since inception of the project there has been no major changes in the design.
  2. There has been no change in design in pre-release (jfreechart-1.0.1-pre1) and the release (jfreechart-1.0.1) for the fact that No.Of Package, No.Of Interfaces and DIT have remained constant between these versions.
  3. In general it can be said that though there are design changes to meet various functional/feature additions, the original design concept has remained intact throughout these versions.

Package Level: To understand package level design changes, a quality analysis graph of number of interface vs number of classes gives instant visibility of changes that has happened across the version.

Interfaces vs Classes

*Graph generated by Analyst4j, clicking on the graph points lists the candidates of the value in QA Cases View.

It can be observed that

  1. There is a visible difference in the  package org.jfree.data (the distinct red square on the top right of the above graph), had around 34 interfaces in jfreechart-0.9.15 and that in subsequent versions it has been distributed into one or more sub packages. We can understand from the Tabel2 , that the total number of interfaces had always been increasing with each newer version.
  2. The package org.jfree.chart.renderer (red square on the left top of the above graph) which had 55 classes(concentrated implementation) has been distributed into more sub-packages in the subsequent versions(dilution).

Class Level: If design patterns are best practices to be followed during design, antipatterns on the other hand find faulty design patterns.  Analyst4j introduced the idea of identifying antipatterns in Java using software metrics, the following graph is an application of the idea to the case study.

Class Antipatterns

*Generated by Analyst4j, clicking on the bars in graph lists the candidates of the value in QA Cases View.

Report

*Report generated using Analyst4j. Download Complete Report (rtf) 

It can be observed that

  1. Referring to the total number of classes of each version in Table1, it can be understood that though there was a decline in number of classes in jfreechart-1.0.1-pre1(388) compared to jfreechart-0.9.15(480) the number of antipatterns classes have increased. This indicates that there has been high consolidation effort amongst these versions and a similar pattern can be seen when the system moved from jfreechart-1.0.1-pre to jfreechart-1.0.1.
  2. Swiss Knife Classes have remained same 2nd and 3rd version and we could see an addition(SpiderWebPlot Class) in the latest version. This means that SpiderWebPlot has undergone rapid functionality addition or it is totally new in the latest version.

Method Level: One way to understand design changes / regression in methods is to measure its structuredness(Essential Complexity) and complexity (Cyclomatic Complexity). The following graph represents the number of methods whose CC(Cyclomatic Complexity) is greater than 40 and EC(Essential Complexity) > 1. The fundamental in the thresholds being that we want to find structural changes in highly complex methods across versions.

Search using metrics

Search

Search results across projects graph, shows increase in unstructured code across versions.

Spaghetti Code

*Generated using Analyst4j, Search using metrics feature

 

Report

*Report generated using Analyst4j. Download Complete Report(rtf)

It can be observed that:

  1. Spaghetti code or unstructuredness is slowly increasing with each newer version.
  2. Although unstructured code is not a good sign for good maintainability, under inevitable circumstances such code could be known exceptions. Also, the graph reveals that there is no sudden increase of spaghetti code which is a good sign maintainable code.

 

Measuring Effort Change
We shall attempt to use one or more automated software metric to evaluate and understand the effort changes at different level of code. The levels being Project , File , Class & Method.

Project Level: The following table (Table 3) represents list of effort change (using Halstead Effort) across the four versions of our case study.

 Metricjfreechart-0.9.15jfreechart-1.0.1-pre1jfreechart-1.0.1jfreechart-1.0.5
Analyzed Files 538 454 464 503 
HE / File 237608.58
298603.55
308292.38
325631.26
Total Effort 127833421.42135566011.7143047668.96163792528.81
Effort IncreaseN/A
+7732590.28
+7481657.26
+20744859.85
Release Date
2003-11-28
2004-11-29
2006-01-27
2007-03-23

HE = Halstead Effort, Total Effort(TE) = HE * Analyzed Files.

Effort Increase = Current Version TE - Previous Version TE.

It can be observed that:

  1. Effort Increase indicates that there has considerable additions/improvement of functionality in the system.
  2. The surge in effort in the latest version (jfreechart-1.0.1 to jfreechart-1.0.5) compared to previous effort increase (jfreechart-1.0.1-pre to jfreechart-1.0.1) indicates that there has been more additions / improvement lately.
  3. Though the variation in Number of Analyzed Files suggests that some files were removed, the constant increase in Average Halstead Effort per file and Total Effort indicate that there has been some restructuring and that the functionality of the files where not actually removed.

File Level: To understand the change in effort at file level we plot a graph using NLOC Vs Halstead Effort in File, which should reveal file level difference in effort. The following QA graph is generated using Analyst4j represents the change in NLOC and HE for our case study.

NLOC Vs HE
A closer study of the graph and individual cases

Study

We can track the changes in effort in two prominent cases

  1. CategoryPlot.java, some amount of code had been removed/commented out in version jfreechart-1.0.1-pre1. The change in HE for CategoryPlot.java can be observed in the above graph.
  2. XYPlot.java, effort on this file has always been increasing and that there is noticeable increase in HE in the latest version (jfreechart-1.0.5) compared to its previous version.

Class Level: Effort change at class level can be measured using Weighted Methods Per Class, the increase/decrease in WMC would reflect the changes made in a class. The following distribution graph of WMC across the four version of our case study gives us a picture of the changes.

WMC
*Generated using Analyst4j, Threshold categorization can be customized and cases are listed in QA Cases View when the graph bars are clicked.

Method Level: The following comparison analysis  graph of NLOC Vs Cyclomatic Complexity and Decision Density represent the effort changes across two versions of our case study(jfreechart-1.0.1 and jfreechart-1.0.5).

NLOCVsCC
NOSVsCC

We can track changes in some prominent cases from the above graphs.

Decision Density: "draw" method of XYPlot class has increase in decision, "executeQuery" method of JDBCXYDataSet class has remained the same and "drawItem" method of CyclicXYItemRenderer class decision density has reduced.

Cyclomatic Complexity:  The complexity of "drawSide", "draw" methods of PiePlot3D have remained the same though there has been considerable reduction in NLOC. However the complexity of "draw" method of XYPlot has increased considerably.

Conclusion

The case study demonstrates that using software metrics during development or maintenance to evaluate the quality of code is found to be very useful to quickly understand the changes in the code. 

 
Next >
[ Back ]