Measurement in the Software Engineering Process - METRICS (1-day course)

Note: this course is one of five available from the PICO project. This is an EU project developing training materials in which Insight Consulting participated. The other courses are 'Process Analysis', 'Goal Based Improvement Planning', 'Experiences with Improvement Projects' and 'Business Goals and Improvement Strategies'.

Summary
This module provides knowledge and understanding of what is involved in introducing and performing measurement in order to improve software processes and products. It explains the types of process and product measurements that can be made during software projects and includes a Case Study that illustrates some of the practical issues arising from a software measurement program. It is based on the 'Goal Question Metric' approach to linking measurement with business goals.

Learning Objectives
Upon completion of this course you will be able to:

• Explain the reasons for measurement within the software process (including software development and software-related services), and its advantages.
• Understand the relationship of software measurement to the achievement of business goals and process improvement.
• Explain the need to specify software fully and to evaluate it against quantitative requirements.
• Define a generic set of qualitative requirements that are common to all software systems.
• Define a set of measures that can be used to express these requirements in quantifiable terms
• Explain the main objectives of cost estimating for software projects
• Describe how cost estimation should be performed during projects.
• Explain how to implement a measurement programme which helps managers with :
  • decision making
  • understanding, controlling and improving
  • setting quantifiable goals
  • identifying where changes are needed

Level and Orientation of Audience
The principal target audience is software development managers, project managers, team leaders and software engineers who will be either:

• directly involved in developing software products and running projects (and therefore performing measurement)
• responsible for product performance and project achievement (and therefore making use of measurements)

It will also be useful for senior managers who need to understand how a software measurement programme can provide value and benefits, and in particular in a way that is linked to their business goals quality or technical staff involved in implementing a measurement programme or improvement initiative.

Topics Covered

Measurement in the software engineering process
This session explains the reasons for measurement within the software process (including software development and software-related services), and its advantages. It relates software measurement to the achievement of business goals and to achieving process improvement.

The Goal Question Metric (GQM) approach - Introduction:
The GQM approach is recognised and built into many successful approaches to software measurement:

• METKIT : identifies 20 practical process/project measures, mostly based on existing information.
• PYRAMID : a software quality metrics framework. Everything is linked to goals.
• AMI : adopts the goals/questions/measures paradigm as 4 distinct activities :
  • Assess
  • Analyse
  • Metricate
  • Improve

Specifying and Evaluating Software
The goal of this section is to explain the need to specify software fully and to evaluate it against quantitative requirements. To define a generic set of qualitative requirements that are common to all software systems. To define a set of measures that can be used to express these requirements in quantifiable terms. Typical requirements discussed include:

1. The development process shall be mature.
2. The software system shall be tested.
3. The software system shall be complete.
4. The software system shall be reliable.
5. The software system shall be usable.
6. The software system shall be robust.
7. The software system shall be structured.
8. The software system shall be repairable.

Cost estimation for software development and maintenance
The goal of To explain the main objectives of cost estimating for software projects. To describe how cost estimation should be performed during projects.

"The main goal of the estimator is not to provide targets for the development teams to strive towards, nor to state what they should do, but rather to provide a reasonable planning projection of what they will do." (Tom DeMarco)

"Unless a systematic approach is used, based on validated historical data, cost estimation will remain an art form rather than a science. Managers must be convinced that it takes more than waving a magic wand to consistently generate cost estimates" (METKIT)

Some of the key points underlying this session:

• A software cost estimate should be based primarily on a forecast of the resources needed to produce a software system and, if relevant, the resources needed to support and maintain the system.
• Estimates for software work are usually quantified as cost, but also need to consider effort and timescale.
• By their nature, estimates have to be made when information is incomplete and uncertainty (risk) is high - and software has a history of significant under-estimation.
• An estimate must be expressed and quantified in such a way that actual performance compared to the estimate can subsequently be measured.

Setting up a measurement programme

This section explains how to implement a measurement programme that helps managers with:

• decision making
• understanding, controlling and improving
• setting quantifiable goals
• identifying where changes are needed

It is based on a real life Case Study. The Data Collection Plan defines what information is to be collected, how it is validated and stored. It includes:

• identity of the item to be collected
• the unit of measure and its bounds
• how it is to be collected
• its frequency of collection
• who will obtain, validate and record it
• where it is currently stored

Process benchmarking and optimisation are also covered.