When you can Measure what you are speaking about, and Express it in Numbers, you know something about it

by Tom & Kai Gilb
(who we are)

Usually, projects fail to deliver value to fixed deadlines. Typically they ship something after the deadline with less value or quite often nothing is ever shipped. If this frustrates you as it frustrates us, keep reading. We cover some fundamentals that are critical to succeed in delivering value on time, fundamentals that are usually absent.

The blog title is an extract from the lord that in 1965 changed my professional life,  Lord Kelvin, that is ;-) Try to understand it, it might change yours too.  Let’s look at a larger section from Lord Kelvin.

"In physical science, the first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it. 
I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it;  but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be.” 

- Lord Kelvin, 1893, Lecture to the Institution of Civil Engineers, 3 May 1883 

Many people mix up or combine, the concepts of ‘quantification’ (express it in numbers)  and ‘measurement.’ 

They typically use the lazy excuse, that ‘perfect measurement’ is too difficult, in order for them to avoid doing ‘quantification.’ Illogical!
There is, of course, a clear enough distinction between a budget and accounting, between a volt and a voltmeter. 
However, people consistently mix up the concepts, to their disadvantage. 

So did I, for a while, before a more-enlightened Swedish professor from Gothenburg pointed out the ‘obvious’ distinction. 
I just repeat the ‘mantra’: ‘volt, voltmeter,’ or ‘speed, speedometer,’ and I have got the difference.

In a single sentence, Lord Kelvin distinguishes between quantification and measurement twice, and three times in the quotation! This is not by accident.

‘Quantification’ alone, has great merit, even if you never actually carry out any measurement! 
A budgeting process, for example, makes you think about what can and might happen: even though the next accounting results might be very different from your budgets. 

Beam Me Up Scottie

If I tell you ‘I want to travel to the moon and back, in 1 second’. I have quantified. 
My idea is clear. In fact, so clear, that you dismiss it, as unrealistic. 
So neither of us is concerned with how we would actually measure how fast I did my moon trip.
Meter not required.

A budgeting process gives you some constraints (budgets) that you later will probably have to respect when real measurement (accounting reports) threatens to exceed those budgets.

The same distinction holds for forming a scientific or engineering hypothesis (quantification), and consequent experimentation to determine if it is proven or not (a scientific experiment).

Quantification is, above all, a useful tool in communication between people. 
Numbers clarify, what words hide and confuse.

We recognize that quantification (in practice, ‘defining a quantification scale,’ and ‘some interesting points on that scale’) alone is useful.
We also know that it is usually also useful, sooner or later, to actually observe reality numerically: to measure in practice. But not ‘always.’

Measurement gives essential contact with the real world. 
If the measurement is early and frequent, then we can usually adjust our plans, to be in better contact with reality, and in contact with our objectives and constraints.

Measurement does not have to be ‘perfect.’ In fact, it cannot be literally perfect, as engineers and scientists acknowledge. Kelvin was not fanatic, as you can read, above. 

So the ‘measurement’ questions and opportunities are:

What exactly is sufficient measurement quality (accuracy, precision, credibility) for our purposes, and what is the lowest-cost measurement process, that has satisfactory quality, for our purposes.

At different stages in the system development process, for different purposes, we can decide to have quite different measurement processes. Which measurement processes do we need to plan for, budget for and use?  ‘Horses for courses’ as the British say.

The choice of the measurement process, since it depends on many scalar attributes (precision, accuracy, setup cost, cost to measure, credibility, repeatability), is really an ‘engineering design’ decision. Can we ‘design’ our measurement processes?

That means it is a matter of finding the measurement process that best fits our objectives, within our constraints. We cannot just take any measurement process that comes to mind.

Figure: The Scale is an abstract concept, to help us think about good or worse attributes of a Function. The ‘Meter’ is a tool or test process to help us get some idea of where on the Scale a real system is. How ‘good’ the system is. The last measurement, the last implementation of the Meter on our system, we can call Status.

Here is a simple example showing the distinction, and the choice, of more than one measuring tool, for a single Quantification Scale specification.

Team Cooperation Capability:
   Type: CTO Level Organizational Improvement Objective
   Ambition: much better and consistent cooperation between team members and between teams in technical projects.
   Scale: average % of Project-Hours spent with Cooperative-Content between Team-Components.
     Meter [Early Stages of a Project] samples of logged hours, by Project Manager, monthly, 1 hour of work.
     Meter  [Analysis of Completed Projects] Database analysis using student trainees, presenting reports and conclusions.
      Status [last measurement] 15%
      Goal [within 2 years]  at least 20%-40%.
     Project-Hours: as logged in project logs, and charged against a project. 
     Cooperative-Content: writing or oral activity directed to others, with the purpose of sharing and getting feedback.
     Team-Components: Any people within a Team communicating with each other. Any part of a team communicating outside the project team, with the purpose of learning or sharing.

Planguage Example:  Two different Meter specifications for the same Scale. One for early project feedback. The other for project completion testing. The Meter specification is usually short (20 words or so), at this stage of specification. Meter statements do try to contain words that are critical for accuracy and cost (for example ‘Student Trainees’, ‘samples’). However, the detailed design of the Meter process is for specialists, such as test planners.

Practical tip:  Keep Measurement As Simple As Possible.

Use the cheapest simplest measuring process that gives you adequate feedback for purpose. 
It is more important to get some early and frequent feedback, week by week, on your critical objectives’ value delivery, than it is to have accuracy greater than ±20%.
This is business, not Nobel Prize Science.
All choices of measurement process need to ‘pay off,’ to give value exceeding their cost.

Policy:  Plan Measurement Formally, and integrated into planning of objectives.

Formal written plans, to measure value delivery in practice, will be integrated with the specification of objectives. 
Why? 
It makes us consider at which points in the development and operational processes we want to measure, 
moreover, makes us consider different levels of measurement capability and their costs.
It helps us avoid excessive measurement. 
How?
Use the ‘Meter’ parameter for specification of different types of measuring processes.

'Principle: Measure Reality

All objectives, 
can and must have sufficient measurement methods, 
based on their defined Scale,
to give knowledge of current levels.

From "Value Planning Book Manuscript" by Tom Gilb