How to measure what we want?

THE MAIN POINT (355 words—71 seconds reading time)

Remember that we get what we measure and we don’t get what we don’t measure. It’s important that we measure the right indicators and compare the indicators to meaningful reference points. Furthermore, the important things to measure are unmeasured and unmeasurable. We measure what we can so we can make time to focus more on things that take intuition, judgment, and experience.Harold Kurstedt

To date, we’ve discussed three methods for implementing a management process for getting control of our work and our life. First, we must set expectations, since meeting expectations leads to success. Second, we must know our work, so we will work on the right things. Third, we must set indicators and reference points, so we can find out how well we’re working on the right things. Remember that we get what we measure and we don’t get what we don’t measure. It’s important that we measure the right indicators and compare the indicators to meaningful reference points. Furthermore, the important things to measure are unmeasured and unmeasurable. We measure what we can so we can make time to focus more on things that take intuition, judgment, and experience.

After we determine what we need to measure, one of the issues is actually making the measurement. The important things can’t be measured with means as reliable as a yard stick or a thermometer. But we need to find a means to measure these important indicators anyway.

We need to identify the indicator and then the means to measure that indicator. We need to sense the indicator. We need a means to make the measurement. For example, an indicator is temperature and the means is a thermometer that we use to collect data on temperature. I’ve recently helped review a questionnaire in Dewberry. Questionnaires are interesting; however, we certainly don’t have the same confidence in the questionnaire accurately capturing the indicator we want as we have in a thermometer accurately capturing temperature data.

Of course, we may be deluding ourselves. Why? Because thermometers don’t measure temperature. Thermometers measure volume of mercury. We happen to have a high confidence level that the volume of mercury can be accurately calibrated to represent temperature. Have you ever verified the calibration on your thermometer? Or have you assumed that the calibration was good? We consider the thermometer’s measurement of mercury volume to be a good surrogate for measuring temperature. However, we don’t have the same confidence in the responses to questionnaires being good surrogates of indicators such as employee satisfaction or client need.

THE FOLLOW UP (356 words—71 seconds reading time)

We need to study, practice, and improve at measurements such as questionnaires because that’s what we have at the moment for measuring important client and employee indicators. The key is that we have to collect data with the ultimate purpose of comparing the data we collect to a reference to make information we can use for good decision making.

What does making a good measurement mean? The measure needs to be measurable and useful. By measurable, I mean that we must have the ability to measure the indicator and we need to have confidence in the measure as a surrogate of the indicator. By useful, I mean the measure must be important among the many different measures and indicators available to us and we need to have the ability to use the measure to collect data to make information leading to a decision.

As we’ve looked at measures and collecting data, I’ve emphasized prevailing measures—those measures that we need to continue to make over time to manage our work. We also have detective measures and corrective measures. We use detective measures to try to locate a problem, its cause, its size, and its consequence. Once we have the answers, we don’t need these measures anymore. We use corrective measures to observe how well we’re fixing the problem once we find it. Once we’ve fixed the problem, we don’t need these measures anymore.

The purpose of measures is to gather data. We need data to make information for good decision making. The method of measures is the method of collecting data. We collect data in tabular or graphical form. We need a consistent (across the organization and across time) form and practice for collecting data. The use of an information system is to take the data we gather and to produce good information. Remember the old adage of “garbage in, garbage out.” The adage means that we must do a good job at measurement and collecting data or our information and our decisions will be at best worthless and at worst harmful.

A question for the week: What measures are we now making that are helpful and what measures aren’t? Why?