How do you know if the efforts that your organization is investing in innovation are delivering the expected results? And what results were you actually expecting? The question of indicators and metrics is, in my experience, the biggest barrier that companies face when deciding to engage in an innovation effort, as well as one of the major causes of failure in these attempts.
The trouble starts with a reasonable assumption, i.e. that if you want to control a process and assess its results, you need to measure some aspects of it. But in the case of innovation it is not totally obvious what exactly should be measured, nor how.
Opinions differ widely, but all of them can be roughly placed on a scale running between two extreme views that we can call “business-is-business” and “just do it“.
<———————————————————————————————–>
BiB says, in effect, that an innovation initiative is no different than any other effort or investment, and therefore, the only kind of results that can justify such a program are its impact on the company’s KPIs, whether they be profit, market share, revenues or any other. Note that holders of this view do not necessarily demand that innovation be measured exclusively in financial terms. In principle, if an organization’s main objective is, say, to spread happiness, then according to this view, the indicator one should use to assess the success of an innovation initiative is the amount of happiness created. There is a very strong rationale for this approach: an innovation initiative does not change the company’s goals, rather, the company engages in innovation in order to further these goals. The indicator for the innovation effort’s success, therefore, must be its impact on these very goals.
Although this argument is hard to refute, adopting this approach is probably a guarantee that your innovation effort will fail.
The reason is simple. Although your innovation effort must eventually impact your business objectives, these objectives cannot be used to monitor the process, since: a) it will usually take months, and in some cases more, for the effects to cascade all the way down (or up) to sales, growth, or profit, and in the meantime there will be no indication of what should be changed or adapted in the program; b) there are so many other factors at play that isolating the net contribution of an innovation initiative on business goals is nearly impossible, and even more so when trying to isolate the contribution of specific elements of such a process.
The other extreme of the scale, “just do it“, recognizes the difficulties mentioned above and, in response, takes the exact opposite view: since we know that innovation is necessary, and as we are aware of the types of actions we have to take, why not just go for it, and measure the inputs rather than wait for vaguely correlated outputs. For example, if we know that training is a necessary condition for innovation, let’s measure direct parameters of the training effort, such as number of people trained, number of hours spent on training, amount of tools and methods taught, and the like. The downside of this approach is clearly evident as well. Who says we need this specific type of training rather than another? How do we know whether trainees understood, and more importantly, to what extent they are utilizing what they learned? What is the optimum number or percentage of personnel trained (and can we suffer from too much training)? And, most importantly, what impact is this training having on performance?
These questions and doubts do indeed challenge the “just do it” approach, but, at the same time, they are already pointing to the solution to the measurement problem, or at least to a combination of approaches that, although not perfect, allows an organization to avoid paralysis and set out on an innovation path:
1) As a rule, start on the JDI pole of the axis, and move along slowly towards BiB. At the outset, any expectation of an immediate business result will just cause frustration, draining the effort of its energy. Measure only that you are indeed performing all the actions that you set out to do and ignore the effects;
2) Even as you start to JDI, remember to talk about and define your ultimate goals. Yes, they are not relevant for the moment, and will not be for a while, but still, it is important to keep in mind that they are the raison d’être of the entire operation;
3) While you measure inputs, try as quickly as you can to also measure some kind of outputs, even if at a very tactical or even technical level. There are many ways to test the success of a training program (feedback forms being one of the least reliable). Try some, such as assigning a specific task to graduates and monitoring their success at performing it; counting the number of ideation sessions that are being held; gauging the number of people that have been “touched” by the innovation initiative around the organization, etc.;
4) Combine quantity with quality. Quantitative measures tend to give a more “objective” feeling, but beware the common mistake of objectively measuring irrelevance. A prime example: number of ideas generated in an innovation session. More often than not, a large number of ideas correlates with poor quality, sloppy filtering and therefore, a low rate of implementation;
5) Measure constantly, but let people work in peace. At least in the first few months, measurement should be used mainly to compliment and encourage, and to make relatively small adaptations to the program, but never to put the entire endeavor in question. Otherwise, no one will have the guts to do anything really useful for fear of: a) being found to fail, and b) being associated with a venture that was “closed down”;
6) When moving closer to the business side of the scale, use a variety of business measures rather than a small number. For example: money saved on restructured processes, number of new products on the market, impact on customers’ perception of your brand as the leader, percentage of revenues due to newly introduced products or services, speed of response in crises or to customers’ complaints etc.
In sum: Yes, innovation is hard to measure, but so is any other deep and meaningful organizational process. And yes, despite the pitfalls, there are quite a few feasible and relevant measures that can allow one to monitor an innovation initiative, enabling course corrections when and where necessary, and providing sufficient parameters to assess success.
This is indeed a key challenge to any large-scale innovation initiative. Another point against measuring too much at first is that what you measure tends to influence what you will produce. Essentially, once the metrics are known, people will try to meet the metrics, and sometimes _only_ the metrics, at the expense of other goals.
I will stick with happiness: If all the value network of the organization is delighted while in, and out of the innovation process, it could be a nice “attractor” for measuring success. Each node of the network might define in advance their own happiness factors, and share them in a common forum. For some, happiness could be within the process (input), for others it could be in the throughput. That would define the variety of indicators. And all the valuenet would feel they own the metrics so there would not exist a feeling of being imposed other people’s indicators. Dont worry, be happy =)
Totally agree with Shlomit’s comment. One of the great fallacies of “quantitative” management. Supposedly, the saying “you get what you measure” implies “you get what you wanted”. More often it means you get what people will do to make you think they are doing what you want from them.