Imagine a chef who desires to create a new, crowd-pleasing recipe. She makes a first attempt at this, then shares it with colleagues and friends to get their feedback. As a result, she may add or subtract ingredients, alter the cooking time or make a number of other adjustments. She then creates a new version of her recipe and again solicits input from taste-testers, which she uses to inform her next version of the recipe. She repeats this until she (and her testers) feel that she has perfected the recipe. This is an example of “iterative product development.” Most CPG companies follow a similar, if more formal, cycle of testing, refining and re-testing when developing their own product ideas. Small teams typically build concepts, get qualitative or quantitative feedback from consumers, refine concepts, get another round of consumer feedback and so on, until they arrive at a “winning” concept.
This technique works well enough, but it has some drawbacks. It often produces ideas that are good enough but not the best. Why? Traditional iterative testing requires that teams considerably narrow their pool of ideas—after all, timing and budgetary constraints only allow for so many rounds of iteration. The result is that the best ideas may not be adequately explored or even considered in the first place. Although the team may still arrive at a successful idea, did they overlook a better one? This constrained exploration translates to potential missed opportunities in market.
Quantity Drives Quality of Ideas
An anecdote in the book Art & Fear, often cited by tech startups, explains the dangers of exploring a very limited number of options. A ceramics teacher divided his class into two groups, telling one that they would be graded solely on the quantity of work produced, and the other that they would be graded solely on its quality. However, something unexpected happened: the works of highest quality were all produced by the group being graded for quantity. Evidently, while the “quantity” group was busily producing one prototype after another—and learning from their mistakes—the “quality” group had sat theorizing about perfection, with little to show for their efforts.
Similarly, in the realm of product development, finding a winning concept is a numbers game. The more concepts tested, the more likely you are to identify a concept that will perform well in market. One study estimated that it takes 3,000 raw ideas to identify one substantially new commercially successful product.1 So you need more ideas—and, ideally, more diverse ideas from people with different roles and perspectives. A recent Nielsen study found that teams of 6 or more generated concepts that were 58% more preferred than the “starting point” concept in pre-market testing, whereas concepts generated by smaller teams were only 32% more preferred than the “starting point” concept.2
How to Iterate at Scale
However, the future will most likely be different—and with better product innovation. It will consist of not just iteration, but “iteration at scale.” This refers to developing and testing not just a few concepts, but tens or hundreds of thousands of them.
To iterate at scale, you need to dramatically:
- Increase the number of people building concepts.
- Increase the number of functions involved in building them.
- Increase the number of concepts created.
- Increase the number of concepts that you can test without expanding your timeline.
The last two requirements seem particularly resource-intensive—and maybe even impossible—at first glance. However, thanks to new collaborative technology platforms and evolutionary algorithms, they’re not.
- Collaborative technology platforms allow remote teams to create concepts more efficiently. Teams can provide input in real-time, so feedback is not dependent on linear email or document review—nor does the project manager need to spend time consolidating all of the feedback. The project manager controls who can be a collaborator, whether or not to accept or reject specific feedback, and when the creative process is “locked.”
- Evolutionary algorithms apply the Darwinian principle of “survival of the fittest” to concept testing and refinement. Instead of building discrete concepts the old-fashioned way, teams spend their time developing an expansive range of dynamic “component parts” or “variants” for their concepts. In testing, these variants combine in different ways to form thousands of concept alternatives. As consumers react to the concept alterna-tives that are created, the system learns which variants (and combinations of variants) are more preferred, evolving ever more appealing concepts in response to real-time consumer feedback. In the end, the system not only scores but actually identifies a few of the most promising concepts.
To use a real world example of evolutionary algorithms, the popular music “personali-zation” app Pandora works in very much the same way. Though most users are unaware, each song in Pandora’s database is composed of a distinct combination of variants—musical characteristics such as the gender of the lead vocalist, level of distortion on the electric guitar, type of background vocals, and so on. As users “thumbs up” or “thumbs down” individual songs, the algorithm learns which elements (and combinations of elements) the user prefers. In the case of product concepts, the variants are not musical characteristics but various product claims, reasons to believe, consumer insights, flavors, and so on. As consumers express a preference for one concept alternative (or song, in the case of Pandora) over another, the system learns.
Why “Iteration at Scale” is the Future
Collaboration technology and evolutionary algorithms sound promising in theory, but the proof is in the volumetric potential of concepts built with these advantages versus those built the traditional way. Data from Nielsen shows that concepts “optimized” in this way yield, on average, 38% more volume potential. For an industry with high rates of new product failure, these new methods are the next iteration of traditional innovation processes.