Test Run Misconceptions

Question:  When is a test run not a test run?  Answer:  When cash-strapped entrepreneurs are involved.  I’ve observed many pilot production runs, for food, health-and-beauty, and CPG products.  Most were mandated by contract manufacturers as a necessary pre-production step.  Tests were scheduled to calibrate machinery, establish final product tolerances, install new tooling, or validate yield assumptions.   All reasonable goals . . . as long as one understands that goals are not always achieved on the first try.

Therein lies the problem.  Inexperienced entrepreneurs too frequently view test runs as having predictable outcomes.  Among their more common misperceptions are these:

Scaling Schmaling
Taking a sample from the bench top to production is not a simple exercise in multiplication. Production equipment can dictate product changes.  Subassemblies and custom tooling are among the many other variables to consider.  Multiple test runs and beta units are the norm, not the exception.  Yet, many a startup team still projects nearly immediate product scaling and ramp up.  Calendars and projections are circulated that fail to reflect the iterative nature of the test process.

Saleable Product
Cash strapped entrepreneurs like to view test run product as saleable.  They purchase custom packaging for that very purpose.  Later, when production realities require recipe changes, that packaging can be worthless.

Worse yet, hastily sold, untested pilot run product can fail later in the field.  Market opportunities can be ruined.

Turn Key Sourcing
Few co-packers offer full turn-key production services to startups.  Not wanting to be caught with ingredients and/or packaging should a product not sell, co-packers prefer to wait until a production history has been established.  Startups must do their own (low volume) sourcing.  The added costs of this inefficient purchasing are often overlooked.  Sourcing from multiple vendors can result in shipping costs that exceed the cost of the ingredients themselves.  This becomes particularly apparent when multiple, unexpected test runs require companies to reorder a particular item.

Normal Yields
Test-run product is ideal for use in sampling and, as the name suggests, testing.  The latter results, including durability and shelf life testing, are required by co-packers prior to an actual production run.  Entrepreneurs, however, tend to overestimate test run yields.  While a >90% yield may be possible in production, test runs are less efficient as equipment is tweaked and employees are trained.  Companies desperate for samples for pre-selling can find themselves with little on hand after allocating product for necessary laboratory use.

Lower Cost Runs
Will a pilot production runs cost less than an actual production run?  It depends.  A test run does not necessarily mean a smaller run.  Manufacturers prefer to simulate actual production results, so batch sizes can be similar.  Moreover, test run set-ups may require the involvement of highly skilled (expensive) R&D and QA personnel.  The costs, particularly if one assumes that multiple runs will be required, can actually be higher.