Related Articles:
	Other Articles

Forecasting Challenges

All forecasting models based on survey data face the same challenge: respondents consistently overstate their intent to purchase new goods and services. Forecasters have developed two very different approaches to the problem:

Using historical data relating survey results to market results to create performance "norms" for key model parameters. By "databasing" these relationships, forecasters can predict future in-market performance based on similar relationships that have existed in the past. Using widely accepted views of buyer behavior to craft logical calibrations from the survey data alone. This approach uses conjunctive logic across four to five basic attitudinal and quasi-behavioral measures to classify respondents into various buyer/non-buyer groups of interest.

The Trouble with Norms

The norms-based approach has enjoyed considerable success in the consumer packaged goods arena, where stable, well-defined categories, limited forecasting goals, and a high volume of testing combine to create a perfect environment for the success of this approach. However even in packaged goods, not to mention more rapidly changing categories, the norms-dependent approach has some significant drawbacks: The approach is fundamentally backward looking, and so does not adapt rapidly to discontinuous product development or fundamental changes in category or business dynamics. It tends to identify "winning" products and services that look like winners from the past, stifling differentiation and leading to a proliferation of "me too" products and services. Reasonably accurate forecasts are always dependent on the existence of appropriate historical analogs in the historical database. In some cases, this means that forecasters cannot forecast at all due to lack of experience in a particular category. Possibly worse, lack of specific experience sometimes leads to the use of other similar categories as a substitute. Some firms use price points as one way of defining which norms to use as the basis for its calibration adjustments. While conceptually forecasts created in this manner are empirically based, the strength of the historical relationships is often illusory due to the lengths to which the database of experience has been stretched. Due to the need to "database" results from each test, tests must be conducted in a very strictly standardized fashion, among a standardized target audience. While standardization of approach has its appeal, business in this century is sufficiently complex that an overly standardized approach can miss important differences across products, categories, and time. It is essentially a "black box" where survey data goes in, adjustments are made inside the box, and a result pops out. This severely limits the amount of learning that can be derived by the client from any given forecasting engagement, while maximizing the potential economic value of the results to the forecasting firm from the enrichment of the results database.

Logic-Based Calibration

SDR's approach to forecasting leverages the logic-based calibration approach. The fundamental basis of the approach is that, for a product or service to be purchased by a particular individual, that product or service must meet several logical criteria:

The potential buyer must: Express an interest in buying the product or service; Feel the product or service provides and important benefit; See that the product provides a reasonable value for the money; Detect some different or unique aspects of the product or service relative to its competition; and Prefer the new product or service to its competition when given a choice.

Clearly these five measures are correlated, yet each yields a slightly different perspective on the appeal of a new offering. By identifying those target buyers who provide consistently positive responses across the five measures, SDR can accurately estimate the level of interest in virtually any product or service among a custom-defined target group.

The approach provides four key benefits: Forecasts of sales for test products and services are always based exclusively on the evaluations of today's buyers in the context of today's market. The approach can be used with confidence on any product or service category, regardless of the level of prior experience forecasting in that category. The approach is flexible with respect to sample design, question wording (and to some degree, order), data collection method, and mode of presentation of concepts and other props. SDR's approach is an open forecasting architecture, meaning SDR prefers to work in a collaborative, open relationship, where each client can learn, to the degree they are interested, exactly how each forecast has been developed, and actively participate in decision-making. We feel this approach maximizes each client's learning and ensures that the forecast incorporates both the client's deep knowledge of their business as well as SDR's forecasting expertise.

Simultaneous Optimization and Forecasting

Conjoint and choice modeling tend to be the tools of choice for most marketers looking to optimize their product offerings. A consensus is emerging in the research community that, of the two, choice modeling is the better forecasting tool, especially when pricing issues are involved. Key advantages of choice modeling are the ability to directly incorporate competitive context and competitive response into the model, and to simulate hundreds or even thousands of market scenarios from the results of one study.

For all the strengths of choice modeling, there are limitations when used alone. Choice models tend to: Overstate demand, since they are based on "perfect world" scenarios where all buyers have perfect information about choices, and all choices are available. Overstate price sensitivity, since respondents can see prices changing relative to each other, and they learn to play along. Overstate the impact of feature changes, for the same reasons.

The key to accurate choice-based forecasting is calibration, using data external to the model. SDR will use two sources of calibration information: Demand forecasts developed separately using the conjunctive logic described above, and Information supplied by the client on current business performance and share.

Using these two external sources, SDR will be able to leverage the advantages of choice modeling for dynamic forecasting, while dampening or eliminating its sources of overstatement.

One final piece of external data that can be incorporated into the simulator is cost/margin information provided by the client. With this information embedded in the simulator, the client can look not only at the revenue and volume implications of a particular portfolio/pricing strategy, but its profit implications as well. This has proven to be a powerful addition to the simulator as a planning tool.

The Impact of Marketing Activity

One final source of potential overstatement needs to be accounted for in the forecast: the impact of marketing activity on awareness and availability. The modeling activities described to this point essentially estimate demand for a service, that is, it sales potential given universal awareness and availability among potential buyers.

Real-world marketers never achieve universal marketing reach, so to convert demand into a sales forecast, some estimate of marketing reach must be made. The best approach is to actually develop one or more marketing plans, and estimate awareness and distribution based on these plans. SDR and other forecasters in consumer packaged goods have developed sophisticated models to predict awareness and distribution based on specific marketing plan assumptions.

Marketing outside of packaged goods does not as readily adapt to this cookie-cutter modeling approach. Business-to-business marketing especially resists strict modeling approaches to estimating awareness and distribution. SDR always works with the client in a collaborative and consultative manner to review its marketing plans and develop reasonable assumptions to use in the final forecasts. Marketing plans and historical results, combined with the judgment and experience of the team, will be used to estimate awareness and distribution.

Putting the Pieces Together

This schematic depicts how the forecasting pieces fit together:

By estimating demand, using choice modeling to simulate the effects of pricing and feature optimization, and calibrating results with external data and marketing plans, SDR provides the most accurate forecast possible, while maximizing the learning derived from its research investment.