In God we trust. All others must bring data.

W. Edwards Deming

Abstract

A method to estimate preferences between product concepts and their aspects is presented. The method is designed for a special case when concepts are fixed and described as a set of aspects. The simplest interview arrangement consists of two blocks. One block is best-choice MaxDiff of concepts or packages, possibly completed with a question on purchase intention. The other block is best-worst MaxDiff of aspects the concepts are composed of. The obtained choice preferences are combined and rationalized as part-worths of the aspects and utilities of the concepts.
The method has been designed to handle more attributes than a CBC.
If a simulation of untested concepts is not required, the traditional compensatory (additive) model of aspects is adequate.
If a simulation of untested concepts is essential, a non-compensatory model should be assumed, which is typical for product packages. An estimation of acceptability thresholds will be obtained from additional interview blocks and included in the analysis.
The test is equivalent to the usual concept test with an advantage to test a higher number of concepts and to compare an influence of aspects on a common scale.


Problem

The choice-based conjoint (CBC) is a widely used research technique for products that can be described by about 6 or fewer attributes each having 2 or more levels. Actual market products and services have often many more attributes that make use of CBC unfeasible. This concerns namely packages that have aspects being either present or absent and not mutually exclusive. Aspects are often grouped by the total value of an offer and relationships between them are complicated. Only some combinations of aspects are allowed and quite a many are prohibited. Generating reasonable product profiles in a conjoint study would require numerous cross-attribute prohibitions that are a frequent cause of inefficient or unrealistic product profiles, of a slow or none estimation convergence, and nearly always of estimation bias. The method of product classes has proved useful for products with only a few attributes, typically for brand-product-price studies. 

The approach of incomplete conjoint profiles is seldom usable. It leads to leveling importance of attributes in general and to an apparently increased importance of the least often shown attributes. The method is unusable in a branded study since respondents implicitly set the missing attributes to the level values they suppose the brand usually has.

Due to the inconvenience with the application and results of a conjoint method, the current common approach remains a traditional concept test, typically a sequential monadic, followed by additional questions on numerous aspects of the concepts. The main disadvantage of this approach is a lack of information on the influence of individual aspects on the acceptability of a product. No simulation of a product set in the frame of the tested aspects is possible.

The objective of the presented method is a concept test providing an estimation of both acceptabilities of the concepts and effects of aspects the concepts are composed of.


Solution

Model

Instead of using profiles generated by an algorithm, the method relies on fixed profiles designed in a managerial way. The range of aspects, their settings and balance between them (typically value and price) are taken as guaranteed since the profiles are provided by a study submitter who is supposed to have good knowledge of the market. By their very nature, the profiles exclude the need to introduce prohibitions between attribute levels. These compensations are not reflected in any known algorithm for generating near-to-orthogonal profiles for a conjoint task.

The method is inspired by the half-forgotten approach of the 1950s, referred to as the SEC - Self Explicated Conjoint (or Self Explanatory Conjoint). The respondent evaluated each attribute level attractiveness and each attribute importance one at a time. The obtained evaluations were transformed into values of an additive variable. The sum of these values characterized a product composed of attribute levels.

As aside to SEC

In our approach, the attractiveness of both profiles and aspects is estimated in the respective MaxDiff tasks. The scale-based questions on concepts in the traditional concept test are replaced by MaxDiff of profiles. The choices can be completed with questions on (stated) purchase intention. The scale-based questions on attribute levels in SEC are replaced by choices from appropriately set aspect selections that collectively make up all aspects of the profiles. The implicit importance of attributes is obtained from uniting the results from both groups of tasks using a common behavioral model. The main principles of CSDCA - Common Scale Discrete Choice Analysis are applied.

In a conjoint-type exercise, the attributes should have about the same number of levels, the differences between effects of levels within an attribute be equidistant, and the spans of attributes be of similar importance. This is required to obtain reliable estimates from responses to profiles generated by an algorithm. The present method does not impose these requirements. The overall scope and importance of the aspects is determined by the provided concepts.

Choice of the behavioral model is important. In the simplest case, the traditional additive model can be used. If there is a suspicion or assumption of non-compensatory behavior, the interview should be extended by a CSDCA PRIORS block of questions to obtain acceptability thresholds.

Improvements in the quality of results is expected to be similar to the transition from scale-based questions to MaxDiff, CSDCA or AGA procedures.

Questionnaire

The tested profiles are fixed and handled as items. To create choice tasks a software for the generation of randomized orthogonal sets of items is required. If software for on-line deployment is unavailable, several versions of a paper-like questionnaire can be prepared. A questionnaire contains two basic blocks of MaxDiff type:
  1. MaxDiff of profiles (product or package concepts).
    • The profiles are supposed to be formulated by a managerial approach concerning both allowed and prohibited combinations of product aspects so that a balance among them is achieved.
    • Only the best offer is marked as chosen. The opt-out option is not offered.
    • Each choice can be accompanied by a stated purchase intention (SPI) or other calibration questions.
  2. MaxDiffs of aspects.
    • The simplest case is a single MaxDiff of all aspects present in the concepts.
    • In some special cases, the aspects may be distributed into several MaxDiff exercises.
    • The aspects are compared in terms of need and convenience. Only the best and worst of 4 or 5 aspects are marked which is easy for respondents.
  3. An additional block of discrete choice tasks
    • An inclusion of an extension depends on requirements from the assumed behavioral model. The most common is a block of PRIORS questions to obtain initial estimates of attribute thresholds.

The actual order of the questionnaire blocks should take account of the learning process of respondents.

As aside to design

Estimation

The mutually exclusive aspects can be grouped into formal attributes. This will allow taking advantage of commercial programs originally designed to analyze conjoint tasks. Among these are constraints between the values of part-worth of ordinal level values, the use of different types of coding for estimation of thresholds, etc.

Utilities of profiles and part-worths of aspects are estimated by the hierarchical Bayes maximum likelihood estimation. The data set is composed of all the discrete choice exercises each of which has a different weight. The weights are determined from the number of tasks and a number of items in a task using the gain in information obtained as a loss of information entropy (cf. How many CBC tasks).

As aside to estimation

The effects of aspects are expressed in terms of perceptance, i.e. difference between the probability of acceptance and (complementary) non-acceptance of an aspect. A profile is characterized by a stated purchase intention computed as its censored perceptance, the censoring being made by considering only non-negative values.

Simulation

As the part-worths of all aspects of the tested concepts are determined a simulation of purchase intentions of untested profiles is possible. However, the reliability of simulation is generally lower than that of a CBC-based simulation. One reason is the prerequisite "considered jointly" is not valid for all combinations of aspects since only a limited number of aspect combinations were present in tested concepts. Another reason is the number of effects summed in the simulation. The present method is designed to handle many more aspects a conjoint method is able to. Utility error of untested concepts grows with the number of changed aspects because of errors of aspects cumulate. 


Properties of the method

The method is a hybrid of a concept test and a choice-based conjoint. It is very flexible and open to various modifications. For example, when price sensitivity is of interest, one or several profiles may be offered at different prices, but, of course, not shown in the same choice set. The advantage is obtaining price sensitivity.

Strengths

Weaknesses

Neutrals


Example: Bank account concepts

The model was applied on a set of several bank account concepts that could be described by 12 attributes with 2 to 5 levels each. This number of attributes was too high to use a CBC methodology. Instead, the levels were treated as 46 aspects. The questionnaire contained a Maxdiff of the concepts and a Maxdiff of the aspects. A web questionnaire was created in Sawtooth SSI ver. 8.4.8 system. Data were processed using Sawtooth CBC/HB 5.5.4. A shortened and obfuscated version of the live questionnaire is available on-line.

The acceptance of each of the concepts was obtained from the MaxDiff of concepts. Each unique concept choice was completed with a calibration question on the intention to switch to the concept so that the raw utilities could be modified to reflect the stated intentions.

The client was interested in which aspects and to what extent they were responsible for an attractiveness or, possibly, a repulsiveness of the concepts. This was achieved by combining the results from both the MaxDiffs. Since perceptance threshold value for aspects was not determined experimentally a common threshold was set arbitrarily to the mean utility value of the price range. The results are in the picture below.


Perceptances of the concept aspects

The interpretation of perceptances is straightforward for most aspects. Nevertheless, it should be remembered a perceptance value is a single number that comprises several attitudes, namely common knowledge, expectation, and satisfaction with the aspect (and its value, if quantifiable).

As aside to perceptance

Among the tested aspects, neither a credit card nor airport lounges are usually offered as a part of a bank account. The explicitly stated absence of an airport lounge has higher expectations compared to the absence of a credit card that respondents are more familiar with. However, including even the lowest credit card option in a bank account leads to a higher satisfaction than providing any of the offered airport lounge options. The most important interpretation value is the difference between perceptances of individual aspects. An aspect with substantially low perceptance of both its absence and presence, i.e. an aspect with high expectations that cannot be satisfied, should be omitted from an offer.

Calibration of profile choices combined with known aspect part-worths allowed for building a simulator of the potential of modified concepts. The simulator is not presented for confidentiality reasons.


Conclusions

The method is seen as very hopeful for testing concepts or packages with many aspects and constraints between them which is the main reason for conjoint-like approach failing more than often. Supposedly some estimation bias may be encountered due to the colinearity of aspect combinations and their missing randomness, typically because of grouping better aspects with higher prices.