Problem

Metric-based conjoint methods suffer from inherent drawbacks:

 

Solution

CBC - Choice Based Conjoint, as a straightforward application of DCM - Discrete Choice Modeling, avoids the problems pertinent to metric methods. Currently, CBC is the most often used method of conjoint study. 

Strengths

Weaknesses

 

Design aspects

Design efficiency

Conjoint of any type is very sensitive to the design efficiency. Metric-based conjoint is especially sensitive to (deficient) orthogonality among the profiles. The orthogonality should be preferred to other design aspects. CBC is much more tolerant in this respect, but other aspects should be accented. Referring to Joel Huber, Duke University, the issues to be considered in CBC design can be summarized as follows.

In practice, the actual experiment has to be modified so that no effect causing a bias would dominate. A pragmatic experimental design balance must be achieved.

Conditional sampling and piping

Users differ in their preferences and, therefore, in their consideration sets. To ensure that every respondent is presented the stimuli in their consideration scope, CBS - Choice Based Sampling can be used.
  • Based on CBS, respondent is presented a CBC version with a narrowed down selection of product classes that are likely to match a consideration set of the respondent. Offers of unacceptable products are greatly avoided.
  • CBS and CBC data can be merged in a common numerical model based on the assumption that the actions observed in both steps follow the same behavioral model.
  • Evaluations of profiles chosen in CBS, if available, can be used as calibration statements.

Conditional attribute levels

The standard orthogonal design of a conjoint exercise assumes that all combinations of attribute levels are allowed. This requirement cannot be met in the design of realistic profiles. In order to remove or at least substantially suppress the detrimental influence of level combination prohibitions, and to have full control over them, a method of hierarchical (nested) design based on product classes of profiles can be used. It is used typically in assigning prices to products with various combinations of benefit attribute levels.

Incomplete profiles

The method of incomplete profiles is inherent to ACA - Adaptive Conjoint Analysis and is the source of equalization of importance of attributes except for the two or at most three dominant ones. The pattern in CBC is similar.

Incomplete profiles with omitted attributes selected from all attributes randomly lead to a decrease of attribute importance differences. The importance of attributes is given by the respondent's ability to discriminate between profiles in the task rather than by the differences in attribute weights. Therefore, the same importance is distributed among the fewer attributes. The attribute importance gets closer to the result of determining importance of each attribute separately.

In many practical cases some attributes cannot be omitted. If some attributes are always shown, their importance will decrease while the omitted attributes will get higher importance. An attribute that is not always displayed will attract more attention when it is displayed and its seeming importance will increase.

Incomplete profiles in a branded study should be avoided completely. Respondents will implicitly substitute the missing attributes with the levels common to the brand. This will sway not only level part-worths of the omitted attributes but also of the other attributes.

Different cardinality of attributes

Problems with nominal attributes differing in number of levels are common. The number of times each level of an attributes will be shown is inversely proportional to the attribute cardinality (the number of levels). Therefore, level part-worths of higher cardinality attributes will be estimated with higher error. A great help in this respect is use of soft constraints between levels of a particular attribute (or a combination of attributes) constructed as tasks involving only levels of the attribute (or the combined attribute). MaxDiff, SCE or simply in-place sorting can be used. Their use will also allow to dramatically decrease the number of CBC task.

Different cardinality of ordinal attributes is much less problematic thanks to possibility to constrain the part-worths estimates. 

Parametric attributes

Most CBC exercises deal with products whose properties are composed of attribute levels directly shown to respondents. In contrast, visible properties of some other product types can be computed or selected parametrically from some starting values revealed by the customer. This is typical for various banking products (accounts, transactions, loans, mortgages), installments sales methods with variable upfront payments, deduction and discount sales methods dependent on a purchased quantity or total value of the purchase, etc. The parameters can be an amount of goods or services, interest rate, up-front payment percentage, length of a contracted period, values in various fee or discount formulas, etc. Parameters can be, and often are, mutually interdependent and/or constrained or bound. The common trait in these problems is the aim to determine influences of the parameters rather than the actually shown attribute values. The parameter values may or may not be shown to respondents.

The starting input values the shown values are dependent on can be entered directly or indirectly by a respondent. An example of a direct input is the desired amount and maturity period of a bank loan, as in the bank loan demo questionnaire. An indirect input can be based on an evoked set of products that enter the actual CBC, as in an installments sales demo questionnaire.

Specific alternatives

Alternatives that do not have common attributes are called specific. Their use has been first introduced in transportation problems where fuel prices, parking fees, transport fares, bike paths, etc. have profound influence on choice of a transportation means such as car, public transport (with sub-alternatives, such as train or bus), motorbike, bike or walk.  Evidently, each of the alternatives has different attributes. In mass product marketing, similar problems are encountered as well. They are most often related to the quality of the products and the extent of services related to them. Again, the product classes approach proved to be adequate.

Choice set arrangement

There are two basic types of choice set arrangement.

Constant alternative

It is generally recommended to use one profile in a choice set as a constant alternative and estimate its utility. While the wording "None of these" is used most often, a constant alternative can have any other meaning such as a status quo ("I would stay with my current product"), a switch to another product from the category, abandoning usage of products from the category, etc. 

A constant alternative is regarded as a way to reveal segments that
  • have lack of interest,
  • are very picky, or
  • are less likely to purchase the product.

The most common alternative "None of these" should serve as a choice option when none of the offered products meets the respondent's needs or expectations. It might mean either a choice of another product based on a new belief acquired during the CBC exercise (acquisition), a choice of another product based on previous experience  (switchback), a continuation of the current usage (retention, loyalty), or their mixture. However, we have often observed choice of the none alternative when a choice of a regular profile could be expected. 

In our view, the none alternative often serves as an escape from a rational answer. Analyses of many studies show that the constant alternative has usually greater variability than levels of any of the attributes. The frequency of "None" choices is increasing with the number of choice tasks to be made. We have never found a significant correlation between the none utility and some other measure of willingness to buy or purchase intention stated in the interview. In non-compensatory modeling, the none choice appears even if a profile with all attribute levels above the thresholds is present. The reason may be fatigue from a long interview, little interest in the object of the study, indecisiveness in selecting from profiles being hard to evaluate or distinguish between, or simply annoyance from a repetitive questioning. We usually leave out the none utility from further considerations.

If no item in the choice set is acceptable, and a "None" alternative or any other way out is not available, the respondent is forced to pick something up. The profile might be chosen for several reasons.
  • It is the cheapest item in the selection that would minimize the loss.
  • It is the best choice in terms of value for money but not fulfilling the needs and expectations.
  • It is the best choice in terms of fulfilling the needs and expectations but too expensive.
  • It is the best choice in terms of exceeding the needs and expectations but not affordable.
  • It might have been somebody else's choice in some situation (this kind of behavior is known as impersonation).

As all the above types of choices violate the preference axioms, no interpretable utility can be estimated from them.

Some authors recommend use of the dual-response none design. With no constant alternative present, some item must be chosen and only then assigned as acceptable or not. Unfortunately, this method introduces the problem of forced choice.

There can be more constant alternatives in a choice set offered simultaneously. E.g. with hired services and the main question "If these were the only services available from your operator, which one would you chose?" there might be constant alternatives "I would switch to another operator" and "I would stop using the service completely". The alternatives might be placed right in the choice task, or when the main "None" choice has been made, or outside the CBC in a separate calibration procedure.  Each of the levels can be characterized by their parameters.

Constant alternative is usually omitted from choice-based studies such as MXD - Maximum Difference Scaling, SCE - Sequential Choice Exercise or CEA - Cross Effect Analysis where it might cause excessive decrease of the collected information, or go against the principle of the method or its goal. The same may be true for electable benefits (freebies, options), necessity products (power, gasoline, staple foods), etc.

 

Performance

Choice tasks are an inefficient way of obtaining quantitative information on preferences. Respondents evaluate multiple concepts in a choice task, but only tell us about which one of them they prefer. It is not known how strong that preference is, relative to the other product concepts. 

Showing more product concepts in a choice-set increases the information content obtained from the task. As humans are quite efficient at processing information about products or their concepts, three to six relatively complicated concepts per task can be shown around. However, many more simple concepts can be shown. This is typical for CPG (Consumer Packed Goods) brand-price studies with concepts arranged as if on a shelf.

Motivations leading to a choice are a mixture of habits, expectations and randomness. It is believed that choices in CBC can represent the behavioral patterns of purchasers. E.g., in a nationwide study of bottled beers, the respondents who stated either (1) frequent visits to two different outlets for a similar purchase purpose or (2) two dissimilar purchase purposes in the same or a different outlet, were asked to do an additional CBC task. The selections of products and their prices were set according to the outlet type. While the filters were an application of CBS - Choice Based Sampling rather than an attempt of a behavioral segmentation, we computed 4 and 8-segment solutions. Percentages of the respondents who were assigned to the identical behavioral segments (table rows) from the two CBC exercises (table columns) are in the table below.

Percentage of assignments to identical behavioral segments
Segmentation solution (a) (1)
An identical purpose of purchase stated;
Different point of sale
(2)
A different purpose of purchase stated;
Point of sale not distinguished
4-Segment 67.0 % 79.1 %
8-Segment 66.0 % 68.0 %
(a) Latent class analysis program (Sawtooth Software, Inc.) was used for LCA - Latent Class Analysis

The behavioral patterns of respondents are strongly persistent even when they state a different purpose for their purchases. The belief in CBC as a robust experimental instrument for behavioral studies is clearly supported.

 

Examples

CBC is the workhorse in most of our studies based on DCM - Discrete Choice Modeling. Examples are scattered over several pages.