> CSDCA - Common Scale Discrete Choice Analysis

CSDCA - Common Scale Discrete Choice Analysis

---

The essence of futility is doing the same thing over and over and expecting a different result. 

[ Attributed to Albert Einstein ]

Problem

The conjoint method is very efficient in obtaining information on psychological trade-offs that consumers make when evaluating several attributes of a product together. Nevertheless, the results are sometimes difficult to use. Often heard complaint is concerned with no indication of perceptual equivalence between attributes. A way to convert perceptions about actual features to perceptions about a reduced set of features needed for product positioning is also missing.

The mentioned problems are due to part-worth reference (zero) points set in attributes. A reference point is chosen either subjectively as one of the attribute levels (when dummy coding is used) or as a middle point set so that average value of all tested levels of an attribute is zero (when effects coding is used). This makes effects of aspects from different attributes incomparable. However, the managerial understanding of an aspect effect is related to effect of any other aspect.

Difference scaling methods, on the other hand, use the same scale for all tested aspects. It is well known that batteries of evaluation questions have small discrimination power and too many features come out as important. MaxDiff (MXD - Maximum Difference Scaling) method does not suffer from this drawback. However, it is not almighty. First, in many practical cases with aspects organized in attributes with mutually exclusive levels, it is nearly impossible to generate a balanced design with satisfactory efficiency. Second, even if the former requirements were achieved, within-attribute precision of estimated part-worths would be heavily impaired. Third, very many tasks would have to be answered to obtain sufficient accuracy.

Solution

CSDCA - Common Scale Discrete Choice Analysis has evolved as an amalgamation of work of many authors in the last 20 years of DCM application in practice. It is based on ability of DCM - Discrete Choice Model to fuse discrete choice data from different sources. The method is fully described in the technical paper Common Scale Hybrid Discrete Choice Analysis [Bryan Orme, Sawtooth Software, Inc., 2013]. It relies on fusing data from two major interview sections, PRIORS and MOTIVATORS, and, optionally, CHOICES:

1. PRIORS
   Determination of order preferences of aspects grouped in an attribute within an attribute (only for attributes that do not have a priori known preference order of aspects). Attributes are tested one by one.
2. MOTIVATORS
   Determination of best-worst preferences of aspects between attributes within a profile. Profiles are presented one by one.
3. CHOICES
   Determination of (best-worst) preferences between profiles. Sets of profiles are presented one by one.

Sections

PRIORS

The preference order of aspects in each attribute is used as soft constraints in the estimation. For a nominal attribute, the order is the result of sorting. For a quantitative attribute, the order is often natural, but not always. Not everybody prefers a 500 Hp car engine to a 200 Hp one.

As aside

• The original CSDCA method (Orme 2013) relies on pairwise comparisons of aspects adjacent in the preference order. Choice sets are created as the first choice with the second, the second choice with the third, etc., until the last choice. The combination of the first choice with the last one is not considered because its information contribution is negligible. If the preference order of all aspects is known the plan is orthogonal and balanced and therefore better than the rank-explosion rule (Chapman and Staelin, 1982), where choice sets are created as a combination of the first choice with all the others, the second choice with all others without the first choice, etc.
• If the preference order of all aspects in an attribute is not known or there are ties in the order, the cyclic plan is neither balanced nor orthogonal. Plans generated by the algorithm used for SCE - Sequential Choice Exercise do not suffer from this drawback.
• If it is assumed that a respondent elected aspects with the same involvement in all the sections, the choices should contain approximately the same amount of information. It is possible to set the number and size of choice sets generated for SCE. This is not possible with the cyclic plan as the number and size of choice sets are fixed.

When aspects of an attribute have a natural order of preferences, typically in a quantitative benefit or detriment such as price, the constraints can be implemented in the estimation procedure directly. If a threshold acceptance level is to be obtained, an interviewing procedure similar to the Gabor-Granger pricing test can be used.

MOTIVATORS

This section known as the Best-Worst Case 2 (Louviere et al., 1995) is the heart of the method. It makes it possible to put part-worths of aspects of all considered attributes on the same scale. Respondents state which aspects of a randomized product profile are most and least preferred or motivating. The exercise may be viewed as a "self-explicated conjoint" transformed from evaluations into a choice format.

The best-worst approach has a theoretical reason but it may may have an additional advantage. E.g. healthcare researchers have commented that learning which conditions or outcomes patients want to avoid is perhaps even more important than learning what they want.

The method has remained out of the scope of practicing researches for some time as it did not provide satisfactory results when used as a standalone tool. However, when completed with the soft constraints from the previous section, it has been shown to give results very similar to those from the "golden standard" CBC - Choice Based Conjoint.  

Depending on the number of attributes, the choice tasks can be designed with aspects of all or reduced number of attributes. In contrast to CBC, it seems reasonable to rely on a (reduced) set of no more than 6 attributes. As there is no "profiles considered jointly" effect a lower number of aspects can make the task easier for respondents. Conversely, a higher number might require additional second best/worst choices with unpredictable influence on the overall performance of the method.

As aside

A CSDCA assembled from PRIORS and MOTIVATORS sections is a scaling rather than a conjoint method.

• In CSDCA, part-worths of aspects are estimated by decomposing preferences between aspects within attributes (section PRIORS) and within profiles (section MOTIVATORS).
• In conjoint, part-worths of aspects are estimated by decomposing preferences between profiles within sets of profiles.

CHOICES

If the purpose of the survey is to identify the main motivators (and/or deterrents) and their relative influences on customer decisions, as in a positioning study, this section is not required. However, if the goal is to estimate market performance of yet non-existent or modified products, their competitive potential and other characteristics available only from a market simulation, choice preferences among fully specified products should be determined.

When a short conjoint exercise is added a refinement of part-worths is achieved by fusing the information obtained from statistical analysis of the whole product preferences. A CBC exercise can be useful if performance of products in full attribute ranges is to be simulated and inspected. If the new product alternatives are given and immutable, a SCE exercise with a set of current and new market products will be sufficient to estimate their market performance.

When estimation of stated acceptances and/or competitive potential of products is desired the choice exercise may be completed with a calibration section.

Properties

The main advantage of the method is a common scale for all attributes. Also simplicity and easiness of answering the survey may be important in deployment on the web. It is well known that web respondents are prone to simplify their decisions in a conjoint exercise if it is long or profiles have many attributes. Tasks in the main two sections of CSDCA do not tempt respondents to shortcut or evasive answers.

By our knowledge, standard CBC works well for up to about 6 attributes. With higher number the compensatory effects of attribute levels are very hard to estimate and respondents tend to resort to simplified decisions, typically to the lowest price. Use of incomplete profiles in CBC cannot be recommended as it leads to excessive equalization of attribute importances (as is common in ACA - Adaptive Conjoint Analysis). In contrast, higher number of attributes is possible in CSDCA as only the best and worst aspects in a single profile are selected. 

The combined PRIORS and MOTIVATORS sections are not intended as a fully equivalent substitute to CBC. While all attributes work together (jointly) in conjoint, aspects in common scale analysis are predominantly referred to and chosen separately. An added CHOICE section can have substantially less tasks than when interviewed alone.

For Czech readers

Krátkou prezentaci v češtině lze stáhnout na odkaze OBIMA_Object-Image-Analysis.pptx

Strengths

• CSDCA allows for a higher number of attributes than a conjoint method.
• Number of unengaged respondents is decreased. 
• Tasks are easy and fast for respondents.
• Simplification choice strategies are greatly avoided.
• Change in types of choice tasks makes interviewing less boring. Fatigue and annoyance from interviewing are lower than in an equivalent conjoint exercise. 
• Unengaged respondents can be detected easier than in conjoint exercise.
• Between-attribute prohibitions do not introduce systematic bias in estimation.
• Part-worths on the common scale are obtained.
• The managerial "attractivity influence" of each aspect is obtained.
• The aspects can be compared across attributes.
• It is possible to determine acceptability thresholds necessary to apply non-compensatory decision models.
• The resolving power of the method is incomparably better than that of batteries of scale questions.
• The method is modular. An interview can be assembled so that inferences can be drawn according to the requirements.
• The tasks are simpler than in a conjoint method. Interviewing on tablets or mobile phones is feasible.

Weaknesses

• Attribute interaction effects are not estimable.
• Part-worths of the levels assigning specific alternatives can take values from the common scale only in special cases.

Extensions

Perceptional thresholds

CSDCA part-worth represent additive increments to the product utility but do not reflect perceptual properties of levels in an attribute. Similar to the OBIMA - Object Image Analysis, the level "None of the [shown] levels [left after the previous choices, if any] is acceptable" can be added to the levels of each nominal attribute tested in the PRIORS section of CSDCA. The part-worth obtained for this level is the value between the just acceptable and just unacceptable levels, and can be interpreted as a stated perceptual threshold. Positive and negative "stated perceptions" computed as acceptances relative to the threshold can be useful for understanding views of the customers. The thresholds can be the basis for non-compensatory modeling and simulation.

Sorting of the levels by the degree of acceptance, attractiveness or suitability (desirability) can be done using SCE - Sequential Choice Exercise. For quantitative attributes, the Gabor-Granger method, originally developed as a price test, is an option. Data from these procedures are equivalent to those from SCE, and can be converted into a format suitable for analysis of discrete choices.

As aside

• Unexpectedly, some problems arose with ordinal attributes, typically price. A simple "price wheel" variant of Gabor-Granger test procedure was not fully satisfactory. Starting the wheel at the lowest price led to unrealistic low-price threshold. Starting at the highest price was dissatisfying from the methodology view. When the attribute represented price of an undesirable option, the respondent was refusing all price levels one by one, and it took the precious interviewing time. A solution was found in using the algorithm that merges both randomized and systematic "price wheel" querying [Lipovetsky, S., Magnan, S., and Polzi, A.Z. (2011). Pricing Models in Marketing Research. Intelligent Information Management, 3, 167-174].
• An early attempt to ask for "the highest price the item would be purchased for" and "the price threshold above which the item would not be purchased" proved to be a complete failure. In too many cases, respondents showed complete ignorance of the actual price range.

Different threshold values can be obtained from CSDCA based product utilities calibrated by purchase intentions statements for complete products. The statements reflect the joint influence of all product attributes. Such thresholds can be called evoked, and the perceptions computed from them called evoked perceptions. Comparison of stated and evoked perceptions can be useful for understanding the differences between statements about isolated features and their projection in complete products.

Optional attributes

A generalized model of bundling or packaging options into a single product is still a problem to be solved. However, whatever modeling approach is adopted, a set of reliable part-worths of isolated options is essential. A full profile CBC with about up to 5 medium or up to 7 easily understood and evaluated attributes can provide reliable estimates of level part-worth. A reduced set of the base attributes, most important for decision makers and, therefore, making the core of the product, can be tested in the CHOICES section, typically a CBC exercise. Many more attributes can be tested in the PRIORS and MOTIVATORS section. The part-worths of the additional, less important attributes, will be dispersed among, and correctly related to, part-worths of the base attributes. It is anticipated that at least some problems with a large number of attributes can be handled this way. 

As aside

• The author has attempted to use a number of methods claimed to be suitable for assessment of several tens of attributes, but with only minor success. CSDCA seems to be a hopeful approach for the future. 
• The approach was partly tested in a study run in 2015. The bundling/packaging conditions could be cleanly depicted with each bundle/package treated as a specific alternative.
• The number of packages was 5, each with 2 or 3 pricing attributes. All pricing attributes had 5 levels. One of the pricing attributes had nominal levels designed as combination of separate payments. Price ranges over the packages were generally disjunctive and were treated as independent for each package. There were 2 additional nominal attributes, namely a discount attribute with 6 levels, and a free-bee attribute with 14 levels. On the whole, there were 10 attributes with 64 level part-worths to be estimated. In addition, 9 attribute perceptual thresholds were estimated.
• The calibration procedure for 8 representative profiles was run as SCE - Sequential Choice Exercise with rating. Median of the Goodman-Kruskal's gamma (a measure of rank correlation) between the actual and computed rankings was 0.87. The median of determination (explained utility variance) was 0.75. Both the values suggest the approach is viable.

Conclusion

It is believed the hybrid CSDCA method that places attributes on a common scale thus allowing direct comparisons of the part-worths across attributes, might make conjoint-like results accessible and appealing to more users. Relative influences of attribute levels on willingness to purchase a product can be presented and compared. The ability to estimate stated and evoked perceptions of attribute levels allows to reveal what respondents think of the isolated or jointly considered attribute levels. 

CSDCA have appeared to be very effective. From researcher's view, more attributes can be tested without putting increased load on respondents compared to the standard CBC. Complaints about annoying repetitive tasks were not encountered. In our practice, CSDCA is already used more often than the standalone CBC method.

A few examples of results are on the page CSDCA Examples.

Live questionnaire

You may experience the look and feel of a (radically shortened) CSDCA live questionnaire for a simple hypothetical product. 

For Czech readers

Jiný přehled a několik výsledků z CSDCA jsou na blogu DCM Cz.