We talk nearly every day with
a conjoint method user who has run into a problem of some kind. The
problem responsible for the most frequent customer support calls is
using too many prohibitions. The system won't be able to produce a good
design, and may fail altogether.
Prohibitions, if at all possible, should be avoided. Too many
prohibitions, in the best case, can lead to imprecise utility estimation
and, in the worst case, unresolvable (confounded) effects and the
complete inability to calculate stable utilities. Allegedly, there have
been entire data sets with hundreds of respondents going to waste ...
[ Freely by
Sawtooth Software, Inc. ]
Problem
A frequent requirement in testing products is concerned with prohibitions
between levels of two or more attributes. Common is conditional pricing
due to presence or absence of some levels such as brand, product
alternative, package, size, etc. However, unwisely set prohibitions can be
detrimental, and sometimes even fatal, for the design
efficiency. As seeing is believing, a simple example is
provided below.
Any experimental plan can be partitioned into smaller design building
blocks. Say one of the blocks is a L4 orthogonal array:
Design
plan
L4
Profile number |
Level of A1 |
Level of A2 |
Level of A3 |
1. |
1 |
1 |
1 |
2. |
1 |
2 |
2 |
3. |
2 |
1 |
2 |
4. |
2 |
2 |
1 |
The block has this covariance matrix:
Covariances
of
design plan L4
Attribute |
A1 |
A2 |
A3 |
A1 |
1.0 |
0 |
0 |
A2 |
0 |
1.0 |
0 |
A3 |
0 |
0 |
1.0 |
Values in the diagonal elements are variance propagation coefficients.
They tell us about the extent by which a random error in the data will be
projected into the parameter estimates. Values of non-diagonal elements
are the same measure for mutual dependence of errors between the
attributes. The design is orthogonal as all non-diagonal elements are
zero, and fully balanced as all diagonal elements are equal.
Now we decide that, for some good reason, we cannot combine all levels
and must introduce a prohibition leading to the change of level of
attribute A3 from 1 to 2 in profile 4. The covariance matrix changes
as follows:
Covariances
of
design plan L4 with a prohibition
Attribute |
A1 |
A2 |
A3 |
A1 |
2.0 |
1.0 |
-2.0 |
A2 |
1.0 |
2.0 |
-2.0 |
A3 |
-2.0 |
-2.0 |
4.0 |
Not only variance of the affected attribute A3 has increased fourfold,
but those of the other attributes have doubled. Even worse, a strong
dependence between attributes has been introduced which means that errors
will not compensate. If there are interactions between attributes, the
introduced bias will accumulate over respondents. Since small interactions
between attributes are common, a design with prohibitions will lead to the
bias that may prevail over the main effects of estimated parameters.
Solution
There are many suggestions in the literature how to deal with
prohibitions and test their influences. We have adopted the standard
method of randomized blocked design made up of "product classes" in
combination with ASD
-Alternative
Specific Design. A class is defined as a complete Cartesian set of
profiles from allowed intervals, i.e. a set with no prohibitions among
attributes allowed. In plain language, a class is the complete set of
products having properties not exceeding the allowed intervals of
properties given a fixed values of the other properties and features.
Randomized representative subsets of orthogonal profiles obtained for each
of the product classes are used in an experiment for each respondent. The
method is statistically clean,
easy to manage, understandable for client to specify and, as of practical
importance, can be converted to the prohibition strategy implemented in
commercially available design programs. The class approach guarantees all
part-worths to be estimable with a minimal bias.
A class for the purpose of a conjoint study design is composed from the
profiles whose certain attributes are limited to fixed intervals. It is
desirable that the defined classes can be ordered in some way, typically
by price range, assumed quality, way of usage, etc. Nominal classes are
usually much harder to be handled with, and, therefore, are often grouped
in ordinal hyper-classes. The number of hierarchical levels is, at least
in theory, unlimited.
Application
In a typical SP (stated preference) experiment, a choice set consists of
several alternatives among which the respondent is asked to discriminate.
The alternatives should (1) provide sufficient variations of attribute
values in the managerially permissible intervals and (2) guarantee
estimability of the model parameters without a danger of having introduced
correlations caused by improper attribute level prohibitions.
A simple representative of a class is a real or suggested product with
the attribute values varied in an allowable range. If some combinations of
attribute levels are still detected as not feasible the set of product
profiles must be partitioned into 2 (or more) classes with attribute level
intervals chosen so that all possible combinations of attribute levels in
a class are allowed. A nested class structure is common.
A conjoint exercise should be composed of the product classes relevant
for a given respondent, i.e. of the alternatives supposed to be from the
respondent's evoked/consideration
set. This can be accomplished with use of CBS
- Choice Based Sampling that allows to assign a range of classes to
a respondent.
Class Properties
Class structure of tested products often comes from the goal of the study
quite naturally. E.g., cars can be classified as commercial and personal
motor vehicles, these as small, lower middle, upper middle, luxury ones,
off-roads, etc., and perhaps further divided into subclasses by make,
equipment features, etc. Consumption goods can be divided into categories,
these into low-level up to luxury classes, then according to the
packaging, etc.The class structure and order of the class levels always
depends on the conditions and objectives of the study in a way
that does not contradict the rules of DOE - Design
Of Experiments.
- A class structure must be strictly hierarchical. No exceptions are
allowed.
- The whole range of tested products (rather than attribute ranges)
is partitioned into product classes such that all combinations of
attribute levels belonging to the class are allowed inside the
class. If this is not possible by classifying products in a single
hierarchy level, a class should be divided into lower-level
sub-classes (nests).
- The ranges of attribute levels in the lowest level class must be
narrow enough not to allow generation of extremely attractive or
totally unacceptable product profiles. All profiles in a class
should be managerially acceptable.
- At the same time, the ranges of attribute levels in the class must
be broad enough to allow estimation of their influences.
- Partitioning of the products into classes does not mean partitioning
of the attribute ranges. No exclusivity of the attribute ranges in
classes is required, i.e., ranges of the respective attribute levels may
(and usually do) overlap.
- Classes should reflect different types of anticipated behavior and
perceptions of respondents.
- Classes may or may not have common attributes. The latter case
requires using ASD
-
Alternative
Specific Design method.
- Each attribute in a class is defined on a given range of values. The
range may contain one level (e.g. of a brand attribute) or more levels
(e.g. of a price attribute).
- If the range is made up of a single level, the class is a specific
alternative.
- In turn, a specific alternative can have only one attribute on a
single level. If there were two or more attributes on a single
level, their utilities would not be estimable. Either a composite
attribute or sub-classes should be created.
- The same number of levels of a common attribute (typically price)
should be used in each class. The number should be very reasonable
so as not to exceed an acceptable total number of levels in the
study.
- The range of level values or items in a class should be broad
enough to asses the effect of attribute levels in the class but
narrow enough not to exceed managerially acceptable values.
- A conjoint study based on product classes can be understood as a
merger of several sub-conjoint studies where there are no prohibitions
between attribute levels in any of the sub-conjoint exercises.
The approach of composing a conjoint exercise from product classes
has appeared as very useful:
- Products often fall into classes quite naturally.
- Ranges of attributes levels can be easily defined and modified.
- The profiles shown to respondents are credible.
- As attribute ranges in a class are usually quite narrow interactions
between attribute levels can be neglected and omitted from the
estimation.
- A blocked design has usually a substantially lower number of
parameters than an equivalent model with interactions. In our
experience, about the same fit of the models is obtained. The model
description is more compact and the precision of a simulation
unimpaired.
- Part-worths from a blocked design are usually more stable (have lower
variance) than those from a non-blocked design.
- Class attribute is usually hidden from respondents. Its level
part-worths may or may not be estimated. The latter case is typical for
the classes serving only as an aid in the design.
- If estimated, interpretation of the part-worths related to the classes
is usually more straightforward than of those based on
interactions. E.g., it is much easier to understand and interpret a
single value of price-based elasticity of substitution or conventional
share-based price elasticity for a brand than, say, values for several
price levels and the same number of values for interaction between the
brand and the price levels.
- The spans of the implicit conjoint blocks, i.e. the classes that make
them up, usually overlap. This contributes to the overall stability and
reliability of the estimates.
- Last but not least advantage is the client can easily see where and
why the design might be deficient and can promptly make the proper
provisions.
To a limited extent, product classes can be given predefined weights to
simulate market distribution and availability of the products the classes
represent.
In the product classes approach, the overall design may be slightly out
of balance in respect to some attribute levels, most often to the outer
levels of a value-based attribute such as price. This effect is often
recognized as positive because the least frequently shown outer levels are
usually the least managerially important. Nevertheless, if the off-balance
should be a problem, the design weight (presentation frequency) of the
affected levels may be boosted.
As aside
- A design made up of classes should not be confused with an
alternative specific design analyzed as a nested "mother logit"
model.
- A typical use of the class approach is for various types of packages
made by bundling products or services. Each package may be of its own
class and have ranges of its properties set independently from
properties of other packages.
- Representatives of all classes in a conjoint exercise are usually
shown to respondent equally often, i.e. with ratio
1 : 1 : 1 : ... . This is in
agreement with the null hypothesis that all classes are (a priori)
equally attractive to the subject. If relevant the frequency can
be set to a ratio of small natural numbers, e.g.
3 : 2 : 1 : ..., e.g. if the class is a
brand with many more varieties than other brands offer.
- The cardinal motivation to introduce the product classes based
construction of a CBC study was to avoid adaptive methods of
interviewing such as ACA
- Adaptive Conjoint Analysis. The usual goal in marketing
engineering is to asses potential of the concepts embodying the
intentions of the vendor as compared to the current products, and the
reason for differences. In adaptive approach, respondents tend to
exclude undesirable levels in addition to those completely
unacceptable. This is in a direct contrast to the fact that customers
tolerate minor disadvantages when the important properties are
satisfying. An adaptive strategy tends to emphasize the idealized
wishes of the customers. The possibility to exclude certain levels of
attributes before the actual presentation and evaluation of concepts
often leads to exclusion of certain groups of common products from the
evaluations, and, consequently, does not provide relevant data to
answer the posed problem.
- The recently introduced ACBC
- Adaptive Choice-Based Conjoint (Sawtooth, Inc., 2009) is an
adaptive method of a new generation. Unfortunately, the author has no
direct experience with it.