CBS - Choice Based Sampling

> CBS - Choice Based Sampling

Irrelevant people give irrelevant answers.

Anonymous

Problem

A range of products tested in a study is often too broad for an individual to be interested in all of them. Individuals differ in consideration sets by their needs, expectations, budgets and alike. So that an interview is efficient and unbiased, the selection of tested products and the related questions should aim at the respondent's consideration scope. Only a product acceptable for the respondent can be chosen with a sufficient reliability. Therefore, the design of a conjoint exercise should allow generation of choice sets given the respondent, and allow to build the situations in which decisions are supposed to be made.

The danger of asking about profiles outside the consideration scope lies in the phenomenon known as impersonation. Instead of refusing profiles not fitting their needs and expectations, people tend to adapt and shift their personal attitudes from their own consideration scope into the frame of the offered profiles. The answers are correct, but do not correspond to consumption habits of the interviewee. A person living mostly on hamburgers and donuts can easily distinguish rise from beens or caviar from lobster, and state his or her preferences but will hardly change habits and contribute to the consumption of the inquired goods.

A common approach to the problem is to split a sample into segments with help of filter questions, and for each segment have a separate version of the questionnaire such as a conjoint block, concept test, battery of question, etc. There are situations where this strategy does not work well because the filter questions cannot be asked jointly. E.g., in case of testing complex products with many attributes, price categories or schemes, component parts of bundles or packages, marketing ideas, etc., formulation of proper filter questions may be tricky.

Solution

A short pseudo-conjoint test with a limited number of items that represent the whole range of the studied items can be designed. Every respondent is presented with a set of product profiles (at best about 12 to 24, but 36 relatively simple profiles have been verified as feasible) where the profiles represent both the reference (usually the current) and the expected state of the market. The set is composed either of ad-hoc profiles reasonably selected from some pool, or, preferably, of items satisfying the rules of efficient design, possibly using a product classes approach. The responses to the stimuli are analyzed in course of the interview, the range of interests is determined, and the respondent is assigned the block of the questionnaire respective to his or her interests. The assignment is based on the sampling scheme known as "independent multinomial sampling" or "product multinomial sampling". The procedure falls to a group of the CBS - Choice Based Sampling methods.

Compared to standard filter questions, CBS has some advantages.

CBS completely removes "selectivity bias" stemming from imprecise answers to (possibly misinterpreted) verbal questions. It is well known that when asked separately nearly all aspects of a product are declared important with a small discrimination among them. In CBS, no "selectivity corrections" are needed.

If the CBS serves just for piping respondents to a questionnaire version or a conjoint block, the requirements of orthogonality and balance of the profiles are very relaxed. The profiles can be devised by the client who can utilize their skintight knowledge of the product category.

If there is a request to extend the range of utility levels used in a conjoint block with the levels of the products used in the CBS block, each of the CBS profiles will have to enter the analysis, and the profiles must obey specific CBS design rules.

If the design of profiles in a CBS is sufficiently balanced in respect to the expected choice probability, and does not exert large deviations from orthogonality, the result from CBS can be combined with the results of the conjoint block it pointed to. The CBS block is usually common to all respondents in a study. In such a case parameters of a common model comprising all conjoint blocks can be estimated and used in a simulation task. Design of a proper CBS block usually requires a close cooperation between the client and the study designer.

The SCE - Sequential Choice Exercise, namely a version with removal of unacceptable items in the first step, has proved useful as CBS. Only several first choices from the set are needed to assign one of the predetermined questionnaire variants to the respondent. The assignment is based on finding the highest inclusive value of the chosen selection as projected onto the the questionnaire variants.

As aside

The request for balance in respect to the expected choice probabilities over the studied sample is a precarious one. As it is hard to predict and can be checked only when the study is on or later, a sample boost or re-weighting of the sample is sometimes necessary.
Respondents usually select profiles from a relatively narrow region of products, typically a price range. This suggest the behavioral model EBA - Elimination By Aspect [A. Tversky, 1972], and allows estimation of the WTP - Willingness To Pay.

The set of profiles used in CBS should respect the following requirements:

The set must create a test market of realistic profiles.
Attractiveness of the profiles representing different classes should be as even as possible. If not, the sampling probability might be disbalanced.

Especially detrimental are profiles with an attractiveness evidently exceeding those of the other offers in the set. They will have tendency to be chosen too often.
Profiles with low attractiveness are not as dangerous since they function as fillers to be removed in the first step of SCE and, actually, will not be considered. They may unnecessarily decrease the number of effective profiles and disbalance the set.

The set must be balanced.

It should cover the needs and expectations of all individuals in the sample. Everybody should be able to chose something from the set.
If labeled profiles are used with labels indicating the classes (e.g. brand, usage, quality, presence/absence of important features, etc.), each label should be represented by a number of profiles approximately proportional to the number of offers available on the market.

A subset of the most important attributes is allowed provided the missing attributes are known to have low importance and/or to be implicitly understood as having the current market values.

In connection with assignment of a conjoint block to a respondent, it is of advantage to represent each class by 3 or 4 representative profiles. With the total number of 12 to 21 profiles, 3 to 7 classes of products can be represented in the set. To make a balanced representation of a class, three-level benefits (with the levels "Low", "Medium" and "High") of the 3 most influential attributes (A#1, A#2 and A#3) may be set according to the following table.

Attribute	1st Concept	2nd Concept	3rd Concept	4th Concept (optional)
A#1	High	Low	Medium	Medium
A#2	Medium	High	Low	Medium
A#3	Low	Medium	High	Medium

The level "high" should be the highest level in the class this sub-block points to. All other attributes should be set so as to achieve the most realistic product profiles. The design in the table is balanced both in theoretical and practical way. Outer levels are shown with the same frequency and all profiles have medium attractiveness on average.

As aside

The above requirements are in opposition to the general specification of SCE - Sequential Choice Exercise where no restriction or limitation of any kind are required.
In the case of merging separate conjoint blocks into a common model covering all classes considered in the study the following requirements must hold. They make a good check of legitimacy of the suggested conjoint attribute ranges.

At least one profile of the subset representing the lowest quality class must have the most influential attribute set on the least beneficial value used in the conjoint.
At least one profile of the subset representing the highest quality class the must have the the most influential attribute set on the most beneficial value used in the conjoint.
Levels of other attributes should be set so that influences of all attributes tend to compensate.