In God we trust. All others must bring data.
W. Edwards Deming
AbstractA method to estimate preferences between product concepts and their aspects is presented. The method is designed for a special case when concepts are fixed and described as a set of aspects. The simplest interview arrangement consists of two blocks.
One block is a best-choice MaxDiff of concepts or packages. Each choice can be completed with a calibration question on purchase intention. The other block is best-worst MaxDiff of aspects the concepts are composed of. The choice preferences obtained from the blocks are combined and quantified as part-worths of the aspects that make utilities of the concepts.
The method has been designed to handle more attributes than a CBC. A non-compensatory model of choice, especially useful for product bundles, is assumed using acceptability thresholds obtained from calibration questions.
The test is equivalent to the usual concept test with the advantage of obtaining the influence of aspects on a common scale.
The choice-based conjoint (CBC) is a widely used research technique for products that can be described by about 6 or fewer attributes having 2 or more levels each. Actual market products and services have often many more attributes that make use of CBC unfeasible. This concerns namely packages that have aspects either present or absent and are not mutually exclusive. Aspects are often grouped according to the total value of an offer, and relationships between them are complicated. Only some combinations of aspects are allowed. Generating reasonable product profiles in a conjoint study would require numerous cross-attribute prohibitions that are a frequent cause of inefficient or unrealistic product profiles, of a slow or none estimation convergence, and nearly always of estimation bias. The method of product classes has proved useful for products with only a few attributes, typically up to 6 attributes.
The approach of incomplete conjoint profiles is seldom usable. It leads to leveling the importance of attributes in general and to an apparently increased importance of the least often shown attributes. The method is unusable in a branded study since respondents implicitly set the missing attributes to the level values they suppose the brand usually has.
Due to the inconvenience with the application and the results of a conjoint method, the current approach relies on product concepts evaluated in a sequential monadic test usually followed by additional questions on numerous aspects of the concepts. The main disadvantage of this approach is a lack of information on the influence of individual aspects on the acceptability of a product. No simulation of a product set in the frame of the tested aspects is possible.The objective of the presented method is a concept test providing an estimation of both acceptance of the concepts and effects of aspects the concepts are composed of.
Instead of using profiles generated by an algorithm, the method relies on fixed profiles designed in a managerial way. The range of aspects, their settings, and balance between them (typically value and price) are taken as guaranteed since the profiles are provided by a study submitter who is supposed to have good knowledge of the market. By their very nature, the profiles exclude the need to introduce prohibitions between attribute levels. These compensations are not reflected in any known algorithm for generating near-to-orthogonal profiles for a conjoint task.
The method is inspired by the half-forgotten approach of the 1950s, referred to as the SEC - Self Explicated Conjoint (or Self Explanatory Conjoint). The respondent evaluated each attribute level attractiveness and each attribute importance one at a time. The obtained evaluations were transformed into values of an additive variable. The sum of these values characterized a product composed of attribute levels.
In our approach, the attractiveness of both profiles and aspects is estimated in the respective MaxDiff tasks. The scale-based questions on concepts in the traditional concept test are replaced by MaxDiff of profiles. The choices can be completed with questions on (stated) purchase intention. The scale-based questions on attribute levels in SEC are replaced by choices from small MaxDiff choice sets composed of the aspects of which the tested profiles are composed. The influence of aspects is obtained from uniting the results from both groups of tasks using a common behavioral model. The main principles of CSDCA - Common Scale Discrete Choice Analysis are applied.
In a conjoint-type exercise, the number of attribute levels should be about the same, the differences between effects of levels within an attribute equidistant, and the spans of attributes of similar importance. This is required to obtain reliable estimates from responses to profiles generated by an algorithm. The present method does not impose these requirements. The overall scope and implicit (or expected) importance of the aspects is given by the provided concepts.
Choice of the behavioral model is important. In the simplest case, the traditional additive model can be used. If there is a suspicion or assumption of non-compensatory behavior, the interview should be extended by a CSDCA PRIORS block of questions to obtain better estimates of acceptability thresholds.
Improvements in the quality of results is expected to be similar to the transition from scale-based questions to MaxDiff, CSDCA or AGA procedures.
The actual order of the questionnaire blocks should take account of the learning process of respondents.
The mutually exclusive aspects can be grouped into formal attributes. This will allow taking advantage of commercial programs designed to analyze conjoint tasks. Among these are constraints between the values of part-worth of ordinal level values, the use of different types of coding for estimation of thresholds, etc.
Utilities of profiles and part-worths of aspects are estimated by the hierarchical Bayes maximum likelihood estimation. The data set is composed of all the discrete choice exercises each of which has a different weight. The weights are determined from the number of tasks and a number of items in a task using the gain in information obtained as a loss of information entropy (cf. How many CBC tasks).
The effects of aspects are expressed in terms of perceptance, i.e. the difference between the probabilities of accepting and non-accepting an aspect. A profile is characterized by a stated purchase intention computed as its censored perceptance, the censoring being made by considering only non-negative values.
A simulation of purchase intentions of untested profiles is possible only for profiles from the close vicinity of the tested profiles. One reason is the prerequisite "considered jointly" is not valid for all combinations of aspects since only a limited number of aspect combinations were tested. Another reason is the number of effects summed in the simulation. The present method is designed to handle many more aspects than a conjoint method is able to. Utility error of untested concepts grows with the number of changed aspects because of the error cumulation.
The model was applied on a set of several bank account concepts that could be described by 12 attributes with 2 to 5 levels each. This number of attributes was too high to use a CBC methodology. Instead, the levels were treated as 46 aspects. The questionnaire contained a MaxDiff of the concepts and a MaxDiff of the aspects. A web questionnaire was created in Sawtooth SSI ver. 8.4.8 system. Data were processed using Sawtooth CBC/HB 5.5.4. A shortened and obfuscated version of the live questionnaire in Czech language is available on-line.
The acceptance of each of the concepts was obtained from the MaxDiff of concepts. Each unique concept choice was completed with a calibration question on the intention to switch to the concept so that the raw utilities could be modified to reflect the stated intentions.
The client was interested in which aspects and to what extent they were responsible for an attractiveness or, possibly, a repulsiveness of the concepts. This was achieved by combining the results from both the MaxDiffs. Since perceptance threshold value for aspects was not determined experimentally a common threshold was set arbitrarily to the mean utility value of the price range. The results are in the picture below.
The interpretation of perceptances is straightforward for most aspects. Nevertheless, it
should be remembered a perceptance value is a single number that comprises several attitudes, namely common
knowledge, expectation, and satisfaction with the aspect (and its value, if quantifiable).
Among the tested aspects, neither a credit card nor airport lounges are usually offered as a part of a bank account. The explicitly stated absence of an airport lounge has higher expectations compared to the absence of a credit card that respondents are more familiar with. However, including even the lowest credit card option in a bank account leads to a higher satisfaction than providing any of the offered airport lounge options. The most important interpretation value is the difference between perceptances of individual aspects. An aspect with substantially low perceptance of both its absence and presence, i.e. an aspect with high expectations that cannot be satisfied, should be omitted from an offer.
The Bank prepared 24 concepts for testing based on a total of 47 aspects. Of these, 2 aspects were optional (insurance with codes 2.2 and 3.2), all others were fixed. Each concept was composed of 11 attributes.
The first block of questions consisted of 9 choice tasks with 4 concepts each. Each choice was completed with a purchase intention question. The next block was a best-worst MaxDiff of 35 aspects in 16 choice sets composed of 4 aspects each. The levels from Condition and Price attributes were not included as they could not be compared with any other aspect, and were treated separately. The obtained perceptance values are shown in the figure below.
Quite as expected, the most important aspects are conditions and price. The first three conditions practically do not reduce the acceptability of the product. The lowest ATM service level is sufficient for most respondents, and its further improvement has only a small effect.
Four types of insurance were offered. The price was increasing with the index behind the dot of the insurance code. The most attractive insurance was insurance 4.0 without an additional fee. Second in order was the simplest 1.1 insurance for the lowest price. Optional insurance contributes more to the attractiveness of the profile than the same fixed insurance. Eligibility, as concluded, contributes to the overall acceptability of the offer.
The offer of financial reserve and bonus proved to be ineffective. The discount program is better, but without it, the offer will still be acceptable. Belief in discount programs is generally low. Specific welcome packages have been welcomed, but it seems the responses were exaggerated. Omitting welcome packages would not fundamentally harm the acceptability of the product.
Gifts were generally welcomed. On the other hand, an explicit statement about the absence of a gift, when offered elsewhere, has a mild negative effect. The effect of the statement "no insurance offered" was even more pronounced, as there was the possibility of optional insurance in other offers. It is believed that not only the context between the aspects in the profile, but also between the profiles, namely those offered by the competition, is important.
The method has proven to be a stable and practical solution for estimating the effects of a large number of attributes with levels understood as separate aspects of the product. It is considered useful for testing concepts or packages with many limitations between aspects. It is less prone to estimation bias due to collinearity of aspects than the conjoint-like approach. Collinearity of aspects is ubiquitous in market products, namely when aspects are grouped according to the product price classes.
During its use, the method has been enhanced with possibilities to randomly modify the default concepts, use optional levels of attributes, create a histogram of acceptable prices, and refine the estimation of thresholds by using a reworked estimation method. The new version has been named CBCT - Choice Based Concept Test and made available for deployment.