Profile Properties

> Conjoint Method Overview > Profile Properties

Profile Properties

Expansion means complexity and complexity decay.

[ C. Northcote Parkinson ]

Problem

The amount of information obtained from a conjoint exercise is dependent mostly on the following:

Design of the exercise, i.e. statistical properties of the tested profiles.
Comprehensibility, credibility and stimulating effects of the profiles as perceived by respondents.

Solution

The standard conjoint model assumes that all pair-wise combinations of levels of all attributes should be generated and presented in a statistically significant number in the study. With today's versatility of goods, achieving such a condition is unrealistic in the frame of a single interview. A conjoint design consists of sets of artificially generated profiles. The profiles can be selected from a single or several product classes. The class approach allows a manageable hierarchical structure of the design. The number of required pair-wise combinations of levels can be decreased and the assumed item correlations, i.e. mutual substitutability, reflected.

Comprehensibility and perceptional properties of profiles can be enhanced by their uniform layout, use of graphic elements instead of descriptive texts, and avoiding any redundant elements or descriptions not necessarily required for understanding and evaluation of the profiles. Simplicity is the essence.

Statistical properties of profiles

The total number of profiles that can be shown in a study is always limited. It is often possible to decrease the required number of profiles by decomposing the design into several smaller conjoint blocks. Each block alone must have favorable statistical properties (see e.g. "Orthogonal Arrays: Theory and Applications" by A.S. Hedayat, N.J.A. Sloane, and J. Stufken):

The block should be balanced.
- All levels should occur equally often and none should be missing.
- The balance is important from a probabilistic point of view. The level shown more often will be chosen more often and get a higher part-worth.
The block should have strength 2 or higher.
- Strength equal to 2 means that all possible pairs of levels are present in the subsets.
The block should be (at least near to) orthogonal.
- Orthogonality means that all possible combinations of levels given the strength (pair-wise for strength equal to 2) occur equally often, i.e. attribute levels in the design are not correlated.
- Orthogonality ensures that all estimated main effects, i.e. part-worth of the levels, will be unbiased and have minimal random error variance for the given number and organization of profiles in the subsets the set is made of.

The required properties on the level related to a respondent are often just a wish-list since the number of profile evaluations or choice-tasks would be unacceptably high. So that the overall design is not deficient, profile sets (possibly composed of sub-sets) must be generated in a way so that orthogonality with at least 90% efficiency is achieved for several respondents. The number should be substantially lower than the size of the smallest behaviorally homogeneous sample segment.

The analysis of the collected data must respect the design. An estimation of part-worths for each individual must be based on a set of profiles that satisfy the above requirements. Neglecting this may have severe consequences up to unusable results. A researcher should not mindlessly rely on modern robust methods such as hierarchical Bayes estimation. If there is not enough valid data from a respondent, the method will borrow too much data from other respondents.

A possible way to meet the balance, strength and orthogonality requirements in order to obtain estimates at a respondent level can be based on product classes.

In direct contrast to an orthogonal design is the "managerial" design. Its inherent property is keeping the trade-offs of all attributes balanced and mutually compensated. Managerially designed profiles nearly never comply with the requirement of attribute separability. Typically, the price is derived from the quality, features, and performance of the product. Certain combinations of attribute levels do not make sense. Removal of such combinations using prohibitions in the design always introduces correlations among attributes. When attributes in profiles are strongly correlated, a conjoint analysis is not the plausible method for the study. Such profiles are straight adepts for a concept test, possibly run as a Sequential Choice Exercise.

As aside

A successful design of profiles for a conjoint study requires iterative communication between the client and analyst. A design proposed in the early stages of a project specification may substantially change later when all the aims, conditions and limitations become fully clear.

Perceptional properties of profiles

A response to a profile or a set of profiles is influenced not only by the just visible profile(s) but by all the profiles shown so far. The perceptional properties of all profiles used in the study are important.

Profiles in a conjoint task must be easily readable, comprehensible and comparable. Combination of attribute levels must be realistic and believable. Difficult to read, too demanding or confusing profiles make respondents ignore the task and resort to evasive answers. The less comprehensible or credible the profiles are the less seriously are taken. Respondents may loose their interest in the study, their answers become unreliable, and their tendency to refusals is increased. While each study requires a specific approach, some non-statistical aspects of profile design and presentation are quite general.

Presentation of attribute levels

Layout of the profiles, namely those to be compared, should be the same.
Long descriptions should be avoided and replaced by formulations resembling allusions or slogans. Respondents will not read long texts when the main task is a profile evaluation, comparison, or selection of the best choice. The short descriptions should be explained before the conjoint exercise is entered. The explanations should be available as a help feature during the exercise.
The descriptions should be of a similar length for all levels of the attribute.
Quantitative attributes deserve a special attention. Presentation form of their levels should serve the purpose of the study.
- In a DCM-based conjoint the interest is only in preferences, i.e. differences in part-worth values among the levels. For example the difference between values 2.99 and 3.99 is the same as between values 3 and 4 which are read and compared much easier. If the values are prices the introduced "psychological cent" will make the later interpolation between non-.99 values unusable.
- On the other hand, if the purpose is to imitate a shelf, calibrate relative utilities, and give up the possibility of interpolation between the interviewed values, values resembling the real ones are the best and should be used.
A visually rich presentation format is recommended. If used, graphic quality such as resolution, size and color rendering of the images must be equal or at least very even.

Combinations of attribute levels

An overly attractive offer evidently exceeding attractiveness of the other offers in a set is useless or even detrimental.
- No useful information is obtained from a choice or evaluation. It could have been predicted without running any exercise.
- Respondents remember the attractive offer and their willingness to a positive choice or evaluation of a regular profile offered later is decreased. Frequency of non-informative answers and refusals increases.
Offers having overly low attractiveness are useless but can be tolerated.
- The offer is usually taken by the respondent for being out of the consideration scope. As such, it is ignored and decreases the informative value of the set. The set becomes unbalanced and the number of effective profiles decreased.
- When the low attractiveness is bound repeatedly to a certain attribute level, e.g. to one of the brands, estimated part-worth utility of the level may get excessively small. When the brand is shown later with a higher attractiveness of the attribute level combinations, the offer may not be accepted as credible. Such a situation should be avoided when the study is composed of several conjoint exercises.

If the study is aimed at a distant future, a possibly radical move in the marketing strategy, or an unexpected change in features leading to profiles strongly differing from the current ones, respondents should be informed about this before entering the test. Respondents always tend to give answers with credibility proportional to the credibility of the objects presented in the study.

Partial profiles

Removal of one or more attributes from all profiles of a choice set, in order to obtain partial profiles with a reduced number of attributes, is the standard method used in ACA - Adaptive conjoint Analysis and an option in CBC - Choice-Based Conjoint. The purpose is to simplify the comparison and evaluation of profiles in the set when the number of attributes is large. A proper method of randomization of the removed attributes allows for balancing the number of times the attributes are shown or hidden.

Partial profiles should be avoided in the studies involving brand and price. Both these attributes implicate the image and quality. The unseen attributes are filled in by respondents with some adequate values quite naturally. The implicit rule "ceteris paribus" (everything else being the same) for the unseen attributes seems to be more often violated than kept.

In analogy with the number of levels of an attribute, the importance of an attribute would seem to grow with the number of times the attribute has been shown. The opposite influence has been observed. Respondents are probably more attentive to the attributes shown less often and react more strongly. When some of the attributes are shown permanently, which is the common approach, the weight of the less frequently shown attributes should be readjusted. Such corrections are possible but are very problematic.

The use of partial profiles adds to the scarcity of the resulting design matrix with a detrimental effect on its definiteness (a property to be successfully inverted). Estimation of parameters requires a robust method relying on a high rate of aggregation and specific settings of the estimation parameters.

As aside

Sawtooth Software, Inc. state that split-sample partial profile vs. full profile methodological studies have shown that the parameters derived from either experiment are quite similar, including price studies. The author of this text does not share this view.

Partial profiles may be very useful in screening studies where the importance of various attributes is virtually unknown. The purpose is not only to locate the important attributes but to provide some measure of their influence on the attractiveness of the core product. Design and analysis methods allow for several tens of attributes. However, the individual-level estimation may not be satisfactory. In many practical cases, a classic way of asking questions for a battery of items and their evaluation as the " self explicated conjoint" may be more adequate for the purpose. A solution named Logit-Cake Method for merging such data with metric conjoint data has been suggested. A DCM solution is MBC - Menu Based Conjoint.

Hold-out profiles

Many authors recommend one or more tasks composed of fixed profiles to be included in the conjoint exercise. The so-called hold-out profiles are not used in the estimation of the model parameters but serve for validation of the estimates based on the other (randomized) tasks, for comparisons among respondents, etc. The profiles are usually designed in a managerial way.

The hold-out tasks can provide a validation of what happened in the interview rather than what might happen on the market. They serve for an internal rather than external validation. In our experience, the prolonged duration of a conjoint exercise is not worth the obtained information. Similar information is readily available from tasks with randomized profiles. It is more efficient to move hold-out tasks out of the main block of the conjoint questionnaire and use them as calibration profiles. Calibration is indispensable in studies where a prediction of the competitive potential is desired.

Constant alternative profiles

A permanent profile with a unique fixed meaning called constant alternative is an often used option in CBC - Choice-Based Conjoint.