> SCE - Sequential Choice Exercise > OBIMA - Object Image Analysis

OBIMA - Object Image Analysis

---

Making money and making a difference are mutually supportive goals.

[ Richard Barrett ]

Problem

Image evaluation of a collection of objects such as brands (brand image), institutions (employee or customer satisfaction), society (life style) a. o., is an important part of market or public opinion research. Understanding strong and weak points of the objects and differences between them is essential for marketing and/or public communication strategies. 

The standard questionnaire is usually built as a sequence of batteries where respondents express their agreement or disagreement with the presented statements, usually on a scale. Results often suffer from low resolution. There are more reasons for that. Even aspects a respondent knows nothing about have to be answered, and many get a more or less random value. With the time spent in useless repetitive activities respondent's annoyance rises and relevant answers are thinning. When an object of respondent's concern finally appears the respondent's  attention have been distracted.

Batteries of scale-based questions concerning aspects can be replaced with MXD - Maximum Difference Scaling method. MXD has significantly higher discriminatory power but the interview is substantially longer. Closer inspection in respect to object image estimation will reveal yet other problems.

• Every aspect should appear about 3-5 times in MXD choice tasks. Should the respondent answer tasks for each of the objects the questioning might last prohibitively long. Objects unknown to the respondent can be excluded, but at the cost of potential information loss. Experienced panelists are well aware of possibility to express some ignorance and make the questionnaire shorter and less strenuous.
• Standard MXD does not provide a threshold value distinguishing important from unimportant aspects. The value can be obtained by "anchoring" the choices using supplementary questions, e.g. on the importance of the chosen aspects. This would inevitably lead to further prolongation of the interview.  
• Each MXD task conducted separately for each object can have a different sensitivity (scaling factor). This might impair comparisons between aspects of different objects.

McCullough (2013) proposed to eliminate the above problems by use of ranking of the tested objects agreement with each of the aspects. The well known Q-Sort method was used for the ranking. Number of MXD choice tasks could be reduced because the loss of information was compensated with information from Q-Sort tasks. The author claims the approach also minimized halo effect of well known objects, namely brands. 

Reference

McCullough, P. R., 2013. Brand Imagery Measurement: Assessment of Current Practice and a New Approach. Proceedings of the Sawtooth Software Conference, October 2013, p. 147.

The principle of the cited solution lies in linking preferences in the dimension of aspects with preferences in the dimension of objects. The proposed Q-Sort method has a weakness in that a formation of adequate choice tasks for discrete choice modeling is inherently arbitrary. Its usage is possible but the outcome will be dependent on the experience of the questionnaire designer and analyst.

Solution

Object image perceptions have two dimensions, objects and aspects. The problem of N objects and M aspects corresponds to elements of the following matrix [b_ij]:

Estimates of b_ij as preferences projected on a common scale can be assessed by linking preferences obtained in the two dimensions. In one group of experiments, preferences among M aspects can be obtained independently for each object of the N objects. In the other group, preferences among N objects can be obtained independently for each aspects of the M aspects. 

As aside

The above matrix can be transposed if it is required by the questionnaire format. This is typical for brands being objects and brand properties being aspects.

The ideal solution would be to use anchored MXD-s in both dimensions. For a medium problem with 8 objects (e.g.. brands) and 16 aspects (characteristics) that would require 8 MXD exercises with 16 aspects and 16 MXD exercises with 8 brands. Such a questionnaire would take unacceptably long time. However, a questionnaire is desirable to be as short as possible, simple to implement, and motivating respondent to reveal object aspects that are important, either appealing and satisfying, or conversely, discouraging or worrisome. Respondents must have chance to skip the items that are irrelevant to them, are uninterested in or know nothing about.

On the page SCE - Sequential Choice Exercise is shown that results from SCE are practically equivalent to those from MXD. For the number of items that can be overlooked on one screen, SCE is much faster than MXD. Furthermore, an importance threshold (anchor) can be represented by one of the items in the SCE exercise and asking additional questions concerning it can be avoided. When the threshold is reached, respondent may terminate the task. This can further reduce the time of questioning. Numerical method for estimating parameters SCE is theoretically justified and, unlike Q-Sort method, unambiguous. In OBIMA method, SCE has been chosen as an appropriate procedure for querying in the both item and aspect dimensions.

Properties

The most distinctive property of OBIMA is its ability to capture small differences between objects. The data format and properties allow for seamless segmentation of respondents by latent class, ensemble cluster analysis or other methods.

For Czech readers

Krátkou prezentaci v češtině lze stáhnout na odkaze OBIMA_Object-Image-Analysis.pptx

Strengths

• Discrimination power of OBIMA far surpasses the standard battery-based image analysis.
• Repeated evaluations are completely eliminated.
• Number of unengaged respondents is substantially decreased.
• Simplification strategies are greatly avoided.
• The design of OBIMA questionnaire is generic and reusable. It is composed of two questions (the best and worst choices) in loops for objects and aspects to generate sequences.
• Determination of perception threshold is inherent to SCE questions. No additional questions or synthetic choice sets are required. 
• Parameter estimates are Bayesian, i.e. reliable and stable.
  
• Compared to conventional scaling batteries, a better discrimination between objects, aspects and individuals is achieved.
• Better discrimination between individuals allows for sharper segmentation.
• The indifference threshold value for a respondent is determined. The value allow for estimation of perception intensity for each element of the parameter matrix.    
  
• The perception intensity expresses acceptance or rejection of each object aspect in percent. A value of zero corresponds to the indifference threshold value.
• Positive values identify the strengths and negative values the weaknesses of objects.
• Normalization of perception intensities is believed to allow comparisons, at least for similar objects, among studies.
• Determination of acceptability thresholds allows for non-compensatory decision modeling.

Weaknesses

• Compared to batteries of scale-based questions the questionnaire is more demanding to respondent.
• Interviewing time is longer.

Limitations

• Both dimensions of the estimated matrix elements are limited to about 15, in the extreme case to 20 items. This arises from use of the SCE sorting method that requires to show all items from a given dimension at the start of choice sequence. At present it is not known if a higher number would be feasible. 
• The total estimated number of elements (dimensions product) should not exceed about 400. This corresponds to e.g.. 20 objects and 20 aspects. Although the respondent has the option to skip any objects or aspects, a greater number would probably reduce the willingness of respondents to answer.

Live demo

You may experience the look and feel of OBIMA among other live questionnaires.

Examples

Example 1: Consumer electronics brands

Questionnaire

A test questionnaire was designed as image evaluation of 4 brands (Apple, Lenovo, Samsung and Sony) based on 12 aspects. Image of these brands is well known. Results from the survey were believed to confirm or refute usefulness of the method.

Colleagues from g82, s.r.o., and about 100 office and business executives from variety of firms were invited to participate. The questionnaire was opened by 57 and completed by 47 of the invited. Enough data for processing were obtained from 52 interviews. The average total interviewing time was about 14 minutes with mode at 11 minutes.

Estimation

Part-worths of aspects were estimated using program CBC/HB 5.5.3 (Sawtoothsoftware, Inc.). The used behavioral model is very robust as can be seen from the picture of estimation process below. Stabilization was achieved already at 1000 iterations. The quality of the estimates is fully comparable with other studies involving nominal attributes. 

Results

There are several ways to present the results. The figure below shows average percentages of "influences" that sum to 100%, as is common in MXD - Maximum Difference Scaling method. The black bar depicts the threshold (anchor) that separates positive from negative perceptions.

As aside

• The figure above is as if from an MXD study for 48 statements (combined 12 aspects for 4 brands). A respondent would have to make best and worst choices from about 48 choice sets of 5 statements. Completing such an exercise would hardly take less than 14 minutes which relates to 9 seconds per best-worst double-choice (including time for generation of pages and sending data from and to the server). However, as such exercise would be very annoying it might lead to complete respondent's resignation. If there were request for determination of perception threshold yet additional about 20 anchoring questions would be needed.
• Using labeled MXD exercises would not help either. About 15 choice sets, each with 4 aspects, would have to be created for each brand. To put the preferences on a common scale, a block of question about preferences of aspects among brands (e.g. Q-Sort) would have to be added. 
• Anchoring questions are sometimes scale-based. A transformation of the scale to some probability value is an unreliable task. Therefore a simple yes/no question that follows every MXD question is used more often.
• Another disadvantage of MXD is that skipping a choice set when it is eventually shown is very problematic.

It is not very easy to interpret results in the form of the figure above, especially in the region of negative perceptions. The problem is the calculated "influences" themselves. They are always positive and the differences between items with small influences are necessarily also small, although their ratio can be as large as for the items with high positive influences. If the threshold value between positive and negative perceptions is known, one can create a percentage scale roughly equivalent to the commonly used top-box percentages with positive and negative values, as in the figure below.

One can figure out that 74% of respondents considered Apple products as [definitely] innovative and technologically advanced, and, at the same time, 66% as [definitely] expensive and unaffordable.

The results are consistent with the well-known character of the assessed brands as perceived in the Czech Republic. Discriminatory ability between brands and their aspects is also good. Typical are the contradictory perceptions of Apple products affirming their exclusivity. The other three brands are seen generally as positive. Samsung dominates, Sony is characterized by quality that must be payed for, and Lenovo offers value at reasonable price.

Perception threshold is determined independently for each respondent. Calculated percentage values of perception will well (though probably not absolutely) correspond to the actual perceptions in the sample. Segmentation may therefore have good predictive value, also thanks to the fact that the underlying values of perceptions are not squeezed between the lower and upper limits used in scale-based questions (utility values have no restrictions).

The OBIMA design and procedure is based on frequency counts and permits a number of other findings, very similar to those available from website service statistics. E.g. the most challenging statement was "reasonably priced and affordable" that most respondents clicked on. In contrast, respondents were least interested in "high quality support and services", probably because only a few had some experience with it.