Document View

There is gold in open-ended survey responses. As opposed to close-ended questions where a respondent is given a finite selection of answers, open-ended survey questions have no specific answers, and respondents are free to share their thoughts. Direct marketers conducting surveys include open-ended questions to gather prospects' or customers' opinions on offers; acquire names; assure quality control; or, sometimes break the ice. The words respondents choose to answer open-ended questions offer valuable insight that can be acted on to improve direct marketing results. Analyzing open-ended survey responses is a labor-intensive and complex process, even with technology. Human speech is unstructured, and many words have several meanings. Linguistics-based analytic software and new technology - such as Natural Language Processing - is now available that can assist in categorizing text responses and make it easier for direct marketers and researchers to pan for the gold in open-ended surveys.

There is gold in open-ended survey responses. As opposed to close-ended questions where a respondent is given a finite selection of answers (e.g. very satisfied, satisfied), open-ended survey questions have no specific answers, and respondents are free to share their thoughts.

Direct marketers conducting surveys include open-ended questions to gather prospects' or customers' opinions on offers; acquire names; assure quality control; or, sometimes break the ice.

The words respondents choose to answer open-ended questions offer valuable insight that can be acted on to improve direct marketing results.

A Wealth of Information

Fairytale Brownies, a mail-order company based in Chandler, Ariz., understands the value of seeking customer opinion through surveys. The company inserts in each package of its brownies a survey card that asks an open-ended question such as, "How did the brownies taste?"

The company initially set up the survey for quality control purposes, but soon learned the responses, particularly to the open-ended question, provided it with useful information such as new product ideas. The survey also serves as a vehicle for building customer relationships. Respondents are asked if they would like a representative from Fairytale Brownies to call them, and if they say "yes," someone does.

Although Fairytale Brownies learned the value of open-ended survey responses, many surveyors choose to ignore open-ended questions because, unlike multiple-choice questions, open-ended answers are difficult to analyze.

Deriving Meaning

The standard method of analyzing open-ended survey responses is to assign codes (usually numbers) to the different responses. The person analyzing the open-ended responses develops a set of response categories that adequately represent the answers given. Then the number of answers that fall into each category is determined. For example, if survey participants were asked, "What features of a product most satisfy you?" their answers probably can be represented on a continuum of response options. Coding typically is done after the survey is completed, because the list of possible answers can only be generated after the survey has been conducted. There are, however, cases when the coding can start before the survey ends. This can be done manually or by using one of a number of software programs that can assist in accelerating the coding process. Coding is critical, because it enables the responses to be statistically analyzed.

Even with technology, analyzing open-ended survey responses still is a labor-intensive and complex process. Human speech, by its nature, is unstructured. Many words have several meanings, and there are countless ways to express a single idea. It is difficult to manually detect the semantic closeness of every possible pair of sentences as well as take into account possible spelling mistakes.

Interpretive Technology

There are new technologies available that assist in categorizing text responses. New solutions combine manual techniques with advanced linguistic technologies designed to extract and classify key concepts from within survey text. Natural Language Processing (NLP) technologies analyze text as a set of phrases and sentences in which the grammatical structure provides a context for the meaning of the text response.

This linguistics approach equates terms that are used in similar contexts. For example, NLP would equate terms like "executive," "manager," and even "mgr," if they are used in similar contexts, whereas a non-linguistics-based solution may not equate manager with executive if it weren't specifically stated. In addition, NLP technologies can interpret the tone of text. A customer comment about "new phone" may be distinct from a related concept, "new phone ASAP" response. The understanding of context cuts through the ambiguity of text, making linguistics-based text analysis the most accurate approach.

Technologies, like linguistics-based analytic software, are making it easier for direct marketers and researchers to pan for the gold in open-ended survey responses. The ability to personalize offers and better understand customers is a key factor in successful direct marketing. And, the best way to find out what is in customers' heads, is to ask them.