In-Depth

Searching for Effective CRM: Customer Relation Management and E-Commerce Work Together to Achieve a Common Goal

• 08/01/2000

Although many business processes are being rapidly Web-enabled, the current integration of e-commerce and customer relationship management (CRM) solutions is a logical match with a potentially dramatic impact on the bottom line. CRM is a customer-focused business strategy to optimize revenue and customer loyalty by offering more responsive and customized service. CRM software needs to be integrated with the Web because that’s where the customers are. Shoppers go to the Web for convenience: Service is a critical competitive differentiator.

To succeed, e-commerce sites need the new eCRM tools that analyze buying habits, personalize marketing offers and even "push" desirable items onto computer screens. But, the current focus on these new tools overlooks one e-commerce technology essential to effective CRM: The search engine. To turn Web shoppers into buyers and buyers into loyal customers, Web shoppers need to find what they want.

That hasn’t been easy. A Resource Marketing analysis of 45 sites disclosed that more than three-quarters of these sites enabled users to type in search requests, but less than one-third generated relevant results. Research and consulting firm Creative Good discovered in its study that 56 percent of search attempts failed. Yet Jupiter Communications found that 48 percent of buyers say that the leading factor influencing them to make future purchases at a Web site is the ability to easily search and find products.

These figures bode ill for an environment in which your biggest competitor is only a click away. Conversely, if your site can provide easy, dependable searching, you can get and keep customers. As a result, it’s important to understand what searching entails – what the challenges are, why early-generation search technologies are prone to failure, and what to look for in terms of new search capabilities in order to satisfy customers and turn failed searches into gold.

Semantic Disjunction. At the heart of the search challenge is a customer issue: communicating with and being able to understand your site visitor. For example, when you expose your products and services on your Web site search engine, do you assume that prospects will know the nomenclature and classification schemes that you used? Maybe they’re looking for pants, but you sell only "slacks" and "trousers." Perhaps they want nails, but will they know to look under "fasteners" to find them in your catalog? What if they ask for 16-ounce beakers, but you offer only 16-oz. beakers? Or suppose they want 14-gauge grn. cable and are pleased to find your 14-gauge grn. cable. Unfortunately, they wanted grounded cable (to finish a project safely), but your cable is green.

These examples show a common problem in searching: semantic disjunction. Simply put, there is great diversity in the ways individuals, an organization’s catalog or a trading exchange’s multi-supplier catalog can describe the same product. As these examples show, the possibilities are numerous – from synonyms (pants vs. slacks), to words whose meanings overlap (nails as a kind of fastener), to variations, including abbreviations ("ounces" vs. "oz."), to the same word or abbreviation – grn. – with different meanings in different contexts.

Now, add to these examples the problem of human error. Even when the user and your catalog "speak the same language," the user will make mistakes in entering the search request – misspelling a word, transposing letters or numbers, or simply leaving out key descriptors.

In many of these cases, traditional text-based search engines may fail to deliver any results – even though the catalog may have the desired item. Their tolerance for misspellings and missing words is too limited. To compensate for the noise and unpredictability of the users’ input, text-based search technologies will "cast a wide net" which delivers too many, often irrelevant results. People shop on the Web for convenience. If they have to scroll through a long list until they find what they want, the experience does not encourage customer loyalty. Shoppers will click away when faced with too many choices.

Second generation B2C and B2B sites have adopted parametric and category searching to compensate for the unreliability of keyword and descriptive text searching. In this mode of searching, the user progressively zeros-in on desired merchandise by navigating through your catalog taxonomy or by entering explicit values in discrete fields such as price, style, color, size, etc. Sometimes these search strategies are a great improvement for the user, other times, they are not. Tolerance for misspelling is still a problem. Equally, an issue is the requirement that the user know your terminology and understand your rationale for categorizing products. Any disjunct will result in a failed search.

The problem is not about the mode of search; both parametric and keyword searching are valuable and necessary. The real problem is the requirement to mediate and reconcile the meaning of two incongruent and incomplete messages; your representation of a product as recorded in the catalog and the user’s representation as expressed in their search terms. Historically, this was the role of the sales or call-center service rep, but Web-based commerce has changed all that.

E-commerce has a self-serve business model. Intermediaries, such as customer service representatives, have been eliminated, leaving users to fend for themselves. Unfortunately, this means that technology must fill the gap and translate discrepancies between the meaning of the company’s language and how the prospect, customer or partner interprets it.

When product data must serve multiple user or supplier communities without a mediator, as it does on the Web, there is great risk that some users will apply a context or interpretation to data that was collected and stored under different circumstances or for totally different purposes. This is especially true for much Web product catalog data, which comes from paper catalogs (in which case you can often call a customer service representative) or, even worse, from different suppliers’ databases or inventories in B2B net marketplace catalogs.

Search descriptions employing several words – which often have multiple meanings, depending on the context – can especially lead to a communications gap without a human intermediary. If you go into a brick-and-mortar store, or even a catalog telesales operation, a good clerk will help you navigate the offerings. If stock is depleted, the clerk may even suggest a substitute or call up another branch store.

Recently, I went to the Web site of one of the country’s largest online sellers of men’s dress shirts. I started by looking for "Egyptian cotton," but only got one page of towels. So I tried to be very specific by requesting an "Egyptian cotton buttondown shirt," but got zero matches. Then, I tried for just a "cottan buttondown shirt," misspelling the first word, but again, no matches. So, typing furiously, I retried with the "cotton buttondwon shirt," this time transposing letters in the longest word; well, that too came back with zero matches. Finally, and with deliberation, I typed simply "buttondown shirt." I was rewarded with 50 pages of results! But, I was out of patience, so I clicked to a competitor’s site.

Some of the more sophisticated sites support Boolean logic as a way of allowing the user to indicate the priority or relationship among the search request words. But, like most users, I can’t be bothered to remember the cryptic Boolean syntax. Furthermore, the language is not completely nor consistently implemented across Web sites, so the technique is of little use to non-programmers. Yet, if Boolean logic isn’t applied during a search, traditional search engines regard all terms as equal. They will simply search their catalog index for all the words in the search string and return results based on the number of "hits" each word receives – often resulting in a deluge of choices. Of course, if the user misspells any search term, that word – however important to the search – is effectively ignored.

There needs to be a better, more automated way of determining which words carry more importance in the search. Search engines should employ advance computer science algorithms to better understand the meaning and relevance of the words that a user enters.

There are both lexical (natural language) and statistical (information content) technologies for determining how much emphasis or importance each individual word should have when evaluating a search request. For example, if you’re looking for yellow, dry-erase markers, markers is a key word. But, dry-erase is important, too – more important than yellow. You don’t want a felt-tip, indelible, or fine-tip yellow marker. Any search that returned all markers, or all yellow markers, or all responses to all three words equally, without being able to select only the best search candidates, would overwhelm a site visitor with choices.

There are more CRM consequences of a failed search than lost sales. If you can’t understand why the search failed, you can’t improve. Conversely, if you know why shoppers can’t find a product you offer, you can change the language of your site, based on their search terms – for example, adding "pants" as a synonym of "trousers." Or, if you know that several searches failed because shoppers wanted a product you don’t carry, you can add it to your inventory. However, if your search engine can’t handle semantic disjunction, it can’t perform the matching necessary to discover what the failures have to tell you.

Being able to log and analyze your users’ search terminology yields more than the ability to reactively alter the search engine behavior or extend your product offerings. It also gives you the proactive ability to customize future client interactions. One-to-one personalization, as well as target marketing to affinity groups takes on new dimensions when you have perspective on what people search for, and the language they use to search. Combined with agent and push-technologies, you are empowered to launch proxy searches that can reach out and notify your customers of items that now satisfy their previously futile inquiries.

Search engine technologies have evolved and can now handle the critical issues cited above. Without getting too technical, here are some of the capabilities needed to address semantic disjunction, noisy and missing data, and filtering results for relevancy. Consider these a checklist for product evaluation when implementing, or upgrading, your Web catalog search strategy.

Context Mediation. This is the determination of the business meaning of a word or value based on the context in which it occurs or on associations of other adjacent data values. This is usually accomplished with compiler technologies, such as parsing and lexical analysis. Context mediation is a prerequisite to the next functions and is essential when dealing with free-form text and multiple word input fields, such as product search descriptions.

Normalization. Next-generation search engines transform terms in the user’s description in realtime to the "standard" terminology adopted by the product catalog and catalog index. For example, if a unit of measure, such as ounces, is included in the index in its normalized form – oz. – when a user searches using the term ounce or ounces or o.z., these are immediately normalized to conform to the index form of oz. Furthermore, next-generation search engines create phonetic and synonym versions of words when building the index, as well as on-the-fly when a search request is entered. These alternate versions provide powerful mechanisms for handling misspellings. Why normalize? Getting the two sides of the search equation to agree in form greatly increases the chances of a successful match – and the likelihood of converting Web shoppers into Web buyers. It’s very significant to note that the same normalization strategies used to condition the user’s search request should also be used when loading and indexing the product catalog.

Fuzzy Retrieval. This is the ability to perform quick and productive retrieval of data even though a precise key, such as a product number, is not available. Keyless retrieval and unstructured queries present a major challenge for e-commerce sites. A user search request must be joined to an optimal set of potential matches from many tens of thousands or millions of possible choices in the database, and in only fractions of a second. Next-generation search software uses sophisticated, often patented, database indexing and search optimization strategies to achieve the right combination of speed and yield. Fuzzy retrieval is designed to operate with noisy and missing data conditions, because even normalizing the input with context mediation still can’t eliminate all non-standard representations of a product.

Fuzzy Matching and Filtering. Once a set of possible matches have been retrieved, it’s important to measure or rank these results to ensure that only the most useful "hits" are returned to the user. Fuzzy matching can apply mathematical information theory and measures of statistical certainty to determine when a set of terms that don’t exactly match the user’s description are a probable match that should be returned to the screen. Records can be further filtered and prioritized by applying your business rules to honor contractual agreements with suppliers and buyer groups, or based on past analysis of the buyer’s demographics and personalized interests. Fuzzy matching and filtering are critical to customer satisfaction. These techniques establish relevancy for the search process – eliminating the frustration and wasted time of presenting your user with pages of irrelevant search results.

Search Analysis and Reporting. Look for tracking and analytic functions to report on search behavior and outcomes. Organizations can use this data to improve their site language, expand their inventory and formulate targeted marketing initiatives.

Search by Proxy. An agent is an automated process that can operate unattended, based on some pre-defined schedule or sequence of events. User-configured agents can continue to search the Web catalog for a shopper’s search request, even after the person has left the site and can notify that person when the item (or a related item) becomes available. Conversely, companies may wish to launch search agents on behalf of users, based on analysis of prior search behavior, analysis of user demographics, or even the acquisition or launch of new product lines.

In other words, new searching technology is critical for effective CRM on the Web. By normalization and fuzzy searching and matching, it bridges the communication gap inherent in a self-service medium. By being efficient and noise-tolerant, it finds the desired product – to form the initial bond with customers and to keep them coming back. It even serves as the basis for advanced CRM initiatives, helping organizations to understand search activity, so they can improve their sites and offerings and drive personalization tools – to produce greater customer satisfaction and more sales.

About the Author: Stephen M. Brown is Vice President of Product Strategy at Vality Technology Inc. (Boston).