Mathematics, Statistics and Sales Chat
A Web Retailer Case Study. A White paper by Ravi Vijayaraghavan, Vice-President and Head - Global Analytics.
Content
24
Mathematics, Statistics and Sales Chat
A Web Retailer Case Study
White paper by: Ravi Vijayaraghavan Vice-President and Head - Global Analytics
February 2010
WHITE PAPER
SEVEN
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Table of Contents
Introduction The Conversion Funnel Statistical Scoring Model Agent Optimization Chat Transcripts Analysis - Text Mining Conclusion 3 4 5 6 7 8
© Copyrights 24/7 Customer, Inc. 2010
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Introduction
With the coming of age of web as a mainstream sales and marketing channel, companies have invested substantial resources in enhancing their web presence. This includes large investments in web advertising. In addition, companies are looking for ways to improve sales conversion and customer experience for web shoppers. Sales chat is a medium that can provide a lift in both these areas. With the growing popularity of chat as a communication medium, particularly among the new generation of consumers, potential for revenue generation from this channel is enormous. The obvious analogy is to consider the sales chat agents as a “virtual sales force” for a” virtual store”. However, a key difference exists. In a real store there are relatively few visitors and a significant fraction of the visitors come with intent to buy i.e. they are “hot” prospects. On the other hand, major web stores such as Amazon, eBay and Overstock have millions of visitors every week and an overwhelming majority of these visitors do not intend to buy. They also have the ability to switch from one store to another at the click of a mouse button. Considering the visitor volumes and the low average likelihood to buy, it is not profitable to randomly engage in chat with every visitor. It is imperative to identify a subset of this visitor population that has a substantially greater likelihood to buy. The following case study presents applications of statistical/mathematical models in identifying “hot” prospects and improving conversion, revenue generation and customer experience for a major web retailer.
© Copyrights 24/7 Customer, Inc. 2010
Page 03 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
The Conversion Funnel
The starting point in understanding and optimizing the performance of this virtual sales force is the Conversion Funnel (Figure 1). The funnel helps visualize the size of the opportunity. Figure 1 represents the funnel for the web retailer in a particular week. Layers 1 & 2 are filters, i.e. these are determined and controlled by the retailers, while layers 3 to 6 are leakages that are essentially decisions made by the customer during the browsing/buying process and are not in the retailer’s control. The science is essentially in determining the appropriate filters to apply in selecting the right customers and matching them up with the right agents to minimize the leakages in the funnel.
Total Number of Visitors 5,154,257
Layer 1 - Filter
• Value of the product • Involvement required from buyer of the product • Product contribution • Cross-sell opportunity • Customer’s web behavior
‘Hot’ Leads 519,080 (10%)
Layer 2 - Filter
• Customer’s web behavior • Resource allocation
‘Hot’ Leads Invited to Chat 390,913 (75%)
Layer 3 - Leakage
Invitations Accepted 30,617 (8%)
Layer 4 - Leakage
• Customer’s comfort with chat channel • Customer’s comfort with sales interaction during shopping • Past experience
No. of Chats Started 23,085 (75%)
Layer 5 - Leakage
No. of Interactive Chats 11,977 (52%)
Layer 6 - Leakage
• Agent competency
Sale!!! 3,566 (30%)
• Pricing • Perceived value • Quality of transaction
Figure - 1: Sales Chat Conversion Funnel – Visitor to Prospect to Customer
© Copyrights 24/7 Customer, Inc. 2010
Page 04 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Statistical Scoring Model – Filter (Layer 1)
The first filter identifies the “hot leads” i.e. the people most likely to purchase via chat. In particular, this identifies customers who have a significantly higher likelihood of purchasing from a chat agent than on their own in a self-service mode. This is an important factor since self-service is obviously a lower cost channel than chat and if a customer is very likely to purchase via self-service then the business case for inviting them to a chat engagement is poor. To avoid cannibalization of a cheaper channel A/B tests are conducted on a regular basis where a fraction of the “hot leads” are not invited to chat and their self-service conversion rates are compared to conversion rates of the remaining “hot leads” who are invited to chat. Typically, conversion rate for chat engagements among this “hot lead” population is substantially higher (5x-10x) than that for self-service engagements. Identification of hot leads is accomplished using a statistical scoring model. The scoring can be done in real time while the visitor/prospective customer is browsing on the website. The scoring is based on a number of attributes including time of the day, day of the week, geographical location of the customer, product category, exhibited behavior on the web site etc. Figure 2 schematically illustrates the scoring model. Essentially certain patterns of behavior exhibit a much greater propensity to buy than others. The scoring model essentially estimates a probability of purchase (P(sale)). Statistical and Data Mining techniques such as Naïve Bayes, Logistic Regression or Neural Networks are used to develop these scores. A threshold score can be set above which customers are invited to a chat. Based on variations in traffic and availability of agents the threshold score can be modified. As more data is generated, the system learns and the scoring model becomes better at identifying the hottest prospects.
Referral Page Search Engine Retailer’s Website Comparison Shopping Site Web Behavior In a Product Page Abandoned Order Process >t sec on Billing and Shipping Page Hour of the Day Day of the Week Product Category Postal Code Connection Type
Hot Lead
0 1 Monday Tuesday Wednesday 2
>t sec on Order Review Page Thursday Recreation & Sports Gifts & Flowers Health & Wellness Home & Garden
Electronics & Computers
10... 11... . . . . 48… 49… . . . 94… 95…
Cable/DSL Corporate
P (Sale) = 0.78
Cold Lead
Jewelry & Watches
Dialup
P(Sale) = 0.03
. . . . . 17 18 19 20 21 22 23
Friday Saturday Sunday
Figure - 2: Schematic of the Scoring Model
© Copyrights 24/7 Customer, Inc. 2010
Page 05 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Agent Optimization - Filter (Layer 2)
Once the customers are scored and the “hot leads” identified, the next step is to invite these “hot leads” to a chat. The number of “hot leads” invited is based on tactical and strategic considerations. On the tactical front, it depends on several factors such as the number of agents available, acceptable abandonment rate without significantly affecting customer experience, average handle time and concurrency (how many chats can an agent handle at a time). This is a routine scheduling problem. The more interesting strategic problem is to determine the right number of agents to maximize profits. The scoring model only prioritizes the visitors to the site. It does not automatically provide a threshold score for the “hot leads”. We provide the threshold score. This in turn determines the number of people invited to chat which establishes our agent staffing levels. But what is the right threshold score? How is it determined? The customers are being prioritized based on how “hot” they are. Based on the average order value for a given product type and the probability of a given “hot lead” to buy, the expected revenue from the transaction can be calculated. To increase the number of chats and hence the overall revenues, we lower the threshold score inviting less qualified leads, at the same time increasing the number of agents. These less qualified customers on an average generate lower revenues per customer i.e. the marginal revenue of these customers is lower. This implies, as we keep adding agents to interact with less and less qualified leads, the marginal profit generated by the additional agents keeps declining. We keep lowering the threshold, increasing the number of “hot leads” and adding agents till we stop making a marginal profit. To estimate the number of agents corresponding to this, we use an optimization algorithm Finally, scoring techniques are also used to match the right agent to the customer. The essential concept is displayed in Figure 3 where we see that the agent Raymond is as such a top performer but is particularly skilled in selling Electronics products. We score various product-agent combinations and manage our chat queues and routing based on not just the overall performance of the agent but also on historical performance in various product categories. The goal here is not just to look at product-agent combinations but to develop a comprehensive scoring model that scores the agent for a set of customer/product attributes and determines the best agent to talk to a given “hot lead”
2 5 .0 % 2 0 .0 % Conversion Rate 1 5 .0 % All Agents 1 0 .0 % 5 .0 % 0 .0 % All Categories Electronics Raymond 30% increase 53% increase
Figure - 3: Matching Customers to the Agent with the Right Skills
© Copyrights 24/7 Customer, Inc. 2010
Page 06 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Chat Transcript Analysis – Text Mining
Once the transaction is completed, the chat transcripts are analyzed for patterns of behavior that does or does not lead to a successful conversion and a quality customer experience. Two kinds of text mining techniques are used - Clustering and Classification. The Classification technique is usually adopted, when the domain of interest (Sales Chat, in this case) is familiar to analyst while clustering is more exploratory. For this case study a classification model was used. Here the analyst defines the categories and the taxonomy for classification (Figure 4) based on a training data set (chat transcripts). Building the text mining model involves training the model to classify the chats correctly into appropriate categories. Once the model is built it is tested on a new set of chat transcripts. Following are some specific categories that are identified with text mining on sales chat. 1. 2. 3. 4. 5. Customer emotions and expectations Agent behavior Effort involved in selling various products Reasons for chats not resulting in sales Opportunities for up selling/cross-selling
Example: A sample of sales chats was analyzed to find the kind of effort required to make a sale. The analysis revealed that it required greater effort to convince people when they could not find satisfying answers for their query. Relating this, with the kind of product in question, it was observed that people took time to make a decision when they were shopping for items in the ‘Home and Garden’ category. It was also observed that people tended to ask for details that could be found only upon close examination of the product. (Some typical questions are, what is the texture of the cloth? what is the feel of the material, How does the back side of the carpet look? What is the material used for the knob?). On the other hand, being well specified, certain electronics items were sold with relatively less effort. It is also important to note that the items that required more details and hence more interaction are ideally sold using the chat medium as opposed to self-service. Consistent with this, it is observed that ‘Home & Garden’ category had the largest volume and the highest expected revenues via chat among all product categories. Text analysis helps optimize these interactions and also help prepare agents with the right information to help customers. Similarly, text mining can be used to identify products for which promotional schemes (club membership, special protection plan, extended warranty plan) were easier to up/cross sell. For example, zodiac pendants often sold with children jewelry.
Figure - 4: Categorization of Customer Emotions and Agent Behaviour to Build the Text Analysis Model
© Copyrights 24/7 Customer, Inc. 2010
Page 07 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Conclusion
Internet chat is a growing channel for sales over the web and retailers are adding this capability to their websites. However, like in self-service web retailing, success in driving up sales chat revenues and profitability will go to players who use advanced data-driven approaches to drive customer intelligence and chat engagement decisions.
References
ITSMA and PAC, How Customers Choose Study, 2009 ITSMA and PAC, How Customers Choose Study, 2009 ITSMA and PAC, How Customers Choose Study, 2009
Confidential Information
Information contained in this document is confidential and proprietary to 24/7 Customer Pvt. Ltd and should not be disclosed to anyone, other than the recipients of this document. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without express written permission from 24/7 Customer Pvt. Ltd 24/7 Customer logo is a trademark of 24/7 Customer, Inc. headquartered at 720 University Avenue, Suite 100, Los Gatos CA 95032
© Copyrights 24/7 Customer, Inc. 2010
Page 08 of 09
Sales Offices:
US
Suite 240, 910 E. Hamilton Avenue, Campbell, CA 95008-0610, USA
UK
Regal House, 70 London Road, Twickenham, Middlesex, TW1 3QS, UK
About 24/7 Customer 24/7 Customer is a predictive interactions solutions provider that guarantees measurable business results across the customer lifecycle. With its patent pending “predictive interactions” SaaS platform coupled with “24/7 Outperformance” framework, 24/7 Customer promises to improve sales by 25% or more, improve customer experience by 10% or more and reduce contact center costs by 20% or more for its clients. Today, 24/7 Customer is the no. 1 partner in contact center operations for 90% of its clients.
(For more information visit: http://www.247customer.com)
© Copyrights 24/7 Customer, Inc. 2010
Delivery Centers:
Guatemala, Nicaragua, China, Philippines, and India
: +1 650 385 0564 : [email protected]
Mathematics, Statistics and Sales Chat
A Web Retailer Case Study
White paper by: Ravi Vijayaraghavan Vice-President and Head - Global Analytics
February 2010
WHITE PAPER
SEVEN
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Table of Contents
Introduction The Conversion Funnel Statistical Scoring Model Agent Optimization Chat Transcripts Analysis - Text Mining Conclusion 3 4 5 6 7 8
© Copyrights 24/7 Customer, Inc. 2010
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Introduction
With the coming of age of web as a mainstream sales and marketing channel, companies have invested substantial resources in enhancing their web presence. This includes large investments in web advertising. In addition, companies are looking for ways to improve sales conversion and customer experience for web shoppers. Sales chat is a medium that can provide a lift in both these areas. With the growing popularity of chat as a communication medium, particularly among the new generation of consumers, potential for revenue generation from this channel is enormous. The obvious analogy is to consider the sales chat agents as a “virtual sales force” for a” virtual store”. However, a key difference exists. In a real store there are relatively few visitors and a significant fraction of the visitors come with intent to buy i.e. they are “hot” prospects. On the other hand, major web stores such as Amazon, eBay and Overstock have millions of visitors every week and an overwhelming majority of these visitors do not intend to buy. They also have the ability to switch from one store to another at the click of a mouse button. Considering the visitor volumes and the low average likelihood to buy, it is not profitable to randomly engage in chat with every visitor. It is imperative to identify a subset of this visitor population that has a substantially greater likelihood to buy. The following case study presents applications of statistical/mathematical models in identifying “hot” prospects and improving conversion, revenue generation and customer experience for a major web retailer.
© Copyrights 24/7 Customer, Inc. 2010
Page 03 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
The Conversion Funnel
The starting point in understanding and optimizing the performance of this virtual sales force is the Conversion Funnel (Figure 1). The funnel helps visualize the size of the opportunity. Figure 1 represents the funnel for the web retailer in a particular week. Layers 1 & 2 are filters, i.e. these are determined and controlled by the retailers, while layers 3 to 6 are leakages that are essentially decisions made by the customer during the browsing/buying process and are not in the retailer’s control. The science is essentially in determining the appropriate filters to apply in selecting the right customers and matching them up with the right agents to minimize the leakages in the funnel.
Total Number of Visitors 5,154,257
Layer 1 - Filter
• Value of the product • Involvement required from buyer of the product • Product contribution • Cross-sell opportunity • Customer’s web behavior
‘Hot’ Leads 519,080 (10%)
Layer 2 - Filter
• Customer’s web behavior • Resource allocation
‘Hot’ Leads Invited to Chat 390,913 (75%)
Layer 3 - Leakage
Invitations Accepted 30,617 (8%)
Layer 4 - Leakage
• Customer’s comfort with chat channel • Customer’s comfort with sales interaction during shopping • Past experience
No. of Chats Started 23,085 (75%)
Layer 5 - Leakage
No. of Interactive Chats 11,977 (52%)
Layer 6 - Leakage
• Agent competency
Sale!!! 3,566 (30%)
• Pricing • Perceived value • Quality of transaction
Figure - 1: Sales Chat Conversion Funnel – Visitor to Prospect to Customer
© Copyrights 24/7 Customer, Inc. 2010
Page 04 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Statistical Scoring Model – Filter (Layer 1)
The first filter identifies the “hot leads” i.e. the people most likely to purchase via chat. In particular, this identifies customers who have a significantly higher likelihood of purchasing from a chat agent than on their own in a self-service mode. This is an important factor since self-service is obviously a lower cost channel than chat and if a customer is very likely to purchase via self-service then the business case for inviting them to a chat engagement is poor. To avoid cannibalization of a cheaper channel A/B tests are conducted on a regular basis where a fraction of the “hot leads” are not invited to chat and their self-service conversion rates are compared to conversion rates of the remaining “hot leads” who are invited to chat. Typically, conversion rate for chat engagements among this “hot lead” population is substantially higher (5x-10x) than that for self-service engagements. Identification of hot leads is accomplished using a statistical scoring model. The scoring can be done in real time while the visitor/prospective customer is browsing on the website. The scoring is based on a number of attributes including time of the day, day of the week, geographical location of the customer, product category, exhibited behavior on the web site etc. Figure 2 schematically illustrates the scoring model. Essentially certain patterns of behavior exhibit a much greater propensity to buy than others. The scoring model essentially estimates a probability of purchase (P(sale)). Statistical and Data Mining techniques such as Naïve Bayes, Logistic Regression or Neural Networks are used to develop these scores. A threshold score can be set above which customers are invited to a chat. Based on variations in traffic and availability of agents the threshold score can be modified. As more data is generated, the system learns and the scoring model becomes better at identifying the hottest prospects.
Referral Page Search Engine Retailer’s Website Comparison Shopping Site Web Behavior In a Product Page Abandoned Order Process >t sec on Billing and Shipping Page Hour of the Day Day of the Week Product Category Postal Code Connection Type
Hot Lead
0 1 Monday Tuesday Wednesday 2
>t sec on Order Review Page Thursday Recreation & Sports Gifts & Flowers Health & Wellness Home & Garden
Electronics & Computers
10... 11... . . . . 48… 49… . . . 94… 95…
Cable/DSL Corporate
P (Sale) = 0.78
Cold Lead
Jewelry & Watches
Dialup
P(Sale) = 0.03
. . . . . 17 18 19 20 21 22 23
Friday Saturday Sunday
Figure - 2: Schematic of the Scoring Model
© Copyrights 24/7 Customer, Inc. 2010
Page 05 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Agent Optimization - Filter (Layer 2)
Once the customers are scored and the “hot leads” identified, the next step is to invite these “hot leads” to a chat. The number of “hot leads” invited is based on tactical and strategic considerations. On the tactical front, it depends on several factors such as the number of agents available, acceptable abandonment rate without significantly affecting customer experience, average handle time and concurrency (how many chats can an agent handle at a time). This is a routine scheduling problem. The more interesting strategic problem is to determine the right number of agents to maximize profits. The scoring model only prioritizes the visitors to the site. It does not automatically provide a threshold score for the “hot leads”. We provide the threshold score. This in turn determines the number of people invited to chat which establishes our agent staffing levels. But what is the right threshold score? How is it determined? The customers are being prioritized based on how “hot” they are. Based on the average order value for a given product type and the probability of a given “hot lead” to buy, the expected revenue from the transaction can be calculated. To increase the number of chats and hence the overall revenues, we lower the threshold score inviting less qualified leads, at the same time increasing the number of agents. These less qualified customers on an average generate lower revenues per customer i.e. the marginal revenue of these customers is lower. This implies, as we keep adding agents to interact with less and less qualified leads, the marginal profit generated by the additional agents keeps declining. We keep lowering the threshold, increasing the number of “hot leads” and adding agents till we stop making a marginal profit. To estimate the number of agents corresponding to this, we use an optimization algorithm Finally, scoring techniques are also used to match the right agent to the customer. The essential concept is displayed in Figure 3 where we see that the agent Raymond is as such a top performer but is particularly skilled in selling Electronics products. We score various product-agent combinations and manage our chat queues and routing based on not just the overall performance of the agent but also on historical performance in various product categories. The goal here is not just to look at product-agent combinations but to develop a comprehensive scoring model that scores the agent for a set of customer/product attributes and determines the best agent to talk to a given “hot lead”
2 5 .0 % 2 0 .0 % Conversion Rate 1 5 .0 % All Agents 1 0 .0 % 5 .0 % 0 .0 % All Categories Electronics Raymond 30% increase 53% increase
Figure - 3: Matching Customers to the Agent with the Right Skills
© Copyrights 24/7 Customer, Inc. 2010
Page 06 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Chat Transcript Analysis – Text Mining
Once the transaction is completed, the chat transcripts are analyzed for patterns of behavior that does or does not lead to a successful conversion and a quality customer experience. Two kinds of text mining techniques are used - Clustering and Classification. The Classification technique is usually adopted, when the domain of interest (Sales Chat, in this case) is familiar to analyst while clustering is more exploratory. For this case study a classification model was used. Here the analyst defines the categories and the taxonomy for classification (Figure 4) based on a training data set (chat transcripts). Building the text mining model involves training the model to classify the chats correctly into appropriate categories. Once the model is built it is tested on a new set of chat transcripts. Following are some specific categories that are identified with text mining on sales chat. 1. 2. 3. 4. 5. Customer emotions and expectations Agent behavior Effort involved in selling various products Reasons for chats not resulting in sales Opportunities for up selling/cross-selling
Example: A sample of sales chats was analyzed to find the kind of effort required to make a sale. The analysis revealed that it required greater effort to convince people when they could not find satisfying answers for their query. Relating this, with the kind of product in question, it was observed that people took time to make a decision when they were shopping for items in the ‘Home and Garden’ category. It was also observed that people tended to ask for details that could be found only upon close examination of the product. (Some typical questions are, what is the texture of the cloth? what is the feel of the material, How does the back side of the carpet look? What is the material used for the knob?). On the other hand, being well specified, certain electronics items were sold with relatively less effort. It is also important to note that the items that required more details and hence more interaction are ideally sold using the chat medium as opposed to self-service. Consistent with this, it is observed that ‘Home & Garden’ category had the largest volume and the highest expected revenues via chat among all product categories. Text analysis helps optimize these interactions and also help prepare agents with the right information to help customers. Similarly, text mining can be used to identify products for which promotional schemes (club membership, special protection plan, extended warranty plan) were easier to up/cross sell. For example, zodiac pendants often sold with children jewelry.
Figure - 4: Categorization of Customer Emotions and Agent Behaviour to Build the Text Analysis Model
© Copyrights 24/7 Customer, Inc. 2010
Page 07 of 09
24
WHITE PAPER
SEVEN
Mathematics, Statistics and Sales Chat - A Web Retailer Case Study
Conclusion
Internet chat is a growing channel for sales over the web and retailers are adding this capability to their websites. However, like in self-service web retailing, success in driving up sales chat revenues and profitability will go to players who use advanced data-driven approaches to drive customer intelligence and chat engagement decisions.
References
ITSMA and PAC, How Customers Choose Study, 2009 ITSMA and PAC, How Customers Choose Study, 2009 ITSMA and PAC, How Customers Choose Study, 2009
Confidential Information
Information contained in this document is confidential and proprietary to 24/7 Customer Pvt. Ltd and should not be disclosed to anyone, other than the recipients of this document. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without express written permission from 24/7 Customer Pvt. Ltd 24/7 Customer logo is a trademark of 24/7 Customer, Inc. headquartered at 720 University Avenue, Suite 100, Los Gatos CA 95032
© Copyrights 24/7 Customer, Inc. 2010
Page 08 of 09
Sales Offices:
US
Suite 240, 910 E. Hamilton Avenue, Campbell, CA 95008-0610, USA
UK
Regal House, 70 London Road, Twickenham, Middlesex, TW1 3QS, UK
About 24/7 Customer 24/7 Customer is a predictive interactions solutions provider that guarantees measurable business results across the customer lifecycle. With its patent pending “predictive interactions” SaaS platform coupled with “24/7 Outperformance” framework, 24/7 Customer promises to improve sales by 25% or more, improve customer experience by 10% or more and reduce contact center costs by 20% or more for its clients. Today, 24/7 Customer is the no. 1 partner in contact center operations for 90% of its clients.
(For more information visit: http://www.247customer.com)
© Copyrights 24/7 Customer, Inc. 2010
Delivery Centers:
Guatemala, Nicaragua, China, Philippines, and India
: +1 650 385 0564 : [email protected]
