As you may know, Win-Loss Analysis is a market research technique companies can use to discover key learnings from customers and prospects. These learnings can then drive improvements in marketing and sales, resulting in more revenue and profits. Most contemporary product management methodologies (Quartz Open Framework, Pragmatic Marketing, 280 Group, and Blackblot) all recommend that companies institutionalize Win-Loss Analysis. A recent survey found that out of 1,700 product managers surveyed, only 25% reported doing Win-Loss Analysis. Since that’s the case, a review of the basic Win-Loss Analysis process is on order. We’ll also review some key lessons learned from years of experience doing Win-Loss Analysis.
Win-Loss Analysis Process
Win-Loss Analysis projects have of five basic steps.
1. Planning
The first step of the process is to do all of the preparatory work required to ensure the success of the project. An important first step is setting research objectives. Setting clearly defined Research Objectives will help you to both target your Win-Loss research as well as set expectations for the success of the program. As with all things Win-Loss, setting research objectives is best done as a team project. And be sure to your Research Objectives with the strategic goals of your organization. There is no sense chasing information from buyers that your organization has no interest in anyways. Research objectives may include:
- Your Product-Market-Price fit
- New market problems that your organization can solve
- Your service levels
- Persona refinement
- Buyer purchase-decision process
- Marketing channel effectiveness
- Sales process
- Communication style
- Better understanding of your place in the competitive mix
In addition to establishing your Research Objectives, you should also define the questions that will be asked during the interviews. The questions should be open ended and designed to encourage a free flowing discussion.
Finally, you will need to design the process you will use to recruit potential interviewees, conduct the interviews, analyze and report the results you have learned. The plan should layout the roles and responsibilities of each participant in the project. You should also develop a schedule and a simple status reporting mechanism so everyone can stay up to date. Create a project charter that contains the research objectives, interview questions, roles, responsibilities, and schedule. Conduct a review meeting with the participants to approve and commit to the plan.
2. Recruitment
The next step in the process is to recruit people for the interviews. Most companies target a mix of net new customers and existing customers who have upgraded or expanded their use of products. You are looking for a mix of won deals and lost deals. Most Win-Loss Analysis projects with a single research objective try to get 15 to 20 interviews.
A portfolio approach to soliciting interviews works best. The tactic that has the highest success rate is when a salesperson makes a personal outreach to a potential interviewee. Generally 25% to 40% of these contacts will convert to an interview. The second most common approach is email. Organizations pull a list of candidates from their CRM or sales force automation system and then email them asking for participation in the project. This approach performs like most email campaigns – a 20% open rate and 2% to 5% click thru rate. 50% of those who click thru convert to interviews. The final tactic is to do phone follow-up with contacts that opened the email but did not click thru. This tactics performs like telemarketing campaigns – a 1% to 2% success rate.
Most Win-Loss Analysis programs use incentives to encourage targets to commit to doing the interview. Companies offer a $25 or $50 gift card. This provides the interviewee with something tangible in exchange for their valuable time.
3. Interviews
The core activity of the project is conducting interviews. Most interviews take 20 to 30 minutes to complete. Companies use internet meeting services that facilitate the interview scheduling process, but also allow the discussion to be recorded. This enables the interviewer to focus on the discussion instead of trying to listen and take notes at the same time. There are services that will transcribe the recordings for you into a Word document for about $1 minute.
Organizations often outsource the entire Win-Loss Analysis process to 3rd parties. Experience has shown that interviewees are more comfortable talking to an independent third party instead of a representative from the company. This results in a free and easy discussion. It also avoids making embarrassing or disparaging comments about their experiences or opinions.
A critical aspect of the interview process is asking why a customer believes certain things they say. Surveys and check lists are one way to get customer feedback, but they lack the ability to follow up on interesting statements. The real value of using experienced interviewers is that they can follow up and explore why a customer believes specific things. Often this is the most valuable outcome from the interviews.
4. Analysis
After the interviews are completed, the recordings are transcribed. Next the team reviews the transcripts to identify common themes. These themes are analyzed and documented into a final report. The report contains a summary of the interviewees – company size, interviewee title, transaction type (new/upgrade and win/loss). For each theme or finding, specific quotes from the interviews are included. This lets the report’s reviewers hear, from the customer’s perspective and in their voice, the exact point they were trying to make. A meeting is held with all of the interested internal organizations to review the report’s findings and conclusions.
5. Action
The final step is to take action on the recommendations. Effective Win-Loss Analysis programs are really part of a cycle to drive improvements in the business. Win-Loss Analysis is a variant of the Six Sigma DMAIC (Define, Measure, Analyze, Improve and Control) methodology. If action is not taken based on the recommendations from the interview analysis then an opportunity to fundamentally improve your business will be missed.
Win-Loss Analysis Lessons Learned
Recruiting Interviewees is Harder than You Think
The biggest challenge many Win-Loss Analysis projects face is recruiting enough quality interviewees. Sending cold non-personalized emails to all of the CRM contacts associated with a customer tends not to be successful. What happy customer wouldn’t want to do a short interview and get a $50 VISA gift card? A lot fewer than you would expect. Experience has shown that a portfolio approach that combines warm introductions, multiple emails, and phone call follow ups works best.
Independent Interviewers are Important
Most organizations choose to use independent third party experts to manage Win-Loss Analysis projects. Interviewees are naturally more comfortable talking to an independent interviewer. Interviewees are more likely to open up and share their true feelings with a third party. Some recent projects again reinforced this for us. There was a customer that chose not to do an upgrade. The sales rep reported in the CRM system that the customer had no budget and went silent after many attempts to engage. In the interview the customer said that their decision was based on price. The upgrade that they were offered cost ten times what they were currently paying. While they saw value in the product, there was not enough value to justify such a steep price increase.
In another project, a customer chose a competitor instead of the company’s solution. When asked why she said that during the sales process she asked both the customer success team and the sales person why their solution was superior to the competitor’s product. They responded that the competitor’s solution was unreliable and prone to outages. When the prospect asked about evidence to back the statement up, the sales person responded ‘it just is unreliable’. The prospect did her own research and found that there were no serious issues with the competitor’s reliability or uptime. She chose the competitor in part because they didn’t try and gaslight her.
Don’t Blame the Sales Guy
There is a natural tendency to blame someone when a sales deal is lost. Sales people are often the target. Interviewers can ascribe comments made by a prospect to the ‘sales guy goofed up category’. Experience has shown that this is often not the case. Sales people are human and do make mistakes. Not all problems that surface during an interview are sales’ fault. Maybe the prospect had a problem with pricing or packaging. Perhaps the core messages pushed by the marketing team did not resonate with prospects in a particular sub-vertical. Or a competitor built a close personal relationship with the prospect that your sales team could not match. While questioning and later analyzing prospect remarks it is important to dig into why they believe what they do. Don’t assume because it is easy to ascribe a problem to the sales team that the root cause of the problem lies there.
High Incentives Can Attract the Wrong Type of Interviewee
Sometimes when a Win-Loss Analysis project has problems in getting sufficient interviews to reach the research objectives. Teams will then increase the interview incentive to $75 or even $100. While this may spur more people to respond to the solicitation, often these individuals are only interested in the incentive payment. These individuals may only have limited exposure to the product and may not have been involved in the actual sales process. This risk can be reduced by selecting the best candidates to target with the solicitation. Sending an email blast to every contact from a company in your CRM system will not produce the quality results you are looking for.
Learn More About Win-Loss Analysis
To learn more about Win-Loss Analysis, download our Win-Loss Primer. In it, we line up all the steps you need to make your first Win-Loss call or set up your first program.
