Customer reference questions.

Freshness Warning
This article is over 10 years old. It's possible that the information you read below isn't current.

My client is in the process of buying some software that will be used extensively across business units. As part of the process we’re talking to customer references from the tool vendors we’re considering.

When interviewing references, people don’t often know what to ask other than non-specific questions about how they like the product and how good the vendor support is. This is certainly the information you want to get, but unless you ask specific questions, you aren’t going to get specific answers.

I’ve put together some sample questions from past projects that should help anyone trying to get good information from references. Not all of these questions apply to every project, but they can be used as a good starting point for your own questionnaire.

Context

  • Describe the way you are using the tool.
    This gives you an understanding about their business needs and how similar or different they are from ours. You should also try and find out how extensively they are using the tool. Is this an enterprise-wide implementation, or a small department only using a limited set of the tool features?
  • Are you a user of the tool?
  • What’s your company’s relationship with this vendor?
    Sometimes vendors will offer the names of strategic partners or even subsidiary companies as references. Or an executive at the reference customer might serve on the board of the vendor (or vice versa).
  • What does your company receive from the vendor in return for being a reference?
    This is trying to establish a context for the references. If this company getting something significant (discounts, early access to new products) from the vendor, then their comments should be viewed with skepticism.

Product

  • What are the best features of the tool?
  • What are the limitations of the tool?
  • Are there the manual processes and workarounds that you must follow in order to complete your tasks using the tool? Describe them.
  • Is there anything that surprised you about the product?

Implementation

  • How long did your implementation project take?
  • What went well during your implementation?
  • What went poorly during your implementation?
  • What did you learn during the implementation that you would have liked to know before you started?
  • If you were doing the implementation again, what would you do differently?

Support

  • How often do you request technical support from the vendor?
  • Describe the process of asking for technical support and receiving a resolution.
  • When you have a question or issue for support, how long does it generally take them to respond to you?
  • Who do you receive resolution from (product developers, tech support reps, professional services consultants, or someone else)?
  • When a bug fix or update is available from the vendor, how do you find out about it?
  • Describe the process you follow to install updates.

Professional services

  • Did you use the vendor’s professional services team to implement the product?
  • How many of the vendor’s consultants were required to implement the product?
  • Did the consultants customize the product for you? Describe that customization.
  • Did the consultants work on-site or remotely?
  • Who provided the project management for the implementation?
  • What type of consultants worked on the implementation (developers, analysts, project managers)?
  • What did the consultant not do (what did you have to do yourself)?
  • Were your expectations for the implementation met by the consultants?
  • Was the project finished on time and on budget? If not, why not?

Conclusion

  • What other tools did you consider when making the decision to use this vendor’s tool?
  • What should we know about this product and company before we make a decision?
  • Would you choose this product again?

dave
June 12, 2004 12:56 PM

Don't forget production: Production - What type of production environment (hw, sw, processes) do you run the product in? - How well did the product fit into your production environment? - What new processes have you created to support the product in your production environment? - When there is a production issue involving the product how are you able to debug it?

Scott
June 16, 2004 8:26 AM

Here are some additional questions I believe are important: -Which of the software modules are being used and what systems do they integrate with? -Were there any integration issues? -How many users regularly utilize the software? -What was the ROI timeframe for the software?

dennis
July 16, 2004 8:35 PM

This is great stuff. I do something similar with customers when trying to flush out product concepts; I should put together a list for that. It'd be extremely useful when utilizing a method such as Ambler's CRC modelling technique ( http://www.ambysoft.com/crcModeling.html ), of which I'll admit I'm still a fan.

David Walker
August 18, 2004 8:54 PM

People who referee potential software providers, like those who referee potential employees, know things that can help you. Your problem is that they've made a vague agreement with the vendor to say nice things about them. Your opportunity lies in the fact that it's a weak agreement: no money has changed hands, and the referee hasn't actually promised to only praise the product in all its aspects.

Maria from California
March 22, 2006 3:33 PM

The Customer reference questions were exactly what I was looking for. It was a great tool in the qualifying process of selecting a software vendor. I am so happy not to have had to reinvent the wheel. You've already done that.

Bhaven Shah
May 4, 2007 9:33 AM

I was asking someone for reference for an AC Installer. The questions didn't necessary pertain to AC Installer, but they helped me come up with questions for an AC Installer from the list you had. Thank You very much. -Bhaven.

Mohit
August 27, 2008 8:13 AM

Very comprehensive list; saved me time and ensured that I covered all inputs required for decision making.

Fallon
June 16, 2009 6:24 AM

Wonderful list of customer reference questions! Thanks for sharing. This helped me out a lot.

Arno
September 24, 2009 8:44 AM

Thanks. Good list.

David Sroka
December 25, 2009 1:54 PM

This is a good set of foundational questions. Several we always include are: 1) Are there areas for improvement?, and 2) What benefits, quantifiable or not, are being realized from the solution?

David Sroka
January 8, 2010 11:58 AM

And there are the business value questions of course. Measurables are always more compelling, but the hard-to-quantify or intangibles shouldn't be left out. Sometimes an example of one of these would be how the solution impacted the customer personally. We've captured comments like, "I have my life back again! I'm no longer working 80 hours a week." Powerful stuff even if you can't analyze it in Excel.

Josh
December 6, 2010 8:38 AM

In addition to asking what the other tools were considered as noted in the conclusion, I like to ask "What evaluation criteria where used in the decision to select this vendor". This is a relatively staight forward way to quickly determine if your own criteria are well aligned with this customer and how well their critique will map to your own.

Your comments:

Text only, no HTML. URLs will automatically be converted to links. Your email address is required, but it will not be displayed on the site.

Name:

Not your company or your SEO link. Comments without a real name will be deleted as spam.

Email: (not displayed)

If you don't feel comfortable giving me your real email address, don't expect me to feel comfortable publishing your comment.

Website (optional):

Follow me on Twitter

Best Of

  • Let it go Netscape 4 is six years old.
  • Debunking predictions Read/Write Web's authors have some goofy predictions.
  • Pitching Bloggers Forget what you learned in your PR classes. Start acting like a human instead of a marketer, and the humans behind the blogs will respond.
  • California State Fair The California State Fair lets you buy tickets in advance from their Web site. That's good. But the site is a horror house of usability problems.
  • The best of 2006 I wrote a lot of drivel in 2006. Here's the things that are less crappy than the rest.
  • More of the best »

Recently Read

Get More

Subscribe | Archives

10

Recently

Travel weight of coins (Aug 26)
My laptop bag was getting heavy, so I emptied it. Found 115 coins from 6 different countries in there. That will do it.
Aereo: a parable (Jun 26)
Here's why people who understand both technology and copyright law think the Supreme Court's Aereo decision defies common sense.
The birth of cubicle hell (Apr 29)
Where do cubicles come from?
Double Opt-In helps you (Apr 2)
A simple reason why double opt-in is a good thing for marketers.
Make the most of opportunities (Mar 5)
Often, part of success is being in the right place at the right time. What will you do with the opportunity when it happens?
Email as a model for future mobile phone service (Feb 10)
What if your phone wasn't tied to your phone number?
MySQL inadequecies (Feb 9)
This guy seriously hates MySQL and helps you understand how to use it better.
invisible Fence (Mar 22)
The New York Times has a paywall now. Sorta. If you don't choose to ignore it.

Subscribe to this site's feed.

Elsewhere

Tropo
Voice and communications platforms, including Tropo and Phono. Work.
SacStarts
The Sacramento technology startup community.
Pinewood Freak
Pinewood Derby tips and tricks

Contact

Adam Kalsey

Mobile: 916.600.2497

Email: adam AT kalsey.com

AIM or Skype: akalsey

Resume

PGP Key

©1999-2014 Adam Kalsey.
Content management by Movable Type.