Customer reference questions.

My client is in the process of buying some software that will be used extensively across business units. As part of the process we’re talking to customer references from the tool vendors we’re considering.

When interviewing references, people don’t often know what to ask other than non-specific questions about how they like the product and how good the vendor support is. This is certainly the information you want to get, but unless you ask specific questions, you aren’t going to get specific answers.

I’ve put together some sample questions from past projects that should help anyone trying to get good information from references. Not all of these questions apply to every project, but they can be used as a good starting point for your own questionnaire.

Context

• Describe the way you are using the tool.
  This gives you an understanding about their business needs and how similar or different they are from ours. You should also try and find out how extensively they are using the tool. Is this an enterprise-wide implementation, or a small department only using a limited set of the tool features?
• Are you a user of the tool?
• What’s your company’s relationship with this vendor?
  Sometimes vendors will offer the names of strategic partners or even subsidiary companies as references. Or an executive at the reference customer might serve on the board of the vendor (or vice versa).
• What does your company receive from the vendor in return for being a reference?
  This is trying to establish a context for the references. If this company getting something significant (discounts, early access to new products) from the vendor, then their comments should be viewed with skepticism.

Product

• What are the best features of the tool?
• What are the limitations of the tool?
• Are there the manual processes and workarounds that you must follow in order to complete your tasks using the tool? Describe them.
• Is there anything that surprised you about the product?

Implementation

• How long did your implementation project take?
• What went well during your implementation?
• What went poorly during your implementation?
• What did you learn during the implementation that you would have liked to know before you started?
• If you were doing the implementation again, what would you do differently?

Support

• How often do you request technical support from the vendor?
• Describe the process of asking for technical support and receiving a resolution.
• When you have a question or issue for support, how long does it generally take them to respond to you?
• Who do you receive resolution from (product developers, tech support reps, professional services consultants, or someone else)?
• When a bug fix or update is available from the vendor, how do you find out about it?
• Describe the process you follow to install updates.

Professional services

• Did you use the vendor’s professional services team to implement the product?
• How many of the vendor’s consultants were required to implement the product?
• Did the consultants customize the product for you? Describe that customization.
• Did the consultants work on-site or remotely?
• Who provided the project management for the implementation?
• What type of consultants worked on the implementation (developers, analysts, project managers)?
• What did the consultant not do (what did you have to do yourself)?
• Were your expectations for the implementation met by the consultants?
• Was the project finished on time and on budget? If not, why not?

Conclusion

• What other tools did you consider when making the decision to use this vendor’s tool?
• What should we know about this product and company before we make a decision?
• Would you choose this product again?