This is the blog of Adam Kalsey. Unusual depth and complexity. Rich, full body with a hint of nutty earthiness.
Freshness Warning
This blog post is over 19 years old. It's possible that the information you read below isn't current and the links no longer work.
10 Jun 2004
My client is in the process of buying some software that will be used extensively across business units. As part of the process we’re talking to customer references from the tool vendors we’re considering.
When interviewing references, people don’t often know what to ask other than non-specific questions about how they like the product and how good the vendor support is. This is certainly the information you want to get, but unless you ask specific questions, you aren’t going to get specific answers.
I’ve put together some sample questions from past projects that should help anyone trying to get good information from references. Not all of these questions apply to every project, but they can be used as a good starting point for your own questionnaire.
Here are some additional questions I believe are important: -Which of the software modules are being used and what systems do they integrate with? -Were there any integration issues? -How many users regularly utilize the software? -What was the ROI timeframe for the software?
This is great stuff. I do something similar with customers when trying to flush out product concepts; I should put together a list for that. It'd be extremely useful when utilizing a method such as Ambler's CRC modelling technique ( http://www.ambysoft.com/crcModeling.html ), of which I'll admit I'm still a fan.
People who referee potential software providers, like those who referee potential employees, know things that can help you. Your problem is that they've made a vague agreement with the vendor to say nice things about them. Your opportunity lies in the fact that it's a weak agreement: no money has changed hands, and the referee hasn't actually promised to only praise the product in all its aspects.
The Customer reference questions were exactly what I was looking for. It was a great tool in the qualifying process of selecting a software vendor. I am so happy not to have had to reinvent the wheel. You've already done that.
I was asking someone for reference for an AC Installer. The questions didn't necessary pertain to AC Installer, but they helped me come up with questions for an AC Installer from the list you had. Thank You very much. -Bhaven.
Very comprehensive list; saved me time and ensured that I covered all inputs required for decision making.
Wonderful list of customer reference questions! Thanks for sharing. This helped me out a lot.
Thanks. Good list.
This is a good set of foundational questions. Several we always include are: 1) Are there areas for improvement?, and 2) What benefits, quantifiable or not, are being realized from the solution?
And there are the business value questions of course. Measurables are always more compelling, but the hard-to-quantify or intangibles shouldn't be left out. Sometimes an example of one of these would be how the solution impacted the customer personally. We've captured comments like, "I have my life back again! I'm no longer working 80 hours a week." Powerful stuff even if you can't analyze it in Excel.
In addition to asking what the other tools were considered as noted in the conclusion, I like to ask "What evaluation criteria where used in the decision to select this vendor". This is a relatively staight forward way to quickly determine if your own criteria are well aligned with this customer and how well their critique will map to your own.
This discussion has been closed.
dave
June 12, 2004 12:56 PM
Don't forget production: Production - What type of production environment (hw, sw, processes) do you run the product in? - How well did the product fit into your production environment? - What new processes have you created to support the product in your production environment? - When there is a production issue involving the product how are you able to debug it?