In the course of my career at Gartner, and my pre-Gartner life as an engineering director who spent giant piles of money on purchasing technology that was often very early-stage, I have spoken to an awful lot of customer references. I’m about to soon dive into the reference-a-thon that a Magic Quadrant represents (we call as many as 5 references per vendor), and it’s leading me to think about what makes for a good or a bad reference, from my personal perspective. So, here are some thoughts, targeted at vendors and service providers.
Make sure your references like you. Nothing will create a worse impression than a reference that isn’t happy with you, and hasn’t been happy with you for some time. It’s fine for a reference to currently be having a transient problem, or even to have experienced some kind of disaster — that sometimes even makes for good stories about how good your support has been during the crisis. But a reference that isn’t a promoter is hugely problematic, because not only does it create a negative impression, it makes it clear that you have failed to keep track of this customer’s sentiment, and to communicate internally about it, to the point where you’re using an unhappy customer as a key reference. You should get in touch with your references on a regular basis to make sure they’re still delighted with you.
Your references should be engaged customers. Engaged customers know why they chose you (and can talk about the competition they looked at and why your solution was the best fit for them), have an opinion on their ongoing use of your product and service, and are passionate enough about it to talk about the good and the bad, what you do well and what they’d like you to improve. Customers who are just, yeah, we selected these guys and it all works okay — fine, you’ve checked the box on “you haven’t actively sucked”, but they’ve really said nothing interesting. This can be fine if you’re just offering a reference to a prospective customer (who wants to make sure that you’ve done an implementation similar to the one he’s contemplating and that it went fine), but it’s deadly in an analyst reference (because the analyst is interested in getting a first-hand picture of what it’s like to deal with you and your product/service, and someone who is neither enthused nor analytical makes for a deathly-dull and not very useful reference).
Your references should be targeted. If you are offering a reference customer to a prospect, the reference should be as similar to that prospect as possible, in terms of solution, industry, approach, and role (likely in that order). If you are offering references to an analyst, they should represent a spread of customers — different use cases, industries, and length of time they’ve been customers (from new implementations to long-term customers).
Your references should be representative. If you’re dealing with an individual customer, your references should be as close to that customer’s expected implementation as possible, even if that is exotic. But if you’re dealing with an analyst, the references should be representative of typical use cases, implementations, and customer types. If you choose to offer some more exotic outliers, great, but make sure the analyst knows that they’re not typical of your customer base. You don’t want to give the analyst the wrong impression about who you normally serve.
Your reference list should be periodically refreshed. You want references that are still actively engaged with you, and represent the current state of your business, in terms of version deployed, use case, and experience with your company. While long-term customers are sometimes nice to talk to (especially in a space where customers sign very long contracts, like 7-year outsourcing deals), for products, or for services bought on shorter contracts, current references are very important. If you are offering references to an analyst, especially in the context of a yearly process like a Magic Quadrant, do not repeat references from year to year; not only will the analyst prefer to talk to someone new, but he will wonder why you can’t easily produce new reference customers.
Ignore client relationships with analyst firms. When offering references to analysts, don’t worry about whether or not a reference has a client relationship with their firm. Reference interviews are typically conducted under NDA, and as far as I know across all research firms, without any regard to client status. Even if an analyst helped a client through the process that resulted in your being selected, he might not have gotten any feedback from the client about what has happened since. Even if the analyst has an ongoing relationship with that client, it’s usually in the context of inquiry, where the analyst, constrained by the 30-minute timeslot and the client’s specific questions, rarely gets to satisfy his curiosity. Reference interviews are very different, and the analyst conducting the interview might not be the same one as previously helped the client.
Consider supplementing references with a customer list. A list of customer logos and, if possible, a one-sentence description of their use case, can be extremely helpful for getting a better general understanding of who you serve, and what they use you for. If providing this list to an analyst, it can usually be done under an NDA.
