The Entrepreneur’s Essentials #12: The importance of reference checking
For this lesson from The Entrepreneur’s Essentials, I’m going with recency bias yet again. At SXSW I was meeting with one of the CEOs we’ve invested in since their seed round about their Series A pitch deck and process. I reminded them that it is all about which partner you get at the reputable firms he was targeting. VCs are like law firms or consulting firms — they are ultimately service firms and the partner that sponsors you and joins your Board really matters. And to make sure you are picking the right partner for your business — and we as entrepreneurs only get so many shots at the goal versus VCs who are naturally diversified — you better make sure you really check their references.
So here we go with this lesson from The Entrepreneur’s Essentials. It was first shared at Lucky7 on November 13, 2013. I made very few edits to the original post, mostly in the area of readability and grammar (not in substance of content):
I’m shocked that more startups, including their Boards and investors, don’t thoroughly check references. That is the subject of this post, and I hope by the end of it you will agree with me that to not check references is both irresponsible — and dangerous.
When I started Coremetrics in 1999, Accel Partners wanted to invest in our Series A alongside Highland Capital Partners. We had already chosen Highland as our lead. We were really impressed with Keith Benjamin in particular and he was joining our Board of Directors (unfortunately Keith passed away in a tragic accident in 2008 as I wrote about in this Bazaarvoice blog post; I think about him often — he was an incredible friend and eCommerce and Wall Street visionary). Accel put forward Arthur Patterson, the co-founder of Accel and a venture capitalist since 1973, to join our Board of Directors alongside me, Keith, and Bong Suh (our independent Director, and a really terrific mentor even to this day). As I had done with Keith, I insisted on checking Arthur’s references. Most people at Accel were surprised, and I think they thought I was naive at the time — I was a 26-year old CEO and they probably chalked it up to inexperience. And when I called his CEO references, some of them expressed a lot of surprise that I had the moxy to do so. But those references turned out to be very helpful to me, especially on how to best work with Arthur as a business partner. I believe Arthur had more respect for me as a result of being one of the first entrepreneurs to check his references. I couldn’t see any other alternative — I deeply loved Coremetrics (I still do) and I wanted to make sure that we fielded the best team possible, and that included our investors and our Board of Directors.
Fast forward to Bazaarvoice, where we were initially obsessive about both our hiring process (which is detailed in #11: The most proven way to hire well) as well as checking references. I would probe and probe on reference checks until I got something useful to work with. Usually I was able to get to some of the “areas for improvement” by saying something like, “Please level with me — I most likely am going to hire them but I could really use your advice about how to best coach them to perform at their best from day one.” That type of personal plea was hard to ignore. They had been there too — with that same person. If that didn’t work, I would ask them to walk me through the areas for improvement on their last performance evaluation. We all have them — no one is perfect.
But I learned later that I wasn’t as thorough as Scott Cook, the co-founder and initial CEO (now Chairman) of Intuit. First, let me take a quick diversion to tell you how I got to know that.
I was fortunate enough to speak at First Round Capital’s CEO Summit earlier this year. First Round did a lot for Bazaarvoice as one of our first-round investors, and especially Josh Kopelman in particular. I spoke on the lessons learned on my journey from founder to CEO and when First Round launched their brilliant Review publication (I highly recommend that all entrepreneurs tune into this valuable — and free — resource), I was happy to see them post a summary of the talk. A more in-depth version of my talk is in #14: Seven lessons learned on the journey from founder to CEO as well as my current reflections on it since beginning the data.world journey.
But the highlight of the Summit for me was seeing Scott Cook present on his lessons learned as the initial CEO of Intuit. He touched on the subject of references (the full summary of his talk, which is well worth the read, can be found here on the First Round Review):
Keeping these four key attributes in mind, Cook gets on the phone with the reference and begins by letting them talk about working with the candidate. However, the reality is people generally want to be nice and don’t have much to gain by being fully transparent and honest, so they generally start by saying how great the potential hire was and how valuable they were to the team. Cook has found that it’s most effective to completely ignore this opening feedback and throw it out. Once they’re done rambling on, Cook asks, “Among all of the people you’ve seen in this position, on a zero to ten scale, where would this person rank?” They go, “Seven.” Cook says, “Why isn’t this person a nine or a ten?” And then you’ll finally start learning about what this person really thinks.
Cook concludes the call by asking this person for other people who can give a reference on the candidate and then begins the process again. Getting as far away as possible from the candidate’s suggested references often leads to the most valuable data.
The brilliance of this approach is the comparative aspect. You are forcing the reference to stack-rank the candidate, and the reference’s integrity will most likely force them to be candid in response to such a direct and comparative question.
Above, I qualified with the word “initially” when I wrote about Bazaarvoice being diligent in checking references. One of the scariest times in the company’s history for me is when I learned that our head of recruiting at the time had decided to stop checking references. I found out about this when one of the people we had hired had started to engage in activities that I choose not disclose here. I started to probe on how that happened when we were so diligent at checking references. “We stopped checking references,” I was told to my shock. I couldn’t believe it! I asked why. “Because they don’t really tell you anything — they are all provided by the candidate.” I then spent the next 30 minutes telling stories about how important it was to always check references, and specifically how to probe. I also talked about the importance of going off-script and finding references that they didn’t provide. We reverted to our original diligent reference-checking practice immediately.
It can be very dangerous not to check references, especially when you are interviewing Board members or raising money from new investors. I’m convinced that the higher level you go — the more like a marriage it will be — the more important reference checking actually is.
Let me tell you two horror stories so that you can understand why I’m so passionate about this.
- I know a CEO that served on the Advisory Board of a startup that had great potential and millions of dollars in funding. But the Board of Directors had lost confidence in the founding CEO, and they decided to recruit a new one. The person they chose to be their new CEO was a management-level person (not an executive) from the company that the Advisory Board member led as CEO. Embarrassed because they were hiring someone away from the Advisory Board member’s company, they did not check the new CEO candidate’s references with him and went ahead and hired him. The new CEO then drove out the founders of the business and the business hasn’t recovered since. The Advisory Board member confronted the investor Board members about this. They expressed embarrassment but also expressed confidence that they had made the right decision (at that time). The Advisory Board member told them that they had made a huge mistake — that management-level person was not performing well in their management duties at their company. They had been a brilliant individual contributor — especially in the area of sales — but had failed to perform well as a manager (this happens with a lot of brilliant individual contributors, and that will be the subject of a future Lucky7 post). Over time, every one of the people that they had hired while they were a manager at the Advisory Board member’s company were let go for performance issues. A’s hire A’s and B’s hire C’s, as the saying goes — and it is very true in my experience. The company was eventually sold for much less than was invested and everyone lost money.
- I once got a reference call for a public-company CEO candidate that I had worked with in the past. Their candidate had not given me a heads up that he was looking for a new job. The person on the other end was an executive recruiter and I remember thinking, “Cool, they are going off-script and I’m a very honest reference.” The recruiter started to ask me their normal “reference check” questions and I quickly became disenchanted. I started to give them some of the cons (there were many pros), and they would quickly change the subject back to the pros. It was clear to me that the recruiter just wanted to check the boxes and get the job done. Anything they learned from me on the cons they would have to report back and so they chose to not hear them at all. And then it hit me: I couldn’t believe that a public-company Board of Directors had delegated the task of checking the references on a new CEO to a recruiter. Did no one on the Board have the time? What about the Chairman of the Board? What about the Governance Committee? It was a bad sign, and although the person they recruited was pretty good (at least in my opinion) — the company didn’t last for very long.
The bottom line is that checking references is a critical activity and therefore it should be a cherished, celebrated practice. You are, after all, recruiting people to join your company and there is a both a real financial cost as well as an opportunity cost — and a cultural cost if they don’t work out — to carefully consider. Thorough reference checking isn’t a foolproof practice — nothing is — but it is a very important one. And don’t forget as CEO it is a lot easier to let go of a team member that isn’t performing than one of your investors or Board members. You can get star-struck with someone’s credentials and awards. The more they have accomplished, the more tempting it is to not check their references. This is counterintuitive but true. Social signaling and intuition will bias you to spend a lot of time checking references on junior — “unknown” — team members and very little time checking references on the most senior. But please follow Scott Cook’s advice and really dig in. The higher level your candidate is, especially potential investors and Board of Director members, the more you should check. Go off-script. You’ll be very thankful that you did.
So that is the end of the lesson as I wrote it almost six years ago on Lucky7, with some edits and seemingly now extraneous omissions. You can see the original post here, and the long comments thread below it.
How did my co-founders and I apply this lesson at data.world? Well, we were very thorough in the selection of our investors. I checked seven of Jason Pressman of Shasta Ventures’ references using the technique above. And to date he has been the best Board of Directors member I’ve ever worked with. I really probed with reference portfolio company CEOs on how Jason behaved when “the chips were down”, and heard story after story about how he lifted up not just the CEO but the rest of the Board. It was inspiring, to say the least. We’ve done the same with our executive hires and so far haven’t made any major mistakes.
What are your best reference-checking practices? What horror stories would you like to share? Please comment — we would all like to hear from you and get a dialog going about this.
