A Litany of Product Management Mistakes | Michael McDerment, Freshbooks | BoS USA 2011

In this talk, Michael McDerment (@mikemcderment) , CEO/co-founder of FreshBooks and serial entrepreneur, talks about how FreshBooks has survived and what they’ve learned. Michael shares his insights on decision making during product management and mistakes made along the way.

Video and Notes Below

Video

Notes

Freshbooks did a lot of things wrong.  Michael looks into the past to see if there are any patterns.

A series of stories in several stages of the company:

• Launch
• Survival
• Scale

Started in 2003, launched in 2004. Over 3.5m users in 2011. 85 people in Toronto. Spent 3.5 years in a basement. Entrepreneurs don’t think they’re going to get into the accounting and billing thing — oh, but you will.

Launch – 2003-2006

• “Decidedly unpretty”
• Radical transparency – going to tell you stuff you’re not going to believe about Freshbooks’ process & technology.
• Team from 2-6 in Michael’s parents’ basement
• Process for software development was “cram it in when we can”
• Had a partner who was a CS professor, had two days of his time per week.
• No source control. No QA. No Unit Testing.
• Deployment was “FTP to production”
• LAMP stack, framework: “not so much”
• There is no Rails yet.
• Core PM Problem: What should you build?
• Build for a purpose, not based on what your developers think they can handle.

Survival – 2006-2010

• Team: 7-50
• Hooray for source control
• Regular releases
• Finally doing QA.
• Story: asking developers to “draw our architecture.” Got back drawings that looked NOTHING alike, plus one photo of a plate of spaghetti.
• Finally building an API.
• Paralyzing debt!
• Core PM Problem: How do you decide who decides?
• Story: Team around 10, moved out of Michael’s parents’ house.
• Had 5 people at a meeting: 3 founders, lead engineer, lead support.
• Meetings were HORRIBLE. Impossible to make decisions.
• Everybody wants a hand in developing a product, circular arguments.
• Reached out to local person who came to speak with them.
• “I know what your problem is, and the solution to it: you’re trying to do design through consensus. 5 people can’t decide. Solution: Design Dictator.”
• Design Dictator needs to listen to feedback and not just advance their own ideas, but somebody needs to move forward.
• Simple command & control stuff, but critical at the time and hard to see.
• Sales Funnel: Visit, Trial, Pay
• You’re measuring “key metrics:” CPA, ARPU, Churn, LTV

Need a system that can effectively TRACK your test for days, months, years. Tools available today kind of don’t do that.

Hypothesis #1.

• Content: screenshots, video, etc.
• Videos lowered conversion by 20%.

Hypothesis #2:

• Pretty looking invoice templates are important.
• Existing template “looked like 2003”
• “Freshen up the template” >> New look 11% lower.

Hypothesis #3:

• A clear upgrade path increases conversions.
• Adding an obvious upgrade button performed 3% worse.

Hypothesis #4:

• Naming pricing packages based on the user type will increase conversions.
• Moped, Shuttle Bus, Limo, Jet, Starship, Time Machine.
• We get fan mail about them sometimes :).
• Changed to “Free, Solo, Solo Unlimited, Team”
• Lost over a million dollars.
• Sales Funnel & split testing during Survival.
• A few tests run in Visit. (Quality of reach)
• A few more in Conversion. (Website conversion)
• Many in Pay.  (Trial activation)
• Let’s look at a conversion example (by Funnel Stage).
• Visit: 10% conversion.
• Trials: 30% conversion.
• Paying: 50% conversion.
• Result is like 1.5% overall conversion rate.
• Where to work to improve this?  How about EVERYWHERE??
• “The biggest and best tests we’ve ever done have nothing to do with the funnel.”
• Balance your time. Focus on your overall offering.

Survival Nuggets

• Very product centric thinking
• Must track all the way through the funnel << Google for “web app marketing metrics” for a great video!
• Biggest impacts outside the product
• Moving one part of the funnel (i.e. increasing trials) doesn’t necessarily help if it simultaneously DECREASES the number of people who pay (because it changes the qualities of people who get through the funnel).

Owning your metrics data is paramount. You may really really want that data later and if the provider you used isn’t around, you’re kinda screwed.

Scale – 2010+

• Only 85 people, so keep it in perspective, but this is still “scaling”
• Team: 50+
• Working toward continuous deployment
• No longer have a clue what folks are up to
• Speedy split tests << it’s amazing how fast results come back from split tests.
• Story: Business advisor sent his friend up to talk to team, provide report back.
• Advisor: “I learned what I figured I’d learn, which is that you guys suck at software. But you’ve figured out the hard part.”
• Core PM Problem: How do you scale decision making?
• Founder can’t make all product decisions anymore!
• (awesome graph of product decisions here, wish I could screenshot it :-/ )
• Business owner sets overall goal
• ProdMgmt breaks goal down into smaller problems.
• BI determines “desired outputs” of those problems.
• Developers do it. (tactics)
• The Open Problem: How can everyone contribute?
• Total of 85 people, 15 support people, everybody wants to help. How?
• Not everybody can be directly in the product cycle, but smart people need to feel like their helping.
• It’s a cultural kind of problem.

After all those mistakes, it’s amazing we’re here to tell the tale…