Guest Author · February 16th, 2010
This guest post is by Ash Maurya, a lean entrepreneur who runs a bootstrapped startup called CloudFire. If you like it, check out Ash’s blog and his tweets @ashmaurya. – Nivi
What you charge for your product is simultaneously one of the most complicated and most important things to get right. Not only does your pricing model keep you in business, it also signals your branding and positioning. And it’s harder to iterate on pricing than other elements of your business. Once you set a price, coming down is usually easier than going up.
Should I charge for my MVP?
Most people choose to defer the “pricing question” because they don’t think they (or the product) are ready. Something I hear a lot is that a minimum viable product is by definition (embarrassingly) minimal. How can you possibly charge for it?
A minimal product is not synonymous with a half-baked or buggy product. If you’ve followed a customer development process, your MVP should address the top 3 problems customers have identified as important and it should do it well. You can ensure that by dedicating 80% of your efforts to improving existing features versus cranking out new ones.
Steve Blank bakes price exploration right into the initial customer interviews. Price, like everything else, is built on a set of hypotheses that needs to be tested early. Steve suggests you ask potential customers if they’d use the service for free. This is to gauge if the product’s value proposition is compelling at all. You then ask if they’d use the service for $X/yr. How do you come up X? You can simply roll the dice and adjust along the way, or use Neil Davidson’s excellent guide to software pricing to start with a more educated guess. Once your MVP is built, Steve asks you to sell it to your early customers. There is no clearer customer validation than a sale.
Sean Ellis, on the other hand, argues that achieving initial user gratification (product/market fit) is the first thing that matters and suggests keeping price out of the equation so as not to create unnecessary friction:
“I think that it is easier to evolve toward product/market fit without a business model in place (users are free to try everything without worrying about price). As soon as you have enough users saying they would be very disappointed without your product, then it is critical to quickly implement a business model. And it will be much easier to map the business model to user perceived value.”
Both Steve and Sean advocate removing price from the equation — but at different points. Steve removes price during the customer discovery process but suggests you charge for your MVP. Sean removes price from the MVP and suggests you charge after product/market fit. I can see the merits of both approaches and wondered which was right for my product: CloudFire: Photo and Video Sharing for Busy Parents.
Why not use freemium?
On the surface, freemium seems like the best of both worlds: Get users to try your service without worrying about price, then up-sell them into the right premium plan later. However, many people make the mistake of giving away too much under the free plan, which leads to low or no conversions. It’s human nature — we all want to be liked.
More important, we don’t yet have enough information to know how to price or segment the feature set. I made this mistake with my first product, BoxCloud: an early visionary customer called me up and said, “I really like your product and want to pay for it but your pricing doesn’t require it.” After a few more iterations of segmenting the feature set, I decided to forgo the free plan and simply offered premium plans with a trial period. Sales went up and so did the quality of feedback, which I attribute to the difference between feedback from customers versus users.
(Hiten Shah shared a similar story with me around his experience with Crazy Egg. Even 37signals has greatly deemphasized their free plans to almost being fine print on their pricing pages.)
Lincoln Murphy just published a timely white paper on “The Reality of Freemium in SaaS” which covers many important aspects to weigh when considering Freemium, such as the concept of quid pro quo where even free users have to give something back. In services with high network effects, participation is enough. But most businesses don’t have high enough network effects and wrongly chase users versus customers. What I particularly liked in this paper is Lincoln’s recongition that “Freemium is a marketing tactic, not a business model.”
I strongly feel that, especially for SaaS products, starting with free and figuring out premium later (all too common) is backwards. If you know you are going to be charging for your product, start by validating if anyone will pay first. There is no better success metric and it leads to less waste in the long run. Focusing on the premium part of freemium first lets you really learn about your unique value proposition — the stuff that will get you paid. You can then come back and intelligently offer a free plan (if you still want to) with more intelligence and the right success metrics clearly defined. Even if you think you have a one-dimensional pricing plan like I did (e.g. number of projects), you’d be better served testing it with paying users because pricing experiments take a much bigger toll than other types of experiments.
Testing price in interviews
How did I put all this to test? The biggest mind shift in following a lean startup process is going from thinking you know something to testing everything you think you know.
So I followed Steve Blank’s advice and built some pricing questions into my initial face-to-face customer interviews. Because CloudFire is a re-segmented product in an existing market, potential customers referred to competitor pricing. This had to be balanced against the perceived value of our unique value proposition – saving time with faster and easier sharing of lots of photos and videos. Through these interviews I determined that, like their sharing needs, my potential customers valued simple hassle-free pricing and $49/year for unlimited photo and video sharing was a fair price they were willing to pay. That is what I charged them once my MVP was ready.
Testing price on the web
I wanted to run the same set of pricing tests with web visitors that I did during my interviews. Short of split testing a free and paid version of the MVP, which is technically illegal (update) and unfair to paying customers, I decided to split-test 3 different products with 3 different prices:
- $49/yr for unlimited photo and video sharing
- $24/yr for unlimited photo sharing
- FREE for 500 photos
All plans have a 14-day free trial with the exception of the free plan which is free forever. Here are the variations I tested:
Original: Single unlimited plan
This is the simple option I discovered during customer discovery interviews. It served as the control.
Variation 2: Multiple plans
I segmented the product into 2 offerings: unlimited photos+video and unlimited photos only. I wanted to test price sensitivity and gauge interest in video sharing. Not many people I interviewed were currently taking lots of videos but they all wanted to be taking more.
Variation 3: Freemium
This has the 2 plans from above along with a limited free plan. Yes, this is a freemium plan. I wanted to measure if a limited free plan would disproportionately drive the right type of traffic (busy parents in my case).
Variation 4: No Price During Introductory Period
I added a fourth variation to test Sean Ellis’ advice on removing price till product/market fit, but I tested this differently. I was not comfortable offering the full product for a price and for free at the same time. So rather than including this page with my A/B tests, I instead tested it with new parents I interviewed.
The Results
First Place: The original single plan — second place in conversions and best overall performer. Surprisingly, the original page was the best overall performer.
Second Place: Variation 3: Freemium – most conversions but second place overall. Not surprisingly, the freemium variation drove the most conversions but only outperformed the original by 12% and had the lowest retention. Referral stats combined with random polling/emailing revealed a majority of the users that signed up were just curious (and not parents).
Third Place: Variation 2: Multiple plans – least conversions and worst overall performer. People reacted least favorably to the two paid plans.
Non-starter: Variation 4: No price during introductory period. Parents I interviewed did not understand the introductory period without explanation and were reluctant to try the service without knowing how much the service was going to cost. Probing further, they weren’t willing to invest the time building up web galleries and inviting others only to find that the service might be priced out of their expectation.
What I learned
It does pay to align pricing with your overall positioning. Our unique value proposition is built around being “hassle-free and simple” and people seemed to expect that in the pricing model as well. A lot of our existing customers were already paying for their existing sharing service so the leap from free to paid was not a big one. While Sean suggests removing price before fit for consumer facing products, he suggests always charging for enterprise customers to gain their commitment. This is another case where pricing needs to be explicit. Using Cindy Alvarez’s model, our customers appear to be Time-Poor, Cash-Rich. Offering no-hassle free trials was sufficient to remove the commitment risk. Money back guarantees might be another way to further lower this risk.
The biggest lesson learned, though, is how accurate my initial customer interview findings were, compared to all the hypotheses that followed. Pricing is more art than science and your mileage will vary, but whenever possible get out of the building, talk to a customer, and consider testing price sooner rather than later.
What do you think? Why do you think these variations finished the way they did? What other variations would you like to see us try? How else do you think we could increase conversions? I’m looking forward to discussing your responses in the comments.
Topics Customer Development · Lean
Great post! Thank you for sharing. Did you use Google Optimizer for the A/B testing? Thanks.
Yes, I used Google Website Optimizer primarily but played around a little with Visual Website Optimizer (http://visualwebsiteoptimizer.com/) too.
VWSO lets you set up your variations inline in the browser so you don’t have to muck around with copy-pasting all the test and control scripts in your pages.
Very good synopsis of your pricing strategy. Curious as to a one aspect of this: Did you purposely select $49 for your Plus service (as opposed to say $50 or $48)? The reason I ask is that the end digit of your price can be a key psychological trigger for people. Discount pricing typically ends in a 9 (as in $49 or $19.99, etc.). Studies have shown that for discount (or highly price-sensitive) shoppers, the strategy of pricing your offering for say, $49 as opposed to $50 is quite powerful. Check it out yourself — go to your local Walmart and you will find that most all of their prices end with the digit “9”.
The question for you is do you (or did you) look at your target buyers to determine what type they are. While discount pricing appeals to discount buyers (duh), it doesn’t appeal to higher-end buyers. To take an extreme case, have you ever seen Tiffany price their products at $199.99? Probably never, because they know that this strategy not only won’t appeal to their buyers, but it also will send a powerful psychological message to these premium shoppers that the Tiffany product is not of the highest quality.
If you determine that you’re dealing with discount buyers, keep the $49.00 price for your Plus offering, but you might want to think about raising the price for your Basic offering to $29.00; you may be pleasantly surprised in the revenue growth that results from this change.
In any case, a great article and I hope my comments give you some food for thought.
Excellent.
While doing the face-to-face interviews have you popped the pricing question with a ridiculously high price as suggested by Steve in his book?
How did you set the A/B testing? Based on regions or simply random?
Patrick –
That is an interesting insight. Yes, I did pick $49 and $24 to stay under the $50 and $25 psychological price. I wouldn’t characterize our customers as discount shoppers but then again we aren’t charging $49.99 and $24.99.
I will consider $29 versus $24 if that plan ever comes back but for right now it’s been nixed.
Yves –
I did try two variations during early interviews. The first was asking a higher price (not out-of-the-question high) of $99/year. The second was not giving a price and asking users what they’d pay, which usually made them a little uncomfortable so I stopped asking it that way.
Steve’s book was written with an Enterprise Software context where pricing is a lot more elastic. In our case, there are a lot of existing price points customers used as reference, so it paid to be a little more “intelligent” about the price we tested.
Ash, you raised an extremely valid point of price testing being an extremely dangerous territory. A number of companies in the past have been caught in a soup because they tried to do a price test across different visitors. Rather, a better approach is to price test across (selling) objects. For example, if you are Amazon, you shouldn’t test a book for $20 v/s $40. Rather you should test two different books on the same subject/topic but different pricing. I think your approach was ideal.
By the way, thanks for mentioning Visual Website Optimizer in your comment. I hope you play with the tool a bit more in future!
“The second was not giving a price and asking users what they’d pay, which usually made them a little uncomfortable so I stopped asking it that way.”
I normally ask them what they think the service will cost. Or what they think other people will think the service will cost.
Ash,
As ever, very well thought out. I’m curious about the results, however. Most pricing studies would lead you to the conclusion that limited choice (the rule of 3) would lead you to better performance than a single binary option.
What do you think led to that result and what was the confidence interval for the results? I’m wondering if there are any possible small sample effects that all of us in the startup world need to be mindful of.
Thanks for sharing.
John –
My theory is that my target demographic (busy parents) valued their time above everything else. They knew what they were currently paying ($24.94 for flickr pro, $39.95 for Smugmug) and wanted a single plan with no hidden gotchas.
These tests were started in December and run through January with ~1000 trials.
At Postling we’ve iterated a number of times on pricing. Our product is a social media management tool for businesses, allowing them to manage their blogs, Facebook, twitter, and Flickr all in one place. Multiple people can manage multiple accounts. It’s targeted at small and medium sized businesses.
We built our billing system so that we can grandfather in any users at any price points, and charge new prices going forward. Not an A/B test, but it allows for an iterative approach.
— Our MVP was a single user tool for managing accounts —
1) Charge $9/month with a 30 day free trial. Ask for credit card upon registration. Credit card was not charged until end of the free trial.
Outcome: Huge drop-off rates during registration from step 1 (username, email address) to step 2 as people were afraid to input their CC information.
2) Charge $9/month with a 30 day free trial, but don’t ask for credit card until the 30th day.
Outcome: We had many, many more users register, but a low conversion rate (~20 paying customers). We suspect this was due to the product lacking key features; the MVP so simple that it did not add enough value over free tools.
We then realized from conversations with users that we had small businesses with tiny budgets and larger companies with much more robust budgets.
— At this point we released a multi-user / multi-account tool —-
3) Permanently free for single users managing a single brand, charge $250/month to add up to 10 more users or brands.
Outcome: Obviously, this got us a lot more users, about 3-4x more and a lot more viral growth. We raised the price intentionally, as the companies who were willing to pay (PR firms, agencies) were willing to pay a LOT more. We’ve been able to get a number of sales at this price, with some saying the price is too low. Sales have required direct contact, and we’re evaluating if a direct sales approach makes sense. Probably not long term, but we are willing to pay a premium now for customer discovery.
4 – Upcoming) Offer individual licenses for $30/month and premium features for the free users to have a path to becoming paying users. Premium features are available to free users for 2 weeks before being hidden behind a paywall.
Desired outcome: Free users will self-service upgrade, as the premium features will be very compelling (search monitoring and analytics). Some businesses who only needed one license (vs. 10) will have a way to test out the benefit of an additional license without requiring massive investment (or VP approval) before rolling out across their organization.
Sorry for the long post, but I thought it would be helpful.
Thanks for sharing, David.
I agree that having a full-featured trial period makes a lot of sense. I have considered trying a variation where rather than expiring at trial, the product would downgrade to a less functional but free state. Would love to hear the results of your latest experiment.
Hi Ash — I think it makes sense to downgrade to Free instead of expiring the service.
I’m also an advocate for pricing models that scale naturally (to infinity) based on a key metric, e.g. number of users, photos, bandwidth etc.
For example, our service is a self-service ad manager (off-the-shelf Facebook Ads) targeted at publishers, so we based the key metric on the number of impressions sold to advertisers.
We target both small (price-sensitive) and large publishers so we needed to make sure our pricing fits both markets.
After multiple tests, we finally settled on a four-plan model.
1. Free: Up to 100,000 impressions
2. Standard: $49/mth for 500,000 impr
3. Pro: $149/mth for 2m impr
4. Enterprise: $499/mth for 10m impr
To scale, we allow clients to mix and match the plans, so if you want to serve 12m, you could mix one Pro and one Enterprise.
As you can see, pricing can be both a science and an art.
Great post Ash,
We spent a lot of time wrestling with the same challenges you mention. Your tips on customer interviews also help a lot. It’s interesting that of your original, single-plan option was the selection that won out. I’m wondering if similar comparisons have been done for B2B and B2C companies. I”m wondering if some sort of guiding principle could be formulated around giving users choice (multiple plans) vs. the simplicity of just one plan. Basically, what’s the cost of adding pricing options vs. the potential revenue they could earn? I’d love your thoughts!
Tim –
I link to Neil Davidson’s excellent guide to software pricing, which is great on pricing theory. Multiple plans make sense where you need to segment your customers (say by features or usage).
I am currently going after a single user type (a busy parent) that more or less has the same usage pattern and wants the same feature set — so a single pricing plan seems to make the most sense.
The key question seems to be, where is the best balance between having more or less pricing options?
Maybe for B2B cases when the rational aspect of the money plays an important role in the buyer’s mind, having more choices could lead to more revenue.
Great post, quick question: Did you try pricing per month instead or per year and look for framing bias this way? I would be curious as to the results.
Any plans to test discounts for pre-paying for two years? Thanks.
Tristan — Pricing per month versus yearly is a currently running experiment.
Would definitely appreciate a follow up post or email if you don’t mind sharing the info. Thanks!
me too 😉
Ash, this is outstanding, thanks! You mentioned that it was technically illegal to split-test a free and paid service simultaneously. This is something I’d never seen mentioned before in the dozens of split test articles I’ve read.
Please could you elaborate on how, specifically, this is illegal? Thanks again!
Andrew — My understanding was that you couldn’t legally price discriminate without workarounds. Car dealers, for example, price at MSRP but negotiate down from there. The equivalent in the online world might be setting a single high price and offering different discount codes to different visitors.
However, a quick search revealed that the same rule doesn’t apply in the online world, and Amazon routinely does this: http://writ.news.findlaw.com/ramasastry/20050620.html
So while “not technically illegal,” it might still be something to consider if you have a close-knit user base or just want fairness in pricing.
Glad you brought that up.
I believe the fact that basecamp still uses freemium means something: They are using it not as a marketing tool but as a lock-in tool, and they won’t remove it because of the lock-in effect. Locking in is like a deferred buy, and you can’t ignore the value of that.
One other thing that strongly influences your experiment is pricing. What would be the difference if you had $9.99 less in some plans? I think there would be some potential on multi-variate testing with pricing as a new dimension to it.
Ash — Great post, thanks for sharing. It looks like you stopped offering Facebook Connect (unless the version of the page I’m looking at here http://www.getcloudfire.com/signup.php is part of a test). Why is that? I assume you tested it and/or are currently testing it?
Chris — We had to temporarily disable it because of a sign-up flow issue (cookies related) we uncovered with IE8. Once we correct that, it will be back.
I’m trying to reconcile the differences between your POV and Sean Ellis’.
I think first off it’s important to establish the goal: To get to product market fit.
Then that means the role of pricing is to maximize learning, which is how you will get to PMF fastest.
In some cases you need to charge users to maximize learning, otherwise they won’t take your product seriously enough to use it.
In other cases you learn more by letting users have access to everything, uninhibited.
I think it’s only useful to test price to maximize learning towards finding PMF, not for the reason to see if people will pay.
If people pay before PMF they are paying for a “nice to have” product by definition and that’s not a scalable, repeatable process. Their purchase is due to extraordinary circumstances, such as a hard sell by the founder, or the user was wealthy and didn’t mind paying for it after the trial was up, but ended up not sticking with the product.
“Will you pay for this?” is really just another way of saying do I have Product Market Fit – But it’s an inferior way of measuring PMF to the question, “Would you be disappointed if you could no longer use this?”. Paying customers are one way to measure if you have PMF but it’s a layer of abstraction above what you actually want. You can try to infer from pricing whether it means you have a must have product or not, but it’s harder to determine causality up a layer of abstraction.
It’s better just to measure directly whether people would be disappointed if they could no longer use it.
I think it’s also worth noting that you may have hit Product Market fit right off the bat with Cloud Fire. If someone is really good at Customer Discovery, which you are, that’s a possible scenario.
And in which case Sean would agree that you need to implement a business model and start charging right away.
Great post! I am facing this kind of problem right now and you gave me some nice insights.
To exemplify, currently, we have worked on two profiles of clients in our MVP, it is a kind of integration for a specific business. Actually we do not know how to charge yet: SaaS, fixed price, price by modules, freemium or percentual of transaction, free until market-fit. I do not know how to test all of them yet and collect relavant data, maybe it will cost too much too, instead I am thinking of choosing the one that makes sense based on previous experiences that we have right now. We runed some tests on percentual and it seems clients did not liked, freemium does not make many sense because our market is huge, modules is a too complicated solution right now, fixed price does not scale and we think that we could try to charge for the MVP.
Well, if we decide to advance with SaaS, we need to find the variables and ranges for the price. I am not sure how to do that right now, I will probably try to do tests on Web (I am not sure if we will get relevant data in our market – hypothesis) or direct interviews. If we go by direct interviews, we need to have a way of conduct and measure that, which I think is the hardest part. I will try to figure out the best way.
Thanks in advance.
Bruno
Hi Ash
It was really interesting to see your experiment.
May I ask you why you currently use “Freemium” despite “Single unlimited plan” performed better in the testing above?
http://www.getcloudfire.com/pricing2.html
Like most of the folks commenting here, we’re in a super competitive space. From day one we’ve been at a disadvantage by offering a subscription service. Most of our competitors are offering a lesser product for free. Now, with the explosion of free apps, it makes it even harder to gain subs.
Here’s how I see most lean startups. You’re not raising outside capital so you’ve got to charge very early in the game. Cash is king and free is a dream. And potentially focusing on free will kill you. Free needs to be subsidized by someone (angels, vc, strategics, cross-selling, etc). If someone is willing to pay $5/mo for your service you’ve, to some point, validated your service. The easiest thing in the world is to get free users. But it’s hard to keep them active.
What I think is lacking in these discussions is how serious you are about integrated SEO from the start. Google will make or break your service-period! You don’t have the luxury of being destination sites like twitter, foursquare, etc. In fact, no one will know about you except you, your engineer, mom and dad. So how you gonna go big?
Yes, validate early if anyone will pay. Then focus on getting found. Right now, that means Google. It’s totally backwards to focus on optimizing and tweaking when you’ve got no traffic. After six years in business, I tell you with conviction to focus on being found. If you have good traffic volume, a monkey can pick three successful business models. But if you have no traffic, you have a cute little site that maybe pays you a few grand a month.