A live mic. They gave me a live mic at last week’s Upfront Summit to interview Reddit’s Alexis Ohanian. Our discussion lasted about 20 minutes but that was sufficient time to cover his open letter on immigration, Reddit’s community standards, his feelings about tech’s role in politics and the opportunity he now has to become more than just “tech famous” if he so desires. But there was one question I’d written down in my notebook that we didn’t have time to cover – did Alexis think the way we measure the success of social products unintentionally result in deprioritizing community safety.
What are the standard KPIs for social, content and community products? The most common for a daily dashboard are a mix of DAU/MAU, page views, impressions, session length, clickthrough, visits per day, engagement. The more sophisticated derivatives of these are retention-related.
What do these measures mostly tell us? That optimizing for clicks, visits, posts and time on site = good. But what happens when these metrics lead us towards more engagement but less happiness?
In the last few months it sure looks like disagreement and flame wars produce more in situ engagement than a simple “like” or read. Your friends rushing to tell off the asshole who just tweeted at you sure looks like a lot of activity when it’s reduced to a daily graph. A hate share of post that makes you go WTF sure does look like virality. Look a lot like a successful product.
But does it make you feel good? Does it make think better or worse of the product you just experienced? Does it increase your NPS for that business? Trolls and harassment drive short-term engagement even though they’re long-term poison.
A few months back I had the chance to speak with the News Product Managers at Facebook. It was when “Trending” felt very TMZ-like (it has since improved in many ways). I told them I clicked on that section a lot but it always left me feeling guilty and shitty for having read some celebrity gossip or inflammatory headline. I clicked more and more until one day I stopped clicking on that section at all. I stopped looking at it. I wished there was a [x] I could use to close the section. It was a piece of candy that Facebook kept putting in front of me. I started off by blaming myself for eating it but you know what, after a while Facebook was to blame as well. Didn’t they care about my health? Didn’t they want the best for me? Then stop giving me candy.
My guess is the best product designers are going to start thinking more and more about the emotional impact of a product feature rather than just the blunt instrument of “did the user interact.” And the next generation of user measurement tools are going to use machine learning to tell us when someone is having a “high engagement, but low quality” experience and try to rescue them before they abandon. Because today I just look like a highly engaged user until the day I stop coming at all.