Remove Chad Remove Distribution Remove Metrics Remove Technical Review
article thumbnail

How to Segment A/B Test Results to Find Gold

ConversionXL

You might also notice that people using the Firefox browser hardly convert at all when they see variation B: this could mean that there are some technical issues with the front-end code that make the treatments not work on Firefox. This is especially true of big businesses: Chad Sanderson, Subway. Chad Sanderson, Subway.

Chad 112
article thumbnail

Lessons Learned: What does a startup CTO actually do?

Startup Lessons Learned

What does your Chief Technology Officer do all day? Often times, it seems like people are thinking its synonymous with "that guy who gets paid to sit in the corner and think technical deep thoughts" or "that guy who gets to swoop in a rearrange my project at the last minute on a whim." But along the way, something strange happened.

CTO 168
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

CXL Live 2018 Recap: Top 5 Lessons from Each Speaker

ConversionXL

Work hard to define meaningful product metrics – enabler of team success. Chad Sanderson – The Statistical Pitfalls of A/B Testing. In an A/A test the distribution of p-values is random. There’s no single metric to track, there are many: Leads. The reasons were all technical (bugs). Engagement.

Retention 106
article thumbnail

Lessons Learned: Continuous deployment and continuous learning

Startup Lessons Learned

Our tests suite takes nine minutes to run (distributed across 30-40 machines). Code reviews and pairing Great practices. Luckily, Chad Austin has recently weighed in with an excellent piece called 10 Pitfalls of Dirty Code. The high level of our process is dead simple: Continuously integrate (commit early and often).

article thumbnail

How to Structure Your Optimization and Experimentation Teams

ConversionXL

In decentralized (or “embedded”) teams, optimization responsibilities are distributed amongst employees under many different umbrellas within the organization, rather than one department. Chad Sanderson, the Experience Optimization Manager for Subway, creates an analogy for how the centralized model is set up: Chad Sanderson, Subway.

Chad 90
article thumbnail

CXL Live 2019 Recap: Takeaways from Every Speaker

ConversionXL

Go beyond standard sources of voice of customer data ; better options are: Interview founders (the original “customers”); Thank-you page surveys; Usertesting.com; Mine sales calls; Mine support tickets; Mine Facebook comments ; Mine online reviews. Setting up your experiments for success: Define success metrics. Do your research!