A-B Test

aka Split Test Experiment

attempting to apply Scientific Method of Hypothesis Testing to Product Development or Marketing actions (Sales Funnel)

often need at least 2 Metrics to consider

• side-effects
  • these are often harder/slower to measure
  • if you keep removing olives you end up with lettuce
  • leads to enshittification?
• "conservation of intent" https://andrewchen.co/conservation-of-intent/

If you don't start from a hypothesis, are you just data dredging?

fancier analysis than Web Traffic tools

tools http://mashable.com/2010/11/04/a-b-testing-resources/

• Feature Toggle

http://b.lesseverything.com/2011/1/28/how-we-refine-our-site-s-content

http://www.quicksprout.com/2013/09/09/7-simple-ab-tests-that-can-increase-conversions-by-10-or-more/

Sample Size calculator: statsig.com or optimizely

Patrick McKenzie provides great details, uses Mix Panel

• his free A Bingo code
• 2010-10-31-MckenzieRailsAbtestAnalyticsCode
• example of testing logo alternatives