You have moved to Continuous Delivery and are delivering features fast. Unfortunately responsibility for features doesn’t end with hitting “deploy”, and when it comes to deciding how best to iterate on your work you are often stumped for what to do.
It’s not that you lack data, but rather that it is muddied by all the things that happened at the same time as the release – other simultaneous feature releases, or even just the difference in behaviour you see on the weekend. This was our experience.
Learn how we have adopted AB testing methodologies to understand feature impact, developing an inhouse tool to automate much of the process and test at the speed and scale Continuous Delivery allows.
Objective: Learn what AB testing is and how we at the Guardian have used it to understand feature impact. You can also expect practical advice on what is and isn’t suitable for testing, and see how it needn’t just apply to incremental changes to a product but may help with decisions about large feature changes and new products too.
Prerequisites: This talk assumes no knowledge of AB testing, but the audience might have an interest in using data to aid decision making, or may be experienced in these methodologies and have anecdotes they can share.