A/B testing for e-business

Make effective A/B testing to verify business hypothesis for your company.

A/B testing is the only tool that makes it possible to test business hypotheses in a relatively fast, cheap and reliable way.

Examples of how A/B testing can be used:

  • safely implementing new functionalities that are profitable,
  • avoiding implementations that bring losses or no benefits,
  • evaluating the gains and losses of implementing these functionalities,
  • testing business hypotheses.

Perfect for:

  • Data scientist,
  • Data analysts,
  • Technical managers.

On the workshop you will

Learn from lecture supported by slides

Group discussions

During the workshop we cover:

  1. Why do you need A/B testing?
  • Sequential testing
  • Split testing without the statistical regime
  • Split testing with the statistical regime
  1. Building a hypothesis and interpreting test results
  • Experimentation process and hypothesis
  • Understanding false positives and false negatives
  • Calculating sample size
  • Calculating and interpreting the p-value
  1. Advance hypothesis testing
  • Multivariate testing
  • Multi-metric experiment design
  • Segmentation
  • Experiment influence lifespan
  1. Data quality
  • Traffic allocation
  • Filtering noise
  • Automatic error detection by sample distribution uniformity checking
  • Simpson’s paradox
  • Cross-device consistency
  1. Frequentist statistics
  • Core assumptions and implications
  • Chi-squared
  • Welch’s t-test
  • Sample size reduction – CUPED technique
  1. Bayesian approach to A/B testing
  • Cons of the frequentist approach
  • Interpreting Bayesian experiment results
  • Pros and cons of the Bayesian approach
  1. Fully automated optimization with multi-armed bayesian bandits
  • Multi-armed bandits use cases
  • How it works mathematically
  1. Implementing A/B testing in an organization
  • Building an A/B testing platform
  • Building a data-driven culture
  1. Final discussion

About the course

The workshop gives you a statistical, technical, and organizational toolbox for implementing A/B testing in a medium to large organization. It is a lecture mixed with discussions, in the context of specific client cases. We run this workshop only for businesses, not individuals.

How long does it take?

It takes 1-2 days, depending on the client's needs and experience.

Requirements

  • Before each workshop, the trainer meets with the group to gather requirements and understand the context.
Bartłomiej Bęczkowski

Bartłomiej Bęczkowski

Bartek is an experienced backend/data engineer, one of the founding fathers of the Allegro experimentation and optimization platform. Currently he is leading the Allegro MLOps transformation. Along the way he has taught people A/B testing and data engineering.

Register

Choose this option if you need a workshop for your team or company. The basic price is 2500 EUR for the group for 1 days.

Reserve now

Choose this option if you need a workshop for yourself or for a small group of people. The basic price is 200 EUR per person for the whole workshop.

Reserve now

Our experts offer consultations for both companies and individuals. The price is set individually.

Contact us