A/B testing is the only tool that allows you to test business hypotheses in a relatively fast, cheap and reliable way. Our training is based on the experience gained in creating an A/B testing platform for Allegro (the 10th largest e-commerce platform in the world). It teaches professional A/B testing, regardless of the technical platform used.

Examples of using A/B tests:

  • safe implementation of new profitable functionalities,
  • avoiding implementations that bring losses or no benefits,
  • evaluation of profits and losses from the implementation of these functionalities,
  • testing business hypotheses.

On the workshop you will

Learn from lecture supported by slides

Solve coding challenges

Complete practical exercises

During the workshop we cover:

  1. Understanding the Testing Problem
    1. data-driven organization
    2. business metrics
    3. the problem of testing changes
  2. Testing process
    1. designing the test
    2. performing the test
    3. interpretation of the results
  3. Building a hypothesis
    1. putting a solid hypothesis
    2. calculation of the required sample size
  4. Measurements
    1. metrics
    2. data segmentation
    3. data filtration
  5. Advanced testing problems
    1. multiple Tests
    2. defensive testing
    3. testing the sample distribution
    4. Simpson's paradox
  6. Statistical foundations
    1. the principle of operation of the statistical test
    2. counting tests for non-binary metrics
  7. Bayesian method
    1. what is the Bayesian method
    2. enrichment of classical reasoning by the Bayesian method
    3. using a Bayesian bandit in testing
  8. Advanced testing process
  9. Choosing the right platform


  • Experimenting process - from the basics to the advanced level + statistical basics
  • Online service measurement
  • Building a data-driven culture in an organization

About the course

The training consists of lectures, discussions and workshops. Trainers have extensive experience in all dimensions of testing: technical, business, product, scientific and organizational. They also have didactic preparation - they have trained hundreds of people internally.

Workshops have various forms, depending on the needs and skills of the participants. They can be based on creating an experiment or performing calculations for tests in the R package or Python.

The training is designed to build knowledge and skills incrementally. It starts with simple but useful processes and ends with advanced techniques.

After this course, you will be able to consciously use platforms such as Optimizely, Google Optimize, VWO. Additionally, for companies implementing A/B tests, we offer post-training consulting with our experts.

How long does it take?

The workshop lasts 2 days, but it may be shortened to 1 or extended to 3 days.

Bartłomiej Bęczkowski

Learn from the best

Bartłomiej Bęczkowski

Bartek is an experienced backend/data engineer, one of the founding fathers of the Allegro experimentation and optimization platform. Currently he is leading the Allegro MLOps transformation. Along the way he has taught people A/B testing and data engineering.


Choose this option if you need a workshop for your team or company. The basic price is 3500 EUR for the group for 2 days.

Private workshop

Choose this option if you need a workshop for yourself or for a small group of people. The basic price is 300 EUR per person for the whole workshop.

Open online workshop

Our experts offer consultations for both companies and individuals. The price is set individually.

Contact us