Metadata
- Author: Itamar Faran
- Full Title:: Why You Should Switch to Bayesian a/B Testing
- Category:: 🗞️Articles
- Document Tags:: Bayesian testing,
- URL:: https://towardsdatascience.com/why-you-should-switch-to-bayesian-a-b-testing-364557e0af1a
- Finished date:: 2024-05-06
Highlights
The Bayesian framework has a completely different point of view. We start by saying “the click rate of the red and green buttons can be any rate between 0% and 100%, with equal chance” (this is what we call the prior). This means that each button initially has a 50% chance to be better than the other. As we start gathering data, we update our knowledge, and we can say things like “Given the data I have observed, I now think there is a 70% chance that the green button is better”. We call this the posterior (View Highlight)
However, Bayesian A/B testing does have a metric that does not have a parallel in the frequentist framework: The Risk. We calculate the risk for both A and B, and it’s interpretation is: “If I chose B when B is actually worse than A, how much will I lose?”. This metric is also used as a decision rule in the A/B test. (View Highlight)