We are sampling a lot of conversions and then comparing them. Basically the posterior distribution is a mapping of conversions, to probability. It tells us how likely it is to observe a certain conversion.
We sample from this distribution to get many conversion estimates. There will be more samples of conversions with high probability and fewer of those with low probability.
We sample for groups A and B. For each we have a numpy array of sampled conversions. Then we just check how many of conversions in B are better than in A.
Thank you for the post! Please explain source code where 95.7% got. What values do we compare here?
You can play around with the code in this collab:
https://colab.research.google.com/drive/1P6OybB5b-fpTQJZy2AD6ZF8q8yGzC5We?usp=sharing
Thank you!
Thanks for the feedback.
We are sampling a lot of conversions and then comparing them. Basically the posterior distribution is a mapping of conversions, to probability. It tells us how likely it is to observe a certain conversion.
We sample from this distribution to get many conversion estimates. There will be more samples of conversions with high probability and fewer of those with low probability.
We sample for groups A and B. For each we have a numpy array of sampled conversions. Then we just check how many of conversions in B are better than in A.