Practical Aspects of Sensitivity in Online Experimentation with User Engagement Metrics

@inproceedings{Drutsa2015PracticalAO,
  title={Practical Aspects of Sensitivity in Online Experimentation with User Engagement Metrics},
  author={Alexey Drutsa and Anna Ufliand and Gleb Gusev},
  booktitle={CIKM},
  year={2015}
}
Online controlled experiments, e.g., A/B testing, is the state-of-the-art approach used by modern Internet companies to improve their services based on data-driven decisions. The most challenging problem is to define an appropriate online metric of user behavior, so-called Overall Evaluation Criterion (OEC), which is both interpretable and sensitive. A typical OEC consists of a key metric and an evaluation statistic. Sensitivity of an OEC to the treatment effect of an A/B test is measured by a… CONTINUE READING
Tweets
This paper has been referenced on Twitter 6 times. VIEW TWEETS

From This Paper

Figures, tables, and topics from this paper.

References

Publications referenced by this paper.
SHOWING 1-10 OF 21 REFERENCES

Similar Papers

Loading similar papers…