Practical guide to controlled experiments on the web: listen to your customers not to the hippo

@inproceedings{Kohavi2007PracticalGT,
  title={Practical guide to controlled experiments on the web: listen to your customers not to the hippo},
  author={R. Kohavi and Randal M. Henne and D. Sommerfield},
  booktitle={KDD '07},
  year={2007}
}
  • R. Kohavi, Randal M. Henne, D. Sommerfield
  • Published in KDD '07 2007
  • Computer Science
  • The web provides an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called randomized experiments (single factor or factorial designs), A/B tests (and their generalizations), split tests, Control/Treatment tests, and parallel flights. Controlled experiments embody the best scientific design for establishing a causal relationship between changes and their influence on user-observable behavior. We provide a practical guide to conducting online experiments… CONTINUE READING

    Citations

    Publications citing this paper.
    SHOWING 1-10 OF 307 CITATIONS

    Experimentation Pitfalls to Avoid in A/B Testing for Online Personalization

    VIEW 12 EXCERPTS
    CITES BACKGROUND & METHODS
    HIGHLY INFLUENCED

    Demystifying dark matter for online experimentation

    Seven rules of thumb for web site experimenters

    VIEW 2 EXCERPTS
    CITES BACKGROUND

    FILTER CITATIONS BY YEAR

    2007
    2020

    CITATION STATISTICS

    • 26 Highly Influenced Citations

    • Averaged 18 Citations per year from 2018 through 2020

    References

    Publications referenced by this paper.
    SHOWING 1-6 OF 6 REFERENCES

    Statistics for Experimenters: Design, Innovation and Discovery

    • J. Pigeon
    • Mathematics, Computer Science
    • Technometrics
    • 2006
    VIEW 5 EXCERPTS
    HIGHLY INFLUENTIAL

    Design of Experiments Using the Taguchi Approach: 16 Steps to Product and Process Improvement

    VIEW 9 EXCERPTS
    HIGHLY INFLUENTIAL

    A Primer on A/B Testing

    • s.l. : Forrester Research,
    • 2004
    VIEW 6 EXCERPTS
    HIGHLY INFLUENTIAL

    Putting A/B Testing in Its Place. Useit.com Alertbox

    • [Online] Aug
    • 2005
    VIEW 4 EXCERPTS
    HIGHLY INFLUENTIAL

    Statistical Design and Analysis of Experiments With Applications to Engineering and Science. s.l

    • 1989
    VIEW 6 EXCERPTS
    HIGHLY INFLUENTIAL