Marketing   293477

« earlier    

p-Hacking and False Discovery in A/B Testing by Ron Berman, Leonid Pekelis, Aisling Scott, Christophe Van den Bulte :: SSRN
We investigate whether online A/B experimenters "p-hack" by stopping their experiments based on the p-value of the treatment effect. Our data contains 2,101 commercial experiments in which experimenters can track the magnitude and significance level of the effect every day of the experiment. We use a regression discontinuity design to detect p-hacking, i.e., the causal effect of reaching a particular p-value on stopping behavior.

Experimenters indeed p-hack, especially for positive effects. Specifically, about 57% of experimenters p-hack when the experiment reaches 90% confidence. Furthermore, approximately 70% of the effects are truly null, and p-hacking increases the false discovery rate (FDR) from 33% to 42% among experiments p-hacked at 90% confidence. Assuming that false discoveries cause experimenters to stop exploring for more effective treatments, we estimate the expected cost of a false discovery to be a loss of 1.95% in lift, which corresponds to the 76th percentile of observed lifts.
testing  research  statistics  marketing 
5 hours ago by Chirael
Twitter
Pinterest Ads: A Step-by-Step Guide To Your First Promoted Pins Campaign by
pinterest  marketing  from twitter
7 hours ago by WeRank
Weekend musing – A simple Kill Switch solution to the mess that is Facebook
"Facebook can't seem to catch a break in the court of public opinion but there may be a simple solution to its woes. Bring on the Kill Switch."
content  marketing  customer  service  management  regulation  social 
10 hours ago by jonerp
100+ Google SEO Success Factors, Ranked Ξ Zyppy
Les facteurs de classement de Google, classés par ordre.
marketing  webdev  guide 
15 hours ago by alssanro

« earlier    

related tags

$2b+  201807  644  a/b  aaa  acceptableusepolicy  activismmodel  adobe  ads  advertising  advocacy  ai  alibaba  amazon  analyse  analytics  and  android  api  appstore  ar  art  article  artificialintelligence  attention  automation  bestof  blog  blogging  book  boosts  bot  brand  branding  bullshit  business  career  casestudy  changemanagement  chart  chatbots  code  coding  comparison  competitive-research  consumer  content  contentmarketing  conversation  copywriting  corporate  create  crisismanagement  culture  customer  customerservice  dashboard  data  data_studio  design  digital  directmail  directory  ebay  ebook  ecommerce  email  embed  entrepreneurship  environment  eu  europe  examples  experience  facebook  feminism  film  firm  focus  free  glossary  google  government  graphic  graphics  growthhacking  gtm  guide  hacks  history  hn  howto  images  in  influence  influencer  influencers  innovation  investment  ios  its  job  kindle  law  layout  linkedin  list  manage  management  market-research  marketer  marketers  marketing  marketplace  martech  media  ml  mobile  modern  nasa  newsletters  offline  open-source  optimisation  outdoor  photographers  photography  pinterest  pixel  plugins  politics  preis  preisfindung  pricing  privacy  product  productivity  programmatic  promise  publicrelations  publishing  purpose  reach  reference  regulation  research  resources  retail  review  role  sales  security  selling  seo  service  shopping  slideshare  social  social_media  socialmedia  socialproof  software  space  speaker  startup  startups  statistics  stats  storytelling  targeting  teams  tech  techcrunch  technology  testing  tips  tool  tools  traffic  trend  tutorial  tv  twitter  uk  video  vr  webdev  webdevelopment  webmarketingfree  websites  with  wordpress  writing   

Copy this bookmark:



description:


tags: