abtesting   3799

« earlier    

The second Ghost of Experimentation: The fallacy of session based metrics
However in an A/B test it is common to allocate users into either group A or B to avoid the same person seeing both versions and disrupting the user experience. Sessions are then not independent since one user can have multiple sessions, and what happens in one session can influence, or change our expectation, of what happens in the user’s other sessions. For example, at Skyscanner we see that a user is more likely to convert if they have more sessions:
abtesting  statistics 
2 days ago by 1luke2
A design is not an answer. It’s a hypothesis. Forgetting this leads to arrogance. Failing to communicate it leads to disappointed stakeholders.
tweetstorm  Design  abtesting 
12 days ago by 1luke2
How annual revenue increased by $50 million using Online Experiments – Methods and tricks | The #Shell
Online tests can help managers figure out how much investment in a potential improvement is optimal. This was a decision Microsoft faced when it was looking at reducing the time it took Bing to display search results. Of course, faster is better, but could the value of an improvement be quantified? Should there be three, 10, or perhaps 50 people working on that performance enhancement? To answer those questions, the company conducted a series of A/B tests in which artificial delays were added to study the effects of minute differences in loading speed. The data showed that every 100-millisecond difference in performance had a 0.6% impact on revenue. With Bing’s yearly revenue surpassing $3 billion, a 100-millisecond speedup is worth $18 million in annual incremental revenue—enough to fund a sizable team.
abtesting  Performance  sitespeed 
4 weeks ago by 1luke2
GitHub - markphelps/flipt: A feature flag solution that runs in your existing infrastructure
:triangular_flag_on_post: A feature flag solution that runs in your existing infrastructure - markphelps/flipt
golang  abtesting  featureflag 
4 weeks ago by geetarista
Growth Hacking for 2019 ⚡️🤓: 42 simple, must-use digital marketing tactics to power your business
First, while A/B tests are amazing, it takes 50–100k visitors per month just to get significant test results for even a couple hypotheses. My old buddy and CRO expert Daniel Roffman recommended I use “an ICE scoring to prioritize what you work on. The challenge isn’t ‘can’ we do it,” Roffo explains. “Rather it’s are we working on the right things that will have tangible impact.”
I love this idea. Using scrum tactics for to get your whole team involved in collaborative product management not only results in the best product, but also a happy and energized team. ICE comes from a similar philosophy.
Sean Ellis (the inventor of the term “growth hacker”) came up with the concept of ICE. It stands for the “Impact, Confidence, and Ease” a test or experiment represents. Get your team together to compile all the desired tests and then collectively rate each one. Use the cumulative score to prioritize.
Growth  SaaS  marketing  abtesting 
5 weeks ago by 1luke2

« earlier    

related tags

a/b  ab  abtest  advertising  algorithm  analytics  android  api  architecture  away  bayesian  bestpractices  blog  cloudflare  commentary  conversion  coordinator  copywriting  criticism  cro  data  dataleakage  deployment  design  devops  divi  ecommerce  edge  email  essay  examples  experiments  facebook  featureflag  featureflags  features  filamentgroup  flags  gem  golang  google  googleanalytics  growth  hn  howto  hypothesen  interesting  ios  javascript  js  kalzumeus  kpis  kubernetes  library  marketing  merchandising  multiarmbandits  multiarmedbandit  navigation  neattrick  omclslack  onpremises  opensource  optimization  optimizely  payments  paypal  performance  persuasion  php  post  ppc  privacy  product  psychology  rails  research  rhp  ruby  run  saas  sales  science  script  security  seo  server-side  servers  service-worker  sitespeed  slack  split  splittesting  startup  startups  statistics  swift  templates  test  testing  third-party  tutorial  tweetstorm  ui  usability  user  ux  web  webadmin  webdesign  webdev  webexperiments  workers 

Copy this bookmark: