Rob Califf estimates that each time you do a Google search you are on average taking part in 3-4 randomised trials. E-commerce has made the randomised trial the cornerstone of website optimisation and a recent report in Trials demonstrated how a trial could be used to optimise the sign-ups for organ donor registration.
In the UK, 17 million people each year renew their car tax online and at the end of the process a request is made to join the organ donor register. This trial tested a number of different messages on signups for the organ donor register against the following control webpage:
The first message was of a social norms type and tested the message on its own (webpage B) and with an image (webpage C) or with the logo for NHS organ donation (webpage D).
The remaining four messages (E-H) tested simple messages on their own without images or logos:
|B||Social norms||Every day thousands of people who see this page decide to register|
|C||Social norms + image||Every day thousands of people who see this page decide to register + image|
|D||Social norms + logo||Every day thousands of people who see this page decide to register + logo|
|E||Loss frame||Three people die every day because there are not enough organ donors|
|F||Gain frame||You could save or transform up to 9 lives as an organ donor|
|G||Reciprocity||If you needed an organ transplant would you have one? If so please help others|
|H||Cognitive dissonance||If you support organ donation please turn your support into action|
The trial was run over a one month period. A randomisation code sequentially assigned individuals a number from 0 to 7 and displayed one of the 8 corresponding webpages shown in the table above. The main outcome measure was the number of sign-ups to the organ donor register.
During the study period, a total of 1,085,322 observations were made and the unadjusted odds ratios for the effect of the various messages on the number of sign-ups compared to control is shown in the table below:
|Unadjusted odds ratio [95% CI]|
|B||Social norms||1.255 [1.196-1.317]|
|C||Social norms + image||0.941 [0.894-0.990]|
|D||Social norms + logo||1.274 [1.214-1.336]|
|E||Loss frame||1.333 [1.271-1.397]|
|F||Gain frame||1.250 [1.192-1.311]|
|H||Cognitive dissonance||1.232 [1.174-1.293]|
All of the test messages resulted in statistically significant improvements in the number of sign ups with the exception of the social norms message with an image which resulted in a decrease in the number of sign-ups. The most effective message was the reciprocity message (“if you needed an organ transplant would you have one? If so please help others”) with individuals 1.38 times more likely to sign up to the organ donor register than controls. Following the results of the trial, the reciprocity message has been implemented across 25 government websites.
You can read the full report here.