Evidence-based Advertising: Frequently asked questions
(click question for a more indepth answer)
Updated January 2013
J. Scott Armstrong, The Wharton School
Kesten C. Green, University of South Australia Business School
What is the purpose of advertisingprinciples.com (AdPrin.com)?
Answer:To improve the effectiveness of advertising though evidence-based principles for persuasion.
The purpose is to summarize all useful knowledge on how to persuade through advertising and to present it in an accessible manner. The knowledge is presented as principles (condition/action statements) that have been drawn primarily from experimental evidence. They are easy to understand and can be used by many groups, including:
- Advertising agencies gain by being able to draw on validated principles for designing and improving ads, and also by being better able to determine which ads are most effective.
- Advertisers gain by being better able to prepare their own persuasive ads, and to select agencies that are familiar with the research on persuasion.
- Government agencies can use these principles to communicate more effectively with the public and to help insure that their regulations on advertising do not harm producers or consumers.
- Students gain as the principles are presented in everyday language so that they can learn how to design persuasive ads and to prepare persuasive management reports.
- Researchers gain as they may be able to find references relevant to their current research. In addition, they can also get ideas about important areas that are in need of further research, as, for example, in the list of “Ten principles in need of experimental evidence” in Table 5 of “Evidence-based Advertising.”
Who started AdPrin.com and why?
Answer:Professor J. Scott Armstrong, who has been teaching advertising at the Wharton School since the early 1970s, started AdPrin.com to help his students.
Professor Armstrong began developing a website for his course in the late 1990s. In keeping with his philosophy on education, the site is concerned with “evidence-based advertising.” As people from outside the Wharton School began visiting the site, he decided to also make it relevant to advertisers, advertising agencies, consumers, and researchers.
What is evidence-based advertising?
Answer:Evidence-based advertising refers to using evidence to derive principles for advertising.
The most important evidence comes from experimental studies that compare the effectiveness of alternative advertisements. In other words, evidence-based advertising is based on the scientific method of testing multiple reasonable hypotheses. The approach was described in Chamberlin 1890.
Where does experimental research in advertising come from?
In the early 1900s, advertising agencies sometimes published their research findings. Since the 1950s, academia has produced nearly all of the experimental research on advertising. Academia encourages publication of such research, as well as full disclosure of the data and experimental methods.
Is evidence-based advertising a new development?
David Ogilvy provided the inspiration for the use of principles. In Ogilvy on Advertising, he provided many generalizations, such as “Do not put a period at the end of a headline.” He did not, however, specify conditions. For example, “Do not put a period at the end of a headline when you want the person to read the body copy.” Experimental research has been used to identify such conditions. Fortunately, much useful evidence has been produced over the past century. The principles in Persuasive Advertising represent the first attempt to summarize knowledge on how to make advertising effective.
Isn’t advertising just common sense?
Many of the principles are counter-intuitive. If you are not persuaded, take the five-minute “Test your advertising IQ” exercise. If you do well, congratulations! On average, people do not do better than if they had simply guessed.
Is the AdPrin.com site recognized as a useful source of advertising knowledge?
The site is first out of about 77,000 sites in a Google Scholar Search for “evidence-based” and “advertising.” The site received the MERLOT Award for the “Best Internet Site in Business Education” in 2004. It has been improved substantially since then and is currently the highest-ranked advertising site on MERLOT.
What do you mean by the effectiveness of advertising?
Answer:Effective advertising is advertising that elicits the desired response (e.g., sales, votes, donations, better behavior, and loyalty in a cost-effective manner.
How should the effectiveness of advertising be measured?
Answer:Return divided by the cost of the ad.
For businesses, the objective of persuasive advertising is to produce a good return on investment (ROI)—in other words, the gross margin obtained from producing and disseminating the advertisement. For non-profits, it depends on the objectives – such as votes per dollar spent for politicians or dollars of donations per dollar spent for charities.
Aren’t there other reasons for advertising than to get a return on the investment?
Answer:Not if you like to spend your money wisely.
In 1905, L. Thomas, head of the largest U.S. advertising agency said, “The key is ‘Does it sell?’ What’s the profit of the ad investment?” This objective has not changed over time.
Why will evidence-based advertising lead to more effective advertising?
Answer:An evidence-based approach is the only way that disciplines concerned with complex uncertain situations can advance.
Think about agriculture, engineering, and medicine, for example. Without experimentation there is no progress. As far as we know, there is no other way that advertising can improve. For a discussion on this, see “Evidence-based Advertising.”
Aren’t evidence-based principles already available in the leading advertising textbooks and handbooks?
To assess the extent to which evidence-based principles are available from other sources, we examined nine popular advertising textbooks and three popular handbooks for advertising practitioners. We did not find any evidence-based advertising principles in these sources. This was because these books:
- failed to describe conditions, and
- seldom used experimental evidence. This analysis is described in “Evidence-based Advertising.”
Surely top advertising experts know what works, so aren’t their opinions a good guide?
Answer:Advertising experts’ key contribution is to suggest creative ways to persuade people.
Top experts such as Bill Bernbach and David Ogilvy were both highly creative and effective. That is because they relied not only on their judgment, but also on research. Their insights provided starting points for developing principles. But it is only through experimentation that the effects of conditions can be assessed.
How can experts design more effective advertising?
Answer:By using the checklist of evidence-based advertising principles.
Given the many possible approaches, checklists serve as a valuable tool for complex problems (See Arkes, Shaffer and Dawes 2006, Shaffer and Dawes 2006 and Haynes et al. 2009). Experience cannot substitute for checklists—as airline pilots know.
Will experience in advertising help to improve my ability to identify effective advertising?
Answer:It will help in simple situations where people use good feedback.
In general, it is difficult to learn much from doing a task as complex as advertising, especially when lacking accurate feedback about what works in comparison with alternative approaches. People who work on direct response advertising get accurate and rapid feedback so they can learn when they systematically track what works. In other words, their experience can be informed by experimental evidence.
People differ and advertisements involve many features. It is therefore difficult to predict the effectiveness of an ad. To see if you have a knack for this, try the short Predict Which Ad Pulled Best test. On average, people do only slightly better than guessing. The “Seer-Sucker Theory” demonstrates people’s inability to use their judgment to make accurate predictions in complex and uncertain situations. The book, Expert Political Judgment, provides further evidence that supports this theory.
AdPrin.com lists 195 principles. Wouldn’t it be more practical to identify the top ten principles?
The conditions for each advertisement (e.g., the objectives, the type of product, and the type of consumer) differ substantially, and the proper application of principles depends on these conditions. Medicine offers an analogy: would it be sensible to pick the top ten drugs and use only those to treat all diseases?
How can I find out which of the advertising principles are relevant to my ad?
Answer:Use the AdPrin Audit.
The AdPrin Audit can guide you to:
- describe the conditions that you face (conditions are based primarily on your objectives, the type of product, and the target market), and then
- use your judgment and the descriptions of the principles to select relevant principles.
Are advertising principles difficult to use?
People with no background in advertising can complete the on-line self-training program in about 90 minutes. They can then use the AdPrin Audit software to rate ads, which usually takes about 30 minutes per ad. Their ratings allow for more accurate predictions of which ads will be more effective than those made by advertising experts. The ratings also lead to suggestions on how to improve the ads.
Does it help to analyze non-experimental historical data to learn which advertising principles have been effective in the past?
Answer:Only in simple situations where there is much data.
For the most part, advertising is complex. When situations are complex, it is difficult to determine which procedures are effective by analyzing non-experimental data; this occurs because advertisers vary their approaches based on the conditions. In addition, many analysts misuse regression for such situations and reach false conclusions. See “Illusions in Regression Analysis.” Moreover, as shown in Using Quasi-experimental Data to Develop Principles for Persuasive Advertising, the findings from analyses of non-experimental data often differed sharply from experimental findings.
Do advertising principles change over time?
Answer:Yes, with respect to our knowledge of principles. No, with respect to changes in the way that people respond to persuasion principles.
Many of the persuasion techniques were developed long ago. For example, Aristotle advised that two-sided appeals are persuasive. Persuasive Advertising traces the history of many of the principles. In the past century, researchers have been using experiments to identify the conditions that affect the application of principles. So the principles do change in the sense that we learn more about how to most effectively use them. However, we have no evidence to suggest that, over time, people respond differently.
Do advertising principles differ across countries and cultures?
Based on the evidence to date, they seem to apply to all people in the same way. A German edition of Persuasive Advertising has been published and a Chinese translation is underway.
Can I be confident that the principles are correct?
Answer:Yes, in the sense that they represent the state of knowledge to date.
The principles are based on all available evidence, and the interpretation of the evidence was peer reviewed by many researchers. Attempts were made to contact all of the researchers whose work was used to develop the principles in order to ensure that their findings were described accurately. Most researchers responded with useful suggestions. This interaction with other researchers was one of the reasons that the Persuasive Advertising book went though 272 revisions to ensure that it was correct. We stand ready to make corrections if need be.
Is the list of principles complete?
Answer:No. The 195 principles are just the end of the beginning for evidence-based advertising.
We expect that new principles will be discovered and that most of the principles will be revised as further studies reveal better procedures and a better understanding of the conditions under which the principles are most effective. We might even find that some principles are not very useful. We ask researchers to send us relevant findings, especially if the findings disagree with current principles. The new findings will be posted on AdPrin.com.
Might the principles be used to deceive customers?
Answer:The principles are concerned with maximizing the long-term profitability of an advertisement, not with short-term deception of consumers.
In free-market economies subject to the rule of law, when advertisers have the objective of maximizing long-term profitability and understand that to do so requires them to treat stakeholders well, there is little incentive to deceive potential buyers. In other words, the long-term success of the advertiser depends on their ability to provide buyers with benefits that are competitive with the benefits offered by other suppliers.
Moreover, it is illegal to disseminate deceptive advertising, and there are harsh penalties in law and in loss of reputation for advertisers who harm buyers. Buyers are naturally skeptical of advertising, as they should be, in their search for products that will maximize their own welfare. Interestingly, government attempts to increase protection for consumers by restricting free speech for advertisers have failed to improve on the outcomes delivered by free markets and long-standing legal remedies. For a review of the evidence see “Evidence on the Effects of Mandatory Disclaimers in Advertising.”
If every advertiser uses the principles, won’t ads look the same and wouldn’t that be boring?
One purpose of the principles is to expand opportunities for creativity. Given that many principles are being ignored or violated, the use of the principles will lead to ads that differ substantially from current advertisements. In addition, applying the principles themselves requires much creativity. For example, if a principle suggests that the use of humor would be effective for a product, different ad creators are unlikely to use the same humor. Finally, there is nothing wrong with a boring advertisement that has a better ROI than alternative ads.
Isn’t the key issue whether the consumers will like an ad?
Answer:Likeability has little relationship to effectiveness and should not, therefore, be an objective of advertising.
Under many conditions, ads that are disruptive or provocative can be more effective than likeable ads. That said, in some situations, “liking” might be an appropriate element in the design of an ad.
Has advertising been getting more effective over time?
Fields typically reach a point at which progress ends unless experimental evidence is used. Based on surveys, many experts and customers believe that the effectiveness of advertising reached its peak in the 1960s and has declined since then. (Sources are provided in Persuasive Advertising, p. 1)
Are big gains in effectiveness possible from using persuasion principles?
Evidence-based advertising can help in the design of effective ads and in selecting which of a set of alternative possible ads would be most effective. It helps to address John Wanamaker’s famous problem: “I know that half of my advertising is wasted, but I do not know which half.” We now know which half, as explained in this letter to Mr. Wanamaker. (See Predictive Validity of Evidence-based Advertising Principles for supporting evidence)
Given the high benefits and low cost, will evidence-based advertising be rapidly adopted?
Business practices are resistant to change because the complexity of business often obscures the effects of both good and bad practices. Fields such as engineering and agriculture are more amenable to evidence-based approaches because it is easier to test innovations and obtain clear results. Medicine was like business: medical practitioners resisted findings from early experiments because they conflicted with their opinions and practice. The 1925 novel, Arrowsmith, described the culture in medicine at that time. The classic example is Semmelweis’s finding in the 1840s that when doctors washed their hands before inspecting pregnant women, the death rate of the women was dramatically reduced. Practitioners resisted his finding for decades. After 1930, the evidence-based approach became increasingly more important because doctors who did not use such procedures could be sued successfully, as is the case for engineers. The evidence-based movement in management is just getting underway, thanks to work by people such as Denise Rosseau and Jeffrey Pfeffer. See the Wikipedia entry on “Evidence-based Management” and EBMgt.
What are the long-term prospects for the widespread adoption of evidence-based advertising?
Answer:Given the large gains that are possible, ad agencies that offer evidence-based advertising and advertisers that request it will eventually replace firms that fail to adopt the approach.
Are the principles useful outside of advertising?
Answer:Yes. They are based on research on effective persuasion from many disciplines, and they apply to many fields in which persuasion is useful.
A 1995 article in the American Economic Review estimated that the “persuasion business” represents about 25% of the Gross Domestic Product. Persuasion is important in law, education, research, politics, and medicine, in addition to such tasks as getting a job, writing a persuasive management report, and developing personal relationships. They are especially useful when they differ from current beliefs about persuasion. For example, when trying to change someone’s opinion, you should be indirect and gentle. This helps to explain why arguments in face-to-face discussions are generally not persuasive, especially when one has stronger logic, better facts, and better delivery.
How can I keep up with new findings about the principles?
Answer:Visit the AdPrin.com site and, in particular, the “New Contributions” and “Evidence on Principles pages".
The objective of the AdPrin.com site is to provide all useful knowledge about persuasion through advertising. Useful and relevant information is difficult to find in academic literature because it is spread across many journals. In addition, the titles and abstracts of the articles are often uninformative. Once relevant articles are found, they are usually difficult to understand, and seldom does a single article provide a sound basis for a principle. At AdPrin.com, we jump through those hurdles in an effort to make all useful knowledge from research accessible to practitioners, researchers, and students.
What is the most important contribution on AdPrin.com?
Answer:The AdPrin Audit.
After less than 2-hours with the self-training software, subjects can provide ratings, typically in half an hour per ad, that lead to useful predictions of the effectiveness of an ad. Their ratings also lead to suggestions for changes in the ad. (See Predictive Validity of Evidence-based Advertising Principles.).
How can I learn to use evidence-based advertising?
Answer: AdPrin.com offers 12 ways to learn.
This list starts with the most effective ways to learn:
- Apply the principles to designing ads, and get feedback from others regarding your success in applying the principles.
- Conduct AdPrin Audits for a number of ads and describe how to improve them.
- Engage in experiential exercises. In other words, try to solve an advertising problem without assistance, and then learn about other ways to solve the problem, and apply them.
- Read Persuasive Advertising. This book provides a full description including all conditions, along with the evidence.
- Do self-administered practice exams.
- Plan applications of the principles by using the principles checklist for problems. Set a goal on how many principles to apply.
- Listen to lectures about the principles on the AdPrin.com site, and take notes on how to apply them. Ask questions that will help you to plan applications.
- Use the lectures for self-guided learning. Select the “Slide Show” format, and write your answers as instructed. Then click for the evidence-based answers.
- Explain to others how to apply various principles.
- Set up a program to use the “Principle of the Day” on AdPrin.com
- Study the examples of ads that are successful (or unsuccessful) at using the principles.
- Keep up with the news on AdPrin.com
How is the site funded?
Answer:Support is provided mostly by the efforts of volunteers and donations by Professor Armstrong.
In the early years, the web design and maintenance expenses were funded by The University of Pennsylvania’s Wharton School’s Marketing Department, and it continues to provide some IT support for the educational materials on the site. However, the outreach activities have led to additional expenses. Since 2006, most of the funding has come from Professor Armstrong’s research budget at The Wharton School and from his personal funds. The directors of AdPrin.com, Scott Armstrong and Kesten C. Green, have volunteered their time, as have others, including Rui Du and Arry Tanusondjaja, who helped to develop the AdPrin Audit. Royalties from sales of Persuasive Advertising are being used to support the site, so ask yourself: Would you consider buying a copy to help others as you help yourself?” While it currently sells for as little as 30 cents per principle, the cost to produce each principle was amazingly high. Fortunately the benefit per principle is also high.