Evaluating Advertisements

Here are methods to make comparisons of responses for two or more alternative advertisements that are considered for a given situation (i.e., for a given brand, product, market and media). They are listed here from least expensive to most expensive:

1. Adprin Audit: Rate how well ad applies evidence-based persuasion principles. (Also provides advice on how to use principles to also improve each ad.)
2. Copy testing: Test which ads gain the best response. Ignore “liking.” Allows for testing the components of the ads (e.g., headline alone or illustration alone.
3. Field experiment: Test direct-response ads to see which are most effective.

Combine the results from two or more approaches to increase predictive validity.

Be conservative – and humble: Predictions of advertising effectiveness are very difficult. There is an extensive literature showing that people are overconfident. The "letter-F test" illustrates the findings. They apply to all of us.

In addition to testing effectiveness, ensure that your ads follow Ethical guidelines
and Advertising education forum and that they are legal US advertising and marketing laws

Unaided ratings of effectiveness by consumers or experts

Decisions about which ad to use are often based on an expert’s unaided judgments about which ads from a set will be most effective. By “unaided,” we mean that the judgments are not aided by formal evidence-based forecasting procedures. One of the most powerful findings from research on forecasting is that combining independent anonymous forecasts improves accuracy (Armstrong 2001). Structured combining procedures are likely to improve decisions over which ad to run. The primary shortcomings of combining expert opinions are that the procedure provides no advice on how to improve ads and, most important, there is a weak relationship between consensus and effectiveness. Given that, it helps to get opinions from many people, and to focus on experts if inexpensive.

Read More

Adprin Audit^TM

The AdPrin Audit examines the extent to which a given advertisement follows the advertising principles. Its primary purpose is to determine the most effective ad among a set. It can also lead to suggestions for improving ads. Finally, it can help people learn how to design and improve ads.

The audit provides an effectiveness score, the AdPrinIndex (or API), based on how well the principles are applied. This simple approach is in line with recent research on the use of the “index method” (see Armstrong and Graefe, 2010). The score's primary benefit is to compare ads for the same product; for example alternative ads for a given campaign, or ads between competitors for the same product category and target market. It also provides a Weighted API-W; the weights, which are applied to each principle, are based on three sources: 1) coder’s judgment that a principle is especially important in the situation, 2) the amount and quality of prior evidence, and 3) the effect size based on prior published research (along with judgments by Scott Armstrong).

If you are new to the Adprin Audit, go to the Self-training module. Otherwise, go to Rate Ads, directly below.

Rate ads

There are Excel spreadsheets for “Still” (print or internet) and “Motion” (radio, TV, and Internet) ads. When feasible, it is recommended that multiple raters be used for each ad, especially when the raters are inexperienced. The Administrator’s Summary helps to summarize across raters.

Rating Sheet for Still Ads
Administrator’s Summary of Still Ratings
Rating Sheet for Motion Ads
Administrator’s Summary of Motion Ratings

Report Writing

The report should provide specific operational changes that focus on the most important changes. Avoid mentioning what was done poorly; simply focus on how the ad might be improved. Provide reasons why each change would improve the ad. Provide support with links to original sources. If possible, show what the ad might look like given the changes; cutouts are especially effective for still ads.

Use  Suggestions on Writing the AdPrin Audit Report to structure your report. Use the  Checklist for Writing Management Reports and Oral Presentations: An Evidence-based Checklist.

Ethical guidelines

Consider the following procedures early in the development of the ad, say at the storybook phase:

Techniques for Avoiding Legal/Ethical Problems

1. Use an independent review board to evaluate whether ads violate good taste. These boards could be internal to the agency or include representatives of key interest groups, such as customers of the brand in question.
2. Conduct copy-testing. This is likely to be less costly than producing the ad and spending money for placement only to find that the ad offends people and generates bad publicity.
3. List the interest groups that might be affected and try to anticipate their reactions. Or, what is more effective, ask people to take on the roles of those in key interest groups and to express their opinions about the ad.
4. Develop a code of ethics with respect to tastes and ask those who develop the ads to sign off on each ad.

Advertising education forum

The Advertising Education Forum focuses on advertising and children. It provides academic and scientific research, including documents published by government, regulators, academics, industry, consumer groups and advertising self-regulatory organizations.

US advertising and marketing laws

• Business Ethics (Better Business Bureau website)
• Federal Trade Commission Documents