How You Can Use Evaluation Data To Fine-Tune Your PSA Program
by Bill Goodwill Chairman, Partners in Public Service
National Broadcast Association for Community Affairs Conference
Houston, TX 1Developing a comprehensive evaluation system for your PSA program is critical to measuring your success and being able to justify the worth of your efforts. In the old days getting reliable evaluation data was difficult if not impossible. However, there are now at least two major electronic tracking systems for TV and even a system that can track radio in major markets that provide very reliable evaluation feedback.
Since broadcast television accounts for the lion’s share of PSA exposure, we spend more time tracking and analyzing TV data than that resulting from other forms of media exposure. Following are just a few of the key trends we measure for client campaigns, most of which rely on electronic data provided by A.C. Nielsen’ SIGMA electronic tracking service.
Campaign Benchmarks
One of the most frequently asked questions posed by clients is: “How is my campaign doing….?” And of course, we are thinking….”compared to what….” since it is a relative question. To provide a meaningful response, we create campaign benchmarks for each media type that permit us to compare any given campaign against a standard. Not only does it give us as evaluators and our clients some basis of comparison; it also allows us to watch trends develop over time and apply corrective actions when we see a campaign performing significantly under the benchmark.
The data in the foregoing graph also provides meaningful feedback for future budgeting purposes also. While it is true that radio and print are going to help you reach discrete audiences, the benchmark clearly shows that broadcast television is going to generate the lion’s share of reach for your campaign.
Daypart Analysis
One of the most frequently repeated mis-statements about PSAs is that "they are only aired during junk time"' which is usually referred to as very late in the evening and early AM. As this graph shows, that is clearly not the case. The electronic data provided by the two major monitoring services breaks down PSA usage by different times within a 24 hour period and this campaign nearly two thirds of PSAs air in the best dayparts, which start with the Early AM daypart up to the first half of the Late Evening daypart (10PM-midnight).
Depending upon your particular target audience, you can track the percentage of PSAs aired during the times when your target audience is most likely viewing TV, which provides an important quality indicator. For example, women (yes there are still stay-at-home moms - just ask Proctor & Gamble) are heavy TV watchers during the day.
Spot Length Analysis
Perhaps the second most widely held misperception about PSAs is that stations use mostly shorter spot lengths. Again, the data shows just the opposite.
Referring to our benchmark again, SIGMA data from campaigns which included sixties in their package, shows that station using sixties ranged from 56% on the low end, to 61% on the high end, with an average of 57%. This graph has rather obvious implications to your creative team. If they are producing only shorter length spots, you are missing an opportunity to register a more compelling message that is possible through longer lengths. Perhaps more importantly, a :60 is worth twice a :30 and four times a :15, meaning you will generate much greater value for your campaign, to many people the most important determinant of success.
Finally, if you are using a tag with a website or an 800 number in your PSAs, you need the longer length messages to adequately register the information on the tag.
Analysis By Market Size
Another way to look at PSA data is to analyze usage by market size. Nielsen uses DMAs - Designated Market Areas which range from 1 (New York being the largest population center) to 212 the least populated market. By examining the percentage of your PSAs that are being used in various markets, which are correlated to population, you can determine areas where your exposure is strongest or weakest. This data provides feedback for taking corrective action in markets where your exposure is weak or nonexistent. In the case of the Peace Corps campaign, nearly two-thirds of all exposure occurred in the top 100 markets which comprise about 80% of the U.S. Population.
Demographic Analysis
Electronic monitoring data - specifically Nielsen's Post-buy package, which works in tandem with SIGMA, can permit you to measure impact upon a specific target audience. By examining demographics, you can track the number of gross impressions delivered to these audiences.
Usage By Type
It is important to know as much as you can about the type of TV outlets that are using your PSAs, and what they contribute to your total exposure. You can analyze usage by media type, i.e, broadcast versus cable TV, radio or by specific types of media as shown in the next graph.
You should know specifically the type of stations using your PSAs, so you can fine-tune future distribution strategy. For example, are they network affiliates…. independents... cable networks....or individual cable systems? What are the formats of the radio stations using your PSAs and are they the ones that reach your primary audiences? Perhaps you are interested in reach a youth audience, in which case you want to examine the number of college stations using your PSAs, or perhaps audiences by ethnicity such as Hispanic, African-American or other minorities. By analyzing the level of usage among different types of stations, you can determine if you reached your objectives.
Level Of Response
Perhaps the most widely held misperception about PSAs is that "they just don't work... if you want to get exposure for your issue, you have to buy the time." Once again, TV PSA data paints a different picture.
Unknown to many PSA planners, PSA campaigns can even generate excellent levels of response. For years the primary method the Peace Corps used to communicate its message to young prospective volunteers was a national PSA campaign. They carefully tracked the source of all incoming leads and applications and shown here they generated 782,000 leads during the six year period we worked with them, resulting in 58,588 applications.
The foregoing graph also shows another interesting trend demonstrating the efficacy of PSA campaigns. In the first five years, they distributed two campaigns a year which are the years when leads were on the rise. In 1998-99, they scaled back to only a single campaign and experienced a reduction in both leads and applications.
One of the most definitive studies showing the impact of a toll-free number used in TV PSAs was done by the National Cancer Institute (NCI) which measured calls to its Cancer Information Service.
NCI distributes public information via a number of different media and was trying to determine the impact of TV PSAs in general and the affect they had on phone calls to its toll-free number. In the months prior to the release of the TV PSA, phone calls averaged 1,872 per month as shown by the flat yellow line for July-November. After the TV PSAs were distributed, over 71,000 calls were logged during the eight month test period, or an average of 8,951 calls per month. Also, as the graph shows, there was a direct correlation between calls and level of PSA activity.
By using a dedicated 800 line and tagging your PSAs with the phone number or a website URL that is specifically set up to track “hits” resulting from your campaign, you can get excellent feedback on PSA effectiveness, as well as demographic data on typical responders. Or, you can ask for people to write for specific pieces of literature that have been prepared exclusively for use with your PSA campaign and track the number of requests you receive.
Geographic Tracking
PSA usage data can be broken out to demonstrate where you are getting the most or least exposure by field office, chapter, district or some other geographic configuration as shown in these graphs.
Obviously there are parts of the country where there are more media outlets than others, or you may have distributed
more PSAs to certain markets or regions. However, by tracking usage by some discrete geographic method, interesting patterns begin to emerge and by sharing this trend data with your community partners, hopefully proactive tactics can be employed to reverse negative patterns. In these two scenarios, you can graph the number of PSAs distributed versus the amount of exposure you get which tells an interesting story and provides the basis for follow-up.
By sharing this type of data with your community partners, it helps involve them in your campaign strategy, and shows specifically where they need to concentrate future media contacts.
Applying the Previous Usage Index
On each of our client reporting portal websites we provide distribution lists indicating which stations have or have not used a client’s PSAs in the past and the Previous Usage Index (PUI) for that station. By looking at these two columns as shown here, we can formulate a follow-up strategy very easily. For example, in this case, KDMD has not used this client’s PSAs, yet they have used our other clients’ PSAs 50 times in the past.
Another way to look at users versus non-users is shown in this graph. For a Social Security TV PSA campaign, there were 46 stations that did not use SSA PSA but have used other client PSAs from one to more than 25 times. These are examples of looking at "actionable" evaluation data that can make a very significant difference in the ultimate usage and impact of any PSA campaign.
Confirmation Of Shelf-Life
Before electronic monitoring data was available, everyone estimated the shelf life of PSA campaigns to be about 12 weeks, because that's when bounce-back cards began to dwindle.
Now, through electronic monitoring, we know that campaigns still get fairly good exposure at 26 weeks, 52 weeks and even up to a year and a half after they are released. As shown in this graph, a campaign for Volunteers of America generated 55% more value at the end of two years compared to the value after only tracking for one year.
An Army National Guard TV PSA campaign was generating significant levels of exposure 17 months after release. This is useful intelligence because if your campaign is still doing very well at the time when tracking is about to expire, you may want to consider extending tracking for a longer period.
Program Effectiveness
Perhaps the most important reason for evaluating any campaign is to substantiate the overall effectiveness of your program. If you can't prove the effectiveness of your program to your boss, or whoever else is funding your campaign, it probably won't survive in these lean times.
There are lots of ways to do this:
- Produce a monthly management summary of the dollar values generated, the number of PSAs aired, and the number of stations and markets where your PSAs were used, the types of audiences reached and gross impressions. (We do this for all our client campaigns.)
- Circulate the summary to all stakeholders in your organization who has any interest in your brand image... board members, volunteer leaders, senior management, field public affairs personnel and funding sources.
If you use an 800 number in your PSAs, you should be able to track the number of phone calls generated, website visits or literature requests received. All of these organizational goals or campaign objectives.
- Compare key trends data from one campaign to another, from one year to another to determine whether you are making progress over time.
- Calculate the advertising equivalency value – the value you would have spent if you bought the airtime - and compute a cost-benefit ratio by dividing the production and distribution costs for your campaign by the amount of exposure received.
Keep in mind that there are limits to corrective actions that you can take at the national level since real change takes place out in the countless communities across our great land. However, by sharing national campaign strategy with your community partners, analyzing the data carefully and sharing results with your field staff, you can generate the kind of support you need to reach your goals.
And most importantly, remember that evaluation data should be actionable. Compiling evaluation data just for the sake of having numbers is a meaningless exercise if you don't use the data to change the way you develop future strategy.
1 Article has been updated with later evaluation data and trends.
