Evaluation of the Office on National Drug Control Policy (ONDCP) National Youth Anti-Drug Media Campaign

NOTE - All data for this initiative was transferred to the University of Michigan and can be found at http://www.icpsr.umich.edu/icpsrweb/ICPSR/studies/27868?archive=ICPSR&q=ONDCP

In September 1998, NIDA awarded a competitive contract to Westat, Inc., a health survey research company, and subcontracts to The Annenberg School for Communication, University of Pennsylvania, and the National Development Research Institute (NDRI), to undertake a science-based evaluation of the National Youth Anti-Drug Media Campaign sponsored by the ONDCP. During the development year (1999), Westat and colleagues designed and implemented a nationally-representative household survey of youth and parents living in the same household, entitled National Survey of Parents and Youth. After Round 1 of interviews, which occurred in Waves 1, 2, or 3, youth and parents were re-interviewed three additional times, approximately once a year, in regard to exposure to the Campaign through June 2004. Westat has produced produce semiannual, annual and special reports that provide data on the impact of the Campaign on attitudes, beliefs, intentions and behaviors on these youth and parents over time. The Public Use Files (PUF) containing data for the first three rounds of the survey are now available for download from this site. Each of the three files includes a representative but reduced sample of youth responses for that round. Identifying information and links both to the respondent's data for other rounds of the survey and his/her parent's surveys have been eliminated to protect respondent confidentiality. Selected parent variables have been appended to the records in the file.

In 2005, NIDA awarded a contract to the NSPY Center - a joint venture between AMSAQ, Inc. and Westat. The NSPY Center is charged with housing, managing, and analyzing restricted use data collected via the National Survey of Parents and Youth (NSPY) during Phase III of the Evaluation of the National Youth Anti-drug Media Campaign.

Note - The NSPY Center was discontinuted in July 2010, information is now available through the Inter-University Consortium for Political and Social Research (ICPSR) at https://www.icpsr.umich.edu/web/pages/ICPSR/index.html

The following Reports are currently available on-line:

2004 Report of Findings: Evaluation of the National Youth Anti-Drug Media Campaign - (Released August, 2006)

All Files and Reports (PDF Format, ZIP Compressed, 8.3 MB)
Executive Summary (PDF Format, 504 K)
Full Report (PDF Format, 4.5 MB)
- C2 Tables (PDF Format, 92 Kb)
- C3 Tables (PDF Format, 848 Kb)
- C4 Tables (PDF Format, 188 Kb)
- C5 Tables (PDF Format, 868 Kb)
- C6 Tables (PDF Format, 1.8 MB)
- Appendix A (PDF Format, 256 Kb)
- Appendix B (PDF Format, 260 Kb)
- Appendix C (PDF Format, 264 Kb)
- Appendix D (PDF Format, 348 Kb)
- Appendix E (PDF Format, 288 Kb)
- Appendix F (PDF Format, 1.2 MB)

2003 Report of Findings: Evaluation of the National Youth Anti-Drug Media Campaign - (Released December, 2003)

Executive Summary (PDF Format, 488 K)
Full Report (PDF Format, 1.5 MB)
- Table of Contents (PDF Format, 108 K)
- Detail Tables (PDF Format, 812 K)
- Appendices (PDF Format, 644 K)

Fifth Semi-Annual Report: Evaluation of the National Youth Anti-Drug Media Campaign - (Released January, 2003)

Executive Summary (PDF Format, 144 K)
Full Report (PDF Format, 17.2 MB)
- Table of Contents and Executive Summary (PDF Format, 488 K)
- Chapters 1-6 (PDF Format, 2.1 MB)
- Detail Tables (PDF Format, 14.2 MB)
- Appendices (PDF Format, 1.2 MB)

Fourth Semi-Annual Report: Evaluation of the National Youth Anti-Drug Media Campaign - (Released May, 2002)

Executive Summary

Highlights of the Report

The National Youth Anti-Drug Media Campaign was funded by the Congress to reduce and prevent drug use among young people by addressing youth directly, as well as indirectly, by encouraging their parents and other adults to take actions known to affect youth drug use. The major intervention components include television, radio, and other advertising, complemented by public relations efforts including community outreach and institutional partnerships. This evaluation report covers the current phase (Phase III) of the project, from September 1999 through December 2001.

Recall of Campaign Messages: Most parents and youth recalled exposure to Campaign anti-drug messages. About 70 percent of both groups report exposure to one or more messages through all media channels every week. The average (median) youth recalls seeing one television ad per week. In previous waves less than 25 percent of parents recalled seeing a TV ad every week; this increased to 40 percent in the second half of 2001. Both parents and youth reported substantial recognition of the Campaign's "anti-drug" brand phrases.
Effects on Parents: There is evidence consistent with a favorable Campaign effect on parents. Overall, there are favorable changes in 4 out of 5 parent belief and behavior outcome measures including talking about drugs with, and monitoring of, children. Moreover, parents who report more exposure to Campaign messages scored better on those outcomes after applying statistical control for confounders. There is no evidence, yet, of indirect effects on youth behavior as the result of parent exposure to the Campaign.
Effects on Youth: There is little evidence of direct favorable Campaign effects on youth. There is no statistically significant decline in marijuana use or improvements in beliefs and attitudes about marijuana use between 2000 and 2001, and no tendency for those reporting more exposure to Campaign messages to hold more desirable beliefs.
For some outcomes, and for some subgroups of respondents, analysis raises the possibility that those with more exposure to the specific Campaign ads at the start of Phase III of the Campaign had less favorable outcomes over the following 18 months. This was true for the youth respondents who were nonusers and aged 10 to 12 at the start of this phase, with regard to their intentions to use marijuana in the future and for all youth 12 to 18 for their perceived social norms about marijuana use. Girls with the highest exposure to Campaign ads at the start were more likely than less exposed girls to initiate marijuana use. This effect was not seen for boys. This unfavorable association with initiation was also significant for the youngest respondents and for the low risk respondents. Further analysis is required before any firm conclusion can be reached to support these unlikely outcomes.

These interim results reflect the first 2 years of Phase III operation. Subsequent semiannual reports may show different effects (including favorable effects on youth). This report provides followup information on less than half of the entire sample; the rest will be available for the next report.

Executive Summary

The number one goal of The National Drug Control Strategy is to "Educate and enable America's youth to reject illegal drugs as well as alcohol and tobacco." One of the objectives in support of that goal includes, "Pursue a vigorous advertising and public communications program dealing with the dangers of drugÉ use by youth." Under the Treasury-Postal Appropriations Act of 1998, Congress approved funding (P.L. 105-61) for "a national media campaign to reduce and prevent drug use among young Americans." Pursuant to this act, the Office of National Drug Control Policy (ONDCP) launched the National Youth Anti-Drug Media Campaign (the Media Campaign).

The Media Campaign has progressed through three phases of increasing complexity and intensity. Phases I and II are not discussed in this report. ONDCP has available other reports that evaluate those phases. This report focuses on Phase III, which began in September 1999 and is planned to run at least through spring 2003. An evaluation of Phase III is being conducted under contract to the National Institute on Drug Abuse (NIDA) by Westat and its subcontractor, the Annenberg School for Communication at the University of Pennsylvania. Funding of the evaluation is provided by ONDCP from the appropriation for the Media Campaign itself. This is the fourth semiannual report of the Westat and Annenberg evaluation of Phase III of the Media Campaign.

The primary tool for the evaluation is the National Survey of Parents and Youth (NSPY). This survey is collecting initial and followup data from nationally representative samples of youth between 9 and 18 years of age and parents of these youth. This Fourth Semiannual Report presents analyses from the first four waves of NSPY, covering the period from September 1999 through December 2001.

This executive summary focuses on evidence for Campaign effects on youth and parent outcomes. There have been about 18 months for the Campaign to produce detectable effects on the outcomes since the midpoint of the first wave of interviews, in March 2000, through the midpoint of the fourth wave of interviews, in September 2001. This report includes evidence about temporal changes in behavior and attitudes and beliefs, focusing on changes between 2000 and 2001. The report also includes evidence for cross-sectional association of exposure to Campaign advertising and attitudes and beliefs and, in some cases, behavior. In addition, this report provides, for the first time, evidence from the cohort of youth and parents interviewed during the first half of 2000 and reinterviewed during the last half of 2001. The repeated interviews of the same respondents permits examination of the ability of earlier exposure to predict later outcomes, a stronger procedure for making claims about potential Campaign effects. The next report will strengthen these lagged analyses because the youth and parents first interviewed either in the last half of 2000 or in the first half of 2001 are all being interviewed for a second time during the first half of 2002, thus increasing the sample size for these analyses of temporal ordering. In the subsequent periods, each of these youth and parents will be interviewed for a third time during the final two waves of data collection, that is, between July 2002 and June 2003. The final evaluation report is scheduled for spring 2004. At that time, the sample youth and their parents will have been studied for 2 to 3 years.

This report by Westat and Annenberg provides six types of information about the campaign and its effects:

A brief update and description of the Media Campaign's activities to date.
A review of the logic and approach of the evaluation.
Statistics on the level of exposure to messages achieved by the Media Campaign during Phase III.
Estimates of change in the drug use behaviors of youth between 2000 and 2001.
Estimates of Campaign effects on youth from three different approaches: (1) estimates of association between exposure to the Campaign and simultaneously measured outcomes, including attitudes, beliefs, and intentions, with statistical controls for confounders; (2) estimates of change between 2000 and 2001 in these outcomes; as well as (3) estimates of any association of early exposure and later outcomes for the youth interviewed twice. The report also includes analyses of change and of associations for various subgroups of the population.
Estimates of Campaign effects on parents. These include association between exposure to the Campaign and parents' talk about drugs with their children, their monitoring of their children's behavior, and their engaging in fun activities with their children, as well as their beliefs and attitudes about talk and about monitoring, and estimates of association between parent exposure and youth drug use behavior. It also includes estimates of trends between 2000 and 2001 in the parent outcomes. Both change and association data are reported for various subgroups of the population. In addition, the lagged associations of early parent exposure to Campaign advertising with later outcomes are presented.

Background on the Media Campaign

The Media Campaign has three goals:

Educate and enable America's youth to reject illegal drugs; population.
Prevent youth from initiating use of drugs, especially marijuana and inhalants; and population.
Convince occasional users of these and other drugs to stop using drugs.

The Media Campaign originally targeted paid advertising to youth aged 9 to 18 (with a current focus on youth 11 to 17), parents of youth in these age ranges, and other influential adults. Phase III advertising is being disseminated through a full range of media or "channels" following a Communications Strategy developed by ONDCP. Phase III also includes components other than advertising. There are outreach programs to the media, entertainment, and sports industries, as well as partnerships with civic, professional, and community groups. These other components, which are being coordinated by a public relations firm, include encouraging entertainment programs with antidrug themes, coverage of the anti-drug campaign in the news media, community activities, corporate co-sponsorship, and special interactive media programming on the Internet.

ONDCP performs overall management of the Media Campaign in collaboration with the following groups:

The Partnership for a Drug-Free America (PDFA), which provides the creative advertising for the Media Campaign through its existing relationship with leading American advertising companies;
A Behavioral Change Expert Panel (BCEP) of outside scientists who help to inform the content of the advertisements to reflect the latest research on behavior modification, prevention, and target audiences;
Ogilvy, a national advertising agency, which has responsibility for media buying (as well as for carrying out some supportive research and assuring a coherent advertising strategy);
Fleishman-Hillard, a public relations firm, which coordinates the nonadvertising components of the Media Campaign; and
The Ad Council, a coordinator of national public interest advertising campaigns, which supervises distribution of donated advertising time to other public service agencies under the "pro bono match" program (see below).

For Phase III, advertising space is purchased on television, radio, newspapers, magazines, billboards, transit ads, bus shelters, movie theaters, video rentals, Internet sites, Channel One broadcasts in schools, and other venues as appropriate. The television buys include spot (local), network, and cable television. One of the requirements in the Media Campaign appropriations language is that each paid advertising slot must be accompanied by a donation of equal value for public service messages from the media, known as the pro bono match. The pro bono match involves one-to-one matching time for public service advertisements or in-kind programming. The pro bono spots may include other themes including anti-alcohol, anti-tobacco, and mentoring, but such themes are not part of the paid advertising.

Methodology

The report presents results from four waves of the National Survey of Parents and Youth (NSPY), an in-home survey designed to represent youth living in homes in the United States and their parents. Each of the first three waves of NSPY enrolled nationally representative samples of youth aged 9 to 18 and their parents. The respondents at these waves represent the approximately 40 million youth and 43 million of their parents who are the target audience for the Media Campaign. Wave 1 included 3,312 youth from 9 to 18 years old and 2,293 of their parents, who were interviewed between November 1999 and May 2000; Wave 2 included 2,362 youth and 1,632 of their parents interviewed between July and December 2000. Wave 3 included 2,459 youth and 1,681 of their parents interviewed between January and June 2001.

Sampling of eligible youth in Waves 1, 2, and 3 was designed to produce approximately equal-sized samples within three age subgroups (9 to 11, 12 to 13, 14 to 18). One or two youth were randomly selected from each eligible sample household. One parent was randomly chosen from each eligible household. A second parent was selected in the rare event when two youths who were not siblings were sampled. Wave 4 conducted followup interviews with the youth who were sampled in Wave 1 and were still eligible, and with their parents. Later waves will follow up the Wave 4 sample again and also follow up samples from Waves 2 and 3. While the focus of the Campaign is on youth older than age 10, the inclusion of 9- and 10-year-old children at Waves 1, 2, and 3 provides a sample of those who will age into the primary target audience at the times of the followup interviews. Wave 4 comprised followup interviews with 2,478 youth and 1,752 parents of those sampled at Wave 1; the interviews were conducted between July and December, 2001.

NSPY achieved a response rate of 65 percent for youth and 63 percent for parents across Waves 1 through 3 of data collection (the recruitment waves), with little response rate variation by wave. In Wave 4, the interviewers for NSPY successfully reinterviewed 82 percent of responding Wave 1 youth who were interviewed in Wave 1 and were still eligible for the survey (primarily still under age 19) and 80 percent of responding Wave 1 parents. The cumulative response rates for Wave 4 were necessarily lower than the rates for the prior three waves due to the followup nature of this wave of data collection. In preparing the respondent data for analysis, adjustments were made at all four waves to compensate for nonresponse and to make certain survey estimates conform to known population values. Confidence intervals for survey estimates and significance tests are computed in a manner that takes account of the complex sample design.

NSPY questionnaires were administered in respondents' homes using touch-screen laptop computers. Because of the sensitive nature of the data to be collected during the interviews, a Certificate of Confidentiality was obtained for the survey from the Department of Health and Human Services, and confidentiality was promised to the respondents. All sensitive question and answer categories appeared on the laptop screen and were presented orally to the respondent over headphones by a recorded voice that could be heard only by the respondent. The responses were chosen by touching the laptop screen.

The NSPY questionnaire for youth included extensive measurement of their exposure to Media Campaign messages and other anti-drug messages. It also included questions about their beliefs, attitudes, intentions, and behaviors with regard to drugs and a wide variety of other factors either known to be related to drug use or likely to make youth more or less susceptible to Media Campaign messages.

The NSPY questionnaire for parents also included measures about exposure to Media Campaign messages and other anti-drug messages. In addition, it included questions about parents' beliefs, attitudes, intentions, and behaviors with regard to their interactions with their children. These included talking with their children about drugs, parental monitoring of children's lives, and involvement in activities with their children. Parent and child responses can be linked for analysis.

Ad exposure was measured in NSPY for both youth and parents by asking about recall of specific current or very recent TV and radio advertisements. The TV and radio advertisements were played for respondents on laptop computers in order to aid their recall. Youth were shown or listened to only youth-targeted ads, and parents were shown or listened to only parent-targeted ads. In addition, both youth and parents were asked some general questions about their recall of ads seen or heard on TV and radio, and in other media such as newspapers, magazines, movie theaters, billboards, and the Internet.

Media Purchases and Evidence about Exposure

Media Purchases

Across its multiple media outlets, the Media Campaign reports that it purchased enough advertising time over the 28-month period covered by this report (September 1999 through December 2001) to achieve an expected exposure to 2.5 youth-targeted ads per week for the average youth and to 2.2 parent-targeted ads per week for the average parent. These estimates include Campaign advertisements intended for either all youth or all parents; they do not include exposure by youth or parents to advertisements intended for other audiences, often called "spill," or separate advertising targeted to specific race- or ethnicity-defined audiences.

Figures ES-1 and ES-2 present the weekly totals for expected youth-targeted and parent-targeted exposures, respectively, where 100 means that the average person in the audience would be exposed once per week. Both the actual weekly media purchases and a smoothed line averaging over 3-week periods are presented. Both graphs show that purchases varied a good deal both between and within the periods corresponding to the NSPY waves of data collection.
Figure ES-1. Weekly youth-targeted general market GRPs (September 1999 through December 2001)

Figure ES-2. Weekly adult-targeted general market GRPs (September 1999 through December 2001)

Table ES-1 summarizes the variations across broad 6-month periods. The table shows that expected weekly exposures of 2.6, 2.5, and 2.8 for youth across the first three waves are followed by a sharp decline in purchases during the second half of 2001, with the average falling below an expectation of 2.0 exposures per week. Purchases of ad time for parents also fell during the second half of 2001; it was also low in the second half of 2000.

Table ES-1. Distribution of youth and parent average weekly exposures across waves
	Wave 1 9/99-6/00	Wave 2 7/00-12/00	Wave 3 1/01-6/01	Wave 4 7/01-12/01
Youth	2.59	2.54	2.80	2.09
Parents	2.75	1.52	2.30	1.94

About 35 percent of youth advertising time was purchased on network or "spot" television and about another 25 percent was purchased on network and "spot" radio. Thus, about 60 percent of total exposures were on media with the potential to reach a wide portion of youth. The rest of the GRPs were purchased on channels that reach narrower audiences, including in-school television (19%), magazines (10%), and other media: basketball backboards, Internet, nontraditional, and arcades (all less that 5% apiece).
For parents, averaged across the four waves, almost 60 percent of the primary media buys were in potentially wider-reach media, that is, network radio (30% of all expected exposures) and network television (28%). Forty percent of the primary media buys were in narrower-reach media, that is,outdoor media (26%), magazines (11%), newspapers (4%), the Internet, and movie ads (1%).
The decline in total media purchases for parents in the second half of 2001 may have been counterbalanced by the reweighting of the media channels used. While over the first three waves 51 percent of all parent media buys were in wider-reach channels, for the final period 85 percent of buys were in those channels.

For both youth and parents, Campaign advertising buys were mostly directed to a small number of platforms or themes. The focus on each platform varied across time, as presented in Tables ES-2 and ES-3, which present the percentage of all television and radio ad buys in each wave dedicated to each platform. For youth, an early focus on Negative Consequences of drug use had almost disappeared by Wave 3 but was revitalized in Wave 4. A focus on Normative Education/ Positive Alternatives was strong across all four waves while Resistance Skills were emphasized in Waves 1 and 3 but not included in Waves 2 or 4. For parents, the Parenting Skills/Personal Efficacy platform was maintained through all four waves and was especially strong in Wave 4. On the other hand, "Your Child at Risk" received substantial weight only at Wave 3, and "Perceptions of Harm" was included only at Wave 1. Some of the "Your Child at Risk" platform advertising in Waves 3 and 4 focused on the risks of inhalants.

Table ES-2. Advertising buys per week purchased for specific youth platforms across waves (TV and radio)
Platform	Wave 1 - 2000 (%)	Wave 2 - 2000 (%)	Wave 3 - 2001 (%)	Wave 4 - 2001 (%)
Negative Consequences	24.7	16.6	0.0	61.3
Normative Education/ Positive Alternatives	40.1	71.1	41.6	34.6
Resistance Skills	33.0	3.0	46.5	3.0
Other	2.2	9.2	11.8	0.01

Table ES-3. Advertising buys per week purchased for specific parent platforms across waves (TV and radio)
Platform	Wave 1 - 2000 (%)	Wave 2 - 2000 (%)	Wave 3 - 2001 (%)	Wave 4 - 2001 (%)
Parenting Skills/Personal Efficacy	54.2	98.8	48.6	91.3
Your Child at Risk	13.6	0.0	51.4	7.9
Perceptions of Harm	31.0	1.0	0.0	0.0
Other	1.2	0.0	0.0	0.0

Anti-inhalant advertising directed toward youth represented less than 0.1 percent of all TV and radio exposures, and even during its heaviest period (the second half of 2001) was only 4 percent of all buys. In contrast, parent-focused anti-inhalant advertising was 11 percent of all purchased radio and television exposures and was included in 43 percent of the ads purchased in the first half of 2001.
Ecstasy-focused ads were broadcast only on radio and only during 2001. They made up about 8 percent of all expected exposures during 2001 for parents. For youth, they appeared only in the second half of 2001, making up 10 percent of all exposures during that period.

Recall of Exposure

NSPY used two measures of exposure; the first is based on general recall of anti-drug ads through all media, and the second is based on specific recall of currently broadcast ads on television and radio. All of the following results relate only to youth aged 12 to 18 and their parents (i.e., children under 12 in NSPY and their parents are excluded).

General exposure recall to all anti-drug advertising was fairly stable for parents and for youth across the four waves. This stability occurred despite the variation in purchases of targeted advertising by the Campaign. The general exposure measures, which may include exposure to advertising targeted to the other audience and advertising placed by other institutions, did not appear to relate closely to changes in Campaign-targeted buys across the four waves. About 70 percent of all parents and 76 percent of all youth recalled weekly exposure to any anti-drug ads (Table ES-4). These estimates suggest that the median monthly exposures are about 10 ads for parents and 13 ads for youth, and the corresponding median weekly exposures are about 2.5 and 3.25 ads.

Table ES-4. Exposure to campaign advertising by wave
Population	Exposure measure: Percent seeing/hearing ads 1 or more times per week	Wave 1 (%)	Wave 2 (%)	Wave 3 (%)	Wave 4 (%)
Parents	General Exposure: Across all media Specific Exposure: TV ads Specific Exposure: Radio ads	71 26 10	71 23 11	72 20 17	68 39* 15*
Youth 12 to 13	General Exposure: Across all media Specific Exposure: TV ads Specific Exposure: Radio ads	74 40 NA	78 43 4	76 51 10	69 60* 3**
Youth 14 to 18	General Exposure: Across all media Specific Exposure: TV ads Specific Exposure: Radio ads	77 34 NA	79 37 4	77 47 13	74 53* 3**
* Significant change between Waves 1 and 2 versus Waves 3 and 4, p<0.05. ** Significant change between Waves 2 and 3, and between Wave 3 and 4 p<0.05. NA: Radio use not measured for youth during Wave 1.

Estimates of specific recall of Campaign ads among parents and youth provide an alternative view of exposure to the estimates generated from the general recall measures. Parents reported a median of 3 exposures and youth reported a median of 7.5 exposures to the TV ads "in recent months." This roughly translates into medians of 0.35 and 0.9 exposures per week for parents and youth, respectively. Radio recall was lower than TV recall: On average, over the 2-year period, about 16 percent of parents recalled general exposure to radio ads in the past week, and over the final three waves of measurement about 6 percent of youth recalled such exposure. About 50 percent of parents and 60 percent of youth recalled none of the specific radio ads played for them.
Specific recall of televised Campaign ads increased significantly between 2000 and 2001 for youth in both the age groups in Table ES-4; the recall increased from 37 percent weekly recall to nearly 51 percent weekly recall for the overall sample of 12- to 18-year-olds. The decline in television ad purchases in Wave 4 had not yet been seen in recall of those ads, perhaps because many youth were still recalling messages they saw during the higher Wave 3 exposure period (Table ES-4). There was a sharp increase between Waves 2 and 3 in the recall of the radio ads by youth in both age groups, but that increase disappears in Wave 4. In all cases, radio recall remained much lower than television ad recall. (Statistically significant findings are presented in bold in the tables.)
As was the case with youth, specific recall of television advertising by parents increased in Wave 4. Although overall parent advertising buys had fallen in Wave 4, television buys were up. This rise in TV buys is consistent with the rise in exposures to specific TV ads. Parent recall of specific radio ads, while still lower than TV ad recall, showed a significant increase between 2000 and 2001, from about 10 percent recalling weekly exposure to about 16 percent.

"Brand" Recall

One of the innovations of Phase III has been the inclusion of a Campaign "brand"-for example, "the anti-drug." A brand is used in many advertising campaigns to provide a recognizable element to coordinate advertising as well as nonadvertising components of the campaign. Insofar as the brand is recognized and positively regarded, its familiar presence may create some initial positive response to any new ad or increase the perception that each ad is part of a larger program. Such effects may, in turn, influence acceptance of the Campaign's message.

The NSPY started measuring brand phrase recall in Wave 3. The data provide evidence for brand phrase recall, particularly among youth, with stronger evidence in Wave 4 than in Wave 3:

Over Waves 3 and 4 combined, approximately 68 percent of 12- to 18-year-olds recalled the Campaign brand phrase targeted at youth, and 55 percent of their parents recalled the Campaign brand phrase targeted at parents. Because some of the claimed recall could have been due to false recollection, true recall cannot be precisely estimated at this time.
There is good evidence that the more individuals were exposed to Campaign advertising the more likely they were to recall the brand phrase, which supports the idea that the phrase was learned as the result of Campaign exposure. Figure ES-3 shows the relationships between recalled exposure of TV ads for youth and TV and radio ads for parents, with the level of brand recognition. For youth, only 39 percent of the lowest TV ad exposure group said they recognized the brand, while 83 percent of the highest exposure group-those who had seen television ads more than 12 times per month-did so. For parents, where recall of both television and radio ads are included in the exposure measure, 35 percent of the lowest exposure group and 74 percent of the highest exposure group recalled the brand phrase. These are large and statistically significant differences.

Figure ES-3. Recall of brand phrase by specific ad recall (%)

Exposures to Other Drug Messages

Both youth and parents receive messages about drugs from other public sources besides Media Campaign paid advertising. Those other sources of messages are themselves the target of Campaign efforts, and they also create a context for receiving the Campaign's purchased anti-drug media messages. Exposure to messages through these other sources is high, but, with a few exceptions, there was not much change between waves (Table ES-5).

One other potential source for providing drug-related messages is the variety of programs that exist for youth and parents. The Campaign's focus in working with youth-serving organizations and parent groups is to encourage them to integrate drug use prevention messages and strategies into their existing educational programs and extracurricular activities, rather than to increase their participation in anti-drug programs per se. With regard to youth and parent involvement in such programs:

Around two-thirds of youth reported having attended anti-drug education in school during the past year, a rate that remained unchanged across the four waves. Out-of-school drug education was much rarer, and it declined slightly, but significantly from about 8 percent in 2000 to 6 percent in 2001.
A little less than one-third of parents reported attending anti-drug and parental effectiveness programs. This did not change across waves.

Other sources for messages about drugs are public drug-related discussions and mass media stories. The NSPY findings relating to this source are as follows:

There was a jump in the percentage of parents recalling community-level drug-related discussion of anti-drug programs between Waves 1 and 2, but the percentage had returned to the Wave 1 level by Wave 3 and stayed there for Wave 4. The net result is a small but statistically significant decline in recall between 2000 and 2001.

Table ES-5. Exposure to drug-related communication by wave
Measure	Population	Wave 1 (%)	Wave 2 (%)	Wave 3 (%)	Wave 4 (%)
Percent in-school drug education in the past year	Youth	67	66	65	65
Percent extracurricular drug education in the past year	Youth	8	7	6	6*
Percent recalling weekly exposure to stories on at least one medium with drugs and youth content	Youth	52	53	53	44*
Percent recalling weekly exposure to stories on at least one medium with drugs and youth content	Parents	66	62	65	62
Percent hearing a lot about anti-drug programs in community in the past year	Parents	31	38	31	29*
Percent attending drug prevention programs in the past year	Parents	29	32	30	30
Percent attending parent effectiveness programs in the past year	Parents	27	30	29	28
* Significant change between Waves 1 and 2 versus Waves 3 and 4, p<0.05.

Weekly exposure to mass media stories about drugs and youth was reported by 64 percent of parents. There was little change in this across waves.
However, the approximately constant rate of around 53 percent of youth reporting such media exposure in the first three waves of measurement was broken at Wave 4, when it fell to 44 percent. As a result, the rate for 2001 was significantly lower than that for 2000.

Drugs are not only a public topic, they are also a common topic for private conversation between parents and children, and among youth and their friends (Table ES-6):

A slightly increasing proportion of parents reported conversations about drugs with their children across years; in 2000 around 80 percent and in 2001 around 83 percent of parents claimed to have had two or more conversations with their children about drugs in the previous 6 months. There were no important differences in reported conversation with children according to the age of the child.
In contrast, youth reported a different pattern of conversation. The percentage of youth reporting such conversations with their parents was lower -- only about 52 percent reported two or more such conversations in the past 6 months. The percentage also declined between 2000 and 2001, a decline that was significant for the entire group of 12- to 18-year-olds and for the 12- to 13-year- olds. In addition, fewer of the younger children (aged 12 to 13) reported such conversations with friends in 2001 than in 2000.

Most youth say they have conversations about drugs with parents and/or friends, and many of them have such conversations frequently. The partners for such conversations shift sharply as youth mature. As they mature, youth are less likely to talk with their parents and more likely to talk with friends.

Table ES-6. Drug-related conversations by wave
Percent with two or more conversations in past 6 months	Population	Wave 1 (%)	Wave 2 (%)	Wave 3 (%)	Wave 4 (%)
Youth with friends	Youth 12 to 13 Youth 14 to 15 Youth 16 to 18	44 69 68	44 52 71	39 65 70	40* 65 71
Youth with friends	All youth	62	57	59	60
Youth with parents	Youth 12 to 13 Youth 14 to 15 Youth 16 to 18	59 58 48	56 52 52	53 53 45	51* 50 48
Youth with parents	All youth	55	53	50	50*
Parents with children	Parents of 12 to 13 Parents of 14 to 15 Parents of 16 to 18	80 82 78	78 79 80	81 82 83	81 86 82
Parents with children	All parents	80	79	82	83*
* Between Waves 1 and 2 and Waves 3 and 4 change significant at p<0.05.

In the course of conversation about drug use, 12- to 18-year-old youth discuss negative things about drugs, but many older youth also speak positively about drugs. Only 9 percent of 12- to 13- year-olds had conversations with the theme "marijuana use isn't so bad" as compared with 44 percent who had conversations about "bad things that happen if you use drugs." In contrast, pro- marijuana conversations are reported by 34 percent of 16- to 18-year-olds, as compared with 55 percent who had conversations about bad things that can happen if you use drugs. There was no substantial change in the balance of pro-drug to "anti-drug" comments between waves.

Estimates of Youth Drug Use

Following the goals of the Media Campaign given earlier, NSPY was designed to assess the influence of the Media Campaign on initial use (i.e., using at least once in a lifetime) and the shift from initial to regular use (i.e., using at least 10 or more times in a year) of marijuana and inhalants. The primary purpose of including questions about drug use in NSPY was not to provide estimates of youth drug use, but rather it was to enable the correlations of cognitive variables (such as attitudes, beliefs, social norms, self-efficacy, and intentions) with actual usage to be studied. Furthermore, NSPY was designed to measure linkages in a theoretical model for Media Campaign action, that is, linkagesbetween ad exposure and attitudes, between attitudes and intentions, and between intentions and actions (drug use). Measures of drug use are needed for an evaluation of this model.

Because it has a larger sample and a long trend line, another survey sponsored by the Federal Government the Monitoring the Future (MTF) study provides better measurements of drug use behaviors and changes in them. The National Household Survey of Drug Abuse (NHSDA) also provides important information about drug use and, as a household survey rather than a school survey like MTF, has much in common with the NSPY. However, 2001 NHSDA data are not available. NHSDA 2000 data were presented in the previous semiannual report. The 2001 MTF data, reflecting data collected through the spring of 2001, show a fairly stable pattern of marijuana use since the start of Phase III, and indeed back through 1998 before the start of the national Campaign. The proportions reporting past year marijuana use from 1991 through 2001 are presented in Figure ES-4.

Figure ES-4. Percentage of 8th, 10th, and 12th graders reporting past year marijuana use: MTF 1991-2001

The NSPY comparisons between 2000 and 2001, although based on smaller samples, take the use data through the end of 2001. They show essentially the same age differential and trend results as the MTF data: stability in annual marijuana use for all of the age subgroups (Table ES-7).

Table ES-7 - Annual use of marijuana by age: NSPY reports
Age Group	Wave 1 11/99-6/00 %	Wave 2 7/00-12/00 %	Wave 3 1/01-6/01 %	Wave 4 6/01-12/01 %
12 to 13	3.3	3.2	2.0	3.2
14 to 15	11.2	11.5	14.4	13.1
16 to 18	28.9	29.3	27.6	26.1
12 to 18	15.9	15.8	15.6	15.3

NSPY also examined rates of change in three other measures of marijuana use - ever use, regular use (almost every month), and use in the previous 30 days. For all ages and for all of those measures, use was unchanging between 2000 and 2001, with two exceptions. Reports of regular use and last 30 days use, while still rare, were significantly increasing among youth who were 14- to 15-years-old. Reports of past month use increased from 3.6% to 7.2%, and regular use (defined as use every month or almost every month) increased from 2.2% to 5.4%.

Campaign Effects

The remainder of this Executive Summary presents evidence obtained to date regarding Campaign effects. The discussion first summarizes the logic adopted for claiming effects. It then presents the findings regarding Campaign effects on youth followed by the findings for Campaign effects on parents.

The Logic of Claiming Campaign Effects

The analysis of Campaign effects in the report involves three components: (1) examining trends over time, (2) examining how exposure to the Campaign that individuals report is associated with their outcomes measured at the same time, and (3) examining how individuals' reported exposure at one wave predicts their outcomes at a later wave, among youth and parents who were measured at two waves (in this report Waves 1 and 4).

If the Campaign has been successful, it would be desirable to see favorable trends in the outcomes over time. However, change in outcomes over time (or a lack of change despite positive Campaign effects) may be due to influences besides the Campaign. Thus, if effects are to be definitively attributed to the Campaign, other supporting evidence is also needed.

Another form of evidence is an association between exposure and outcome, measured at the same time. However, evidence of the presence or absence of a simple association is inadequate for inferring that exposure has, or has not, had an effect on an outcome. The main threat to such an inference is that a positive association may be due to the influence of other variables (confounders) on both exposure and outcomes. This threat to inference can be substantially lessened by applying statistical controls for the confounders, as described below. However, even when controls have been applied for all known, measured confounders, there remains the possibility that unmeasured and perhaps unknown confounders are the cause of the adjusted association. Furthermore, even if controls were fully applied for all the confounders, there remains an alternative explanation for the adjusted association, namely that it is outcome that is the cause and (recall of) exposure that is the effect. Thus, an association between exposure and outcome, controlled for all known confounders, cannot alone definitively determine that the campaign has had an effect on an outcome.

The threat of ambiguity of causal direction that exists with a cross-sectional association can be overcome when longitudinal data are available. If, after controlling for all confounders, exposure measured at time 1 is associated with outcome measured at time 2, then the causal direction is from exposure to outcome since an effect cannot precede its cause. With this report, longitudinal data are available for the first time. Therefore, it is now possible to establish time order between variables that is, to examine whether a prior state of exposure affects a later outcome measure.

There is another constraint on the analysis of associations that needs to be considered. The analysis addresses only the direct effects of exposure. Associations between exposure and outcomes are expected only if individuals personally exposed to Campaign messages learn and accept those messages in the short term. This form of analysis does not reflect any indirect effects that might occur through other routes. Therefore, this report includes analyses that assess only one important route for indirect effects, that is, those mediated through parents.

For youth, analyses of Campaign effects are limited to 12- to 18-year-olds who report never having tried marijuana (referred to as "nonusers" in this report) and concerns their attitudes, beliefs, and intentions ("cognitions") about possible initiation of marijuana use in the subsequent year, and in the case of the longitudinal analyses, their actual initiation of use between Waves 1 and 4. There were not enough occasional users (i.e., those using marijuana one to nine times in the past year) among the youth to examine Campaign effects on their cognitions. The parent analysis includes all parents of 12- to 18-year-olds and focuses on the target parenting behaviors (and their supporting cognitions) including talk, monitoring, and engaging in fun projects or activities with their children in or out of the home. In addition, the analyses examine the association between parent exposure and youth behavior.

All analyses of associations between exposure to Campaign messages and outcomes use a method called "propensity scoring" to control for the possible influence of a very wide range of possible confounding variables. The analyses began with tests for any preexisting differences among the exposure groups on a large number of variables. The parent analyses were corrected, among other factors, for observed differences on race, ethnicity, gender, age of parent, income, marital status, strength of religious feelings, age of children, neighborhood characteristics, media consumption habits, language, and parental substance use (alcohol, tobacco, marijuana, and other illegal drugs). The analyses of youth associations were controlled for parent characteristics and further controlled for any preexisting difference among exposure groups on school attendance, grade level, academic performance, participation in extra-curricular activities, plans for the future, family functioning, personal antisocial behavior, association with antisocial peers, use of marijuana by close friends, personal tobacco and/or alcohol use of a long-standing nature, and sensation-seeking tendencies. For the cross-sectional analyses, the propensity scores were based on measures of these characteristics taken concurrently with the measures of exposure and outcome. For the longitudinal analyses, these characteristics were measured at Wave 1, concurrently with the exposure measure at that wave, but prior to the Wave 4 outcome measures.

The third semiannual report (Hornik et al., 2001) found evidence consistent with a Campaign effect on parents, including evidence of positive change in parent outcomes over the first three waves of measurement, and evidence for cross-sectional associations between exposure and most of those outcomes. The patterns were particularly strong for fathers. In contrast, there was little evidence consistent with a positive Campaign effect on youth. There was little evidence for changes in youth beliefs, attitudes, intentions, or behaviors, or for associations between Campaign exposure and outcomes. The current report extends these analyses to incorporate data from Wave 4 and also adds longitudinal analyses of linked Wave 1 and Wave 4 data for both youth and parents.

Campaign Effects on Youth

The analysis focuses on five outcomes for youth: initiation of marijuana use, intentions to avoid initiating marijuana use, and three indices - attitudes and beliefs about marijuana use, perceptions of social norms about marijuana use, and self-efficacy to avoid marijuana use if it is available. The intentions outcome focuses on the proportion of youth who said "definitely not" when asked about the likelihood of their using marijuana in the next year. This measure has proved to be highly predictive of subsequent use. Among nonusing 12- to 18-year-olds at Wave 1 who said they would "definitely not" use marijuana in the next year, 12 percent reported at Wave 4 having ever used marijuana (i.e., 18 months on average after their Wave 1 interview). In contrast, among nonusers who said "probably not," "probably yes," or "definitely yes" to the intentions question, about 45 percent reported having initiated use.

The attitude and belief index includes questions about eight specific consequences of marijuana use for the respondent as well as general attitudes toward marijuana use; the perception of the social norms index includes questions about what parents and friends would expect the respondent to do about marijuana use, and the self-efficacy index assesses the respondent's confidence that he or she could refuse marijuana in a variety of circumstances. Each of the three indices is substantially related to intentions to use marijuana. The intentions measure is presented as the percentage of youth who said "definitely" not. The other three indexes are calibrated so all 12- to 18-year-old users at Wave 1 had a mean score of 100 and a standard deviation of 100.

Table ES-8 presents a summary of the trend and cross-sectional association data separately for 12- to 13-year-olds and 14- to 18-year-olds. The table shows statistically significant negative trends for 12- to 13-year-olds with regard to (1) attitudes and beliefs, and (2) social norms. The other year-to-year trends are stable. There is only one significant monotonic association,1 which is negative: 12- to 13- year-olds who report higher general exposure measure report less strong rejection of marijuana use in the next year. However, that result is not repeated for the specific exposure index for that age group, and it is not repeated for either exposure measure for the 14- to 18-year-olds. In the absence of any trend data or replication with the other measures or populations, this unlikely finding is probably best interpreted as anomalous rather than as a basis for inferring negative Campaign effects.

Table ES-8 - Trend and cross-sectional association evidence about youth Campaign effects
	12- to 13-year-olds				14- to 18-year-olds
Outcome measure	Change		Associated with exposure?		Change		Associated with exposure?
Outcome measure	2000	2001	Specific exp.	General exp.	2000	2001	Specific exp.	General exp.
Percent definitely not intending to try marijuana	92%	91%	No	Yes↓	85%	84%	No	No
Belief/Attitude Index	129	122*	No	No	97	93	No	No
Social Norms Index	137	130*	No	No	91	85	No	No
Self-Efficacy Index	101	101	No	No	103	110	No	No
*Significant change between 2000 and 2001, p<.05. ↓ This arrow indicates that youth with more exposure were less likely to report that they would definitely not use marijuana in the next year.

These trend and cross-sectional analyses were repeated for important subgroups defined by gender, sensation seeking (a personality characteristic defined by an interest in engaging in novel, intense, and risky experiences, including illegal drug use), race/ethnicity, and a composite measure of risk of marijuana use. These subgroups were not further subdivided by age. Nine subgroups were analyzed for each of the four outcome measures: for 2000 to 2001 trends, and for the associations between the specific and the general exposure measures and the outcomes. A total of 108 analyses were examined. Of the 108, seven trend or association analyses were significant at the P<0.05 level. Of the seven, three were favorable trends for self-efficacy (among boys, low risk, and low sensation-seeking youth) and three were unfavorable trends for girls (for intentions, and the attitude/belief, and social norm indexes). Only one cross-sectional association was significant out of 72 examined, and that one was unfavorable. A small number of significant effects may be detected when a large number of tests are undertaken, simply by chance. The trends of the negative pattern among girls and for the 12- to 13- year-olds presented previously, and the positive pattern for self-efficacy, are worth some attention. However, absent any credible evidence of association between Campaign exposure and the outcomes, there is no firm basis for a claim of positive (or negative) Campaign effects based on these trend and cross-sectional association data.

This report introduced one additional form of analysis: lagged associations. These analyses are restricted to the youth who were interviewed at Wave 1 and again at Wave 4, and who were nonusers at Wave 1 and aged 12 to 18 at Wave 4. The interval between the two interviews was on average 1 years. The analyses ask whether level of exposure to advertising at Wave 1 - both general and specific exposure - predicts subsequent important outcomes.

The results for 12- to 13-year-olds at Wave 4 are displayed in Table ES-9, and those for 14- to 18-year- olds are displayed in Table ES-10. The exposure columns represent the level of exposure reported by these youth at Wave 1 to Campaign television advertising. The rows represent measures of four of the five outcomes of interest at Wave 4 for the same youth. The estimates in the cells are adjusted, through the propensity scoring methodology, for a wide variety of potential confounders as well being survey weighted to represent the U.S. population. The statistical significance tests take the complex sample design into account.

Table ES-9 Exposure per month at Wave 1 and outcomes at Wave 4 among 12- to 13-year-olds who were nonusers of marijuana at Wave 1
		Exposure at Wave 1
Wave 4 Outcome (average)		<1 expo- sure	1 to 3 expo- sures	4 to 11 expo- sures	12+ expo- sures	Spearman rho*	Significance**
% Not intending to use	General exposure	92.7		91.8	85.4	-.10	P=.02
% Not intending to use	Specific exposure	94.2	90.1	85.7		-.11	P=.02
Attitude/Belief Index	General exposure	126.1		129.8	111.1	-.07	NS
Attitude/Belief Index	Specific exposure	126.1	120.3	106.1		-.05	NS
Social Norms Index	General exposure	122.8		143.1	115.9	-.01	NS
Social Norms Index	Specific exposure	137.1	120.2	112.4		-.08	NS
Self Efficacy Index	General exposure	101.8		118.9	98.2	-.05	NS
Self Efficacy Index	Specific exposure	112.6	104.2	96.2		-.05	NS
* Spearman rho is an estimate of the association of two ordered variables and varies between -1 and +1. ** The significance is based on the Jonkheere-Terpstra test for monotonic association. NS denotes not significant at the 5 percent significance level.

Table ES-10 - Exposure per month at Wave 1 and outcomes at Wave 4 among 14- to 18-year-olds who were nonusers of marijuana at Wave 1
		Exposure at Wave 1
Wave 4 Outcome (average)		<1 expo- sure	1 to 3 expo- sures	4 to 11 expo- sures	12+ expo- sures	Spearman rho*	Significance**
% Not intending to use	General exposure	71.3		76.9	71.5	.00	NS
% Not intending to use	Specific exposure	75.4	74.4	68.3		-.06	NS
Attitude/Belief Index	General exposure	65.6		72.3	74.0	.02	NS
Attitude/Belief Index	Specific exposure	68.3	76.8	63.6		.00	NS
Social Norms Index	General exposure	60.2		66.7	57.4	-.02	NS
Social Norms Index	Specific exposure	76.4	62.5	49.5		-.09	NS
Self Efficacy Index	General exposure	94.2		110.6	108.5	.01	NS
Self Efficacy Index	Specific exposure	115.7	104.4	100.6		-.03	NS
* Spearman rho is an estimate of the association of two ordered variables and varies between -1 and +1. ** The significance is based on the Jonkheere-Terpstra test for monotonic association. NS denotes not significant at the 5 percent significance level.

The principal conclusions to be drawn from Tables ES-9 and ES-10 are the following:

There is evidence for a lagged association for the 12- to 13-year-olds for both the general and specific measures of exposure with one of the outcomes, intentions, but not for the other three outcomes. This association goes counter to expectation since those with higher exposure at Wave 1 are less likely to report that they do not intend to use marijuana at Wave 4.
There is no evidence for a lagged association for the 14- to 18-year-olds for either measure of exposure for any of the outcome measures. That is, regardless of their level of exposure at Wave 1, 14- to 18-year-old youth hold not significantly different scores on each of the outcome measures at Wave 4.
There is also evidence of a significant lagged association for the 12- to 18-year-olds between specific exposure and the social norms measure (data only shown in report). Those youth with more exposure at Wave 1 report poorer norms at Wave 4.

These unfavorable results were produced through the set of analysis procedures that had been set in place prior to examining the data, as was true for all other reported analyses. However, because these results were contrary to expectations, they stimulated additional analyses to check for their robustness. The results of these alternative procedures found in all cases that the statistically significant negative associations remained. None of the alternative procedures produced any positive Campaign lagged effects. The schedule for this report did not permit further investigation of these results. Wave 5 will provide both a substantially larger longitudinal sample and more time to investigate additional alternative analytic approaches to investigate these effects.

The same form of analysis was conducted to examine the association of Wave 1 exposure with the initiation of marijuana use between Wave 1 and Wave 4 by these youth, all of whom were nonusers at Wave 1. None of the associations of general exposure and use were significant, consistent with the results for the cognitive outcomes just presented. Table ES-11 presents the results for specific exposure for all 12- to 18-year-old youth, for gender subgroups, for age subgroups, and for groups that were at higher and lower risk for marijuana use. (Risk for marijuana use was a summed score that incorporated both personal and parent characteristics that were predictive of marijuana use.)

The association for the entire sample is not statistically significant. Consistent with the results shown previously, there is a statistically significant association for 12- to 13-year-olds, with higher exposure being associated with a higher proportion of youth initiating marijuana use. This trend is not significant for 14- to 18-year-olds. When the data are presented by gender, males show no effect; however, females show a significant unfavorable association. Low risk youth also show a statistically significant unfavorable association, but high risk youth (a small sample) show no significant association.

Table ES-11 - Specific exposure per month at Wave 1 and initiation of marijuana use by Wave 4 among nonusers of marijuana at Wave 1
Outcome (average)	<1 expo- sure	1 to 3 expo- sures	4+ expo- sures	Spearman rho*	Significance**
All 12 to 18 year olds	10.4	14.4	16.3	.07	NS
12 to 18-year-old males	15.9	16.0	11.4	-.05	NS
12 to 18-year-old females	3.7	12.9	21.6	.22	P<.01
12 to 18-year-old Whites	11.0	16.4	18.8	.09	NS
12 to 13 year olds	1.2	5.8	5.2	.09	P=.04
14 to 18 year olds	15.7	18.2	21.9	.07	NS
Higher risk youth	35.8	39.4	37.0	-.00	NS
Lower risk youth	5.4	9.6	11.8	.09	P=.02
* Spearman rho is an estimate of the association of two ordered variables and varies between -1 and +1. ** The significance is based on the Jonkheere-Terpstra test for monotonic association. NS denotes not significant at the 5 percent significance level.

The results about youth are complex. Unequivocally, there is no evidence yet consistent with a desirable effect of the Campaign on youth. The trends in behavior and in the beliefs that underpin behavior are either flat or, in a few cases, in a direction that suggests that the Campaign is having an unfavorable effect. There is no evidence that those who have been more exposed to the Campaign espouse desired beliefs more than others. And there is no evidence that those who were more exposed to the Campaign at Wave 1 moved subsequently in Wave 4 to more positive views or behavior than those who were less exposed. In addition, there is a suggestion at least for the 12- to 13- year-old subgroup that those who were more exposed to the Campaign moved toward less favorable beliefs, and for 12- to 13-year-olds and for all girls that they had increased levels of initiation. However, a finding that the campaign has an unfavorable effect should be seen as interim, with further elaboration and testing needed. In addition, it will be important to examine whether the apparent unfavorable finding holds up once the full sample is available following completion of the Wave 5 data collection.

Campaign Effects on Parents

There are five outcome indices that are the focus of analysis for the parent data in the report: (1) parent reports of talking with their children about drugs; (2) an index of attitude and belief items concerning talk (talk cognitions); (3) parent reports of monitoring their children; (4) an index of concerning monitoring (monitoring cognitions); and (5) parent reports of engaging in fun activities with their children in and outside of the home.

As with the youth results, the analyses searched for three supportive findings as the basis for a claim for a Campaign effect: a favorable trend on a target outcome, a favorable cross-sectional association between exposure to the Campaign and the outcome, and evidence for a lagged association between exposure at Wave 1 and outcomes at Wave 4 for the parents interviewed on both occasions (where the associations are controlled for confounders).

There was a good basis after Wave 3 to claim some support for an inference of Campaign effects on the first two of those grounds. The inclusion of Wave 4 data supports this position. Table ES-12 summarizes the evidence for the five focus indices for trend and for cross-sectional associations. In two of the cases (talking behavior and cognitions), there is both a significant trend and statistically significant associations with both the general and specific exposure measures for the overall population of parents of 12- to 18-year-olds. For monitoring cognitions, there is a favorable trend and positive associations with general exposure and with specific exposure for fathers. For monitoring behavior, there is a favorable trend, although no overall association of such behavior for either specific or general measures of exposure. However, as was observed during Wave 3, specific exposure is again associated with monitoring behavior for fathers and for parents of male youth. For fun activities, there are clear cross-sectional associations; however, there was no favorable trend overall or for any subgroup. It is worth some emphasis that for both monitoring cognitions and monitoring behavior there are favorable associations with specific exposure for fathers, even when there was no overall association. These results confirm (and incorporate) the results from the previous report.

However, the above results do not extend to suggest that parents' exposure has affected youth behavior. There was no cross-sectional associational evidence for any group that parent exposure was associated with lower marijuana consumption among youth.

The lagged analyses for parents examined the association of Wave 1 exposure with Wave 4 outcomes, controlling for Wave 1 measures of those outcomes, as well as the set of other potential confounders. The existence of significant associations of parental exposure at the first time point (Wave 1) with parent outcomes at the second time point (Wave 4) would be supportive evidence for a Campaign effect on parents. Table ES-13 summarizes these results.

Table ES-12 - Summary of cross-sectional trend and association results for parents
	Parents of 12- to 18-year olds				If no significant trend or association for for all parents of 12- to 18-year-olds, is there a significant association for a subgroup of parents?
Index	2000	2001	Association		Trend	Association
Index	2000	2001	General	Specific	Trend	General	Specific
Talking behavior	2.26	2.36*	Yes	Yes	--	--	--
Talking cognitions	96.80	102.90*	Yes	Yes	--	--	--
Monitoring behavior	1.41	1.46*	No	No	--	No	Fathers, parents of male youth
Monitoring cognitions	87.10	92.70*	Yes	No	--	--	Fathers, parents with college education
Doing fun activities	63.50	62.70	Yes	Yes	No	--	--
Youth marijuana use in the previous year	15.80	15.50	No	No	12-13 year old Afr.-Am. youth (favorable)	Parents of Hispanic youth, (unfavorable)	Parents of higher risk and White youth (unfavorable)
* Significant difference between 2000 and 2001 at p<.05. Yes: Significant monotonic association at P<0.05. --: Subgroup test not statistically different from result for full sample.

Table ES-13 - Summary of lagged association results for parents (Wave 1 exposure and Wave 4 outcomes)
Outcome		<1 expo- sure	1 to 3 expo- sures	4 to 11 expo- sures	12+ expo- sures	Spearman rho*	Significance**
Talking behavior (0 to 3 scale)	General exposure	2.35		2.35	2.35	0.00	NS
Talking behavior (0 to 3 scale)	Specific exposure	2.34	2.33	2.40	2.23	-0.02	NS
Talking cognitions Score on index with:	General exposure	103.23		90.67	106.60	0.01	NS
Talking cognitions Score on index with:	Specific exposure	99.83	96.56	98.13	119.88	0.06	NS
Monitoring behavior	General exposure	1.48		1.39	1.50	0.01	NS
Monitoring behavior	Specific exposure	1.52	1.41	1.45	1.41	-0.03	NS
Monitoring cognitions	General exposure	100.97		88.81	87.90	-0.04	NS
Monitoring cognitions	Specific exposure	98.48	88.21	86.61	93.71	-0.03	NS
Doing fun activities	General exposure	65		65	66	0.01	NS
Doing fun activities	Specific exposure	67	63	66	73	0.05	P=.05
* Spearman rho is an estimate of the association of two ordered variables and varies between -1 and +1. ** The significance is based on the Jonkheere-Terpstra test for monotonic association. NS denotes not significant at the 5 percent significance level.

If the analyses of the above lagged associations had found significant effects, this would have supported a claim that the Campaign caused the outcomes rather than that the outcomes (which represent parent engagement with their children) caused recall of campaign exposure. Either of these interpretations is consistent with the observed strong patterns of cross-sectional associations. Table ES-13 shows that significant lagged associations were not found, except in one case. Thus, these analyses do not provide support for a claim that exposure causes outcomes, rather than outcomes cause exposure. Only in the case of fun activities was the lagged association statistically significant, and that result reflected the difference between the highest level of exposure and the other three categories, which were essentially the same. There is no clear interpretation of these findings. They are consistent with the possibility that the causal chain runs from outcomes to recalled exposure. They are also consistent with the possibility that there was not enough additional influence of Wave 1 exposure over and above that already seen in the Wave 1 outcome to be detected.

The parent analyses in this semiannual report had two new elements. The first, just discussed, was the lagged analysis that attempted to help sort through the causal order question. The second innovative element was the incorporation of a youth behavior measure as an outcome in the parent cross- sectional analysis. The youth behavior variable was introduced as an ultimate outcome that extends the evidence for a Campaign effect beyond the associations between exposure and the intermediary outcomes. It was natural to ask the next (and crucial) question as to whether the possible changes produced by the Campaign in the intermediary outcomes had translated into actual behavior change. To date, there is no evidence supporting that claim.

Overall, there are trends and cross-sectional associations consistent with Campaign effects on parent outcomes, including talking behavior and cognitions, and monitoring cognitions. These associations are most consistent for fathers. The longitudinal data do not as yet provide the hoped for additional evidence to rule out reverse causation as an explanation for the observed cross-sectional associations. Also, the evidence does not as yet support an effect of parent exposure on youth behavior. This may reflect the apparent lack of relevance of some of the parent outcomes to youth behavior (talking) or the weak associations of exposure and outcome (monitoring behavior). It may also be that the 18-month interval between NSPY Waves 1 and 4 is insufficiently long for the Campaign to have a measurable influence on youth behavior through this indirect parent exposure route. With the next wave of data, the sample for the longitudinal analyses will grow by 150 percent. This may make it possible to detect longitudinal effects, particularly in subgroups, that were not apparent in the limited analyses that could be performed for this semiannual report. Indeed, the parent cross-sectional associations that are described in this chapter appeared only when the Wave 2 and Wave 3 data were joined to the Wave 1 data. A parallel effect may also occur when the full longitudinal sample becomes available. Yet subsequent waves of data collection will increase the time interval between NSPY's youth and parent respondents' first interview and their most recent interview, thereby providing 2 to 3 years to influence youth marijuana use behavior.

The Campaign's eventual success depends on its influence on youth behavior. The above discussion of possible effects of the Campaign on youth showed little evidence that youth exposure to the Campaign is as yet affecting their behavior directly. Thus, evidence in this section for effects on parents, and for potential eventual indirect effects on youth, are particularly important. However, the argument for indirect effects requires that there be evidence that parent behaviors are protective, showing that children whose parents talk with them about drugs, monitor them, and engage in fun activities with them are less likely to use drugs. There is good correlational evidence that monitoring (both the behavior and the cognitions) and fun activities are related to drug use behavior and intentions. There is not evidence yet that youth have been affected by parent exposure to the Campaign.

In summary, these findings for parents continue to provide some basis for optimism about Campaign effects on parents. Still, there remain possible challenges to claims of Campaign effects. Subsequent reports will be able to investigate these issues more thoroughly. By the end of the evaluation, each respondent will have been measured three times. With those data, claims of effects can be based on a higher standard of inference. Thus, while still subject to future confirmation, at this stage in the Campaign evaluation there are some positive indications of Campaign effects on parents, but not on youth.

Reference

Hornik, R. et al, (2001). Evaluation of the National Youth Anti-Drug Media Campaign: Third Semiannual Report of Findings,Report prepared for the National Institute on Drug Abuse (Contract No. N01DA-8-5063), Washington DC: Westat.

Full Report (PDF Format, 6 MB)
- Table of Contents and Executive Summary (PDF Format, 401 K)
- Chapters 1-6 (PDF Format, 1.6 MB)
- Detail Tables (PDF Format, 1.6 MB)
- Appendices (PDF Format, 1.1 MB)
National Survey of Parents and Youth (PDF Format, 1.3 MB)

Third Semi-Annual Report: Evaluation of the National Youth Anti-Drug Media Campaign - (Released October, 2001)

Executive Summary

The number one goal of The National Drug Control Strategy is to “Educate and enable America’s youth to reject illegal drugs as well as alcohol and tobacco.” One of the objectives in support of that goal includes, “Pursue a vigorous advertising and public communications program dealing with the dangers of drug… use by youth.” Under the Treasury-Postal Appropriations Act of 1998, Congress approved funding (P.L. 105-61) for “a national media campaign to reduce and prevent drug use among young Americans.” Pursuant to this act, the Office of National Drug Control Policy (ONDCP) launched the National Youth Anti-Drug Media Campaign (the Media Campaign).

This program has progressed through three phases of increasing complexity and intensity. Phases I and II are not discussed in this report. ONDCP has available other reports that evaluate those phases. This report focuses on Phase III, which began in September 1999 and is planned to run at least to spring 2002. An evaluation of Phase III is being conducted under contract to the National Institute on Drug Abuse (NIDA) by Westat and its subcontractor, the Annenberg School for Communication at the University of Pennsylvania. Funding of the evaluation is provided by ONDCP from the appropriation for the Media Campaign itself. This is the third semiannual report of the Westat and Annenberg evaluation of Phase III of the Media Campaign.

Highlights from the Third Semiannual Report include:

A brief description of the Media Campaign’s activities to date;
A review of the logic and approach of the evaluation;
Statistics on the level of exposure to messages achieved by the Media Campaign in the first 9 months of Phase III; and
A description of baseline behaviors, beliefs, attitudes, and intentions of both parents and youth. These descriptions focus on the outcomes that will be monitored over time for possible changes that might be brought about by the Media Campaign.

This report from the Westat and Annenberg evaluation presents a first round of measurement. It includes early estimates of exposure to the Media Campaign, and it identifies anti-drug beliefs and drug use behaviors that will be watched over time both for movement and for their association with exposure. It thus sets the stage for the evaluation. This report contains no findings about the effectiveness of the Media Campaign. Such findings after only 9 months of operation of Phase III of the Media Campaign would be premature. This reflects both substantive and technical concerns. From the substantive perspective, effects are expected to be achieved and measurable after a longer period of Media Campaign operations. From the technical perspective, there would be little confidence in inferences from a simple cross-sectional analysis, without even accompanying evidence for change over time in outcomes.

The first report on tentative analyses of effects will be issued after the next wave of data collection in March 2001. At that time, there will be some evidence presented about changes, if any, in outcome measures like the cognitive variables of interest such as beliefs about the consequences of marijuana use at least once or twice in a lifetime. This evidence about change will be complemented by evidence about association of exposure with the outcome measures. However, it is possible that Media Campaign-produced change will take longer to achieve and/or to detect. Indeed, conclusive evidence will take several years to accumulate and analyze. The final report is scheduled for March 2004. At that time, the sample youth and their parents will have been studied for 3 to 4 years.

This report by Westat and Annenberg provides six types of information:

A brief update and description of the Media Campaign’s activities to date;
A review of the logic and approach of the evaluation;
Statistics on the level of exposure to messages achieved by the Media Campaign during Phase III;
Evidence for change in the drug use behaviors of youth between Wave 1 and Wave 3, with an average elapsed time between measures of about 12 months.
Preliminary evidence for Campaign effects on youth. This includes evidence for association between exposure to the Campaign and beliefs, attitudes, and intentions, with statistical controls for confounders and evidence of change between the first and third waves of data collection on these outcomes. In addition, there is analysis of change and associations among subgroups of the population.
Preliminary evidence for Campaign effects on parents. It includes a presentation of evidence for association between exposure to the Campaign and parents’ talk about drugs with their children, their monitoring of their children’s behavior, and their engaging in fun activities with their children, as well as their beliefs and attitudes about talk and about monitoring. It also includes evidence for change between Waves 1 and 3 in these outcomes. Also, both change and association data is reported for some subgroups of the population.