Statement for the Record
Mr. Chairman and Members of the Subcommittee, I am Susan David, Deputy Chief of the Prevention Research Branch at the National Institute on Drug Abuse (NIDA) and the NIDA Project Officer for the Evaluation of the Office of National Drug Control Policy's (ONDCP's) National Youth Anti-Drug Media Campaign. I am pleased to be here on behalf of NIDA to discuss the status of the Phase III Evaluation of the Campaign. While it is this final stage of the campaign (Phase III) in which NIDA is most closely involved, we have provided technical input from the Campaign's onset to ensure its strong scientific foundation. I would like to start by bringing you up to date on the quality and breadth of the evaluation that we are currently undertaking; then provide you with some preliminary findings; and finally discuss future directions for the evaluation.
The Evaluation Design and Methodology
As the Nation's leading research experts in drug abuse and addiction, NIDA was asked by the Office of National Drug Control Policy (ONDCP) in January 1998 to evaluate the outcomes and impact of the television and radio components of the Campaign on parents and children. The following September, NIDA awarded a contract through a fair, open and competitive process to the nationally known health survey research company, Westat. Westat teamed with communication research experts at the Annenberg School for Communication at the University of Pennsylvania, and substance abuse experts at the National Development and Research Institute to assemble a team of nationally recognized experts to lead various aspects of the evaluation.
As I stated earlier, NIDA and the Westat team are currently involved in the Campaign's third fully operational phase. The Phase III evaluation uses a very different methodology from that used for Phases I and II. To ensure a strong science-based evaluation, NIDA and Westat sought guidance from panels of experts on program evaluation, survey research, and communications research, while also building on the knowledge and lessons learned from the earlier phases of the campaign.
Since most of the country was exposed to a national campaign in Phase II, it was recognized in planning the Phase III evaluation that there would be no opportunity to use a more standard evaluation approach where it would be possible to compare those exposed to the campaign against those who were not. Consequently, Westat developed a complex research design that depended heavily on the specific targets, messages, and content of the ONDCP campaign, while also carefully measuring the differences in outcome associated with different levels of exposure to the media.
The overall objective of the Phase III evaluation is to measure the extent to which the television and radio advertising in the Campaign impacts the knowledge, beliefs, attitudes, and behaviors of parents and their children in regard to illegal drug use. The evaluation will also provide information on overall cumulative changes in these factors related to drug abuse as well as exposure to campaign media messages.
Because the media campaign is only one piece of a larger effort that involves a multitude of federal, state and local agencies, and public and private organizations that are working to prevent and reduce drug use in this country, it is extremely challenging to determine the exact causes for any changes in drug abuse rates. To ensure that changes can be attributed to the Campaign, the evaluation was designed to go well beyond the analysis of trends from existing data. The researchers developed a new evaluation survey that uses state-of-the art technology, named the National Survey of Parents and Youth (NSPY). Over the life of the Campaign, NSPY will measure changes among youth, in knowledge, beliefs, attitudes and intentions related to drugs; family and other risk factors; drug use behaviors; and exposure to the Media Campaign. Among parents, the NSPY measures beliefs and attitudes about drugs; behaviors directed at preventing drug use among their children; and exposure to the Media Campaign. Other long-standing national surveys such as NIDA's own Monitoring the Future Survey and the Substance Abuse and Mental Health Services Administration's (SAMHSA's) National Household Survey on Drug Abuse will continue to be used to monitor overall drug use trends.
The NSPY involves conducting interviews of both youths and their parents or caregivers from the same household three times over the course of the 3 _ year evaluation period. The first, baseline recruitment phase consists of three national cross-sectional surveys, or Waves, that each last about six months. The follow-up phase begins with Wave 4, and consists of two additional follow-up interviews with the same youths and their parents.
The NSPY is designed to assess the effect on knowledge, beliefs, attitudes and behaviors that are caused by exposure to the messages in the Campaign advertisements. Because these messages are quite specific, our questions attempt to capture that specificity. For instance, we try to determine and measure whether parents talk to their children, whether children's attitudes about marijuana are positive or negative, and whether teens think their friends approve or disapprove of drug use. Parents and children are also asked whether they recognize specific television and radio advertisements from the Campaign and other general anti-drug advertising, as a self-reported link to Campaign exposure. Since the evaluation is utilizing state-of the-art technology, in this case computers with audio and visual displays, the evaluators can actually show the study participant the media ads that are currently running in the Campaign. This computer technology also allows the subjects more privacy while responding to questions that are prompted by the computer.
One of the unique features of the evaluation is that we are interviewing parents and children in the same household. This improves our ability to measure how parents and children respond to Campaign messages, and how that in turn affects youth drug use. For instance, in response to the Campaign, when parents start talking to their children more about drug abuse, how does that affect their children's attitudes and behavior? How do those attitudes and behaviors change over time?
The analysis of the results will compare groups with high exposure to the campaign with groups who have had lower exposure to assess differences in outcome. To determine if there are any pre-existing differences between high- and low-exposure groups, the survey also includes questions that help to define their risk for drug use, including personal and family history, beliefs, attitudes, perception of social norms, and sensation seeking behaviors.
Evaluation Findings to Date
To keep this committee up-to-date on the status of the evaluation, NIDA has been submitting monthly contractor progress reports. In addition, the Westat evaluation team has completed three in-depth reports: The first was a special report on the overall design of the evaluation, and the others, two semi-annual reports on evaluation results. All three reports have been submitted to Congress by ONDCP, and are displayed on the ONDCP web site. Because there are many researchers interested in this evaluation, NIDA has also posted the reports on the NIDA Archives web site.
The first Semi-Annual Report was released in November 2000. It is based on data collected between November 1999 and May 2000 and serves as the baseline for the Phase III evaluation. Although data collection actually began in November 1999, respondents were asked about advertising they had seen that preceded their interview by two monthsÑi.e., September 1999, when the first Phase III advertising appeared. In this wave, approximately 5,600 interviews were conducted to obtain early estimates of exposure to the Campaign and to identify the beliefs and drug use behaviors that we will be observing and measuring throughout the evaluation.
The Second Semi-Annual Report was released in April 2001. Approximately 4,000 interviews were conducted with parents and youth during this Wave 2 data collection which began in July 2000 and ran through the end of that year. The results were compared to Wave 1 data to analyze changes since that time. Since there were only six months between Waves 1 and 2, we were not surprised that we found few differences in results in that short time frame. Although at this time there is insufficient data to determine if the following results can be directly attributable to the Campaign, here are some of the preliminary findings:
- For both waves, approximately 71% of youth and 70% of parents reported seeing general anti-drug advertisements on a weekly basis.
- Reported marijuana use among youths remained stable over the two waves, which is consistent with data in the Monitoring the Future Survey.
- Among 14-18 year old non-users, several measures showed a movement toward anti-drug attitudes, beliefs, and intentions; one measure, not intending to use marijuana in the next year, increased from 83% in Wave 1 to 87% in Wave 2 in this group; while 12-to-13 year olds did not change significantly in this measure, their intentions were already at a very high level, 92% in Wave 1 and 93% in Wave 2.
- Among 14-18 year old non-users, there was an increase in the belief that their parents would "strongly disapprove" of their trying marijuana, from 92% in Wave 1 to 97% in Wave 2.
- Among parents, there was an increase in the percentage reporting "hearing a lot about anti-drug programs in the community" in the past year, from 32% in Wave 1 to 36% in Wave 2.
Completing the Evaluation
The next Semi-Annual Report, #3, which is scheduled for release in October 2001, will reflect data collected from 4,200 parents and youth during the last wave of new NSPY parents and youth to enter the study. We expect to be able to report on effects for subgroups, such as rural youth and African-American and Latino populations. Because of the additional Campaign exposure time, we will have a greater possibility of detecting change in knowledge, attitudes, beliefs and behaviors than we did for previous waves.
Starting with Wave 4, we will be conducting the follow-up interviews with the respondents who were first interviewed in Wave 1 to determine any changes since that first interview session. The Semiannual Reports for Waves 4 and 5, due in May and October 2002, will yield more definitive results than those from earlier waves. These reports will be based on comparisons between the baseline survey and the first follow-up interview with all respondents and will allow us to more clearly associate changes in drug use attitudes, beliefs, intentions, and behaviors with the exposure to Campaign messages over this time period.
NIDA has also asked the Westat/Annenberg team to prepare data on a variety of environmental influences that may have some effect on Campaign impact. Examples include key informant discussions with leaders of national organizations and State prevention coordinators about major policy and programming initiatives, monitoring media coverage of drug abuse prevention, and special neighborhood analyses based on Census data.
To conclude, I would like to thank you on behalf of NIDA for providing us with this opportunity to personally keep this Committee informed about the progress of the Phase III evaluation. By relying on proven research models, state-of the art survey instruments, and technical expertise in an array of disciplines, this evaluation should be able to meet everyone's overall objective - that is, to assess the impact of the national Campaign on preventing youth drug use nationwide. I will now be happy to answer any questions.