Longitudinal assessment of the SHAHRP program (Australian)
Formative development
Fundamental to intervention research is a formative phase of development to ensure that the intervention is attune with consumer needs and activities. During the development of the SHAHRP intervention, particular attention was given to ensuring the intervention was based on the latest evidence by incorporating results from a systematic literature review of school drug education, was appropriate and relevant to young people by incorporating realistic scenarios and situations based on focus groups conducted with young people, and was tested with teachers and students in the school setting and incorporated modifications to make the program workable in schools and to enhance the potential for success.
Focus Groups
To ensure that the SHAHRP intervention was sensitive to the concerns of the young people it sought to influence, the SHAHRP researchers conducted a series of focus groups with 80 Western Australian teenagers. Young people were allocated to the focus groups based on gender as it was felt that different issues could be raised. The focus groups aimed to identify various issues including young people's alcohol use experiences, alcohol related harms of particular concern to young people, harm reduction strategies adopted by young people, and educational approaches likely to be effective with young people.
The young people were generally keen to express their opinions about alcohol and issues associated with its use. There was consistency in the issues that young people raised in the focus groups.
Piloting Intervention Program
Pre-testing of the SHAHRP intervention played an important role in refining the activities and materials by incorporating teacher and student responses, while maintaining the integrity of the evidence-based components. Three Western Australian secondary schools were recruited to pilot the SHAHRP intervention. SHAHRP was implemented in six year eight classes (12 - 13-year-olds) involving approximately 200 students and six teachers. Before implementation, pilot teachers attended a two-day training workshop based on interactive modelling of intervention activities. Pilot teachers provided written and verbal evaluation of this workshop.
Process evaluation, fidelity of implementation and other monitoring procedures were also piloted during this initial formative stage of the study. Pilot teachers were asked to complete a process evaluation form for each activity immediately after conducting the lesson. The process evaluation forms required comment on level of completion, student response, teacher response and suggestions to retain or modify each activity. Teachers sent these forms to the research team immediately after completion of each lesson. This information provided an ongoing list of suggested modifications and impressions that were used to revise the SHAHRP intervention before the main study.
Evaluation Workshop
An evaluation workshop was conducted at the conclusion of the pilot to further refine the SHAHRP intervention. The workshop was attended by the pilot teachers, the primary writer of the intervention, and the SHAHRP research team. A triangulation of measures was adopted to assess the intervention including teacher and student monitoring. Each activity was discussed and assessed by the pilot teachers, according to intention, content, methods, relevance to age, timing and training. The process evaluation forms, which required comment on level of activity completion, student response, teacher response and suggestions to retain or modify for each activity, were reviewed. Each teacher also provided written student comments about what the young people liked and disliked about the intervention and recommendations for improvements.
These recommendations were incorporated into the SHAHRP intervention prior to longitudinal assessment of the program. One of the most valuable aspects of the pilot process was ‘testing the SHAHRP intervention in the real world' of the classroom, thereby gaining insight into the potential effectiveness of the harm reduction program that incorporated evidence within the school setting.
Study Design
The SHAHRP Randomised Controlled Trial was an efficacy study based on a quasi-experimental, intervention research design. The study explored the effects of a student focused, secondary school, alcohol curriculum program on young people’s alcohol-related knowledge, attitudes, patterns and context of alcohol use and alcohol related harm. The study reflects the Australian National Drug Strategy by adopting a harm minimisation approach; in this case, aiming to reduce the level of alcohol-related harm in young people who drink alcohol, and to reduce the harm experienced by those young people who do not drink alcohol, but socialise with others who do drink. Figure 1 provides an overview of the study design including phases of intervention and data collection.
Figure 1: Overview of Study Design
Month and year |
2/97 |
5/97
8/97 |
10/97 |
5/98 |
10/98 |
10/99 |
10/00 |
10/01 |
Intervention group |
O1 |
X1 |
O2 |
X2 |
O3 |
O4 |
O5 |
O6 |
Control group |
O1 |
|
O2 |
X |
O3 |
O4 |
O5 |
O6 |
Key: |
X: Intervention |
X control school regular alcohol education |
|
O: Observation |
Sampling: The study sample was selected using cluster sampling, with stratification by socio-economic area. The baseline sample size of 2300 young people considers the design effect created by cluster sampling. The fourteen schools involved in the SHAHRP study represent approximately 23% of government, secondary schools in the Perth metropolitan area. Random allocation to intervention and control conditions occurred by school.
Power calculations suggested that recruitment of a minimum of 800 students was required to achieve statistical power greater than 0.9 to detect an effect size of 0.15 with a coefficient variation of 25% through simple random sampling. The SHAHRP sample of 2343 students (intervention n=1111, control group n=1232) considered the design effect created by cluster sampling (design effect = 1.48; minimum sample required = 1184) and allowed scope for an expected attrition rate of fifteen percent per year.
Retention: The retention rate of study students was 75.9% over 32 months.