Low-Level Classroom Disruption: Characteristics, Measurement, and Implications, Study Guides, Projects, Research of Accounting

Download Low-Level Classroom Disruption: Characteristics, Measurement, and Implications and more Study Guides, Projects, Research Accounting in PDF only on Docsity!

The Development and Validation of a teacher-reported Low-Level Classroom Disruption Scale (LLCD-S)

Abstract Low-level classroom disruption (LLCD) is characterised by pupils swinging on chairs, whispering or fidgeting in class. This paper provides initial data on the development and validation of the teacher-rated Low-Level Classroom Disruption Scale (LLCD-S), with two samples of primary pupils. Exploratory factor analysis in Study 1 ( N =120) revealed one factor accounting for 61% of the variance; supported by confirmatory factor analysis in Study 2 ( N =274), with one factor accounting for 63% of the variance. Both studies reported high Cronbach’s alpha values of .82 and .93. The evidence supports LLCD being a unidimensional construct, measured by the eight item LLCD-S. Weak convergence validity was found between the LLCD-S and the Strengths and Difficulties Questionnaire’s (SDQ, Goodman, 1997) externalising behaviours: conduct problems and hyperactivity. This preliminary evidence indicates that LLCD-S is a valid and reliable measure of low-level classroom disruption. Further research is needed to test the utility of the LLCD-S across different levels of education, cultures and as a pupil-reported measure. Keywords: low-level classroom disruption scale (LLCD-S); low-level classroom disruption, teacher-report scale; scale validation; primary school. Word Count: 7310

meta-analysis, Aloe, Amo, and Shanahan (2014) demonstrated a link across several studies between teachers’ low classroom management self-efficacy and three dimensions of burnout including emotional exhaustion, depersonalisation and lowered personal accomplishment. In line with this, Skaalvik and Skaalvik (2017) found significant correlations between teachers’ perceived negativity towards interruptions caused by LLCD and their feeling of emotional exhaustion. The impact of LLCD on pupil’s learning is also significant. Emmer, Everston and Worsham, (2009) suggest that minimal distraction enables effective teaching and learning to take place, with more on task time correlated with greater learning gains. In contrast, a dysfunctional atmosphere in a classroom can negatively affects pupil attainment and academic success (Haydn et al. 2014). Longitudinal research by Duncan et al (2007) found that early disruptions to attention in class at 5–6 years old strongly predicted poor reading and maths achievement at 11–12 years old. Importantly, this result controlled for cognitive ability and was similar across gender and socioeconomic status. Furthermore, the relationship between a disruptive classroom environment, poor attainment and lower academic success has been found to extend into early adulthood; with attention problems at primary school predicting lower academic achievements at 17 years, whilst controlling for socioeconomic status and IQ (Breslau et al. 2009). Although the negative effects of LLCD on teaching and learning are well-documented in educational reports and policy documents, empirical research specifically quantifying and reporting on LLCD is sparse. The notable absence of research could be due to the lack of a suitable tool specifically designed to measure LLCD. Elton (1989) highlighted educational concerns over the accurate recording of LLCD concluding that, ‘in the absence of national statistics the problem [LLCD] itself could not be directly measured. Any estimate would have to be based mainly on teachers’ perceptions’ (p.59). More recently, Bennett (2017) has called

for ‘a national standardised method for capturing data on school behaviour’ (p.9) in order to record a range of behaviours including LLCD. The present study sought to address these gaps by designing a standardised scale to specifically measure low-level disruptive behaviours in the classroom. First, it is important to outline the behavioural exemplars that define LLCD, and to differentiate LLCD behaviours from other forms of classroom disruption, named here as high-level behaviours. A delineation of LLCD characteristics now follows. Swinging on a chair, or fidgeting, can comprise a single act of LLCD which is typically low in intensity or power (Sullivan et al. 2014). LLCD has been described as presenting no physical threat or destruction to others or to school property (Kreisberg 2017) and innocuous and/or passive in nature (Beaman and Wheldall 1997). Conversely, a single act of high-level disruption (e.g. such as a kicking/ shouting at a peer or bullying) is of a high intensity and power, typically aggressive, non-compliant and extreme in nature (Wallace 2017). A single episode of high-level disruption will, as a rule, result in a high enough disturbance for teaching to be suspended, and the perpetrator excluded from the room (Hayden and Dunne 2001). These behaviours tend to involve only a single child and are relatively infrequent, which can reduce their overall impact on teaching and learning. In contrast, and fundamental to its definition, LLCD occurs at a high frequency, thus effecting classroom functioning more regularly. Although low in intensity, the rate at which incidents of LLCD occur can result in teachers having to implement a range of behaviour management strategies, which interrupts and reduces instruction time. Ofsted (2014) reported that 20% of teachers identified interruptions caused by LLCD in every lesson, accounting for up to an hour a day of lost learning time for some pupils. Moreover, due to the high frequency of LLCD and its management at a classroom level, the impact of LLCD greater and is felt across the whole class (Hall and Hayden 2007; Swinson 2010).

There are many teacher/carer - rated measures readily available to assess a variety of pupil behaviours. For example, the Children’s Behaviour Questionnaire (Rothbart et al. 2001) assesses temperament (extraversion/surgency, negative affectivity and effortful control) in children aged 3–7 years old and the Sutter-Eyberg Student Behaviour Inventory (Eyberg and Pincus 1999) that captures both the degree to which a behaviour is problematic and its intensity in children aged 2–16 years. The mostly widely used measure is the Strengths and Difficulties Questionnaire (SDQ: Goodman 1997), a behavioural screening tool for 3–16 year olds, which is is often used in clinical settings to assess positive and problematic behaviours across five sub-scales (emotional problems, peer problems, conduct problems, hyperactivity, and prosocial behaviour). All of these aforementioned measures quantify aspects of childhood behavioural problems; however, none specifically measure behaviours associated with the characteristics of LLCD. An exception to this, is the recently developed Pupil Behaviour Questionnaire (PBQ: Allwood et al. 2018). This is the first scale aimed specifically at quantifying behaviours which are related to LLCD within a community sample (N= 2074, age range 4 to 9 years). This teacher-rated scale includes the following items: talking out of turn, interrupting other pupils, making unnecessary noises, making cheeky or rude remarks to the teacher, verbal abuse towards other pupils, and physical aggression towards other pupils. Although these six items achieved Cronbach’s Alpha values of .70 to .90, indicating good to very good internal consistency, the two items relating to verbal abuse and physical aggression are more closely aligned with high-level behaviours and diverge from the characterisation of LLCD identified by teachers (Ofsted, 2014). Furthermore, Allwood et al (2018) compared the scale to the clinically-based Strengths and Difficulties Questionnaire (SDQ: Goodman 1997) to assess the construct validity of the PBQ. They found moderate convergence between the PBQ score and the SDQ total difficulties score ( r = .59). On closer inspection, strong associations were

evident between the PBQ and conduct problems ( r = .67) and hyperactivity ( r = .72) which together indicate externalising behaviour. These moderate to strong convergent associations suggests that both scales may be measuring similar underlying constructs. Moderate divergence was found between the PBQ score and prosocial behaviour ( r = −.53), suggesting that both scales may be measuring opposing underlying constructs. Weak to no associations were evident between the PBQ and peer problems ( r = .19) and emotional symptoms (problems) ( r = .01) which indicate internalising behaviour. These results suggest a strong association between externalising behaviours and LLCD as measured by the PBQ. This diverges from a definition of LLCD as presenting no physical threat or destruction to other pupils or to school property (Kreisberg 2017), and as being more passive in nature (Beaman and Wheldall 1997). Although Allwood et al (2018) concluded that the similarities with the SDQ represented good construct validity of the PBQ, we argue this is does not support the notion of LLCD as being distinct in nature and impact from high-level disruptions, in which are behaviours more closely associated with conduct problems and hyperactivity (Bennett 2017; Ofsted, 2014; Sullivan et al. 2014; Wallace 2017) and that the PBQ may not be a reliable measure of LLCD. The present research The present paper aimed to report on the construction and validation of a scale to quantify levels of teacher-reported LLCD in primary schools. Eight items taken directly from the Ofsted report (2014), which specifically reported on LLCD, were used to construct this new LLCD-S. These were (Q1) talking and chatting, (Q2) disturbing other children, (Q3) calling out, (Q4) not getting on with work, (Q5) purposely making noises to gain attention, (Q6) fidgeting and fiddling with equipment (Q7) answering back or questioning instructions and, (Q8) swinging on chairs. As with exploratory analyses no priori hypothesis relating to these factors and patterns was predicted.

School 2 ‘satisfactory’ (2012), School G ‘good’ (2014) and School M ‘good’ (2015). No Ofsted data was available at that time for School L. Table 1 details the Office of National Statistics (2016) data, highlighting the characteristics of the geographical areas the schools were drawn from and the national figures. [Insert Table 1 here] The sample selection was determined by age. Adolescence (12 years +) can be regarded as a turbulent period in the behavioural trajectory of childhood (Steinberg 2005; Youniss and Smollar 1985). The onset of puberty has been associated with hormonal changes that can influence behaviour (Steinberg 2005), sometimes generating behaviour problems not previously presented (Harms et al. 2014). Therefore, it was advantageous to recruit a pre- adolescent sample, in order to limit such behavioural disturbances. To allow for familiarity to have formed between the class teacher and their pupils, all data for both studies were collected during the final academic summer term. Study 1 took place across two primary schools located in the county of Kent, UK. LLCD data was collected from the class teachers ( N = 4) reporting on the pupil sample ( N = 120). The pupils self-reported their year group, age and gender. The pupil sample was spread across two year groups (5 and 6) with an age range of 9–11 years old ( Mage = 10.29, SD = .64). Of the total pupil sample, 49.2% were in year 5 and 50.8% were in year 6, 59% of pupils identified as male and 41% identified as female. Study 2 took place across three primary schools located in the counties of Kent and Cambridgeshire, UK. Using the LLCD-S, data was collected from the teachers ( N = 8) reporting on the pupil sample ( N = 248). The pupils self-reported their year group, age and gender. The pupil sample was spread across two year groups (year 4 and

5. with an age range of 8–10 years ( Mage = 9.34, SD = .66). Of the total pupil sample, 47.8%

were in year 4 and 52.2% were in year 5, 49% of pupils identified as male and 51% identified as female. Measures Low-level classroom disruption. The intent of this scale construction was to generate items to measure low-level classroom disruption, specifically. Past literature has defined LLCD as having the characteristics of low intensity, high frequency and as having impact across the classroom as a whole unit (Kreisberg 2017; Sullivan et al. 2014; Swinson 2010). To reflect this definition, and to ensure a sufficient breadth of LLCD content was included, the eight highest teacher-rated behavioural issues as highlighted in the Ofsted report (2014) were selected as items. For the present study, teachers were instructed to rate how often each individual pupil in their class carried out the following eight acts: 1) talking and chatting, 2) disturbing other children, 3) calling out, 4) not getting on with work, 5) purposely making noises to gain attention, 6) fidgeting and fiddling with equipment, 7) answering back or questioning instructions and 8) swinging on chairs. Responses were rated on a three-point scale of 1 (never), 2 (sometimes), 3 (a lot), with a higher score indicating a higher presentation of LLCD. Following guidance of prior scale development work (Clark and Watson 1995), all eight items were positively worded to avoid ambiguity in the interpretation of meaning. Limiting the scale to eight items also enables the teachers to complete quick assessment on every child in the class (Slade, Thornicroft, and Glover 1999). Cronbach’s alphas values were very good to excellent, with Study 1 equal to .82 and Study 2 equal to .93. Behaviour. For Study 1, teachers also completed The Strengths and Difficulties Questionnaire (SDQ: Goodman 1997) for each pupil. The SDQ is a well-validated behavioural screening questionnaire for 3–16 year olds, typically completed for clinical diagnostic purposes. The SDQ measures emotional and behavioural changes. Consisting of five subscales (emotional problems, peer problems, conduct problems, hyperactivity, and pro-

recording each pupil’s personal code along with a personal code of their own. On completion of the data collection, all participants (parents, pupils and teachers) received debriefing forms. These contained full details of the study, ethical issues such as post hoc withdrawal from the study and information about help/support lines should they require this. Results: Study 1 Descriptive statistics On average, pupils in the sample were 10.29 years old ( SD = .64; range 9–11) at the time of the data collection. The sample was made up of 59 boys and 61 girls ( N = 120). Table 3 presents the means, standard deviations and bivariate correlations of the LLCD-S items from Study 1. All items of the LLCD-S were positively correlated and larger than .3. [Insert Table 3 here] Exploratory factor analysis (EFA) EFA assessed the eight items of the teacher-rated LLCD-S with maximum likelihood estimator, using SPSS (IBM). The sample size and the strength of relationship between the items indicated suitability of the data for EFA. With a Kaiser-Meyer-Olkin value for the data set of .8, being greater than the recommended .5 (Kaiser 1970). The strength of the relationship between the items considered Pearson’s correlations and revealed the presence of all coefficients larger than .3. Bartlett’s test of sphericity (Bartlett 1954) tested the overall multivariate correlations within the correlation matrix and was significant ( χ2 (28) = 542.64, p < .001). Thus, indicating normality of distribution, supporting the factorability of the data (Table 3). Following the eigenvalue rule, stating only eigenvalues larger than one retained (Howitt and Cramer 2017), the EFA analysis identified the existence of one factor. With an eigenvalue equal to 4.88 achieving a total variance in the data of 60.92%. As Table 4

indicates, the component matrixes revealed very strong loadings on this one factor for all eight items of the measure (>.50). [Insert Table 4 here] Inspection of the scree plot revealed a clear elbow with one point above this, supporting a one-factor solution (Figure 1). Parallel Analysis further supported these results, which showed only one component with eigenvalues exceeding the corresponding criterion values for a randomly generated data matrix of the same size (8 variables x 120 respondents). These results demonstrate that all eight items converge on the same factor, indicating one salient construct underling the LLCD-S item scores. Rotation did not take place, as all eight items loaded sufficiently onto one factor. The LLCD-S demonstrated excellent internal consistency with a Cronbach’s alpha value of .82. This result compared very favourably with the recommended value for scales used in research of above .6 (Nunnally 1978). [Insert Figure 1 here] Convergent Validity Convergent validity was investigated by calculating Spearman’s correlations coefficients between the LLCD-S and the SDQ total scale, and sub-scales of the SDQ (Goodman 1997). As was predicted small positive correlations were found between the LLCD-S score and the SDQ total difficulties and the hyperactivity sub-scale scores, indicting weak similarities. These similarities were noticeably weaker than the moderate similarities that were found for these convergent correlations by Allwood et al (2018). Contrary to the prediction, a medium positive correlation was found between the LLCD-S total score and the conduct problems score, as measured by the SDQ sub-scale. This correlation was similar to the correlation found between LLCD, as measured by the PBQ, and conduct problems as measured by the

Confirmatory Factor Analysis (CFA) Considering the one-factor solution identified in Study 1, CFA was conducted to test the following hypothesis: Low-level classroom disruption, as measured by the eight item LLCD- S, is a one-dimensional construct. Replicating the model from Study 1, the CFA model for the teacher-reported LLCD-S constrained all eight items to load onto one factor. Model fit assessed the CFA indices, indicating a good fit: X^2 (272) = 174.33, p< .001, SRMR = .052, CFI = .90 and TLI = .86 (Hu and Bentler 1999). Moreover, all standardized factor loadings were statistically significant, ranging from .64 to .88 (Table 7). Overall, CFA results indicate adequate factor structure for the cross validation sample. Reflecting Study 1, the Cronbach’s reliability coefficients for Study 2 recorded an alpha value of .93 indicting excellent internal consistency. [Insert Table 7 here] Discussion These studies describe the construction and factor structure of the teacher-reported Low-Level Classroom Disruption Scale (LLCD-S), providing preliminary evidence of the reliability and validity of one factor. First, the study presented previous literature and outlined differences between the concept of low-level classroom disruption and high-level classroom behaviours. LLCD has been consistently defined by teachers as being of low intensity, passive in nature, high frequency and typically disruptive for the whole classroom (Bennett 2017; Ofsted, 2014; Wallace 2017). Whereas, high-level classroom behaviours are conversely characterised by their high intensity, low frequency and typically disruptive for only the perpetrator of the maladaptive behaviour (Bennett 2017; Ofsted, 2014; Sullivan et al. 2014; Wallace 2017). Based on this differential and evidence reported in the Low-level disruption in classrooms: below the radar report (Ofsted, 2014), which specifically investigated LLCD, the eight items capturing the behaviour exemplars of LLCD were defined

as: (Q1) talking and chatting, (Q2) disturbing other children, (Q3) calling out, (Q4) not getting on with work, (Q5) purposely making noises to gain attention, (Q6) fidgeting and fiddling with equipment, (Q7) answering back or questioning instructions, (Q8) swinging on chairs. These eight behaviour exemplars were included as the items for the development of the LLCD-S. Due to the exploratory nature of this new scale no priori hypothesis was forecast. The Study 1 values for the KMO and the Bartlett’s sphericity test revealed that the sample of 120 was large enough for exploratory factor analysis (EFA) to take place and that scores were normally distributed. EFA yielded a one-factor structure, with all eight items loading significantly onto this one factor, explaining 61% of the total variance. As for any scale development it is imperative that the internal consistency of the scale properties is tested on additional data sets; therefore, Study 2 evaluated the LLCD-S with a new sample of 274 primary pupils. The hypothesis that low-level classroom disruption (as measured by the eight item LLCD-S) is a one-dimensional construct was upheld. Confirmatory factor analysis (CFA), based on the previous EFA results from Study 1, supported a single factor model and explained 63% of the total variance. Estimates of the internal consistency of a scale should range from .7 to .9 to indicate reliability (McCrae et al. 2011). Encouragingly, the single factor scale showed strong internal consistency for both Study 1 and Study 2 (.82 and. respectively), indicating that all eight items were measuring the same underlying concept of LLCD and that the LLCD-S is a highly reliable scale across two different samples of primary aged pupils. These excellent internal consistency results allow for the preliminary conclusion that the LLCD-S is an accurate measure of the presentation of low-level classroom disruption with a primary school sample. It therefore provides education practitioners with a much needed and long awaited means of systematically capturing LLCD (Bennett 2017; Elton 1989).

samples were restricted to pre-adolescent primary pupils aged between 8–11 years; however, this limits the generalisability of the results. As LLCD is also reported as a significant issue at secondary education, it is highly recommended that future research should look to expand the sample age range to include adolescence and/or post-adolescence. This would enable important investigations to observe the influences on LLCD, and the changes to LLCD both during and across key developmental stages. Second, the research locations were limited to the counties of Kent and Cambridgeshire, UK. Future investigations of LLCD should look to widen the research areas to include school samples from across the UK, and beyond; in order to capture a more diverse sample, which would also enable the exploration of socio-economic factors. Third, this paper only reports on the observer-rated scores of the LLCD-S from class teachers. Future studies could evaluate a pupils’ self-reported LLCD-S in order to reduce the risk of common method variance. Finally, the cross-sectional nature of this study does not allow for test-retest assessment of the external reliability of the LLCD-S. Having the same sample report levels of LLCD, over two or more separate data collection waves, would allow correlations between the time points to be calculated. Therefore, future studies could overcome this limitation by implementing a longitudinal design. Concluding Remarks In conclusion, LLCD has been consistently emphasised as the number one behavioural issue in primary schools, with a negative impact on both teachers and pupils (Bennett 2017: Elton 1989; Ofsted, 2014; Steer 2005). Considering this, and addressing a recognised gap in the literature, the LLCD-S can be effectively utilised for screening and/or as an outcome measure to accurately record low-level classroom disruption presentation at primary school level (Elton 1989; Ofsted, 2014). Importantly, the LLCD-S focuses specifically on LLCD clearly addressing only low-level maladaptive behaviours, which differentiates the LLCD-S from prior measures that include high-level maladaptive

behaviours. This scale would be beneficial to quantify levels of LLCD across individual pupils, classrooms and schools. Further exploration of the scale is required across time with various age populations, across the UK and beyond, and additionally as a pupil self-report measure. Disclosure statement No potential conflict of interest was reported by the authors.