The director of MAEL is embarking on a major research project. It centres on the fact that there are many different ways of implementing the Montessori approach. As a result, it is hard to say which is the most authentic, even harder to tell which model gives the best results for children. There is very little research on this question (at least in the English language.) This project is designed to try and provide some empirical data, but I need some collaborators.
In a review of Montessori Research in the U.S.A., Chattin-McNicholls (1990, 2009) drew attention to several systematic problems with product-research in this field. Among these issues was the concern that “the confusion between programs and the model they supposedly represent has been a major source of confusion in the research on Montessori.” (2009:26). (The other issues were small sample sizes, short-term studies, confounding the effects of a model with parent selection effects and the use of measuring instruments which do not relate to the unique Montessori goals.) It is widely known in the Montessori community, but perhaps less so generally, that there are striking contrasts between differing modes of implementation of the Montessori approach. Daoust (2004) studied 66 Montessori pre-schools, using telephone interviews with randomly selected staff members, to explore their implementation of Montessori practice along 5 dimensions. She found 4 distinct clusters, (which she termed traditional, contemporary, blended and experimental) which conforms with a personal, informal impression based on extensive links within the movement, although the sample base (half the Montessori schools in one county in California) is rather localised. This is the only (Daoust, 2013, pers. comm.) published academic study into this aspect of Montessori education. This variability of implementation was also noted by Sammons & Eliot (n.d.:4) who estimated that around 25% of the variation in children’s progress (specifically based on the Marie Clay assessment measure) could be attributed to the “significant differences between schools.” Most studies of the effectiveness of the Montessori approach make no such distinction, and thus it is impossible to tell if their data is skewed by this variability.
The initial phase of the research will focus on the analysis of Montessori implementational models, and is basically a replication of Daoust’s work with an international data set. The research question will be ‘Are there identifiable clusters of modes of implementation in Montessori pre-schools?’ Survey data may be collected from three main sources: firstly from visits to and interviews with personnel in English Montessori pre-schools in an attempt to replicate Daoust’s study in a British context and see whether Daoust’s clusters are confirmed, and also to compare the delimiting factors; secondly from a questionnaire survey of international settings and thirdly from a survey of the literature describing early (pre-1914) Montessori schools.
On the assumption that clusters will be identifiable, the second research question will be ‘Can reliable indicators be identified for each cluster?’ Any indicators, to be useful, will need to be reliable but also easy to measure. The identification of such indicators would enable effectiveness research to be more closely focused, thus avoiding the potential bias due to conflation of models. Goldstein (1997) has drawn attention to the inherent limitations of indicators as a measure of effectiveness, and this area will need to be addressed in detail.
The third phase of the research (and possibly the central strand) will be to look at the different effectiveness of the clusters identified, using the indicators to assign settings to clusters. The research question could be ‘Does the model of implementation impact on the effectiveness of Montessori pre-schools?’ This stage of the research could start with a meta-analysis of effectiveness studies which included Montessori settings (focussing particularly on the EPPE data), using the identified indicators (subject to the limitations mentioned above), to see if there is any significant link between effectiveness and the assigned cluster. There are many issues with the methodologies of school effectiveness research (SER) and these have been summarised by Sammons (n.d.), particularly focusing on problems around added-value. .
There are also issues with this approach in that the usual measures of effectiveness in a mainstream context (frequently used in such studies, e.g. Lillard & Else-Quest, 2006) are not necessarily applicable to a Montessori setting, although many studies, including Sammons & Eliot and Lillard & Else-Quest, have shown a positive effect (Lillard (2012) identified 8 peer-reviewed published studies of Montessori effectiveness in the last 15 years, of which 3 did not show any positive effect.) The few cases where alternative measures have been used have led to inconclusive results. Sammons & Eliot (n.d.:3, 13) found these measures “less clear-cut” and a longitudinal study of the Franciscan Montessori Earth school in Portland (Glen, 2003) also found results, whilst positive, were not conclusive and he suggests the influence of Montessori is ‘subtle’. A similar finding (in the context of Belgian Steiner and Frienet schools) was reported by de Bilde et al (2013) where independence and engagement was greater in mainstream schools. There are unexplored issues here.
Lillard (2012) has recently undertaken a study of the effect of implementational variation on the effectiveness of Montessori education. She studied two types of implementation (which she called classic or high-fidelity and supplemented – which relate to Daoust’s traditional and contemporary – and which are particularly promoted by the two major Montessori training organisations in the USA.) using a single measure (equipment availability) to differentiate them. She also studied some highly regarded mainstream pre-schools as a control. Lillard used 6 measures of school readiness over a school year and found, in all but one case, that the classic implementation was the most effective. Some differences (e.g. in the measure of executive function, using the HTKS task) were striking. In that example, children in a classic implementation increased their scores by 13.72 points, in supplemented approaches by 7.34 and in the mainstream by 7.85. Other results were less clear cut, and in one case (theory of mind, tested by a false belief task followed by either a hidden emotion or a perceptual access task) there were no significant differences. Overall, though, the classic approach showed the best results. Interestingly, the marked gains from the classic approach appeared to significantly dissipate over the Summer recess and Lillard, whilst proposing several possible mechanisms, was unable to use her data to choose between them. This is the only study to attempt to elucidate any implementational effect, although it only involved 172 children in 18 classrooms (only 3 of which were ‘classic’ Montessori.)
Different measures of ‘Montessori effectiveness’ may then be investigated to see if they could be used in a sample of the settings used in the initial survey. These could include:
- concentration (possibly using the Leuven scale for involvement – although there are contextual issues here)
- executive function (as used by Lillard, 2012)
- independence (possibly using an existing measure of teacher intervention)
- peacefulness (possibly using an existing measure of conflict) or social problem solving (as used by Lillard, 2012)
- freedom (measures to be investigated as the Montessori concept of freedom is closer to Friere’s than to many other alternative approaches.)
- environmental structure
- curriculum navigation
The null hypothesis would be that there are no significant differences between the clusters. If this is confirmed then it would suggest that personnel issues are more significant than methodology. This is probably what Montessori herself would have expected. She laid great stress on what she termed the ‘spiritual development’ of the teacher and less than some of her followers on the minutiae of the presentations. However Lillard (2012) dismisses any effect of teacher experience, on the basis of one quoted research paper. This conflicts with findings from mainstream research, e.g. Sylva et al (2004). If that is the finding, the whole emphasis of the rest of the programme would have to be re-thought. The research expectation, prior to any empirical study but based on years of visiting Montessori schools, is that the traditional and contemporary clusters would be relatively similar in terms of effectiveness, and considerably better than either the blended or experimental clusters. Lillard (2012) found that the traditional approach produced better results overall. However she drew attention to the ‘fidelity paradox’ wherein a more exact reproduction of a model usually produces better results but which tends to result in the approach dying out, whilst amendment tends to ensure longevity. She notes this has been documented in the health sector but is only mentioned in one Montessori text.
If a differential effect of implementational model is confirmed then an instrument to help Montessori settings improve their outcomes could be adapted from the ECERSR, as proposed by Sammons & Eliot (n.d.;5). Rigg (2010) has published an attempt at this project but unfortunately she shows little appreciation of the nature of a rating scale or of the usual norms of validity assessment. This final phase might be based on the question ‘Can the identified indicators be used as the basis for a Montessori environmental rating scale?’ This would need to be trialled and, presumably, validated by the Montessori community.
At this point in time, I need collaborators who would be prepared to administer the questionnaire (for the first research question) to a selection of pre-schools. The questionnaire is included as appendix A. The questionnaire may be administered through an interview (e.g. by phone) or by being given to the classroom directress. They can be returned as hardcopy or by email. One thing I would stress is that I need a broad and representative sample, i.e. including weak and also eccentric pre-schools as well as authentic ones. If anyone would like to take part in the research, please email MAEL on info@montessoriassessment.com
References
Chattin-McNicholls, J. (1990). What does research tell us about Montessori. in Loeffler M.(ed.) (1992) Montessori in Contemporary American Culture. Portsmouth NH: Ginn; revised & reprinted in Clarkson J. (ed.) (2009) A Brief Guide to Collaborative Practitioner Research in a Montessori Context Westhorpe, MAEL.
Daoust, C.J. (2004). An Examination of Implementation Practices in Montessori Early Childhood Education. unpublished Ph.D. thesis University of California, Berkeley. Available at http://www.amshq.org/Publications%20and%20Research/
de Bilde, J. et al. (2013). Can alternative education increase children’s early school engagement? A longitudinal study from kindergarten to third grade. School Effectiveness and School Improvement 24(2): 212-233
Glenn, C.M. (2003). The Longitudinal Assessment Study (LAS), Cycle 6 (Eighteen Year) Final Follow-up. Portland, OR: Franciscan Montessori Earth School. (ERIC Document Reproduction Service No. ED478792) .
Goldstein, H. (1997). Methods in school effectiveness research. School effectiveness
and school improvement. 8: 369-95.
Kramer R. (1976) Maria Montessori: a Biography New York, G.P. Putnam’s Sons (2nd ed. 1988, Reading MA, Addison-Wesley, with intro. by Freud, Anna).
Lillard, A.S. (2012). Preschool children’s development in classic Montessori, supplemented Montessori, and conventional programs. J. School Psych. 50: 379-401.
Lillard, A.S. & Else-Quest, N. (2006). Evaluating Montessori Education. Science 313(5795): 1893-1894.
Montessori Schools Association (2011). Member schools Ofsted Grades Tally 2011. London: Montessori St. Nicholas Trust..
Rigg, P. Z. (2010). Montessori Early Childhood Environmental Rating Scale. San Leandro, CA; Montessori Research & Development.
Sammons, P. (n.d.). School Effectiveness and Equity: making connections. London: CfBT (accessed 13.6.13 from http://www.cfbt.com/evidenceforeducation /pdf/ Full%20Literature%20Review.pdf)
Sammons, P. & Eliot, K. (n.d.). Investigating the Impact of Montessori Schools on Children’s’ Educational Outcomes. London: Institute of Education.
Sylva, K. et al. (2004) Effective Pre-School Education London: The Institute of Education.
Category: Uncategorized