Martin Sutton writes about his Highly Commended dissertation, submitted as part of his MAEd (Geography) from UCL Institute of Education.
Teaching about summative assessment with trainee teachers has long intrigued me, especially the tension between accurately assessing work and the inherent challenges that any assessment process can entail. My interest in geography assessment, coupled with my joint roles of being both a PGCE teacher educator and also a secondary school geography teacher, meant that I could draw upon both perspectives to inform my research. I have often wondered how trainee teachers are ‘taught’ to summatively assess work and how well they can transfer what they have learned into practice.
Popham (2011, p.267) used the phrase ‘assessment literacy’ to describe the understanding and ability of an educator to grapple with the theoretical and practical demands of this field. This study set out to investigate assessment literacy with both trainees and also their school mentors. The relationship between mentor and trainee is well researched (Lord, Atkinson & Mitchell, 2008; Rehman & Al-Bargi, 2014; Roberts, 2019; Healy et al. 2022). The mentoring role is clearly valued by the DfE, who have constructed non-statutory Mentor Standards (DfE, 2016) which stipulate that trainees should receive support from mentors around assessment and marking. My research addressed how this mentor-trainee interaction worked in terms of summative assessment literacy.
Lambert (2011, p.5) summarises the geography assessment landscape when he claims that “assessing progress is particularly challenging in a subject like geography which is not learned in a cumulative or linear sequence.” This is a notion to which many classroom teachers can empathise with. The interconnectedness of the subject has been widely agreed upon from Massey’s “a sense of the global” (2014, p.36) with porous boundaries, to Jackson’s “geographies of connection” (2006, p.199). It is this synoptic characteristic which makes assessing progress in the acquisition of knowledge and skills a demanding task, as the Geographical Association has for many years attempted to clarify (GA undated).
The discussion around formative assessment (for learning, rather than of learning) is a well-trodden path. Black and Wiliam’s (1998) seminal meta-analysis of assessment for learning, sparked the publication of geography specific assessment research, by Weedon and Lambert (2006). They champion the use of formative feedback in a subject specific context and point to further work that suggests that feedback should not be accompanied by a mark or grade (Butler, 1987). In line with work of their predecessors (Black & Wiliam, 1998; Bloom, 1969; Popham, 2008), Weedon and Lambert (2006) outline the advantages of peer assessment, self-assessment and encouraging the pupils to reflect on their own work (such as the use of traffic lighting their own performance, confidence or knowledge). Pupils were positioned at the heart of the assessment process, in line with later work by Weedon, co-authored with Hopkin (2006; Figure 1). However, there is relatively little literature that concentrates on summative assessment or the assessment of learning (although, see Lambert and Lines, 2000).
Dempsey et al. (2009) identified a clear need for improved assessment skills among trainee teachers and tested a web-based coaching software to address this. They found that by exposing the marker to short prompts and hints, coupled with the marks from peers, could help trainee teachers to better assess children’s work at a primary school level. Inspired by their work, my study investigated whether a similar tool could benefit UK trainee geography teachers and their school mentors.
Considering the time constraints reported by both mentors and trainees, I conducted research to evaluate the potential effectiveness of a web-based solution in this context.
By using a collection of carefully designed coaching comments, the 15 trainee teachers were exposed to a coaching intervention in an attempt to teach them how to mark examination answers. The trainees were shown an exam answer and asked to suggest both a raw score out of 9 and also a more general level (1-3), based on a rubric written by the exam board. The software then displayed a coaching suggestion for the trainee to read, that was specifically written for the question, prompting them to look at a specific part of the rubric. These statements were written based on the feedback from a cluster of qualified teachers. The trainee then had the option to re-score the answer if they wished to.
The research focused on measuring both the self-efficacy and also the accuracy of the trainees’ marking, across a set of 7 sample GCSE examination answers. Additionally, the views of 37 qualified geography teachers, who all work in teacher teaching, was collected and analysed.
By undertaking a pre- and post-questionnaire, the trainees’ change in self-efficacy was found to have significantly improved (p<0.05). Furthermore, their ‘gain scores’ could be calculated by comparing their judgement before and after the coaching intervention. The trainees were shown to have significantly improved their marking accuracy when they were shown a coaching comment, when they were a mark or more away from the ‘correct’ score (p<=0.01). The teachers were asked for their opinion on the coaching software in comparison to their current practice and reported a strong preference for this novel pedagogy (p<0.05). Although nothing can replace to experience gained through actually becoming an examiner, it appears that this coaching intervention was valued as opening up this particular ‘black box’.
The trainees and teachers cited an increase in independent practice and time efficiency as the two main strengths of the experience. They suggested that the face-to-face element of a post task discussion should be maintained in future practice. The study suggests that online coaching software should be used within the ITT year and additionally across the wider subject community, such as the RGS or the GA, to deliver powerful geography CPD to trainees and qualified teachers.
It has become clear to me that since submitting my dissertation, that there has been a recent explosion in the use of generative Artificial Intelligence in education. It would be perfectly feasible for this technology to be coupled with the software that I have designed, so that the intervention statement given to the trainee was generated by AI, based upon their view of the work itself. This would lead to an exceptionally bespoke supportive prompt that would demand more research and thought into its use in geography education.
Note: Please feel free to contact the author should you wish to know more about the software that he developed as part of his research. m.sutton@reading.ac.uk
References
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi delta kappan, 92(1), 81-90.
Bloom, B.S. (1969). Some theoretical issues relating to educational evaluation. In Educational evaluation: New roles, new means. The 63rd yearbook of the National Society for the Study of Education, part 2 (Vol. 69), ed. R.W. Tyler, 26–50. Chicago, IL: University of Chicago Press.
Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest, and performance. Journal of Educational Psychology, 79(4), 474.
Dempsey, M. S., PytlikZillig, L. M., & Bruning, R. H. (2009). Helping preservice teachers learn to assess writing: Practice and feedback in a Web-based environment. Assessing writing, 14(1), 38-61.
DfE (2016). National Standards for school-based initial teacher training (ITT) mentors. Available at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/536891/Mentor_standards_report_Final.pdf (Accessed: 05/06/2024).
Geographical Association (undated) Progression in geographical learning, Geographical Association. https://geography.org.uk › students-learning-in-geography
Jackson, P. (2006). Thinking geographically. Geography, 91(3), 199-204.
Healy, G. Hammond, L., Puttick, S., and Walshe, N. (eds) (2022) Mentoring Geography Teachers in the Secondary School. Abingdon: Routledge
Lambert, D. (2011). The Geography National Curriculum: GA curriculum proposals and rationale. Geographical Association. Available at: https://www.geography.org.uk/download/ga_gigcccurriculumproposals.pdf (Accessed: 05/06/2024).
Lambert, D. and Lines, D. (2000) Understanding Assessment: purposes, perceptions, practice. Abingdon: Routledge
Lord, P., Atkinson, M. and Mitchell, H., (2008). Mentoring and coaching for professionals: A study of the research evidence. Variations, 1(4).
Massey, D. (2014). Taking on the world. Geography, 99(1), 36-39.
Popham, W. J. (2011). Assessment literacy overlooked: A teacher educator’s confession. The Teacher Educator, 46(4), pp.265-273.
Popham, W.J. (2008). Transformative assessment, Alexandria, VA: ASCD.
Rehman, A. A. & Al-Bargi, A. (2014). Teachers’ Perspectives on Post Observation Conferences: A Study at a Saudi Arabian University. Theory and practice in Language Studies, 4(8), p.1558.
Roberts, R.L. (2019) in Hickman, D. (ed) Mentoring English teachers in the secondary school: a practical guide. London: Routledge.
Weedon, P., & Lambert, D. (2006). Geography inside the black box: Assessment for learning in the geography classroom. Sheffield: The Geographical Association.
Weedon, P., and Hopkin, J. (2006). Assessment for learning in geography. In Jones, M. (2017). Handbook of Secondary Geography. Geographical Association.