Eur J Pediatr Surg 2022; 32(05): 399-407
DOI: 10.1055/s-0041-1736387
Original Article

Validation of a Newly Developed Competency Assessment Tool for the Posterior Sagittal Anorectoplasty

1   Department of Pediatric Surgery, Radboud University Medical Center, Nijmegen, the Netherlands
,
Guus M.J. Bökkerink
2   Department of Pediatric Surgery, Princess Maxima Center for Pediatric Oncology, Utrecht, Utrecht, the Netherlands
,
Jonathan Sutcliffe
3   Department of Paediatric Surgery, Leeds Children's Hospital, F Floor, Martin Wing Leeds General Infirmary GT George Street, Leeds, W Yorkshire, United Kingdom
,
Marc A. Levitt
4   Department of Surgery, Colorectal and Pelvic Reconstructive Surgery, Children's National Hospital, District of Columbia, Washington, United States
,
Karen Diefenbach
5   Department of Pediatric Surgery, Nationwide Children's Hospital, Columbus, Ohio, United States
,
Carlos A. Reck
6   Department of Pediatric Surgery, Medical University of Vienna, Wien, Wien, Austria
,
6   Department of Pediatric Surgery, Medical University of Vienna, Wien, Wien, Austria
,
7   Department of Pediatric Surgery, Radboud Medical Centre, Nijmegen, the Netherlands
,
Sanne M.B.I. Botden
8   Department of Pediatric Surgery, Radboudumc-Amalia Children's Hospital, Nijmegen, the Netherlands
› Author Affiliations
Funding The authors have no financial ties to disclosure. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Abstract

Introduction The correction of an anorectal malformation (ARM) is complex and relatively infrequent. Simulation training and subsequent assessment may result in better clinical outcomes. Assessment can be done using a competency assessment tool (CAT). This study aims to develop and validate a CAT for the posterior sagittal anorectoplasty (PSARP) on a simulation model.

Materials and Methods The CAT-PSARP was developed after consultation with experts in the field. The PSARP was divided into five steps, while tissue and instrument handling were scored separately. Participants of pediatric colorectal hands-on courses in 2019 and 2020 were asked to participate. They performed one PSARP procedure on an ARM simulation model, while being assessed by two objective observers using the CAT-PSARP.

Results A total of 82 participants were enrolled. A fair interobserver agreement was found for general skills (intraclass correlation coefficient [ICC] = 0.524, p < 0.001), a good agreement for specific skills (ICC = 0.646, p < 0.001), and overall performance (ICC = 0.669, p < 0.001). The experienced group scored higher on all steps (p < 0.001), except for “anoplasty (p = 0.540),” compared with an inexperienced group.

Conclusion The CAT-PSARP is a suitable objective assessment tool for the overall performance of the included steps of the PSARP for repair of an ARM on a simulation model.

Ethics Approval and Consent to Participate

Written informed consent was obtained from all participants. According to national law and legislation, ethical board approval of the ethics committee of Arnhem and Nijmegen was deemed unnecessary. Approval of the ethics committee of the institution Radboudumc was waived according to national regulations.[26] The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.


Availability of Data and Materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.


Authors' Contributions

Study concept and design: M.J., I.B., and S.M.B.I.B.


Data acquisition: M.J., G.M.J.B., J.S., M.A.L., K.D., C.A.R., W.K., I.B., and S.M.B.I.B.


Analysis and data interpretation: M.J.


Drafting manuscript: M.J.


Critical revision: G.M.J.B., J.S., M.A.L., K.D., C.A.R., W.K., I.B., and S.M.B.I.B.


All authors read and approved the final manuscript.




Publication History

Received: 21 May 2021

Accepted: 20 August 2021

Article published online:
25 November 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 Baldwin PJ, Paisley AM, Brown SP. Consultant surgeons' opinion of the skills required of basic surgical trainees. Br J Surg 1999; 86 (08) 1078-1082
  • 2 Brennan MF, Debas HT. Surgical education in the United States: portents for change. Ann Surg 2004; 240 (04) 565-572
  • 3 Collins JP, Civil ID, Sugrue M, Balogh Z, Chehade MJ. Surgical education and training in Australia and New Zealand. World J Surg 2008; 32 (10) 2138-2144
  • 4 Frank JR, Langer B. Collaboration, communication, management, and advocacy: teaching surgeons new skills through the CanMEDS Project. World J Surg 2003; 27 (08) 972-978 , discussion 978
  • 5 Smith AJ, Aggarwal R, Warren OJ, Paraskeva P. Surgical training and certification in the United kingdom. World J Surg 2009; 33 (02) 174-179
  • 6 Epstein RM. Assessment in medical education. N Engl J Med 2007; 356 (04) 387-396
  • 7 Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB. Observational tools for assessment of procedural skills: a systematic review. Am J Surg 2011; 202 (04) 469-480.e6
  • 8 Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 2001; 193 (05) 479-485
  • 9 Aggarwal R, Grantcharov TP, Eriksen JR. et al. An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 2006; 244 (02) 310-314
  • 10 Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg 2012; 99 (10) 1322-1330
  • 11 Eriksen JR, Grantcharov T. Objective assessment of laparoscopic skills using a virtual reality stimulator. Surg Endosc 2005; 19 (09) 1216-1219
  • 12 Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg 2004; 91 (12) 1549-1558
  • 13 Ganni S, Chmarra MK, Goossens RHM, Jakimowicz JJ. Self-assessment in laparoscopic surgical skills training: is it reliable?. Surg Endosc 2017; 31 (06) 2451-2456
  • 14 Anderson DD, Long S, Thomas GW, Putnam MD, Bechtold JE, Karam MD. Objective structured assessments of technical skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res 2016; 474 (04) 874-881
  • 15 Miskovic D, Ni M, Wyles SM. et al; National Training Programme in Laparoscopic Colorectal Surgery in England. Is competency assessment at the specialist level achievable? A study for the national training programme in laparoscopic colorectal surgery in England. Ann Surg 2013; 257 (03) 476-482
  • 16 Ganni S, Botden SMBI, Schaap DP, Verhoeven BH, Goossens RHM, Jakimowicz JJ. “Reflection-before-practice” improves self-assessment and end-performance in laparoscopic surgical skills training. J Surg Educ 2018; 75 (02) 527-533
  • 17 IJgosse WM, Leijte E, Ganni S. et al. Competency assessment tool for laparoscopic suturing: development and reliability evaluation. Surg Endosc 2020; 34 (07) 2947-2953
  • 18 Levitt MA, Peña A. Anorectal malformations. Orphanet J Rare Dis 2007; 2: 33
  • 19 Peña A, Devries PA. Posterior sagittal anorectoplasty: important technical considerations and new applications. J Pediatr Surg 1982; 17 (06) 796-811
  • 20 van Ling JA, Bökkerink GMJ, de Blaauw I, Botden SMBI. Development of a posterior sagittal anorectal surgical teaching model. BMC Pediatr 2021; 21 (01) 57
  • 21 Joosten M, Bökkerink GMJ, Levitt MA. et al. The use of an inanimate simulation model for the correction of an anorectal malformation in the training of colorectal pediatric surgery. Eur J Pediatr Surg 2021; ( e-pub ahead of print) DOI: 10.1055/s-0041-1723035.
  • 22 Groenier M, Brummer L, Bunting BP, Gallagher AG. Reliability of observational assessment methods for outcome-based assessment of surgical skill: systematic review and meta-analyses. J Surg Educ 2020; 77 (01) 189-201
  • 23 Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15 (02) 155-163
  • 24 Popović ZB, Thomas JD. Assessing observer variability: a user's guide. Cardiovasc Diagn Ther 2017; 7 (03) 317-324
  • 25 Mahtani K, Spencer EA, Brassey J, Heneghan C. Catalogue of bias: observer bias. BMJ Evid Based Med 2018; 23 (01) 23-24
  • 26 WMO. Wet medisch-wetenschappelijk onderzoek met mensen. Accessed on April 16, 2021 at https://wetten.overheid.nl/BWBR0009408/2020-01-01