An instrument for evaluating clinical teaching in Japan: content validity and cultural sensitivity

BMC Medical Education

Albert JJA Scherpbier¹  Sue Roff⁵  Sei Emura⁴  Renee E Stalmeijer³  Makoto Kikukawa² 

[1] Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands;Department of Medical Education, Kyushu University, 3-1-1 Maidashi Higashi-ku Fukuoka, 81-8582 Kyushu, Japan;Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands;Centre for Graduate Medical Education Development and Research, Saga University Hospital, Saga, Japan;The Centre for Medical Education, Dundee Medical School, Dundee, Scotland

关键词: Content validity;    Modified Delphi method;    Culture;    Japan;    Clinical teaching;    Instrument;    Evaluation;   

Others  :  1091680
DOI  :  10.1186/1472-6920-14-179

 received in 2014-01-18, accepted in 2014-08-08,  发布年份 2014

【 摘 要 】

Background

Many instruments for evaluating clinical teaching have been developed but almost all in Western countries. None of these instruments have been validated for the Asian culture, and a literature search yielded no instruments that were developed specifically for that culture. A key element that influences content validity in developing instruments for evaluating the quality of teaching is culture. The aim of this study was to develop a culture-specific instrument with strong content validity for evaluating clinical teaching in initial medical postgraduate training in Japan.

Methods

Based on data from a literature search and an earlier study we prepared a draft evaluation instrument. To ensure a good cultural fit of the instrument with the Asian context we conducted a modified Delphi procedure among three groups of stakeholders (five education experts, twelve clinical teachers and ten residents) to establish content validity, as this factor is particularly susceptible to cultural factors.

Results

Two rounds of Delphi were conducted. Through the procedure, 52 prospective items were reworded, combined or eliminated, resulting in a 25-item instrument validated for the Japanese setting.

Conclusions

This is the first study describing the development and content validation of an instrument for evaluating clinical teaching specifically tailored to an East Asian setting. The instrument has similarities and differences compared with instruments of Western origin. Our findings suggest that designers of evaluation instruments should consider the probability that the content validity of instruments for evaluating clinical teachers can be influenced by cultural aspects.

【 授权许可】

   
2014 Kikukawa et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Lombarts KM, Bucx MJL, Arah OA: Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology 2009, 111:709.
• [2]Ficklin FL, Browne VL, Powell RC, Carter JE: Faculty and house staff members as role models. J Med Educ 1988, 63:392-396.
• [3]Kisiel JB, Bundrick JB, Beckman TJ: Resident physicians’ perspectives on effective outpatient teaching: a qualitative study. Adv Health Sci Educ Theory Pract 2010, 15:357-368.
• [4]Roff S, McAleer S, Skinner A: Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005, 27:326-331.
• [5]Harden R, Crosby J: AMEE guide No 20: the good teacher is more than a lecturer-the twelve roles of the teacher. Med Teach 2000, 22:334-347.
• [6]Sutkin G, Wagner E, Harris I, Schiffer R: What makes a good clinical teacher in medicine? A review of the literature. Acad Med 2008, 83:452-466.
• [7]Fluit CR, Bolhuis S, Grol R, Laan R, Wensing M: Assessing the quality of clinical teachers: a systematic review of content and quality of questionnaires for assessing clinical teachers. J Gen Intern Med 2010, 25:1337-1345.
• [8]Snell L, Tallett S, Haist S, Hays R, Norcini J, Prince K, Rothman A, Rowe R: A review of the evaluation of clinical teaching: new perspectives and challenges. Med Educ 2000, 34:862-870.
• [9]Williams BC, Litzelman DK, Babbott SF, Lubitz RM, Hofer TP: Validation of a global measure of faculty’s clinical teaching performance. Acad Med 2002, 77:177-180.
• [10]Arah OA, Heineman MJ, Lombarts KMJMH: Factors influencing residents’ evaluations of clinical faculty member teaching qualities and role model status. Med Educ 2012, 46:381-389.
• [11]Beckman TJ, Lee MC, Rohren CH, Pankratz VS: Evaluating an instrument for the peer review of inpatient teaching. Med Teach 2003, 25:131-135.
• [12]Copeland HL, Hewson MG: Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Acad Med 2000, 75:161-166.
• [13]Morrison EH, Hitchcock MA, Harthill M, Boker JR, Masunaga H: The on-line clinical teaching perception inventory: a “snapshot” of medical teachers. Fam Med 2005, 37:48-53.
• [14]Smith CA, Varkey AB, Evans AT, Reilly BM: Evaluating the performance of inpatient attending physicians: a new instrument for today’s teaching hospitals. J Gen Intern Med 2004, 19:766-771.
• [15]Stalmeijer RE, Dolmans DH, Wolfhagen IH, Muijtjens AM, Scherpbier AJ: The Maastricht Clinical Teaching Questionnaire (MCTQ) as a valid and reliable instrument for the evaluation of clinical teachers. Acad Med 2010, 85:1732-1738.
• [16]Harden RM, Grant J, Buckley G, Hart IR: Best evidence medical education. Adv Health Sci Educ Theory Pract 2000, 5:71-90.
• [17]Phillips D, Schweisfurth M: Comparative and International Education: An Introduction to Theory, Method, and Practice. New York: Continuum International Publishing Group; 2006.
• [18]Sasahara T, Kizawa Y, Morita T, Iwamitsu Y, Otaki J, Okamura H, Takahashi M, Takenouchi S, Bito S: Development of a standard for hospital-based palliative care consultation teams using a modified Delphi method. J Pain Symptom Manage 2009, 38:496-504.
• [19]Hofstede G: Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations. Thousand Oaks, CA: Sage Publications, Inc; 2001.
• [20]Phuong-Mai N, Terlouw C, Pilot A: Cooperative learning vs Confucian heritage culture’s collectivism: confrontation to reveal some cultural conflicts and mismatch. Asia Europe Journal 2005, 3:403-419.
• [21]Kluckhohn C: The study of culture. In The Policy Sciences. Edited by Lerner D, Lasswell HD. Standford: Stanford University Press; 1951:86-101.
• [22]American Educational Research Association: American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational, and Psychological Testing (US): Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association; 1999.
• [23]Beckman TJ, Cook DA, Mandrekar JN: What is the validity evidence for assessments of clinical teaching? J Gen Int Med 2005, 20:1159-1164.
• [24]Kikukawa M, Nabeta H, Ono M, Emura S, Oda Y, Koizumi S, Sakemi T: The characteristics of a good clinical teacher as perceived by resident physicians in Japan: a qualitative study. BMC Med Educ 2013, 13:100. BioMed Central Full Text
• [25]Tu WM: Confucian Traditions in East Asian Modernity: Moral Education and Economic Culture in Japan and the Four Mini-dragons. Cambridge: Harvard University Press; 1996.
• [26]Iwata Y: Kyoushikyouiku Kyouinyouseikennkyu no Kadai to Houhou. Curriculum Center Teachers Tokyo Gakugei Univ Annu Res Rep 2009, 8:64-71.
• [27]Yum JO: The impact of Confucianism on interpersonal relationships and communication patterns in East Asia. Commun Monographs 1988, 55:374-388.
• [28]Georgakopoulos A: Teacher effectiveness examined as a system: Interpretive structural modeling and facilitation sessions with US and Japanese students. Int Edu Stud 2009, 2:60.
• [29]Kozu T: Medical education in Japan. Acad Med 2006, 81:1069-1075.
• [30]Fujita H: Education reform and education politics in Japan. Am Sociol 2000, 31:42-57.
• [31]Kikukawa M: Igakukyouiku wo subspeciality to shitemanabutoiu sentakushi. 2010. [Igakukaishinbun] 2874http://www.igaku-shoin.co.jp/paperDetail.do?id=PA02874_02 webcite
• [32]Koichi H: Quality assurance of professional education: focusing on medical doctors and legal professions. (Japanese). Tokyo Daigaku Daigakuin Kyoikugaku Kenkyuka Kiyo 2011, 50:45-65.
• [33]Otaki J: Innovation and research in medical education. Zasshi Tokyo Ika Daigaku 2009, 67:275-282.
• [34]Teo A: The current state of medical education in Japan: a system under reform. Med Educ 2007, 41:302-308.
• [35]Iwasaki S: Hospitals providing postgraduate training–improvement in the quality of training hospitals through the third-party evaluation of the training programs (Japanese). Nihon Naika Gakkai Zasshi 2009, 98:199-204.
• [36]Newman LR, Lown BA, Jones RN, Johansson A, Schwartzstein RM: Developing a peer assessment of lecturing instrument: lessons learned. Acad Med 2009, 84(8):1104-1110.
• [37]Boor K, Van Der Vleuten C, Teunissen P, Scherpbier A, Scheele F: Development and analysis of D-RECT, an instrument measuring residents’ learning climate. Med Teach 2011, 33:820-827.
• [38]Bowden J, Marton F: The University of Learning: Beyond Quality and Competence. London: Kogan; 1998.
• [39]Sargeant J, Mann K, Ferrier S: Exploring family physicians’ reactions to multisource feedback: perceptions of credibility and usefulness. Med Educ 2005, 39:497-504.
• [40]Keeney S, Hasson F, McKenna HP: A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 2001, 38:195-200.
• [41]Palisano RJ, Rosenbaum P, Bartlett D, Livingston MH: Content validity of the expanded and revised gross motor function classification system. Dev Med Child Neurol 2008, 50:744-750.
• [42]Jones J, Hunter D: Qualitative research: consensus methods for medical and health services research. BMJ 1995, 311:376-380.
• [43]Ker JS, Williams B, Reid M, Dunkley P, Steele RJ: Attributes of trainers for postgraduate training in general surgery–a national consensus. Surgeon 2003, 1:215-220.
• [44]Martens M, Duvivier R, Van Dalen J, Verwijnen G, Scherpbier A, Van Der Vleuten C: Student views on the effective teaching of physical examination skills: a qualitative study. Med Educ 2009, 43:184-191.
• [45]Huggett KN, Warrier R, Maio A: Early learner perceptions of the attributes of effective preceptors. Adv Health Sci Educ Theory Pract 2008, 13:649-658.
• [46]Yeates P, Stewart J, Barton J: What can we expect of clinical teachers? Establishing consensus on applicable skills, attitudes and practices. Med Educ 2008, 42:134-142.
• [47]Zuberi RW, Bordage G, Norman GR: Validation of the SETOC instrument – student evaluation of teaching in outpatient clinics. Adv Health Sci Educ Theory Pract 2007, 12:55-69.
• [48]Roff S, McAleer S, Harden R, Al-Qahtani M, Ahmed A, Deza H, Groenen G, Primparyon P: Development and validation of the Dundee ready education environment measure (DREEM). Med Teach 1997, 19:295-299.
• [49]McEvoy P: Educating the Future GP: the Course Organizer’s Handbook. Abingdon: Radcliffe Medical Press; 1998.
• [50]Stalmeijer RE, Dolmans DH, Wolfhagen IH, Muijtjens AM, Scherpbier AJ: The development of an instrument for evaluating clinical teachers: involving stakeholders to determine content validity. Med Teach 2008, 30:272-277.
• [51]Delbecq AL, Van de Ven AH, Gustafson DH: Group Techniques for Program Planning: a Guide to Nominal Group and Delphi Processes. Glenview: Scott, Foresman and Company; 1975.
• [52]Paes P, Wee B: A Delphi study to develop the Association for Palliative Medicine consensus syllabus for undergraduate palliative medicine in Great Britain and Ireland. Palliat Med 2008, 22:360-364.
• [53]Cantrill J, Sibbald B, Buetow S: The Delphi and nominal group techniques in health services research. Int J Pharm Pract 1996, 4:67-74.
• [54]Dunn WR, Hamilton DD, Harden RM: Techniques of identifying competencies needed of doctors. Med Teach 1985, 7:15-25.