Measuring evidence-based practice knowledge and skills in occupational therapy—a brief instrument

BMC Medical Education

Nandi Siegfried¹  Jennifer Jelsma²  Helen Buchanan² 

[1] Independent Clinical Epidemiologist, Cape Town, South Africa;Department of Health & Rehabilitation Sciences, University of Cape Town, F45 Old Groote Schuur Hospital Building, Observatory, Cape Town, 7925, South Africa

关键词: Responsiveness;    Reliability;    Psychometric;    Instrument;    Evaluation;    Adapted Fresno Test of Competence in Evidence-based Practice;    Skills;    Knowledge;    Evidence-based Practice;    Occupational therapy;   

Others  :  1233381
DOI  :  10.1186/s12909-015-0475-2

 received in 2015-08-31, accepted in 2015-10-22,  发布年份 2015

【 摘 要 】

Background

Valid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy. The aims of this paper are to: 1) describe amendments to the Adapted Fresno Test of Competence in EBP (AFT), and 2) report the psychometric properties of the modified instrument when used with South African occupational therapists.

Methods

The clinical utility of the AFT was evaluated for use with South African occupational therapists and modifications made. The modified AFT was used in two studies to assess its reliability and validity. In Study 1 a convenience sample of 26 occupational therapists in private practice or government-funded health facilities in a South African province were recruited to complete the modified AFT on two occasions 1 week apart. Completed questionnaires were scored independently by two raters. Inter-rater, test-retest reliability and internal consistency were determined. Study 2 was a pragmatic randomised controlled trial involving occupational therapists in four Western Cape Department of Health district municipalities (n = 58). Therapists were randomised in matched pairs to one of two educational interventions (interactive or didactic), and completed the modified AFT at baseline and 12 weeks after the intervention. An intention-to-treat analysis was performed. Data were not normally distributed, thus non-parametric statistics were used.

Results

In Study 1, 21 of 26 participants completed the questionnaire twice. Test-retest (ICC = 0.95, 95 % CI = 0.88–0.98) and inter-rater reliability (Time 1: ICC = 0.995, 95 % CI = 0.99–0.998; Time 2: ICC = 0.99, 95 % CI = 0.97–0.995) were excellent for total scores. Internal consistency based on time 1 scores was satisfactory (α = 0.70). In Study 2, 28 participants received an interactive educational intervention and completed the modified AFT at baseline and 12 weeks later. Median total SAFT scores increased significantly from baseline to 12-weeks (Z = −4.078, p < 0.001) with a moderate effect size (r = 0.55).

Conclusion

The modified AFT has demonstrated validity for detecting differences in EBP knowledge between two groups. It also has excellent test-retest and inter-rater reliability. The instrument is recommended for contexts where EBP is an emerging approach and time is at a premium.

Trial registration

Pan African Controlled Trials Register PACTR201201000346141. Registered 31 January 2012.

Clinical Trials NCT01512823. Registered 1 February 2012.

South African National Clinical Trial Register DOH2710093067. Registered 27 October 2009.

【 授权许可】

   
2015 Buchanan et al.

【 参考文献 】

• [1]Glegg S, Holsti L. Measures of knowledge and skills for evidence-based practice: a systematic review. Can J Occup Ther. 2010; 77(4):219-232.
• [2]Shaneyfelt T, Baum K, Bell D, Feldstein D, Houston T, Kaatz S et al.. Instruments for evaluating education in evidence-based practice. A systematic review. JAMA. 2008; 296(9):1116-1127.
• [3]Buchanan H, Siegfried N, Jelsma J, Lombard C. Comparison of an interactive with a didactic educational intervention for improving the evidence-based practice knowledge of occupational therapists in the public health sector in South Africa: a randomised controlled trial. Trials. 2014; 15:216. BioMed Central Full Text
• [4]Buchanan H. Evidence-based practice in occupational therapy in South Africa and the Western Cape. PhD dissertation. University of Cape Town, Cape Town; 2011.
• [5]MacDermid J, Solomon P, Law M, Russell D, Stratford P. Defining the effect and mediators of two knowledge translation strategies designed to alter knowledge, intent and clinical utilization of rehabilitation outcome measures: a study protocol. Implement Sci. 2006;1.
• [6]McCluskey A, Lovarini M. Providing education on evidence-based practice improved knowledge but did not change behaviour: a before and after study. BMC Med Educ. 2005; 5:40. BioMed Central Full Text
• [7]Ramos K, Schafer S, Tracz S. Validation of the Fresno test of competence in evidence-based medicine. BMJ. 2003; 326(7384):319-321.
• [8]McCluskey A, Bishop B. The adapted Fresno test of competence in evidence-based practice. J Contin Educ Health Prof. 2009; 29:119-126.
• [9]Lizarondo L, Grimmer-Somers K, Kumar S, Crockett A. Does journal club membership improve research evidence uptake in different allied health disciplines: a pre-post study. BMC Res Notes. 2012; 5:588. BioMed Central Full Text
• [10]Novak I, McIntyre S. The effect of education with workplace supports on practitioners' evidence-based practice knowledge and implementation behaviours. Aust Occup Ther J. 2010; 57:386-393.
• [11]Crabtree J, Justiss M, Swineheart S. Occupational therapy master-level students’ evidence-based practice knowledge and skills before and after fieldwork. Occup Ther Health Care. 2012; 26(2–3):138-149.
• [12]Tilson J. Validation of the modified Fresno test: assessing physical therapists’ evidence based practice knowledge and skills. BMC Med Educ. 2010; 10:38. BioMed Central Full Text
• [13]Spek B, de Wolf G, van Dijk N, Lucas C. Development and validation of an assessment instrument for teaching evidence-based practice to students in allied health care. J Allied Health. 2012; 41(2):77-82.
• [14]Lewis L, Williams M, Olds T. Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals. BMC Med Educ. 2011;11:77.
• [15]Johnston J, Leung G, Fielding R, Tin K, Ho L. The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ. 2003; 37:992-1000.
• [16]Jerosch-Herold C. An evidence-based approach to choosing outcome measures: a checklist for the critical appraisal of validity, reliability and responsiveness studies. Br J Occup Ther. 2005; 68(8):347-353.
• [17]Ilic D. Assessing competency in evidence-based practice: strengths and limitations of current tools in practice. BMC Med Educ. 2009; 9:53. BioMed Central Full Text
• [18]Caldwell E, Whitehead M, Fleming J, Moes L. Evidence-based practice in everyday clinical practice: strategies for change in a tertiary occupational therapy department. Aust Occup Ther J. 2008; 55:79-84.
• [19]McDowell I. Measuring health. A guide to rating scales and questionnaires, 3rd ed edn. Oxford University Press, New York; 2006.
• [20]Streiner D, Norman G. Health measurement scales. A practical guide to their development and use, 4th edn. Oxford University Press, Oxford; 2008.
• [21]Boynton P. Administering, analysing, and reporting your questionnaire. BMJ. 2004; 328:1372-1375.
• [22]Bland J, Altman D. Statistics notes: Cronbach’s alpha. BMJ. 1997; 314:572.
• [23]McGraw K, Wong S. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996; 1(1):30-46.
• [24]Cicchetti D, Sparrow S. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Def. 1981; 86(2):127-137.
• [25]Streiner D, Geddes J. Intention to treat analysis in clinical trials when there are missing data. Evid Based Ment Health. 2001; 4:70-71.
• [26]Cohen J. A power primer. Psychol Bull. 1992; 112(1):155-159.
• [27]DePoy E, Gitlin L. Introduction to research. Understanding and applying multiple strategies, 3rd edn. St Louis, Elsevier Mosby; 2005.
• [28]Flores-Mateo G, Argimon J. Evidence-based practice in postgraduate healthcare education: a systematic review. BMC Health Serv Res. 2007; 7:119. BioMed Central Full Text
• [29]Bennett S, Tooth L, McKenna K, Rodger S, Strong J, Ziviani J et al.. Perceptions of evidence-based practice: a survey of Australian occupational therapists. Aust Occup Ther J. 2003; 50:13-22.
• [30]McColl A, Smith H, White P, Field J. General practitioners’ perceptions of the route to evidence-based medicine: a questionnaire survey. BMJ. 1998; 316:361-365.
• [31]Dysart A, Tomlin G. Factors related to evidence-based practice among US occupational therapy clinicians. Am J Occup Ther. 2002; 56(3):275-284.
• [32]McCluskey A. Occupational therapists report on low level of knowledge, skill and involvement in evidence-based practice. Aust Occup Ther J. 2003; 50(1):3-12.
• [33]Upton D, Lewis B. Clinical effectiveness and EBP: design of a questionnaire. Brit J Ther Rehabil. 1998;5(12):647–50.
• [34]Pain K, Hagler P, Warren S. Development of an instrument to evaluate the research orientation of clinical professionals. Can J Rehabil. 1996; 9:93-100.