A scoping review about conference objectives and evaluative practices: how do we get more out of them?

Health Research Policy and Systems

M Kent Ranson¹  John N Lavis²  Justin Neves³ 

[1] Alliance for Health Policy and Systems Research, World Health Organization, Geneva, Switzerland;Department of Global Health and Population, Harvard School of Public Health, Boston, USA;Bachelor of Health Sciences (Honours) Program, McMaster University, Hamilton, Canada

关键词: Evaluation methods;    Indicators of success;    Learning and behaviour theories;    Program evaluations;    Conference objectives;    Conferences and symposia;   

Others  :  809933
DOI  :  10.1186/1478-4505-10-26

 received in 2012-02-24, accepted in 2012-07-03,  发布年份 2012

【 摘 要 】

Large multi-day conferences have often been criticized as ineffective ways to improve social outcomes and to influence policy or practice. Unfortunately, many conference evaluations have also been inadequate in determining the impact of a conference on its associated social sector, with little evidence gathered or analyzed to substantiate or refute these criticisms. The aim of this scoping review is to investigate and report stakeholders’ objectives for planning or participating in large multi-day conferences and how these objectives are being evaluated. We conducted a scoping review supplemented by a small number of key informant interviews. Eight bibliographic databases were systematically searched to identify papers describing conference objectives and/or evaluations. We developed a conference evaluation framework based on theoretical models and empirical findings, which structured the descriptive synthesis of the data. We identified 3,073 potential papers for review, of which 44 were included in this study. Our evaluation framework connects five key elements in planning a conference and its evaluation (number in brackets refers to number of themes identified): conference objectives (8), purpose of evaluation (7), evaluation methods (5), indicators of success (9) and theories/models (8). Further analysis of indicators of success identified three categories of indicators with differing scopes (i.e. immediate, prospective or follow-up) as well as empirical links between the purpose of evaluations and these indicators. Conference objectives and evaluations were largely correlated with the type of conference (i.e. academic, political/governmental or business) but diverse overall. While much can be done to improve the quality and usefulness of conference evaluations, there are innovative assessments that are currently being utilized by some conferences and warrant further investigation. This review provides conference evaluators and organizers a simple resource to improve their own assessments by highlighting and categorizing potential objectives and evaluation strategies.

【 授权许可】

   
2012 Neves et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

【 参考文献 】

• [1]Rog S, Wolffe S: Economic Impact Study. Washington D.C, United States: Convention Liason Council; 1994.
• [2]Mathieson D: The London Summit: Milestone or stumbling block?. Madrid, Spain: Real Instituto Elcano; 2009.
• [3]Tepper SJ, Hinton S: The Measure of Meetings: Forums, Deliberation, and Cultural Policy. Princeton; United States: Princeton University; 2003.
• [4]Karosas L, Riklikiene O, Suprikiene R: Evaluating international clinical education encounters in Lithuania. J Contin Educ Nurs 2008, 39:274-80.
• [5]Lalonde B, Wolvaart J, Webb E, Tournas-Hardt A: A process and outcomes evaluation of the International AIDS Conference: Who attends? Who benefits most? Medscape Gen Med 2007, 9:6-22.
• [6]Saltiel M: G20: Less than meets the eye. London United Kingdom: Adam Smith Institute; 2009.
• [7]De Vries B, Pieters J: Knowledge sharing at conferences. Educ Res Eval 2007, 13:237-247.
• [8]Severt D, Wang Y, Chen PJ, Breiter D: Examining the motivation, perceived performance, and behavioral intentions of convention attendees: Evidence from a regional conference. Tour Manag 2007, 28:399-408.
• [9]Forsetlund L, Bjømdal A, Rashidian A, Jamtvedt G, O'Brien M, Wolf F, Davis D, Odgaard-Jensen J, Oxman A: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009.
• [10]Dryzek JS, Tucker A: Deliberative innovation to different effect: Consensus conferences in Denmark, France, and the United States. Public Adm Rev 2008, 68:864-876.
• [11]Roseman M, Reichenbach L: International Conference on Population and Development at 15 years: achieving sexual and reproductive health and rights for all? Am J Public Health 2010, 100:403-406.
• [12]Lee JL, Back KJ: A review of convention and meeting management research. J Conv Event Tour 2005, 7:1-19.
• [13]Malekmohammadi A, Mohamed B, Ekiz HE: An analysis of conference attendee motivations: Case of international conference attendees in Singapore. J Travel Tour 2011, 11:50-64.
• [14]Mittal V, Kumar P, Tsiros M: Attribute-level performance, satisfaction, and behavioral intentions over time; A consumption-system approach. J Mark 1999, 41:77-79.
• [15]Yoo J, Webber K: Progress in convention tourism research. J Hosp Tour Res 2005, 29:194-222.
• [16]Rodgers T: Conferences and Conventions: A Global Industry. United Kingdom: Butterworth-Heinemann, Oxford; 2003.
• [17]Haley KJ, Wiessner CA, Robinson EE: Encountering new information and perspectives: Constructing knowledge in conference contexts. J Contin High Educ 2009, 57:72-82.
• [18]Wiessner CA, Hatcher T, Chapman D, Storberg-Walker J: Creating new learning at professional conferences: An innovative approach to conference learning, knowledge construction and programme evaluation. Hum Resour Dev Int 2008, 11:367-383.
• [19]Jaffe DM, Knapp JF, Jeffe DB: Final evaluation of the 2005 to 2007 National Pediatric Emergency Medicine Fellows' Conferences. Pediatr Emerg Care 2009, 25:295-300.
• [20]Pianta M: UN world summits and civil society: The state of the art. Geneva, Switzerland: United Nations Research Institute for Social Development; 2005.
• [21]Portnoy B, Miller J, Brown-Huamani K, Devoto E: Impact of the National Institutes of Health Consensus Development Program on stimulating National Institutes of Health-funded research, 1998 to 2001. Int J Technol Assess Health Care 2007, 23:343-348.
• [22]Wenger E, McDermott R, Snyder W: Cultivating Communities of Practice. Boston, United States: Harvard Business Press; 2002.
• [23]Lund E, Gram IT: Response rate according to title and length of questionnaire. Scand J Soc Med 1998, 26:154-160.
• [24]Storberg-Walker J, Wiessner CA, Chapman D: How the AHRD 2005 conference created new learning: Preliminary results of a case study. Hum Resour Dev Q 2005, 16:547-555.
• [25]Curwick CC, Reeb-Whitaker CK, Connon CL: Reaching managers at an industry association conference: Evaluation of ergonomics training. Am Assoc Occup Health Nurs J 2003, 51:464-469.
• [26]Buyum A, Dubruiel N, Torghele K, Alperin M, Miner KR: Can summits lead to curricula change? An evaluation of emergency preparedness summits for schools of nursing in Georgia. J Contin Educ Nurs 2009, 40:210-215.
• [27]Driever MJ, Perfiljeva G, Callister LC, McGivern S: Creating a context for professional dialogue between United States and Russian nurses: Design of an international conference. J Contin Educ Nurs 2005, 36:168-174.
• [28]Farnum K, McCarthy ML, Beauchesne MA, Lawrence PR: The Primary Care for the Underserved Conference as a building block to social capital: Impact on practice, research, and education. J Cult Divers 2005, 12:126-135.
• [29]Foster J, Guisinger V, Graham A, Hutchcraft L, Salmon M: Global Government Health Partners' Forum 2006: Eighteen months later. Int Nurs Rev 2010, 57:173-179.
• [30]James G: A seven-year follow-up comparing attendees and nonattendees at a statewide, school employee wellness conference. J Sch Heal 2001, 71:127-131.
• [31]Scott EJC: Evaluation of the impact of the recommendations of the Conference for the Development of Nursing Research held in Salamanca. J Res Nurs 2005, 10:693-694.
• [32]Travers R, Wilson M, McKay C, O'Campo P, Meagher A, Hwang SW, Parris UJ, Cowan L: Increasing accessibility for community participants at academic conferences. Prog Community Health Partnerships 2008, 2:257-264.
• [33]Evanoff DD, Bartholomew PF, DeYoung R, Lucaci C, Phillips RJ: Bank Structure Conference impact study. J Financ Serv Res 2008, 34:99-121.
• [34]Kwong Y, Kwong FN, Patel J: Publication rate of trauma abstracts presented at an International Orthopaedic conference. Injury 2007, 38:745-749.
• [35]Bergqvist D: The annual meeting of the European Society for Vascular Surgery – the scientific contents over the years. Eur J Vasc Endovasc Surg 2008, 36:114-117.
• [36]Kho ME, Brouwers MC: Conference abstracts of a new oncology drug do not always lead to full publication: Proceed with caution. J Clin Epidemiol 2009, 62:752-758.
• [37]Ajzen I: Theory of Planned Behaviour. University of Massachusetts, Amherst, United States; 2011. http://people.umass.edu/aizen/tpb.html webcite
• [38]Abbott C, Carin B, Innis L: Breaking Global Deadlocks II. Waterloo, Canada: Centre for International Governance Innovation; 2009.
• [39]Aitken C: Paediatric emergency care: The assessment of an education program on the level of knowledge and confidence of nurses within a mixed emergency department. Australas Emerg Nurs J 2007, 10:224.
• [40]Alefsen H, Raue J: Women's rights are human rights: The UN Women's Rights Convention at 30. Vereinte Nationen 2009, 57:217-222.
• [41]Amadio M: Inclusive education in Latin America and the Caribbean: Exploratory analysis of the national reports presented at the 2008 International Conference on Education. Humanities, Social Sciences and Law 2009, 39:293-305.
• [42]Benchmark Environmental Consulting: Democratic global civil governance report of the 1995 benchmark survey of NGOs. Oslo, Norway: Benchmark Environmental Consulting; 1996.
• [43]Ber R, Grunfeld GB, Alroy G: A multidisciplinary forum for ethics in medicine: Our seven years experience. Isr Med Assoc J 2000, 2:954-956.
• [44]Calder Bateman Communications: Setting the Direction for Special Education in Alberta. Minister's Forum: What We Heard. Edmonton, Canada: Calder Bateman Communications; 2009.
• [45]Davison CM: Benchmarks for social justice: Assessing the fairness of academic conference participation. Promot Educ 2005, 12:71-76.
• [46]Doek JE: The CRC 20 Years: An overview of some of the major achievements and remaining challenges. Child Abuse Negl Int J 2009, 33:771-782.
• [47]Kangas BD, Vaidya M: Trends in presentations at the Annual Conference of the Association for Behavior Analysis. Behav Anal 2007, 30:117-131.
• [48]Miguel-Dasit A, Marti-Bonmati L, Sanfeliu-Montoro A, Aleixandre R, Valderrama JC: Scientific papers presented at the European Congress of Radiology: A two-year comparison. Eur Radiol 2007, 17:1372-1376.
• [49]Mockli D: After Annapolis: A Fragile Peace Process in the Middle East. Cent Secur Stud Confl Res 2007, 2:1-3.
• [50]Muller B, Robinson D: Assessments of Bali. Oxf Energy Forum 2008, 1:16-22.
• [51]Oliveira LR, Figueiredo AA, Choi M, Ferrarez CE, Bastos AN, Netto JM: The publication rate of abstracts presented at the 2003 urological Brazilian meeting. Clinics 2009, 64:345-349.
• [52]Reychav I, Te'eni D: Knowledge exchange in the shrines of knowledge: The ''How's'' and ''Where's'' of knowledge sharing processes. Comput Educ 2009, 53:1266-1277.
• [53]Roch MC, Wilkening KE, Hart P: Global to local: International conferences and environmental education in The People's Republic of China. Int Res Geogr Environ Educ 2007, 16:44-57.
• [54]Saha A, Poddar E, Mankad M: Effectiveness of different methods of health education: A comparative assessment in a scientific conference. BMC Publ Health 2005, 5:88. BioMed Central Full Text
• [55]Saito Y, Yajima S, Kaplan M, Kusano A: Networking and collaboration outcomes at Japan's First National Intergenerational Conference. J Intergen Relat 2009, 7:111-117.
• [56]Secil M, Ucar G, Dicle O: Scientific papers presented at the 2000–2001 European Society of Gastrointestinal and Abdominal Radiology (ESGAR) meetings: Publication rates during the period 2000–2004. Eur Radiol 2007, 17:2183-2188.
• [57]Vogt J, Gasche V, Muris J, Leonhardt J, Mitchell JT: A report on the evaluation of the first European Conference on Critical Incident Stress Management organized by the European Office of the International Critical Incident Stress Foundation. Int J Emerg Ment Health 2009, 11:111-120.
• [58]Walstad WB, Watts M: Closing an International Economic Education Conference in OZ. J Econ Educ 2005, 36:306-308.
• [59]Weale AR, Edwards AG, Lear PA, Morgan JD: From meeting presentation to peer-review publication – A UK review. Ann R Coll Surg Engl 2006, 88:52-56.
• [60]Whitehouse MR, Atwal NS, Blom AW: Publication rates for hip surgery-related abstracts presented at national and international meetings. Orthopedics 2009, 32:407.