Literature review: CME/CPD of anesthesia team in the U.S.

The previous chapter explained socio-economic and political processes that shape the professional and educational landscape of healthcare professions. This section will describe how education is changing in that very dynamic context. It will start with a reflection on more general technology-related changes, which share many similarities with processes affecting our society in general. The focus will continue to be on topics that are specific to education of healthcare professionals in the U.S.

1     Evolution of technology-enhanced learning used by U.S. anesthesiologists

Distance learning of the U.S. healthcare workforce has a long history, starting with correspondence education in the 1960s (Josseran & Chaperon, 2001). Some popular correspondence programs, such as Refresher Courses in Anesthesiology, were initiated in the early 1970s (ASA, 1973).

Online learning has become the dominant way of delivering CPD. Five years ago, Harris, Sklar, Amend, and Novalis‐Marine (2010) predicted that “online CPD is likely to be 50% of all CPD consumed within 7-10 years.” Five years later, in 2015, all education delivered by the American Society of Anesthesiologists (ASA) was online or enhanced by online formatting. That happened significantly faster than expected, and it aligns with the now widely accepted opinion that online CPD programs are as effective as traditional CPD programs (Wutoh, Boren, & Balas, 2004), and that a physician’s time is very expensive.  Consequently, 97% of physicians expect more online CPD in the future (archemedx.com, 2013).

Five generations of distance education, as described by (Taylor, 2001) and later elaborated on by (Bates, 2008), can categorize the evolution of CPD as provided by the ASA.

1. The Correspondence Model, based on print technology, is losing its share and is enhanced with online delivery. However, it still plays a significant part. In 2015, approximately 30% of CPD credits claimed by ASA users were earned through that model.
2. The Multi-media Model – delivery of multimedia content on print, digital storage devices (CD/DVD, flash memory), or through the Internet, but without any communication among humans. It is well-suited for industrial mass production. It is the dominant method of delivery, with around 68% of credit hours offered in this format.
3. The Tele-learning Model delivers synchronous communication, such as webinars, and is used quite rarely in CPD. There were no CPD credits awarded by ASA this year using this model.
4. The Flexible Learning Model is based on asynchronous online communication (Bates, 2008).  In the U.S. CPD context, it is at this moment very rarely used, and there is significant potential to extend usage of that model (Cheston, Flickinger, & Chisolm, 2013). The first ASA course that utilizes a discussion board was launched in March 2016. .
5. The Intelligent Flexible Learning model is being engineered around the new LMS. It builds on the functionality of the Flexible Learning Model. Some of the additions are: easy access to institutional guidelines and resources; computer-mediated communication; user- generated content; and peer assessment. The system will be integrated with the Anesthesia Quality Institute clinical outcomes tracking system (Dutton, 2014), allowing individuals and groups to assess and reflect on their clinical performance and create improvement and learning plans. The system will also deliver a business intelligence layer that suggests future learning topics based on users’ clinical performance, and performance in courses and certification status.

Specific learning theories are associated with each of those generations. Generations 1 and 2 are associated primarily with behaviorism and cognitivism (Bates, 2008). A majority of CPD is delivered through the first two generations of distance education. Generation 3 is not popular anymore and, instead of implementing Generation 4, the goal is to go straight to Generation 5. Simultaneously, Generation 5 utilizes constructivist approaches like collaborative learning, knowledge construction, communities of practice and self-directed learners (Peters, 2002). Between the first two generations and the fifth generation, we have significant technological, theoretical and cultural differences.

As described below, the U.S. healthcare reform and recently adopted educational technology solutions will enable those changes to happen in the form of IPL and QIE. However, the technology is just one element of that formula, and there are numerous challenges that have to be addressed prior to successful implementation. For example, ASA faculty, just like faculty at medical schools, is not well-informed of learning theories used in this context (Flynn, Jalali, & Moreau, 2015). That is a major strategic challenge. Without faculty who know how to lead and give structure to learning activities,  “social media can negatively impact student learning” (Gikas & Grant, 2013, p. 19) and cause significant frustration.

2     Transformation of healthcare CPD

Technology is just one driver transforming healthcare CPD. The list of additional drivers is extensive. They include the evident need for better implementation of adult and collaborative learning principles, the need for more outcome-focused education, and involvement of patients in the learning process (Price et al., 2012).
The CME/CPD model currently used in the U.S. has been heavily criticized (Cooke, Irby, & O’Brien, 2010, IoM, 2010 #456,Hager, 2008 #809; Mehta, Hull, Young, & Stoller, 2013). Weaknesses include low efficiency, inflexibility and not being learner-centered. Mehta et al. (2013) explain that the current teaching methods are often designed to address “arcane assessment methods (e.g., Multiple-choice examinations)” (p. 1418). Consequently, the learning process is focused more on test performance than on development of professional competencies, and grades will reflect more on students’ memory and test-taking skills, rather than behaviors, skills and attributes needed by an effective physician.

CPD focus and cultural change. Historically, the focus of CME/CPD was primarily on content transmission and clinic topics. More recently, strong societal forces are converging in a focus shift toward behavior-changing learning activities with impact on patient population (Donald E Moore, Green, & Gallis, 2009; Russell, Maher, Prochaska, & Johnson, 2012). We can also notice a shift of focus from individuals (CME) toward CPD of groups and organizations (Webster-Wright, 2009). That transformation is part of a focus shift from continuing medical education (CME) toward CPD (image below). In that context, the CPD term serves as an umbrella (Karle et al., 2012) that encompasses formal CME focused on medical practice, and all other forms of medical education – including QIE/IPL. Furthermore, CPD covers multifaceted competencies important for patient care – such as awareness of cultural differences, communication skills, managerial, social and interprofessional education, and humanitarian and psychological aspects of care (WFME, 2003). That is a huge cultural change for all traditional members of the medical education continuum and newly associated groups, such as anesthesiology assistants, technologists, managers and leaders.

Figure 4. Evolution from CME to CPD

Quality vs. Education. Until recently, continuing education of healthcare professionals and quality improvement initiatives existed as two very separate entities. It was common to hear that CME and QI people may have offices next to each other – but they do not talk to each other; they do not speak the same language; they do not have the same focus (Shershneva, Mullikin, Loose, & Olson, 2008). For example, CME focused on credit hours has been awarding credit for seating time. Simultaneously, QI initiatives are focused on implementing sustainable organizational and individual behavioral change. In recent years, we have seen a significant shift (Balmer, 2013). Innovative approaches to integrate education and QI and IPL are being developed and implemented (Shojania, Silver, & Levinson, 2012).

Repeating history? Although recent developments may suggest that integration of education and QI and IPL is a new phenomenon, that is not true. A recently republished article focused on “Relating Continuing Education Directly to Patient Care [Quality]” (Brown & Fleisher, 2014), was first published 45 years ago – in 1971. In the same manner, the first report created by the Institute of Medicine (IoM, 1972) was focused on IPL. Therefore, while analyzing interaction between QI/IPL and education, the question should not be: “Why haven’t we figured that out before?” but “Knowing what we do, why haven’t we made the required changes?” Or even better: “When and why did education and quality improvement become disconnected?”

Interprofessional apprenticeship. Apprenticeships have historically been the main form of medical education (Dornan, 2005). Nowadays, their role in undergraduate and graduate medical education is a bit reduced, but residency programs are created around the apprenticeship model. Rodriguez-Paz et al. (2009) argue that the traditional “see one, do one, teach one” model is not adequate because inexperienced trainees learn by practicing on real patients, making it a safety issue. However, the model should not be replaced, but updated. Oversimplification and disintegration of professional competence in knowledge, skills and attitudes is counter-productive, because they are interwoven parts of the same fabric of competence. If they are learned in isolation from one another, the outcome (Makovsky Health, 2013) will be less than ideal (Dornan, 2005). Furthermore, experts have “tacit competence” – things they can do skillfully but without ability to describe properly. The best way to gain those unteachable competencies is through mentorship in practice settings.

Share the care. A variety of educational tools and concepts, like the competency-based training paradigm, technology-enhanced patient safety and quality-improvement educational interventions, can ensure that trainees practice with real patients without risk. A good example is value-added medical education. It is a team-based “share the care” concept, where numerous clinical and non-clinical professionals, patients and learners work together so that each team member contributes to his or her maximum potential (Lin et al., 2014). Medical and other healthcare students participate in such a team according their competencies. Therefore, instead of shadowing a physician and attempting to do only the things physicians are supposed to do, early medical students can start as health and behavioral change coaches or quality-improvement project administrators or data collectors.

Those concepts, enhanced by technology like high-fidelity simulation, virtual reality, and the collaborative Web healthcare will enable learners from college thorough retirement “to… see one, simulate many, do one competently, and teach everyone.” (Vozenilek, Huff, Reznek, & Gordon, 2004, p. 1153).

3     Failure of didactic format and perpetual status quo

Didactic lectures are still the main learning delivery format, yet the impact of such learning on competencies and patient outcomes is questionable (D. Davis et al., 1999; Holm, 1998). That is not a new debate. Abraham Flexner, the author of the famous Flexner report (Flexner, 1910) and the person who helped change the face of American medical education (Cooke, Irby, Sullivan, & Ludmerer, 2006), was very vocal about it. Flexner criticized the lecture system, stating that although it allows schools to “handle cheaply by wholesale otherwise unmanageable numbers” (Flexner, 1908, p. 194), it doesn’t prepare students for real-life tasks. The programming should be created around integration between formal learning with clinical practice and research. Therefore, Flexner concludes by describing lectures as “an astonishing failure of pedagogic insight” (Flexner, 1908, p. 197). That criticism was muted with the fact that, didactic, content-focused lectures, as a short periodic interaction with a group of unnamed students allow industrialized education. Lecturers can “educate” large numbers of students in a short time. Less time spent on lectures means that the lecturer has more time for the research necessary for career development (Colbeck, 1998).

Flexner explains that, a century ago, increased reliance on didactic lectures was perceived as a sign that the college was “grown-up” (Flexner, 1908, p. 199). At that time, industrialization and mass production were prominent signs of progress. Therefore, industrialization and mass production gained popularity in education and universities started competing in research instead of quality of education. However, Flexner sharply criticized that approach, explaining that “rapidly won distinction as research centers is not compensation for college failure“ (Flexner, 1908, p. 217), and that as soon as people started looking closely at educational function “it will become evident that the college is nowadays educationally headless.“ (Flexner, 1908, p. 218)

Today, more than 100 years after the Flexner report  (Flexner, 1908, 1910, 1912) we can see that the basic teaching model went through only minor changes during past 100 years (Mehta et al., 2013). Furthermore, some of Flexner’s recommendations are in the same stage of implementation as they were a century ago.

4     Back to performance

Learning for quality. Balmer (2013) explains that reduced funding from industry, mainly the pharmaceutical industry, creates a context where CPD of physicians and healthcare professionals is paid for mainly by healthcare institutions or individuals. Therefore, instead of selecting a program because it is free, healthcare professionals and institutions more often select programs because they will improve their performance and have good return on investment. That trend correlates with pay-for-performance initiatives and performance-tracking frameworks that are increasingly being used in the U.S. Through such frameworks, individuals or teams can monitor their performance, diagnose performance gaps, and, based on that plan, their professional development.

Vision of Qualiy.  Those trends align well with the vision proposed in “Health Professions Education: A Bridge to Quality,“ IoM and (2003, p. 45):

“All health professionals should be educated to deliver patient-centered care as members of an interdisciplinary team, emphasizing evidence-based practice, quality improvement approaches, and informatics.”

As Dr. Edward Hundert, M.D., and Mary Wakefield, Ph.D., R.N., explain in the preface to Health Professions Education: A Bridge to Quality, (IoM & 2003, p. ix), the main message of the book is: “…reform of health professions education is critical to enhancing the quality of health care in the United States.” Furthermore, that reform must involve all healthcare professionals.

Changes – Five themes. Numerous themes have been initiated in response to the aforementioned challenges. (Balmer, 2013) described five dominant themes grounded in political, economic and educational U.S. context. They are:

1. Shift of focus from time-based attendance metric (awarding seating time) to measurement of competences with impact on patient care.
2. Common usage of inter-professional education to enhance profession-specific CPD.
3. Integration of quality improvement with continuing education – creating quality improvement education or organization-wide CPD.
4. Increased focus on the big picture where CPD is seen as a tool to address public health and population issues.
5. Defining and standardization of professional competencies needed for successful healthcare services, as well as needed for CPE interventions.

5     Assessment drives learning – in wrong direction?

In previous chapters, a few conflicts inherent in the current political, social or research culture have been described. Those conflicts inhibit change. Could it be that in the same manner a cultural conflict hidden inside the educational system is blocking change? Could it be that the most popular assessment method is sending us in a wrong direction? Let’s check it out.

Debate on multiple choice questions (MCQ) has a long history (Anderson, 2004; Pickering, 1979). MCQ exams are known to be a reproducible, cost-effective and reliable tool to test medical knowledge. However, they have limited validity in assessing clinical competencies, have limited flexibility in different settings, and are not comprehensive as a single assessment tool (Tetzlaff, 2007).

In other words, as the table below illustrates, MCQ, the most commonly used assessment method, has many features different from what is perceived as optimal for continuous quality improvement. Therefore, it is fair to ask if part of the challenges we are experiencing are caused by the current assessment methods. The table below illustrates differences between continuous quality improvement and MCQ. Since “assessment drives learning” (Frederiksen, 1984; Wood, 2009), if we change what and how learning is assessed, learning practices would be changed. Is it possible that MCQ exams, which are  often stressful, isolating, competitive, content-focuse learning experience with known potential negative effects on learning (Bailey, Mossey, Moroso, Cloutier, & Love, 2012; Roediger III & Marsh, 2005), are contributing to the challenges we face today?

Table 3.

6     Assessment pyramid

Miller’s pyramid of clinical competencies (Figure 5) links assessment tools with evidence we can use to determine if the healthcare professional is “competent to practice.” The pyramid (Miller, 1990) was developed with the individual practitioner in mind. Now we can observe it through lens of team-based QIE/IPL.

Knowledge is at the bottom of the pyramid. Assessing knowledge is quite easy (Miller, 1990). With MCQ we can easily do it on a mass scale. It can be automatic. Furthermore, “very precise” numbers we can get as outcomes can overshadow questions about the impact of MCQ. Although knowledge is essential to function as a professional, merely knowing is insufficient for practicing good medicine. Therefore, if we are focused on assessing knowledge, we will not be able to distinguish candidates who can, from those who cannot, practice medicine well.

Assessing teams. Looking at Miller’s pyramid through the lens of QIE/IPL, Level 1 and Level 2 (Knows and Knows how) are primarily focused on individuals. Therefore, although the two levels can help assess individuals who will participate in the QIE/IPL, they are only of limited value to assess teams or performance of individuals in team-based activities.
The top levels of the pyramid focus on how “Shows how” and “Does” can better assess individual and group competencies. However, those assessment modalities are more time-consuming, cannot be automated as MCQ, and as a result are less often used. Therefore, more focus on skill-based assessment (Levels 3 and 4) may be needed for QIE/IPL.
Clinical performance assessment tools used as part of performance improvement CME are valuable assets. At this moment, they are used in limited scope, mainly because the performance assessment methodology is still maturing, and QI CME courses are not very popular.

Figure 5. Miller’s model of clinical competence.(Miller, 1990; Mitchell et al., 2015)

Evolving professionalism. Currently professionalism is taught to healthcare students through the continuum of healthcare education. It is based on the belief that healthcare professionals must come together to continuously research, debate and improve competencies and beliefs (Wynia, Papadakis, Sullivan, & Hafferty, 2014) so they are better prepared for the societal expectations. In that context, our focus is on what a team does, and then shows how it succeeds.

Professionalism as a belief and value system, that describes professional identity or “being” of individuals and groups, is according many authors stronger basis for consistent professional behavior than “doing.”(Goldie, 2012; Weaver, Peters, Koch, & Wilson, 2011). In an era when transformative changes are common, when knowledge is growing exponentially, and graduation or specialty board certifications are important steps in continuous professional development, “doing” should be constantly re-evaluated and improved to better reflect who we are and how we as individuals, teams and professions contribute to the society. To address that, Cruess, Cruess, and Steinert (2015) proposed updating Miller’s pyramid by adding the professional identity of “Is” to the top of the pyramid.
A team-based lens adds more complexity to this model. As team-based care becomes standard, we can see that participation in a team becomes the first identity; team members perceive themselves first as a team and then as a specialty (Hlede, 2015). One interviewee noted:

“Instead of, ‘I’m an anesthesiologist,’ or, ‘I’m a nurse anesthetist,’ it’s, ‘I’m a member of the joint replacement team.’ ‘I’m a member of the spine team.’ ‘I’m a member of the cardiac team.’ You can see that in the hospitals now in some very focused areas. … That’s the first identity.” (Hlede, 2015, p. 17)

7     CME as human capital vs. a requirement for licensure

One of the bigger obstacles to wider implementation of QIE/IPL has been accreditation requirements and ways to measure completion and award credits. The impact of traditional CME formats is heavily debated (Hager, Russell, Fletcher, & Macy Jr, 2008; IoM, 2010), and critics argue that the majority of CME credits are awarded for “seat time” (Schmitt, Baldwin, & Reeves, 2012). Despite that, nobody can argue that if your main goal is to get the required number of credits, this approach is very convenient. You were sitting in this lecture room for one hour, please claim your credits; you completed that MCQ quiz, please claim your credits. Quite often, online MCQ quizzes allow an unlimited number of attempts per each question. Therefore, users can select A-B-C-D until they get the correct answer, and then move to another question. The simplicity of that system – while users are focused primarily on getting credits instead of learning outcomes – creates a combination that is hard to match in the IPL context.

Fortunately, challenges associated with credit-focused CME are well-recognized. E. G. Campbell and Rosenthal (2009) convincingly argue that a huge positive and transformational driver would be a situation where healthcare professional perceive CME primarily as a tool to improve their personal and team human capital. The current model where CME credits serve as a requirement for licensure foster negative selection, and learners, rather than look for the best course, seek the easiest way to gain credits. (Cook et al., 2015).

Pharma is/was shaping CME. Sponsorship from the healthcare industry contributes to the aforementioned challenges. For example, in 2009 the Institute of Medicine published extensive research on conflict of interest in medical research, education and practice (IoM & 2009). The conclusion was that continuing medical education “has become far too reliant on industry funding” (p. 161). The industry funding fosters CME as a marketing tool where the primary focus is on promoting products, while broader education, alternative methods to improve healthcare, and system-based issues like prevention or communication are often ignored.

In reflecting on that situation, (E. G. Campbell & Rosenthal, 2009) used arguments from the Flexner report (Flexner, 1910), saying, “A century later, another component of the continuum of medical education requires equally sweeping reform – continuing medical education.” (p. 1807) They explained that three of Flexner’s main criticisms of the undergraduate medical education in 1910 are applicable to CME now. The aforementioned excessive comercialization is one. Nonstandardized curricula is another. Lack of impact on patient care is the third.  E. G. Campbell and Rosenthal (2009, p. 1807) explain: “Traditional CME is not adequately focused on improving patient outcomes. In fact, there is scant evidence that CME actually improves patient outcomes.“

References 

• Anderson, J. (2004). Medical teacher 25th anniversary series multiple-choice questions revisited. Medical teacher, 26(2), 110-113.
• archemedx.com. (2013). 2013 Healthcare Professional Continuing Education Preference Survey.   Retrieved from http://www.archemedx.com/blog/2013-clinician-continuing-education-preference-survey/
• ASA, Americaln Society of Anesthesiologists (1973). ASA Refresher Courses in Anesthesiology – Volume 1. ASA Refresher Courses in Anesthesiology, 1(1), 1-167.
• Bailey, P. H., Mossey, S., Moroso, S., Cloutier, J. D., & Love, A. (2012). Implications of multiple-choice testing in nursing education. Nurse Education Today, 32(6), e40-e44. doi:http://dx.doi.org/10.1016/j.nedt.2011.09.011
• Balmer, J. T. (2013). The transformation of continuing medical education (CME) in the United States. Advances in medical education and practice, 4, 171.
• Bates, A. W. T. (2008). Transforming distance education through new technologies. In T. Bates (Ed.).
• Brown, C. R., & Fleisher, D. S. (2014). The Bi‐Cycle Concept—Relating Continuing Education Directly to Patient Care. Journal of Continuing Education in the Health Professions, 34(2), 141-148.
• Campbell, E. G., & Rosenthal, M. (2009). Reform of continuing medical education: investments in physician human capital. JAMA, 302(16), 1807-1808.
• Cheston, C. C., Flickinger, T. E., & Chisolm, M. S. (2013). Social media use in medical education: a systematic review. Academic Medicine, 88(6), 893-901.
• Colbeck, C. L. (1998). Merging in a seamless blend: How faculty integrate teaching and research. Journal of Higher Education, 647-671.
• Cook, D. A., Holmboe, E. S., Sorensen, K. J., Berger, R. A., & Wilkinson, J. M. (2015). Getting Maintenance of Certification to Work: A Grounded Theory Study of Physicians’ Perceptions. JAMA internal medicine, 175(1), 35-42.
• Cooke, M., Irby, D. M., & O’Brien, B. C. (2010). Educating physicians: a call for reform of medical school and residency (Vol. 16). San Francisco, CA, USA: John Wiley & Sons.
• Cooke, M., Irby, D. M., Sullivan, W., & Ludmerer, K. M. (2006). American Medical Education 100 Years after the Flexner Report. New England Journal of Medicine, 355(13), 1339-1344. doi:doi:10.1056/NEJMra055445
• Cruess, R. L., Cruess, S. R., & Steinert, Y. (2015). Amending Miller’s Pyramid to Include Professional Identity Formation. Academic medicine: journal of the Association of American Medical Colleges.
• Davis, D., O’Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey, A. (1999). Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA, 282(9), 867-874. doi:10.1001/jama.282.9.867
• Dornan, T. (2005). Osler, Flexner, apprenticeship and’the new medical education’. J R Soc Med, 98(3), 91-95.
• Dutton, R. P. (2014). Quality management and registries. Anesthesiology clinics, 32(2), 577-586.
• Flexner, A. (1908). The American college: a criticism: Century Company.
• Flexner, A. (1910). Medical education in the United States and Canada bulletin number four (The Flexner Report). New York (NY): The Carnegie Foundation for the Advancement of Teaching.
• Flexner, A. (1912). Medical education in Europe: a report to the Carnegie Foundation for the Advancement of Teaching: Carnegie Foundation for the Advancement of Teaching.
• Flynn, L., Jalali, A., & Moreau, K. A. (2015). Learning theory and its application to the use of social media in medical education. Postgraduate medical journal, postgradmedj-2015-133358.
• Frederiksen, N. (1984). The real test bias: Influences of testing on teaching and learning. American psychologist, 39(3), 193.
• Gikas, J., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. The Internet and Higher Education, 19, 18-26.
• Goldie, J. (2012). The formation of professional identity in medical students: Considerations for educators. Medical teacher, 34(9), e641-e648.
• Hager, M., Russell, S., Fletcher, S. W., & Macy Jr, J. (2008). Continuing education in the health professions: improving healthcare through lifelong learning: Josiah Macy, Jr. Foundation.
• Harris, J. M., Sklar, B. M., Amend, R. W., & Novalis‐Marine, C. (2010). The growth, characteristics, and future of online CME. Journal of Continuing Education in the Health Professions, 30(1), 3-10.
• Hlede, V. (2015). Interprofessional Learning: Anesthesiologists’ Perspectives. Assignment, Doctoral Programme in E-Research and Technology Enhanced Learning. Department of Educational Research. Lancaster University.
• Holm, H. A. (1998). Quality issues in continuing medical education. British Medical Journal, 316(7131), 621-624. doi:10.1136/bmj.316.7131.621
• IoM. (1972). Institute of Medicine: Educating for the health team. Washington, DC.: ERIC Clearinghouse.
• IoM. (2010). Institute of Medicine: Redesigning Continuing Education in the Health Professions (9780309140782). Retrieved from http://www.ama-assn.org/resources/doc/cme/iom-report-cme.pdf
• IoM, & , Institute of Medicine. (2003). Health Professions Education: A Bridge to Quality (E. Knebel & A. C. Greiner Eds.): National Academies Press.
• IoM, & , Institute of Medicine. (2009). Conflict of interest in medical research, education, and practice (M. J. Field & B. Lo Eds.). 500 Fifth Street, N.W. Washington, DC 20001: National Academies Press.
• Josseran, L., & Chaperon, J. (2001). History of continuing medical education in the United States. Presse medicale (Paris, France: 1983), 30(10), 493-497.
• Karle, H., Paulos, G., & Wentz, D. K. (2012). Continuing Professional Development: Concept, Origins, and Rationale. In D. K. Wentz (Ed.), Continuing Medical Education: Looking Back, Planning Ahead (pp. 281-289). Hanover, NH, US and London: Dartmouth College Press.
• Lin, S. Y., Schillinger, E., & Irby, D. M. (2014). Value-Added Medical Education: Engaging Future Doctors to Transform Health Care Delivery Today. Journal of general internal medicine, 30(2), 150-151.
• Makovsky Health. (2013, Sept. 9, 2013). Online Health Research Eclipsing Patient-Doctor Conversations.   Retrieved from http://www.makovsky.com/insights/articles/25-insights/articles/article/229-as-the-web-goes-mobile-healthcare-stands-still
• Mehta, N. B., Hull, A. L., Young, J. B., & Stoller, J. K. (2013). Just Imagine: New Paradigms for Medical Education. Academic Medicine, 88(10), 1418-1423 1410.1097/ACM.1410b1013e3182a1436a1407.
• Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63-67.
• Mitchell, S., Simonds, A., Andreas, S., Bonsignore, M. R., Cooper, B., Donic, V., . . . Prest, G. (2015). Introducing a core curriculum for respiratory sleep practitioners. Breathe, 11(1), 50.
• Moore, D. E., Green, J. S., & Gallis, H. A. (2009). Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. Journal of Continuing Education in the Health Professions, 29(1), 1-15.
• Peters, O. (2002). Distance education in transition: New trends and challenges: BIS Verlag.
• Pickering, S. G. (1979). Against multiple choice questions. Medical teacher, 1(2), 84-86.
• Price, D., Havens, C., & Bell, M. J. (2012). Continuing Professional Development and Improvement to Meet Current and Future Continuing Medical Education Needs of Physicians In D. K. Wentz (Ed.), Continuing Medical Education: Looking Back, Planning Ahead (pp. 1-14). Hanover, NH, USA, and London: Dartmouth College Press.
• Rodriguez-Paz, J., Kennedy, M., Salas, E., Wu, A., Sexton, J., Hunt, E., & Pronovost, P. (2009). Beyond “see one, do one, teach one”: toward a different training paradigm. Quality and safety in health care, 18(1), 63-68.
• Roediger III, H. L., & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(5), 1155.
• Russell, B., Maher, G., Prochaska, J. O., & Johnson, S. S. (2012). Strategic approaches to continuing medical education: applying the transtheoretical model & diffusion of innovation theory. CE Measure, 6(3), 27-31.
• Schmitt, M. H., Baldwin, D. C., & Reeves, S. (2012). Continuing Interprofessional Education; Collaborative Learning for Collaborative Practice. In D. K. Wentz (Ed.), Continuing Medical Education: Looking Back, Planning Ahead (pp. 281-289). Hanover, NH, USA, and London: Dartmouth College Press.
• Shershneva, M. B., Mullikin, E. A., Loose, A.-S., & Olson, C. A. (2008). Learning to collaborate: A case study of performance improvement CME. Journal of Continuing Education in the Health Professions, 28(3), 140-147. doi:10.1002/chp.181
• Shojania, K. G., Silver, I., & Levinson, W. (2012). Continuing medical education and quality improvement: a match made in heaven? Ann Intern Med, 156(4), 305-308.
• Taylor, J. C. (2001). Fifth generation distance education. Instructional Science and Technology, 4(1), 1-14.
• Tetzlaff, J. E. (2007). Assessment of competence in anesthesiology. Current Opinion in Anesthesiology, 22(6).
• Vozenilek, J., Huff, J. S., Reznek, M., & Gordon, J. A. (2004). See one, do one, teach one: advanced technology in medical education. Academic Emergency Medicine, 11(11), 1149-1154.
• Weaver, R., Peters, K., Koch, J., & Wilson, I. (2011). ‘Part of the team’: professional identity and social exclusivity in medical students. Medical Education, 45(12), 1220-1229.
• Webster-Wright, A. (2009). Reframing professional development through understanding authentic professional learning. Review of Educational Research, 79(2), 702-739.
• WFME. (2003). Continuing Professional Development (CPD) of Medical Doctors – WFME Global Standards for Quality Improvement. Retrieved from http://www.who.int/workforcealliance/knowledge/toolkit/47/en/index.html
• Wood, T. (2009). Assessment not only drives learning, it may also help learning. Medical Education, 43(1), 5-6. doi:10.1111/j.1365-2923.2008.03237.x
• Wutoh, R., Boren, S. A., & Balas, E. A. (2004). ELearning: a review of Internet‐based continuing medical education. Journal of Continuing Education in the Health Professions, 24(1), 20-30.
• Wynia, M. K., Papadakis, M. A., Sullivan, W. M., & Hafferty, F. W. (2014). More than a list of values and desired behaviors: A foundational understanding of medical professionalism. Academic Medicine, 89(5), 712-714.