Learning modalities

Moodle Project Inspire – transparent next-generation learning analytics

Moodle/Totara community started Project Inspire. The goal is to create “transparent next-generation learning analytics that go beyond simple descriptive analytics to provide predictions of learner success, and ultimately diagnosis and prescriptions (advisements)” to learners, teachers, content experts and CME/CPS professionals. That sounds great. Right?

As a vibrant open source community, with 106+ million users, Moodle is well positioned to create a powerful learning analytic engine. At this moment, Project Inspire is collecting anonymized user data they can use to train the machine learning engine (+ develop and test additional analytic algorithms). So far bunch of schools and universities contributed anonymized copies of their sites. I’m proud to say that my Alma Mater, the University of Zagreb, contributed the first full data set to Project Inspire in April.

If you use Moodle/Totara 2.7+ you can join the project. Here you can find more about it:

Interviews

Sample Size

The literature states that there is “no prescriptive sample size for a phenomenographic study” (Yates, Partridge, & Bruce, 2012, p. 103). Bowden (2005) suggests that the sample should be large enough to find sufficient variation in perceptions but small enough that the amount of data is manageable. A concept close to sufficient variation in perceptions is the saturation point. The research will achieve the saturation point when additional perceptions cannot be detected (Kaapu & Tiainen, 2012).

Study participants. Following that recommendation, I interviewed five to eight  members of each of the four groups: physician anesthesiologists, nurse anesthetists (NA), anesthesiologist assistants (AA), and surgeons. Majority of the participants are clinicians and leaders who are clinically active. The rest are CPD professionals and staff leaders.

This study analyzed twenty-two transcripts, gained through interviewing

• five anesthesiologist assistants (AAs),
• four surgeons and one CPD expert involved in the education of surgeons,
• five anesthesiologist physicians and two CPD professionals, and
• seven certified, registered nurse anesthetists (CRNAs).

In addition to that, because of technical issues, the recordings with one anesthesiologist and one CRNA were damaged and were not included in the analysis.

The in-depth, open-ended interviews were recorded and transcribed verbatim. The interviews lasted between 22 minutes and 115 minutes, and in all cases, a state of mutual understanding was achieved (discussion was exhausted) (Booth, 1997). The variation of length was because of different backgrounds and experiences with IPL. The participants involved in education and leadership had longer interviews than those working solely as clinicians. There are two possible explanations for that: (a) they are more familiar with the complexities of QIE and IPL, and (b) clinicians, especially physicians, usually give very precise and short answers that, in many ways, mimic the way they talk in OR.

Additional materials

The opening sessions during the last three ASA Annual Meetings (2014, 2015, and 2016) have tackled answers to the questions in this research. Since the speeches were delivered as opinion-shaping contemplations of leaders—and the profession in general—I decided to treat those presentations as separate interviews. Additional reasons for that decision are the following:

• The actions of top leaders are critical to the success or failure of IPL (IoM, 2013).
• The provided messages were very clear and can be paired with the questions asked in this research, and those messages have influenced how IPL and QIE are viewed.
• The key speeches delivered during yearly meetings of professional societies and promoted on their websites suggest the direction a profession is taking.
• The videos of those sessions are available online.

The interviews I did for my research on perception of IPL among anesthesiologists (Hlede, 2015) provide information that complement well with this research; therefore, I included them in the analysis.

Special label. Knowing that it is not a standard phenomenographic procedure, I clearly labeled all the comments I received through those channels.

Recruitment

The participants were recruited through references and direct personal contact, mainly through LinkedIn. A few leaders were willing to help me recruit participants. In comparison to my previous research participant recruitment campaigns, the references had a much smaller impact. Prior to this research, I have completed three phenomenographic papers, and I did not have problems recruiting participants. Actually, while I was doing phenomenographic research on anesthesiologists’ perspectives of IPL (Hlede, 2015), I had more leaders willing to be interviewed than I was able to handle. I was not able to replicate that ease of recruitment in this case.

Updating recruitment protocol. In an attempt to improve recruitment, on July 7, 2016, I requested permission to

1. update the research participant recruitment scripts,
2. offer participants’ insight into the summary of research data (described in the scripts), and
3. reward participation in the research with $60 Visa gift cards.

As reasoning for that change, I used the following:

1. This research requires interviews with eight groups of people. These include both clinicians and education professionals from four professions—physician anesthesiologists, surgeons, nurse anesthetists, and anesthesiologist assistants—a total of eight groups of participants. The recruitment of participants from all groups can be challenging.
2. Sensitive nature of the study topic. Interprofessional politics is an important factor at this moment. As health-care reform progresses and team-based care become standard, anesthesiologists, surgeons, and nurse anesthetists have very passionate discussions about the new team structure and payment models. For example, the debate on the nursing scope of practice (what nurses can do without the supervision of physician anesthesiologists) (Nader, Massarweh, & Safety, 2016) was culminating during this research; therefore, the mere mention of IPL and QIE as elements that assume more intensive collaboration with other professions may be perceived negatively.
3. Not comfortable talking about the topic. Interprofessional learning and QIE are still emerging phenomena; therefore, those topics are not commonly discussed among potential interviewees who are not accustomed to explaining their positions on this topic (Namageyo-Funa et al., 2014).
4. Extensive health-care reform creates instability, and users are exposed to numerous surveys and interviews related to the reform; therefore, they are not interested in additional surveys/interviews.
5. Concerns about confidentiality, a common issue, are exacerbated by the perception that I am an employee of a competing professional group (ASA) with an ulterior motive.
6. Perceptions of bias may be exacerbated because of my work for the ASA.
7. No-participation policy. Wiebe, Kaczorowski, and MacKay (2012) found that more than one-third of physicians may have an office policy of not participating in research. It is fair to assume that the same policy affects all clinicians in the organization.
8. Make participation a revenue-neutral experience. U.S. clinicians are well-paid and busy professionals; however, in the case of anesthesiologists or surgeons, they start their career in their early thirties with significant student debt that, with interest, could exceed $350,000 (AAMC, 2014). Therefore, they are more likely to prefer opportunities where they will be paid for their time.
9. Reducing nonresponse bias. Nonresponse bias or the likelihood that the survey respondent group may be significantly different from the population in this research (VanGeest & Johnson, 2013) can be reduced if payment can encourage additional participants.
10. Build trust by showing respect and underline the seriousness of this research. Trust is the key in participant recruitment. This initiative can contribute to trust by respecting their time and presenting this as a serious research with a budget.

On July 18, 2016, the change was approved.

As a result, one surgeon and one anesthesiologist assistant were recruited through references. All other participants were recruited through LinkedIn or through previous contacts. The representatives of all organizations were willing to participate in the research except the AANA. That was at odds with the fact that the recruitment of nurse anesthetists through LinkedIn was the most productive.

Interview Questions

The questions below were selected to provide critical variation among participants. Those variations were categorized and organized in an outcome space (Cope, 2004):

1. Can you please reflect on your previous experience of working in multi-professional teams?
2. How would you describe IPL and QIE? For example, what is the purpose of each? Is it the same purpose?
3. What are the differences or similarities?
4. How about benefits and challenges?
5. Can technology help us address those challenges or enhance the benefits? If yes, how?
6. What, in your opinion, are the most important benefits and challenges associated with the perioperative surgical home (PSH)?
7. How is PSH related to IPL and/or QIE? Is it related?
8. Is there something QIE/IPL related that members of your profession can learn from those other professions?

The in-depth, open-ended interviews were recorded and transcribed verbatim.

Since this is a phenomenographic research, the participants were informed that there are no right or wrong answers (Daly, 2008).

References

1. Booth, S. (1997). On phenomenography, learning and teaching. Higher Education Research & Development, 16(2), 135-158.
2. Bowden, J. A. (2005). Reflections on the phenomenographic research process. In J. A. Bowden & P. Green (Eds.), Doing Developmental Phenomenography. Melbourne, Victoria: RMIT University Press.
3. Cope, C. (2004). Ensuring validity and reliability in phenomenographic research using the analytical framework of a structure of awareness. Qualitative Research Journal, 4(2), 5-18.
4. Daly, S. R. (2008). Design Across Disciplines. Purdue University. Engineering, Education, Ann Arbor, MI, U.S. Retrieved from http://books.google.hr/books?id=dSE4uvnBskMC
5. Hlede, V. (2015). Interprofessional Learning: Anesthesiologists’ Perspectives. Assignment, Doctoral Programme in E-Research and Technology Enhanced Learning. Department of Educational Research. Lancaster University.
6. IoM, Institute of Medicine. (2013). Interprofessional Education for Collaboration: Learning How to Improve Health from Interprofessional Models Across the Continuum of Education to Practice: Workshop Summary: National Academies Press.
7. Kaapu, T., & Tiainen, T. (2012). Phenomenography: Alternative Research Approach for Studying the Diversity of Users’ Understandings. Paper presented at the European Conference on Information Systems, http://aisel.aisnet.org/ecis2012/29.
8. Nader, N., Massarweh, M., & Safety, M. E. (2016). Veterans Affairs Proposed Rule for Advanced Practice Registered Nurses in the Operating Room A Step Forward or Overstepping?
9. Namageyo-Funa, A., Rimando, M., Brace, A. M., Christiana, R. W., Fowles, T. L., Davis, T. L., . . . Sealy, D.-A. (2014). Recruitment in qualitative public health research: Lessons learned during dissertation sample recruitment. The Qualitative Report, 19(4), 1-17.
10. VanGeest, J. B., & Johnson, T. P. (2013). Surveying clinicians: An introduction to the special issue. Evaluation and the Health Professions, 36(3), 275-278.
11. Wiebe, E. R., Kaczorowski, J., & MacKay, J. (2012). Why are response rates in clinician surveys declining? Canadian Family Physician, 58(4), e225-e228.
12. Yates, C., Partridge, H., & Bruce, C. (2012). Exploring information experiences through phenomenography. Library and Information Research, 36(112), 96-119.

Data analysis

Data analysis will rely on three segments (Figure 5):

1. Phenomenographic research will draw a picture of how QIE/IPL is perceived. My goal is to make the process as collaborative, iterative and transparent as possible/approved. Therefore, in addition to the standard phenomenographic method illustrated below, to evaluate the findings I will 1) organize a follow-up interview with selected participants and 2) share findings with a few leaders and the public (if approved) and ask for feedback. If there are moderate differences between groups, I will reflect on them. If those differences are significant, it may be possible to create separate outcome spaces for each group.
2. A case study of each professional organization involved in the research will be focused on QIE/IPL-related practices and technology, and the official policy used in the organization. That can help us better interpret data from phenomenographic research and get insight into what is possible in reality. For example, in my most recent research (Hlede, 2015) I found that in the ASA all research participants indicated that IPL is the preferred way to go. However, the official policy of the organization doesn’t reflect that.
3. Interaction between groups and existing and potential QIE/IPL projects will be analyzed through activity theory, so we can get a better picture of interprofessional activities that at this moment shape perceptions of each profession.

Figure 5. Data analysis.

Reference

Hlede, V. (2015). Interprofessional Learning: Anesthesiologists’ Perspectives. Assignment, Doctoral Programme in E-Research and Technology Enhanced Learning. Department of Educational Research. Lancaster University.