Understanding Secure Messaging in the Inpatient Environment: A New Avenue for Communication and Patient Engagement

Abstract

Background Patient portals, and the secure messaging feature in particular, have been studied in the outpatient setting, but research in the inpatient setting is relatively less mature.

Objective To understand the topics discussed in secure messaging in the inpatient environment, we analyzed and categorized messages sent within an inpatient portal.

Materials and Methods This observational study examined the content of all secure messages sent from December 2013 to June 2017 within an inpatient portal at a large Midwestern academic medical center (AMC). We analyzed a total of 2,598 messages, categorizing them by sender (patient, family, or care team member), type, and topic, and conducted a descriptive analysis of categories and an examination of code co-occurrence.

Results Patients were the most frequent message senders (63%); family members sent the fewest messages (10%). We identified five types of messages: Alert/Request; Thanks; Response; Question; and Other (typo/test message). Patient messages included Alerts/Requests (38%), Questions (31%), Statements of Thanks (24%), Response (1.2%), and Other (5%). We also identified 14 nonmutually exclusive message topics: Medication; Procedure/Treatment Plan; Schedule; Pain; Results; Diet; Discharge; Non-Medication Questions; Provider Requests; Symptoms; Custodial; Technical Issues; Potential Error; and Contact Information. Patient message topics most commonly discussed Symptoms (18%), Procedure/Treatment Plan (14%), or Pain (12%).

Conclusion Our analysis of secure message content suggests certain message types and topics such as Alerts/Requests and Questions about symptoms and treatment plans are particularly important to patients. These findings demonstrate that both patients and family members utilize the secure messaging function to engage in the care process by posing questions, making requests, and alerting staff to problems. As this technology is implemented in additional facilities, future work should examine how use of secure messaging may be influenced by factors including patients' demographics, reasons for hospitalization, and length of stay.

Keywords

patient portals - hospitalization - medical informatics - engagement

Background and Significance

In the outpatient setting, patient portals have been linked to improved patient outcomes and disease management[1] [2] [3] as they have increasingly been integrated into the delivery of care.[4] [5] By offering patients easy access to their health records, the ability to schedule and view their appointments, laboratory and test results, access to educational materials, and a way to communicate with providers, portals may facilitate patient engagement and increase many collaborative aspects of health care encounters.[6] [7] However, challenges may exist in using a portal, particularly for patients with chronic illness making attention to design choices critical.[8] [9] [10] [11] Communication between the patient and their care team has been an area of particular interest, with direct engagement through portal communication a component of meaningful use standards set forth to incentivize implementation and use of electronic health records.[12] [13]

Patient portals, and the secure messaging feature available within those portals in particular, have the potential to encourage a greater sense of trust in providers.[5] While much of the research on patient communications through portals has occurred in the outpatient setting,[12] [14] [15] [16] [17] [18] [19] study of inpatient portals containing this function has been limited.[20] This shift in context may be of importance, however, in how these tools are both perceived and used. Chen et al describes three main benefits of outpatient secure messaging including: asynchronous communication, the ability to discuss sensitive subjects with greater privacy, and the ability to retain messages to review later.[21] While existing outpatient coding schemes may inform the analysis of messages sent in the inpatient environment, given the nascent stage of inpatient secure messaging, it is unclear whether these same benefits apply to inpatient portals. In addition, while in the hospital, patients may use a portal designed for the outpatient environment to send messages to providers who care for them in the outpatient environment. However, outpatient portals do not allow communication with the team caring for them in the hospital. In addition, research about the inpatient environment suggests that hospitalized patients may have different health information technology (HIT) needs compared with patients in outpatient settings. For instance, Prey et al identified important facilitators of patient engagement in the inpatient setting including providing patient-specific education and the presence of additional communication opportunities associated with that environment.[22] Thus, both the members with whom a patient communicates and the patient HIT needs may differ in the inpatient and outpatient environments.

Early research on inpatient portals has been limited to studies focused on either specific conditions or smaller-scale studies assessing usability and satisfaction.[23] [24] [25] [26] [27] Collins et al conducted a mixed methods analysis of stakeholder perspectives related to inpatient portal use and reported that patients valued access to an inpatient portal.[4] In another study, Woollen et al identified unmet patient information needs in the hospital setting that inpatient portals can provide, such as easy access to laboratory results.[28] Yet given the novelty of this technology, researchers note that we currently lack understanding both about the optimal inpatient portal experience and how patients and providers can maximize inpatient portal use.[4] [22]

Differences between inpatient and outpatient contexts are particularly relevant when considering the secure messaging feature of patient portals. One prior study of the content of secure messages sent within the outpatient environment used automated classifiers and natural language processing, and reported that patients most frequently used the portal tool to seek medical information, send social messages, or elicit logistical and clinical information.[29] In another study, Shimada et al coded secure messages sent to primary care providers via an outpatient portal and found that they were most frequently transactional in nature, involving requests for medication refills and questions about scheduling.[30] However, needs specific to hospitalized patients may be appreciably different in this context and merely providing portal functions in the inpatient environment similar to those available in outpatient settings may be insufficient. Hospitalization may serve as a cue to action, increasing the patient's focus on their health and potentially increasing their willingness to use a portal to engage in their care.[5] [31] [32] Secure messaging in the inpatient setting may also offer an additional benefit to families who are not always present when the care team is in the patient's room. Using this tool, family members can monitor their loved one's condition, ask questions of the care team, and provide information to the care team.[33] Further, in the outpatient environment, a patient's relationship with their physician has been found to be an important predictor of use of secure messaging[34] [35] [36]; however, hospitalized patients may have a different experience. Inpatients typically see multiple providers during a single hospital stay, some of whom may be new to the patient. Without established relationships with these providers, patients' willingness to send secure messages in the inpatient environment may be impacted. This new technology in the inpatient environment thus introduces a new avenue for communication between patients or family members and care team members and raises new questions about how and by whom this tool is used.

Objective

To date, little is known about secure messaging in the inpatient environment. We undertook a mixed methods study of secure message usage within the inpatient portal implemented in a large academic medical center (AMC) to develop an understanding of how secure messages are used and examine this use in the context of inpatient care. Our study was designed to explore how patients and care team members use the secure messaging feature by examining the content of individual messages and categorizing them by sender, type, and topic. Understanding the secure messaging function in the inpatient setting can help to inform future efforts to support patients' use of inpatient portals as well as help hospitals prepare for this new form of communication with their hospitalized patients.

Materials and Methods

Study Design

This observational study examined the content of secure messages sent within an inpatient portal across a large Midwestern AMC. Messages were examined both qualitatively and quantitatively to permit both categorization and descriptive analysis of message sender and message content.

Study Setting

This AMC provides health services across the continuum of care at 7 hospitals and 53 ambulatory care locations, including 30 community-based clinics. The AMC implemented MyChart Bedside, an inpatient portal developed by Epic Systems (Verona, Wisconsin, United States) that offers a secure messaging feature, across the cancer hospital and labor and delivery units in late 2013. This was followed by implementation across all general hospital units in the AMC in a staged rollout that began during the summer of 2016.[37] The MyChart Bedside application is made available to inpatients on tablet computers that are provisioned upon admission. Patients who are English-speaking, over 18 years of age, nonprisoners, and have no cognitive impairments are offered a tablet for use during their hospital stay.

Patients and family members can send messages via the MyChart Bedside portal with messages going to all members of the care team assigned to that patient. In addition, care team members can initiate messages to the patient, which the patient will be able to view upon opening the application. Policies for care team members responding to messages, such as which team member responds and how frequently team members must check their messages, are established at the unit level and vary across the health system. Further, across the health system, care team members are able to respond electronically or in-person to the patient; the health system does not require an electronic response or documentation of an in-person response.

Study Data

All inpatient secure messages sent between December 2013 and June 2017 across the 7 hospitals of the health system (n = 2,599) were provided to the research team via secure link from the AMC's Information Warehouse. Data were stored in a secure directory accessible only to the research team and requiring institutional credential sign-in. Messages were deidentified and not linked to any patient information for the analyses we conducted in this study.

Qualitative Analysis

A coding team consisting of three experienced coders (C.S., J.V., and J.H.) jointly reviewed 300 messages to develop a preliminary coding dictionary based on sender (originator of the message), message topic (issues addressed in the message), and message type (goal of the message). Emergent codes were proposed and discussed as the preliminary coding progressed.[38] Then, following the methods of thematic analysis outlined by Constas, two coders (C.S. and J.V.) each coded the same 200 messages using the preliminary coding dictionary in increments of 50 messages until 100% agreement between coders was reached. Frequent discussions between the two coders enabled identification of messages in which coding did not align and differences were resolved so that consensus about the codes and a final coding dictionary was achieved. The same two coders then each coded half of the remaining 2,399 messages using the finalized dictionary, continuing to meet throughout the process to ensure agreement in coding. The qualitative analysis software Atlas.ti (version 6.0) was utilized to facilitate the coding process.[39]

Coders categorized messages on three dimensions: sender, type, and topic. Message sender was first identified by the source recorded in the portal log file output (e.g., patient vs. care team member). Care team members were identified in the file name by their role on the care team (e.g., physician, nurse, patient care associate). Our analysis then further distinguished sender reading each message and determining whether the message was sent by the patient using personal pronouns (e.g., I have a question for my doctor) or identifying oneself by name (e.g., this is John Smith) versus those sent from the patient's account but clearly originating from a patient's family member by referring to the patient by role (e.g., my mom's medications) or by name (e.g., John's blood pressure). Message type and topic were determined by reading each message determining the appropriate code. All messages were assigned a type categorizing the intent of the message, with more than one type possible in a single message. Message topics described the specific issue being expressed in the message. Individual messages could contain more than one topic. Type and topic categories emerged from the message content.

Quantitative Analysis

We conducted a descriptive analysis of the frequency of messages by sender, type, and topic. We also examined the number of messages sent and number of topics addressed per message by sender. Within topic, we reviewed the co-occurrence of codes to understand which topics were discussed together in the same message. Co-occurrence was calculated for each message topic as the number of messages with the focal topic of each calculation plus each other topic individually, divided by the total number of messages with the focal topic of the calculation.[40] For example, to determine the co-occurrence of Treatment Plan with Symptoms (focal topic), we utilized the following formula:

Results

Secure Message Senders

We found that the majority of patients (74.3%) sent one message during admission, and fewer than 3% sent over 5 messages. Most messages, 63%, were sent by patients, with family members sending the fewest messages, at 10%. Within care team categories, nurses sent the majority of messages. We present this overall distribution of messages by senders in [Table 1].

Table 1 Total secure messages by sender

	N	%
Patient	1,632	62.8
Family member	261	10.0
Care team (total)	705	27.1
Nurse	540	76.6
Physician/Medical student	38	5.4
Patient care assistant	34	4.8
Other	93	13.2
Total	2,598	100.0

Secure Message Types and Topics

Our qualitative analysis identified five types of messages: Alert/Request, Thanks, Response, Question, and Other. [Table 2] provides a definition of each message type along with an example of that type of message. Because secure messages are considered protected health information, the examples included in this table are fictional.

Table 2 Inpatient secure message types

Message type	Definition of message type	Example of message type
Alert/Request	Identifying a new condition, symptom, or problem or asking for a service or device	“I have been having more frequent headaches”
Thanks	Offering thanks (e.g., for care received)	“Thank you all for taking such great care of me”
Question	Seeking information about any topic	“Will I be having an MRI today?”
Response	Responding to a previous question or request	“The doctor will be in to see you shortly”
Other	Sent in error or to test this feature of the device	“Hi, just testing out this new tablet”

Abbreviation: MRI, magnetic resonance imaging.

We also identified 14 distinct topics included in the secure messages as shown and defined in [Table 3]. As with message types we presented above, the examples we included are fictional. Also, as previously noted, messages could contain more than one topic. However, 29% of messages were not assigned a topic due to their content. These messages were typically Thanks, Response, or Other.

Table 3 Inpatient secure message topics

Topic code	Topic definition	Example message
Medication	Includes requests for medication, asking what a particular medication treats, or asking when the next dose of medication will be administered	“Why am I taking aspirin?”
Treatment Plan	Addressing issues related to possible tests, medications, procedures, or other treatments	“Will I need an MRI?”
Symptoms	Discussing physical or mental symptoms the patient is experiencing or is concerned about experiencing	“I have a headache and my stomach hurts”
Scheduling	Asking when a test, medication administration, procedure, or treatment will take place	“Is my operation scheduled for Tuesday?”
Dietary Issues	Discussing a patient's diet or meal delivery	“I would like to be taken off of a low fat diet”
Results	Addressing results of laboratory tests, procedures, or other tests	“Please explain the results of my blood test”
Pain	Mentions that the patient is experiencing pain or concerned about pain	“My arm is hurting”
Discharge	Discussing timing of or plans for discharge including postdischarge treatment	“Can I be discharged on Thursday?”
Non-Medication Questions	Addressing items or services a patient needs that are not related to medication	“I would like an extra pillow”
Provider Requests	Requesting assistance from a particular provider or type of provider	“I would like a social worker to stop by my room”
Custodial	Discussing problems with the physical environment of the patient's room	“Please clean the TV screen”
Technical Issues	Discussing problems with technology in the patient's room, including the tablet with the MyChart Bedside application	“My tablet needs to be charged”
Potential Error Identification	Alerting the care team to a potential discrepancy in medical history, medication, or expected procedures	“My allergy to amoxicillin is not listed in my allergies”
Contact Information	Providing information about persons relevant to the patient's care	“My primary care physician's number is …”

Abbreviation: MRI, magnetic resonance imaging.

Messages by patients typically addressed only 1 topic, and we found a maximum of 5 topics in a single message. Family members addressed slightly more topics per message, with an average of 1.5 topics and a maximum of 5 topics covered. Care team members included 1 topic per message on average, with a maximum of 4 topics in a single message.

Secure Messages Types, by Sender

We found that types of messages varied by sender. As [Table 4] shows, most patient messages were Alerts/Requests (40%), Questions (28%), or Thanks (25%). Family members sent messages that asked Questions (45%) or Alerted the care team/made a Request (41%). The majority of care team messages were Responses to patient or family messages (74%), followed by Responses to patient Thank you messages (20%).

Table 4 Secure messages by type and by sender

	Patient		Family		Care team
Message type	N	%	N	%	N	%
Alert/Request	699	40.7	119	41.0	6	0.7
Thanks	425	24.7	40	13.8	168	19.8
Response	21	1.2	1	0.3	629	74.2
Question	487	28.3	130	44.8	7	0.8
Other	86	5.0	0	0.0	38	4.5
Total	1,718	100.0	290	100.0	848	100.0

Secure Message Topics, by Sender

[Fig. 1] shows the percentages of messages sent by each sender on each message topic. The most common message topic from patients was related to discussing Treatment Plan, followed by Medication and Symptoms. Very few patient messages were related to Custodial issues or Potential Error Identification. Family members sent messages most frequently about the same three topics. Messages from the care team also most frequently focused on the Treatment Plan, but messages related to the provider coming to see the patient were the second most frequent topic addressed by the care team.

Fig. 1 Counts of secure messages sent by sender type and by topic.

Code Co-occurrence

To gain a thorough understanding of secure messaging, we also examined how frequently message topics occurred concurrently. The results of our code co-occurrence analysis are presented in [Fig. 2] which shows the percentages of co-occurring message topics, by topic category. We found that among messages sent related to Potential Error Identification, 54% addressed Medication, 19% addressed Procedure/Treatment Plan, and 12% discussed Symptoms. However, of the messages that related to Medication, only 4% were also related to Potential Error Identification; most pertained to Pain (34%), Symptoms (28%), or Procedure/Treatment Plan (27%). Messages related to Pain were most frequently also related to Medication (58%) and Symptoms (46%). Technical Issues were infrequent but most commonly co-occurred with messages about Diet (28%) and Results (17%).

Fig. 2 Code co-occurrence of focal topics.

Discussion

While other researchers have examined the content of secure messages exchanged between providers in the inpatient setting,[41] this study is the first to analyze secure message content exchanged between patients or family members and care team members exclusively through an inpatient portal, a tool increasingly being introduced in hospital settings.[17] [18] Outpatient portals have seen an increase in secure messaging over time, with messages often related to appointments and medication refills. Our findings suggest that hospitalized patients use the secure message feature differently than in the outpatient setting, with inpatients using the feature mainly to discuss their care plan and ask questions. We found that just over one-quarter of patients sent more than one message during their admission, suggesting acceptance of this portal feature, even in this early stage of inpatient portal use. In addition, we found that patients, families, and care team members in our study used the secure messaging function differently with respect to both message type and topic. For instance, patient messages focused predominantly on health questions, logistical/scheduling concerns, and technical problems, while messages from family members frequently alerted the care team to symptoms the patient was experiencing or asked for additional information about symptoms or the procedure/treatment plan. Notably, timely access to health information, such as explanation of results and clarification of medication schedules, is particularly relevant to the inpatient environment, and may decrease patient and caregiver anxiety,[27] thus the availability of this information via a portal may be an important opportunity to increase the patient-centeredness of care delivered, an important goal of health care organizations.

Many messages from both patients and family members in our study involved asking questions (28 and 45%, respectively), providing early evidence that patients are willing to use this communication modality to seek health information. Moreover, secure messaging in the inpatient environment allows family members to communicate directly with the care team asynchronously. Prior to the availability of this technology, family members might leave notes at the patient's bedside to which the care team could respond later, or they might attempt to connect with the care team via telephone. The secure messaging function, however, allows family members to send a message while they are visiting the patient and receive a response when they next visit and use the inpatient portal. While other studies suggest mixed results related to whether providing health information through an inpatient portal facilitates or replaces in-person communication,[28] [42] our findings demonstrate that both patients and family members may use the secure message function to ask questions and this may help them to remain engaged in the care process. Further, in the outpatient setting, patients viewed secure messaging as an important opportunity to communicate with their providers at a time that was convenient for them and that offered a record of the communication[5] [21]; we expect similar benefits would be perceived by caregivers in the inpatient environment as well.

While each patient-generated message in our study was sent to all members of the patient's care team, nurses were the most frequent respondents to these messages. This finding may reflect the typical pattern of in-person communication during hospitalization as nurses tend to have the most frequent contact with patients and therefore are well-positioned to respond to patient messages. Nonetheless, 25% of messages from the care team were sent by other team members, including physicians. Future research should monitor patterns in care team responses as both patients and care teams gain greater experience with secure messages in the inpatient setting. Moreover, studies in the outpatient setting document provider concerns about the possible increase in workload from secure messages in the outpatient setting,[18] [43] and we might expect similar concerns among hospital care team members. As secure messaging in the inpatient environment increases in use, future studies should explicitly consider issues such as how secure messaging impacts provider time and workflow and how best to train both patients and providers in using this type of tool.[44]

Our code co-occurrence analysis highlighted patterns that may be useful for understanding which topics are most important to patients, as well as providing insight into which features of the inpatient portal might need improvement. Our analysis found that half of all messages referring to procedure/treatment plan co-occurred with message topics about pain and symptoms. This suggests patients may be utilizing the secure messaging feature to keep their providers better informed and updated on their health status, which may alter their treatment plan.

Although messages reporting potential errors were infrequent, the majority of these types of messages addressed possible medication errors, lending support to the idea that inpatient secure messaging may present another avenue to explore in the context of patient safety improvement efforts.[45] [46] In addition, patient technology-related questions most frequently focused on challenges with using the food ordering function and with viewing laboratory test results, suggesting a need to improve these features and perhaps enhance training around their use.[11] [21] [47]

Secure messaging in the inpatient environment, as in the outpatient setting, has the potential to improve patient care by increasing patient engagement in their care.[6] [22] [48] [49] [50] The secure messaging function also enables patients to retain an electronic record of their communications with care team members. In practice, the availability of this new asynchronous communication avenue in the hospital context may provide the opportunity to enhance patient care by improving patient and family understanding of the patient's condition and treatment plan, facilitate recognition of changes in symptoms particular to that patient, and even assist with postdischarge treatment by allowing caregivers who may be caring for the patient after discharge another venue for asking questions.

As Collins et al note, however, there is a need to understand how use of the secure messaging feature enhances in-person communication but does not replace it.[4] Patient portal technology is becoming more integrated within the inpatient environment and understanding how patients, families, and care team members are using the secure messaging feature to communicate can enhance our ability to answer patient questions and respond to patient needs. Our study results thus help to improve the understanding of the use and potential for this portal feature in the inpatient environment and may inform future interventions focused on improving patient engagement in the inpatient setting.

Limitations

Our study faces some important limitations. First, our study offers insight based on secure messages sent within a single AMC that was an early adopter of an inpatient portal. As a result, our study findings may have limited generalizability to the broader population of health care organizations and later adopters of this tool. As inpatient portals are implemented in more hospitals, researchers should examine how the secure message types and topics evolve.

Second, this study utilized deidentified data that limited our ability to match messages with patient demographic and clinical information. Our emphasis in this study was on describing how hospitalized patients use secure messages. However, investigating the effect of patient characteristics on message attributes remains an intriguing area for future inquiry.

Conclusion

Inpatient portals are increasingly available and offer many features to hospitalized patients and their families, particularly the ability to communicate via secure messaging. We found that patients, families, and care team members are willing to use secure messages as an additional means of communication, and we characterized the content of the messages sent, describing topics of importance to patients, families, and care team members. As this technology is increasingly implemented in the inpatient environment, future work should examine how patterns of use evolve, as well as how use may be related to patient outcomes such as examining whether greater use of the message feature is related to improved understanding of discharge instructions or fewer readmissions.

Clinical Relevance Statement

This study provides a preliminary framework to help health care systems and clinicians better understand how to engage patients using secure messaging in the inpatient environment. We found that patients and family members are willing to use secure messaging in a variety of ways, thus requiring increased attention by hospitals and health systems to understand how best to support this function. Understanding the types of messages patients send and topics that are important to patients can help hospital clinicians better adapt to this new technology as it becomes more frequently used in the inpatient setting.

Multiple Choice Questions

1. What are two benefits of secure messaging in the inpatient setting?

   • Having a record of communication.

   • Facilitates asynchronous communication between families and care team members.

   • Backup to the call light for urgent medical needs.

   • It replaces in-person communication with the care team.

   Correct Answer: The correct answers are options a and b because the secure message feature should be not used for urgent medical needs (c) and should not replace in-person communication with the care team (d).

2. Which of the following would be applications of an analysis of secure messages?

   • To improve the functionality of the secure messaging feature of the portal.

   • To help patients view their laboratory results.

   • To help care team members prioritize responses.

   • a and c.

   Correct Answer: The correct answer is option d. An analysis of secure messages does not assist patients in using other features of the tool such as viewing laboratory results.

Conflict of Interest

None declared.

Acknowledgment

The authors would like to thank Robert Taylor who was affiliated with our institution at the time of this research for his assistance with managing the data for this study.

Protection of Human and Animal Subjects

This study was approved by the Institutional Review Board of The Ohio State University.

• References

• 1 Krist AH, Woolf SH, Rothemich SF. , et al. Interactive preventive health record to enhance delivery of recommended care: a randomized trial. Ann Fam Med 2012; 10 (04) 312-319
• 2 Osborn CY, Mayberry LS, Mulvaney SA, Hess R. Patient web portals to improve diabetes outcomes: a systematic review. Curr Diab Rep 2010; 10 (06) 422-435
• 3 Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res 2012; 14 (06) e162
• 4 Collins SA, Rozenblum R, Leung WY. , et al. Acute care patient portals: a qualitative study of stakeholder perspectives on current practices. J Am Med Inform Assoc 2017; 24 (e1): e9-e17
• 5 Sieck CJ, Hefner JL, Schnierle J. , et al. The rules of engagement: Perspectives on secure messaging from experienced ambulatory patient portal users. JMIR Med Inform 2017; 5 (03) e13
• 6 Walker DM, Sieck CJ, Menser T, Huerta TR, Scheck McAlearney A. Information technology to support patient engagement: where do we stand and where can we go?. J Am Med Inform Assoc 2017; 24 (06) 1088-1094
• 7 Shah SD, Liebovitz D. It takes two to tango: engaging patients and providers with portals. PM R 2017; 9 (5S): S85-S97
• 8 Ali SB, Romero J, Morrison K, Hafeez B, Ancker JS. Focus section health IT usability: applying a task-technology fit model to adapt an electronic patient portal for patient work. Appl Clin Inform 2018; 9 (01) 174-184
• 9 Wieland D, Gibeau A, Dewey C, Roshto M, Frankel H. Patient portal readiness among postpartum patients in a safety net setting. Appl Clin Inform 2017; 8 (03) 698-709
• 10 Walker DM, Menser T, Yen PY, McAlearney AS. Optimizing the user experience: identifying opportunities to improve use of an inpatient portal. Appl Clin Inform 2018; 9 (01) 105-113
• 11 Walker DM, Hefner JL, Sieck CJ, Huerta TR, McAlearney AS. Framework for evaluating and implementing inpatient portals: a multi-stakeholder perspective. J Med Syst 2018; 42 (09) 158
• 12 Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med 2010; 363 (06) 501-504
• 13 Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res 2015; 17 (02) e44
• 14 Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs). J Am Med Inform Assoc 2008; 15 (06) 729-736
• 15 Nelson JA. Personal health records. Home Health Care Manage Pract 2009; 21 (02) 141-142
• 16 Murphy J. Engaging patients and families in ehealth. Nurs Econ 2011; 29 (06) 339-341
• 17 Halamka JD, Mandl KD, Tang PC. Early experiences with personal health records. J Am Med Inform Assoc 2008; 15 (01) 1-7
• 18 Kelly MM, Dean SM, Carayon P, Wetterneck TB, Hoonakker PL. Healthcare team perceptions of a portal for parents of hospitalized children before and after implementation. Appl Clin Inform 2017; 8 (01) 265-278
• 19 Steitz B, Cronin RM, Davis SE, Yan E, Jackson GP. Long-term patterns of patient portal use for pediatric patients at an academic medical center. Appl Clin Inform 2017; 8 (03) 779-793
• 20 Kelly MM, Coller RJ, Hoonakker P. Inpatient portals for hospitalized patients and caregivers: a systematic review. J Hosp Med 2018; 13 (06) 405-412
• 21 Chen C, Garrido T, Chock D, Okawa G, Liang L. The Kaiser Permanente Electronic Health Record: transforming and streamlining modalities of care. Health Aff (Millwood) 2009; 28 (02) 323-333
• 22 Prey JE, Woollen J, Wilcox L. , et al. Patient engagement in the inpatient setting: a systematic review. J Am Med Inform Assoc 2014; 21 (04) 742-750
• 23 Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc 2009; 16 (05) 660-669
• 24 Masterson Creber R, Prey J, Ryan B. , et al. Engaging hospitalized patients in clinical care: study protocol for a pragmatic randomized controlled trial. Contemp Clin Trials 2016; 47: 165-171
• 25 Dykes PC, Stade D, Chang F. , et al. Participatory design and development of a patient-centered toolkit to engage hospitalized patients and care partners in their plan of care. In AMIA Annual Symposium Proceedings; 2014 . American Medical Informatics Association
• 26 Greysen SR, Magan Mendoza Y, Rosenthal J. , et al. Using tablet computers to increase patient engagement with electronic personal health records: protocol for a prospective, randomized interventional study. JMIR Res Protoc 2016; 5 (03) e176
• 27 Kelly MM, Hoonakker PL, Dean SM. Using an inpatient portal to engage families in pediatric hospital care. J Am Med Inform Assoc 2017; 24 (01) 153-161
• 28 Woollen J, Prey J, Wilcox L. , et al. Patient experiences using an inpatient personal health record. Appl Clin Inform 2016; 7 (02) 446-460
• 29 Cronin RM, Fabbri D, Denny JC, Jackson GP. Automated classification of consumer health information needs in patient portal messages. In AMIA Annual Symposium Proceedings; 2015 . American Medical Informatics Association
• 30 Shimada SL, Petrakis BA, Rothendler JA. , et al. An analysis of patient-provider secure messaging at two Veterans Health Administration medical centers: message content and resolution through secure messaging. J Am Med Inform Assoc 2017; 24 (05) 942-949
• 31 Holtrop JS, Corser W, Jones G, Brooks G, Holmes-Rovner M, Stommel M. Health behavior goals of cardiac patients after hospitalization. Am J Health Behav 2006; 30 (04) 387-399
• 32 Coulter A, Ellins J. Effectiveness of strategies for informing, educating, and involving patients. BMJ 2007; 335 (7609): 24-27
• 33 Kelley T, Docherty S, Brandon D. Information needed to support knowing the patient. ANS Adv Nurs Sci 2013; 36 (04) 351-363
• 34 Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med 2011; 26 (10) 1112-1116
• 35 Ralston JD, Rutter CM, Carrell D, Hecht J, Rubanowice D, Simon GE. Patient use of secure electronic messaging within a shared medical record: a cross-sectional study. J Gen Intern Med 2009; 24 (03) 349-355
• 36 Weppner WG, Ralston JD, Koepsell TD. , et al. Use of a shared medical record with secure messaging by older patients with diabetes. Diabetes Care 2010; 33 (11) 2314-2319
• 37 Huerta TR, McAlearney AS, Rizer MK. Introducing a patient portal and electronic tablets to inpatient care. Ann Intern Med 2017; 167 (11) 816-817
• 38 Constas MA. Qualitative analysis as a public event: the documentation of category development procedures. Am Educ Res J 1992; 29 (02) 253-266
• 39 Scientific Software. D., Atlas.ti Berlin: Scientific Software Development; 2008
• 40 Namey E, Guest G, Thairu L, Johnson L. Data reduction techniques for large qualitative data sets. Handbook Team Based Qual Res 2008; 2 (01) 137-161
• 41 Dalal AK, Dykes PC, Collins S. , et al. A web-based, patient-centered toolkit to engage patients and caregivers in the acute care setting: a preliminary evaluation. J Am Med Inform Assoc 2016; 23 (01) 80-87
• 42 O'Leary KJ, Sharma RK, Killarney A. , et al. Patients' and healthcare providers' perceptions of a mobile portal application for hospitalized patients. BMC Med Inform Decis Mak 2016; 16 (01) 123
• 43 Nazi KM. The Personal Health Record Paradox: Health Care Professionals' Perspectives and the Information Ecology of PHR Systems in Organizational and Clinical Settings. In Medicine 2.0 Conference; 2012. . JMIR Publications Inc., Toronto, Canada
• 44 Hefner JL, Sieck CJ, McAlearney AS. Training to optimize collaborative use of an inpatient portal. Appl Clin Inform 2018; 9 (03) 558-564
• 45 Pell JM, Mancuso M, Limon S, Oman K, Lin CT. Patient access to electronic health records during hospitalization. JAMA Intern Med 2015; 175 (05) 856-858
• 46 Yoo S, Lee KH, Baek H. , et al. Development and user research of a smart bedside station system toward patient-centered healthcare system. J Med Syst 2015; 39 (09) 86
• 47 Hefner JL, Sieck CJ, Walker DM, Huerta TR, McAlearney AS. System-wide inpatient portal implementation: survey of health care team perceptions. JMIR Med Inform 2017; 5 (03) e31
• 48 Vawdrey DK, Wilcox LG, Collins SA. , et al. A tablet computer application for patients to participate in their hospital care. In AMIA Annual Symposium Proceedings; 2011 . American Medical Informatics Association
• 49 Health in the United States; 2016 Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/nchs/hus/contents2016.htm#older . Accessed November 20, 2017
• 50 O'Leary KJ, Lohman ME, Culver E, Killarney A, Randy Smith Jr G, Liebovitz DM. The effect of tablet computers with a mobile patient portal application on hospitalized patients' knowledge and activation. J Am Med Inform Assoc 2016; 23 (01) 159-165