Provider Preferences for Patient-Generated Health Data Displays in Pediatric Asthma: A Participatory Design Approach

Abstract

Objective There is a lack of evidence on how to best integrate patient-generated health data (PGHD) into electronic health record (EHR) systems in a way that supports provider needs, preferences, and workflows. The purpose of this study was to investigate provider preferences for the graphical display of pediatric asthma PGHD to support decisions and information needs in the outpatient setting.

Methods In December 2019, we conducted a formative evaluation of information display prototypes using an iterative, participatory design process. Using multiple types of PGHD, we created two case-based vignettes for pediatric asthma and designed accompanying displays to support treatment decisions. Semi-structured interviews and questionnaires with six participants were used to evaluate the display usability and determine provider preferences.

Results We identified provider preferences for display features, such as the use of color to indicate different levels of abnormality, the use of patterns to trend PGHD over time, and the display of environmental data. Preferences for display content included the amount of information and the relationship between data elements.

Conclusion Overall, provider preferences for PGHD include a desire for greater detail, additional sources, and visual integration with relevant EHR data. In the design of PGHD displays, it appears that the visual synthesis of multiple PGHD elements facilitates the interpretation of the PGHD. Clinicians likely need more information to make treatment decisions when PGHD displays are introduced into practice. Future work should include the development of interactive interface displays with full integration of PGHD into EHR systems.

Keywords

analysis - design - data visualization - self-management - mHealth - smartphone

Background and Significance

Pediatric asthma, an airway disease characterized by wheezing and chest tightness, is the most common chronic disease in children.[1] Despite treatment improvements, asthma prevalence continues to increase and mortality rates have remained stagnant.[2] [3] In attempts to mitigate worsening outcomes, initiatives now include the use of mobile health (mHealth) applications (apps) to encourage self-management.[4] [5] Such apps facilitate personal health tracking with the collection of patient-generated health data (PGHD), and may include device data, medication history or other health data, and patient-reported outcomes.[6] Adolescents, the largest segment of smartphone owners, are well positioned to collect and share PGHD with their providers, given the need to establish ongoing asthma control early in life.[7] [8]

PGHD shared during clinical encounters could facilitate assessment and modification of treatment plans.[9] Presenting PGHD within the electronic health record (EHR) would give providers a longitudinal view of health between visits.[10] [11] However, clinicians have expressed concerns with presentation formats and the prioritization of PGHD.[12] EHR interfaces in general are criticized for poor usability, not matching clinical workflows, and having issues with information presentation.[13] To integrate PGHD into EHRs, an understanding of clinical workflows and advances in data visualization are needed.[14] Iterative design approaches can be used to accommodate various PGHD data types and use cases.[15] There is a need to translate PGHD into actionable insights for providers,[16] but design of PGHD interventions seldom involves providers.[17] In a scoping review,[18] we were unable to find guidance for the incorporation of PGHD into clinical workflows or the optimal approaches to display and visualize such data.

Objective

The purpose of this study was to investigate provider preferences for the graphical display of pediatric-asthma PGHD to support decisions and information needs in the outpatient setting. We aimed to design and assess the usability of low-fidelity prototypes, assess preferences for PGHD visualizations, and obtain insights to guide future interactive-display development.

Methods

We conducted the study in two phases. The first phase focused on design and creating case-based vignettes and initial low-fidelity prototypes for information displays. The second phase was a formative evaluation of the displays. The study procedures are depicted in [Fig. 1].

Fig. 1 Study procedures. *The information needs assessment is described in a separate publication.

Development of Vignettes and Displays

Vignette Development

We developed vignettes to anchor the participatory design approach and formative evaluation. Vignettes are brief, written scenarios about hypothetical characters to simulate the features of specific, real-world situations, and are used to elicit responses from research participants that can then be generalized.[19] [20] Vignette-based methodologies provide insights into thoughts, behaviors, and information-seeking strategies.[19] In the outpatient setting, researchers have used vignettes to measure physician practice.[21] We followed guidelines on how to construct and present vignettes as outlined by Kim,[20] and recommendations for vignette content provided by Evans et al.[19]

One member of the research team (V.L.T.) drafted two vignettes based on a set of decisions, information needs, and PGHD derived from evidence-based clinical guidelines for pediatric asthma.[22] The vignettes contained clinical content that emulated pediatric-asthma events in the outpatient setting. The focus of each vignette was on the PGHD needed to support decision making rather than the quality or outcome of decisions. The vignettes were reviewed by a senior member of the research team (C.W.) with extensive experience in vignette development and were revised until both authors agreed on the clarity and consistency of the content ([Table 1]).

Table 1 Vignette descriptions, decisions, and types of patient-generated health data

Vignette	Description	Decision or task	PGHD type
1	A 7-year-old girl presents as a new patient to your Salt Lake City practice for an asthma evaluation. She has a BMI at the 90th percentile, but active. She reports few symptoms during the winter, but in the spring, when her allergies are severe, she takes an Albuterol inhaler before outdoor activities. She had one exacerbation about 5 months ago but has had no symptoms in the past month. She has not needed recent urgent care or prednisone therapy. Upon examination, no wheezing is noted. Her mother is helping her track symptoms using a smartphone app, and every month she uses the app to calculate her ACT score. Her most recent score was 22. In addition, the app collects outdoor air quality, and local pollen counts.	Identify level of symptom control. Assess extent of exposure to risk factors.	Symptoms, Asthma Control Test Exposures, symptoms, environmental factors (air-quality index, pollen count)
2	A 15-year-old boy with a long history of asthma arrives for his follow-up visit at an NYC clinic. He complains of daily wheezing and episodes of nighttime coughing. He reports missed school days, is frustrated by schoolwork, and cannot keep up on the basketball court with his friends. He tries to remember to use his Flovent daily and Albuterol inhaler before exercise. No other health issues are noted. You have been unable to identify any specific triggers. He carries his smartphone with him everywhere, and for the last 6 months, has been using a smart inhaler connected to a mobile health application to collect all medication doses. In addition, he uses the app to document symptoms and exposures on a weekly basis.	Determine adjustments to the medication regimen. Identify exposure to risk factors.	Symptoms, inhaler use Exposures, symptoms

Abbreviations: ACT, Asthma Control Test; BMI, body mass index; NYC, New York City.

Display Development

Consistent with Gestalt theory and visualization principles,[23] [24] we chose to focus on graphical displays for the information. Graphical displays are ideal for use in data-rich environments and consist of a combination of object-based and text-based information to reduce cognitive load for decision makers.[25] For tasks requiring information filtering, low-fidelity prototypes of user interfaces can be an effective first step.[26] [27] [28]

We developed one display for each vignette in the form of two-dimensional wireframe mockups. Display 1 (D1) corresponded with the first vignette and Display 2 (D2) corresponded with the second vignette. The following criteria were used to guide the design. First, we wanted the displays to be clinically relevant and to reflect the types of PGHD needed to support each vignette. Second, using the strategy outlined by Shneiderman's Visual Information-Seeking Mantra (overview first, zoom and filter, then details-on-demand),[29] we created an overview of all available PGHD, with opportunities to zoom, filter, and access details on demand planned for future iterations. Third, we sought to ensure that each display had the highest-possible concentration of PGHD and contained multiple features to support the decisions described in each vignette.

We employed theoretically grounded visualization principles to assemble the PGHD in the displays.[30] At the overall display level, we used the proximity–compatibility principle by grouping related items together and displaying information relevant to a common task.[31] The displays were organized so that users would see information at a glance in a timely, organized manner. At the display feature level, we leveraged Gestalt principles and laws of visual perception for usability and design, such as common fate, element connectedness, proximity, and similarity.[32] For example, the laws of common fate and element connectedness group together PGHD elements that move in a similar trajectory. These principles influenced the design of the air-quality index and pollen-count elements as aligned line graphs.

We used color to facilitate information processing, as well as trends and patterns to display time-oriented data elements.[30] [33] [34] [35] As an example, the Asthma Control Test (ACT) scores were stratified and sequentially color coded from green (best) to yellow to red (worst) to correspond with the level of control.[36] We deliberately used the existing ACT color scheme in the display to match the clinician's mental model and allow for rapid pattern matching. Consistent with information-visualization literature, we included simple line graphs and bar graphs.[13] [37] [38] Numeric representation was incorporated where needed, but we used text sparingly. A full description of the design features is found in [Table 2].

Table 2 Design of display features

Feature	Feature description	Visualization principle(s)
Display 1
ACT score	Color-coded line graph; part of line or bar chart with symptoms	Aigner et al (2011) Tufte (2001) Tufte et al (1990)
Symptoms	Bar graph with number of symptoms; part of line or bar chart with ACT	Aigner et al (2011) Tufte (2001) Tufte et al (1990)
Air quality and pollen count	Line graphs without axis, numbers only	Gestalt laws Proximity–compatibility principle Tufte (2001)
Overall	One x-axis to align all features temporally	Gestalt laws Aigner et al (2011)
Display 2
Rescue-inhaler doses	Icon as repetitive elements	Gestalt laws
Total medication doses	Colored line graph with x- and y-axes	Gestalt laws Aigner et al (2011) Tufte (2001)
Exposures	Colored line graph with x- and y-axes	Gestalt laws Aigner et al (2011) Tufte (2001)
Symptoms	Colored line graph with x- and y-axes	Gestalt laws Aigner et al (2011) Tufte (2001)
Overall	Colored lines and combined line graph to show relationship between elements	Proximity–compatibility principle Gestalt laws Aigner et al (2011) Tufte (2001)

Abbreviation: ACT, Asthma Control Test.

Participatory Design of Displays

In the early development stages, participatory design methods are well suited for obtaining feedback, exploring user needs, and generating knowledge.[39] [40] Our iterative evolutionary approach used multiple cycles to incrementally adjust the prototype display features guided by user consensus. By including users in the process, we sought to gain an understanding of the preferred features for PGHD display.

Participant Recruitment

We recruited study participants from two academic medical centers with multiple outpatient clinic locations—one in Salt Lake City and the other in New York City. Inclusion criteria were adult clinicians who practiced as a physician or nurse practitioner and made treatment decisions for patients with pediatric asthma in the outpatient setting.

Using purposive and snowball sampling, we emailed invitations to providers who had experience managing the care of patients with pediatric asthma. Although there is debate on the number of participants for usability studies, testing with a sample size of at least five uncovers most usability problems.[41] [42] The Institutional Review Board of the University of Utah approved this study.

Procedures

We elicited feedback on the prototype displays through a series of individual design sessions with semi-structured interviews. We held sessions over the phone that lasted for 15 to 20 minutes. Before each session, we emailed a document containing the two vignettes, with corresponding displays, and provided a link to an online questionnaire. To ensure consistency in our process, one member of the research team (V.L.T.) conducted all sessions and read instructions from a predefined interview guide ([Appendix A]). We digitally recorded all interviews after obtaining verbal permission from the participants.

Utilizing the think-aloud protocol, we asked a series of questions and probes with the goal of a more in-depth exploration of the participant's interpretation of the data. As part of each cycle, we conducted interviews until we achieved target-user response saturation, which we defined as no new information or repeated responses to the interview questions.

After the participants reviewed the prototype display, we conducted formative usability testing which tends to be exploratory, making it well suited for rapid, iterative display design.[43] We asked participants to complete a 10-item questionnaire derived from the IBM Post Study System Usability Questionnaire (PSSUQ) to elicit user satisfaction, usefulness, and intention to use.[44] The PSSUQ instrument was modified to remove items that were not relevant to the prototype display.[45] To supplement the remaining nine PSSUQ items, we added one intention-to-use item with responses on a Likert-type scale (1 = strongly agree to 5 = strongly disagree). We administered the questionnaire for each display in each cycle using Research Electronic Data Capture (REDCap).[46] [47] Once participants reviewed the first display, answered the questions, and completed the questionnaire, they followed the same steps for the second display. This resulted in the completion of one cycle.

Due to the formative and diagnostic nature of this study, we were primarily interested in discovering severe usability problems. After the first cycle, we conducted a content analysis of participant responses to the interview questions and used the recommendations to refine and modify the prototypes for subsequent cycles. We switched the order of the vignette and display presentation for the second cycle. On completion of the second cycle, we assessed the need for a third cycle using the results of the PSSUQ and the content analysis. The criteria to terminate the design cycles after the second round were a mean score of ≤2 (agree) and no major display modifications identified.

Data Analysis

Data analysis consisted of calculating the mean scores for each questionnaire item for each display in each cycle. In addition, we calculated the mean of the nine PSSUQ items. To assess reliability, we calculated the Cronbach's α for the items measuring user satisfaction and usefulness.[48] For each display in each cycle, correlations examined the associations between the total results for the nine PSSUQ items and the intention-to-use item.

We used a professional transcription service to transcribe the audio recordings from the individual interview sessions and all transcripts were stored securely. Two members of the research team (V.L.T. and S.E.W.) used conventional content analysis to independently code the transcripts and derive content from participant responses.[49] [50] Key words were highlighted from the transcript text, coded, and sorted into categories. Regular meetings were held to resolve discrepancies through discussion until consensus was reached.

Results

We conducted two individual design sessions with six participants for a total of 12 interviews. The participants were physicians with pediatric-asthma experience who practiced at an academic medical center (two in Salt Lake City; four in New York City). The formative evaluation of the display relied on two components: the results of the participant survey and the analysis of the individual interviews.

Analysis of Survey Data

The mean scores of each item and the total mean of the nine PSSUQ items for each display in each cycle are available in [Table 3]. In the second cycle, the nine PSSUQ items attained a total mean score that was ≤2 for both displays, indicating good usability. We ended the design cycles after the second cycle because we had met the criteria for termination.

Table 3 Mean participant ratings of prototype displays (1 = strongly agree to 5 = strongly disagree)

Criteria	Display 1		Display 2
	Cycle 1	Cycle 2	Cycle 1	Cycle 2
Q1. Overall, I am satisfied with how easy it is to use this display.	2.2	1.3	3.0	2.2
Q2. I was able to complete the tasks and scenarios quickly using this display.	2.8	1.5	2.6	2.0
Q3. I felt comfortable using this display.	2.0	1.3	2.8	1.8
Q4. It was easy to learn to use this display.	2.4	1.5	2.8	2.0
Q5. It was easy to find the information I needed.	2.2	1.7	2.4	2.0
Q6. The information was effective in helping me complete the tasks and scenarios.	2.0	1.5	2.8	1.7
Q7. The organization of information on the display was clear.	3.3	1.3	3.4	2.0
Q8. This display has all the functions and capabilities I expect it to have.	3.5	1.5	3.0	2.0
Q9. Overall, I am satisfied with this display.	2.5	1.3	3.4	2.0
Grand mean of nine modified PSSUQ Items	2.5	1.4	2.9	2.0
Q10. If this display were made available to me, I would incorporate it into my practice.	2.2	1.3	2.8	1.8

For the modified PSSUQ items measuring user satisfaction and usefulness, the estimated Cronbach's coefficient was α = 0.97, indicating a single construct. For D1, the nine-item mean strongly correlated with the intention-to-use item in the first cycle with a value of 0.96 and in the second cycle with a value of 0.97. For D2, the correlation between the nine-item mean and the intention-to-use item was 0.98 in the first cycle and 0.78 in the second cycle, indicating a strong positive correlation between the constructs.

Analysis of Interview Data

Using the qualitative data, we made iterative changes to each display between cycles. We included all modifications as part of the second cycle, except for suggestions that were unrelated to the vignette content or display features. The displays used for each cycle are shown in [Figs. 2] and [3].

Fig. 2 Display 1. The left display (A) is the initial prototype used in the first cycle: the top portion of the display depicts the monthly score of the Asthma Control Test using connected colored dots and the bar graph depicts the number of exposures for each month. The lower portion of the display depicts line graphs for air quality and pollen count for the same time period as reported by the mHealth apps. The right display (B) was refined after the second cycle: the line graph for the Asthma Control Test scores was separated from the symptom bar graph. The symptom bar graph was converted to a multiple bar graph to differentiate the types of symptoms. Additional colors and legends were added to all graphs.

Fig. 3 Display 2. The left display (A) is the initial prototype used in the first cycle: the line graphs represent the total number of symptoms, doses of controller medication, and exposures to triggers as reported in the mHealth app monthly. The pill bottle icon represents the administration of rescue inhaler doses. The right display (B) was refined after the second cycle: the top portion of the display is a bar graph depicting the number of controller medication doses versus the number of rescue doses each week. The bottom left portion of the display depicts two-line graphs that show the relation of symptoms and triggers reported on a weekly basis. The bottom left of the display contains three pie charts depicting the percentages of day symptoms, night symptoms, and triggers for the 4-week timeframe.

Through content analysis, we identified display preferences for pediatric-asthma PGHD displays and then categorized into two higher-order categories: display features and display content ([Appendix B]). Participants expressed preferences for display features (line graphs, pie charts, and bar graphs) used to depict the PGHD in the displays ([Table 4]). Participants also reported that the use of colored dots for the ACT scores, legends to explain the trendlines, and symbols for medication doses were desirable. Although most participants felt that trendlines were more helpful than numbers, a few of the participants requested that numbers be added to the trendlines. The preferred frequency of data points was monthly, although some participants requested both yearly and daily views. All participants mentioned the benefit of using color to denote abnormality levels and strongly preferred color for this purpose, where possible.

Table 4 Interview excerpts for display features and content

	Display features	Display content
Display 1
	I like that it's a visual graph. Almost like a run chart or a dotted line that's connected so that you can see the shape of the increase or decrease. You don't have to read actual numbers . . . you can see at a glance with the shape if it's getting worse or better. So I like that.	The red dots are going up as air-quality index goes down. That's a very helpful thing to know because I'm not usually familiar with the local air-quality index or pollen count so this is really helpful to see it all in one place.
	It's nice to have the bars. It's nice that the shading on the bar is alternated. . . . So it's easy to see	As air quality improves his ACT score goes up and the symptoms better as pollen count goes down. . . . So this is cool, this is super good.
	The way it's all displayed is nice and crisp and clean and easy to interpret which is nice, which I like . . . trying to use color effectively, and not too distracting.	Looks good and it makes sense . . . just makes you think about all of the things contributing to asthma control.
	I really love the first graph with the ACT . . . also, the color coding is amazing. . . . I really like the color-coding 'cause we don't see it too often, and of course the proper labeling.
Display 2
	I think that trend is good . . . you can use some of the trends to help with the management . . . the yearly trend is good. . . . It's good to have a longer-picture understanding.	Interested in knowing if you are compliant with controller med. . . . I think that's a valuable piece of information to have there.
		It's nice to have the controller and rescue side by side.

Abbreviation: ACT, Asthma Control Test.

Participants expressed specific preferences for types of content, the stratification of the content, and the ability to see the relationships between items in the content. In particular, participants commented positively on the importance of environmental content, and on the advantage of having a large amount of relevant patient data displayed in one place. Participants also requested the addition of other types of PGHD that were not a part of the vignette, such as allergy medications and amount of exercise. Participants indicated the need to understand the underlying data, the method of collection, and the functions of a smart inhaler.

Responses to the final open-ended question provided suggestions for future features regarding the use of PGHD in clinical care. Two participants suggested that it would be helpful to know about comorbidities, such as obesity, and to have the ability to see the body mass index in the display. Participants were interested in seeing activity data, such as the schedule for gym classes or soccer practice, to assist with planning for asthma control. Other participants commented on the benefit of having PGHD instead of relying on recall. However, one participant expressed concern for Medicaid populations and the added expense of smart inhalers or mHealth tools, coupled with health-literacy issues. One participant thought that the user should be able to calculate the level of asthma control for the patient within the display, indicating the desire for embedded decision support.

Discussion

To our knowledge, this is the first study to design and conduct a formative evaluation of PGHD displays for providers using participatory design methods. The identification of clinician preferences for PGHD displays provided insights into how to help clinicians meet their information needs for pediatric asthma and highlighted opportunities for future research.

Similar to previous studies on display design, most of the participants in this study preferred the use of patterns and trendlines for visualizing PGHD.[13] [51] Because providers assess the level of asthma control for pediatric patients during monthly visits, it is understandable that trending views of longitudinal data would be preferred.[52] It may be worthwhile to explore whether providers prefer trendlines for other chronic-disease PGHD, as well. We incorporated suggestions for display improvements that included legends and the use of color to indicate control levels. Although the ACT score has a particular color scheme, not all participants were familiar with the displayed colors as expected; however, participants indicated that the use of red coloring to indicate a control problem and green to indicate good control is standard practice.

Once we made modifications to the display features for the second cycle, and display content became more relevant, participants found it easier to interpret the relationships in the data. This finding indicates an interrelation of display content and display features. Given the importance of data interpretation, a deeper exploration of display content and features within the context of decision making is essential for future research.

In general, participants requested more content and more detail in each display. This was not surprising, given the tradeoff between having all data in one place and the limitation of one display being able to show all possibly relevant data. There might be a role for more features, like filtering and details on demand, in the current design of EHRs to address this tradeoff. However, given that we explored the use of PGHD displays outside of EHR workflows, user preferences may evolve as they adapt. An exploration of display use over time may also influence design features.

Because we did not perform a task analysis as part of this study, it is unknown whether more content and more details are needed to support the vignette decisions. In most cases, the requested data types would not be available from the asthma apps that we explored. But we expected the request for PGHD related to the patient's daily activities, given their importance in chronic disease management. With the promise of health-monitoring sensors, it is reasonable to expect that clinicians will soon have access to these PGHD. Application developers should explore gaps in required PGHD types for potential inclusion in asthma mHealth apps as these technologies mature.

Although most of the participants lacked knowledge on the numeric representation of a positive or negative air-pollen count, there was interest in seeing details related to the air-quality index and pollen counts. This finding demonstrates a growing concern for the association between asthma and outdoor environmental factors, in addition to clinician continuing education. It may be helpful to include geolocation data within pediatric asthma mHealth apps to estimate the amount of time spent in green spaces or near cut grass. As more mobile data capabilities enter the health care market, future work should explore the collection of other types of environmental PGHD to support asthma treatment planning.

The previous use of PGHD was not an inclusion criterion for this study; however, some participants did not have a thorough understanding of the available types of pediatric-asthma PGHD or methods used to collect PGHD. One participant was interested in learning more about smart inhalers, including the mechanisms used for medication compliance. There was also an interest in understanding the expense associated with device acquisition, especially for patients who need them most. Support for providers should include an assessment of readiness to answer questions from patients and families, along with information about available PGHD, the methods for collection, and the medical evidence used to create the app features. An investment in resources, allowing providers to partner with patients on PGHD activities, may encourage increased use of PGHD in the future.

We also found a great interest in visually integrating PGHD with relevant EHR data. Although this study examined preferences for PGHD visualizations in isolation, clinicians likely need more information to make treatment decisions. Many historical data already reside in the EHR or in health records from other specialists, and patients do not always have the ability to share with other providers. As we transition to a more comprehensive flow of EHR data and PGHD given advances to facilitate interoperability, future design work should examine the impact of and preferences for the viewing of PGHD and EHR data integrated on the same display.

Limitations

The participants included in this study represent a sample of physicians from urban academic medical centers and may not be representative of all providers. However, we recruited from two regions of the country and reached saturation of responses. Another limitation is that the preferences reflect the views of only one provider type. Although we invited both physicians and nurse practitioners, only physicians participated.

We used a methodological approach which assumed that all clinicians consider PGHD to be a reliable information source, whether direct from devices or entered by patients. Although we did not assess the level of trust, we believe that participants with interest in PGHD self-selected to participate, which served to support the participatory nature of the study.

The prototype displays were limited to PGHD in isolation, did not include EHR data, and we did not use EHR software. The lack of EHR data may have impacted our findings; however, leaving out EHR data was intentional. We believe that a full understanding of PGHD, a relatively new information source, is required before combining with EHR data.

Recommendations for Future Research

Future iterations of interactive PGHD displays for pediatric asthma should continue to involve participatory design. Interactive capabilities would allow for an examination of visualization principles such as zoom and filter and, subsequently, details on demand. Displays that integrate EHR data with PGHD and that are available within the EHR during clinic visits should also be explored along with a multifaceted usability evaluation.

Researchers should investigate the use of case-based vignettes requiring providers to make decisions using PGHD. These types of studies would allow for an examination of the features and functionalities needed for optimal decision making or improved patient outcomes. We recommend further exploration of preferences for PGHD displays using other chronic diseases, such as congestive heart failure or cancer, to determine the generalizability of results for longitudinal, chronic-care activities. Future work should also include multiple members of the care team, participants of different ages, and those serving diverse patient populations.

Conclusion

Providers treating patients with pediatric asthma expressed preferences for the features and content used in PGHD displays such as color, trendlines, and environmental data. Although providers felt the visualizations served as a useful summary, they also expressed a need for greater detail, additional data sources, and visual integration with relevant EHR data. Therefore, future research should examine interactive PGHD displays integrated into EHRs and evaluated within the context of clinical workflows to promote team-based care and shared decision making using PGHD.

Clinical Relevance Statement

Participatory design approaches are beneficial in the design of data displays. The visual synthesis of multiple PGHD elements facilitates the interpretation of the PGHD. Clinicians likely need more information to make treatment decisions when PGHD displays are introduced into practice.

Multiple Choice Questions

1. When integrating PGHD into clinical workflows, what is an important first step?

   • Recruiting providers.

   • Designing displays.

   • Creating vignettes.

   • Assessing information needs.

   Correct Answer: The correct answer is option d. Assessing information needs. Prior to introducing new data into clinical workflows, it is important to assess information needs and how the data will contribute to clinical decision making. Although creating vignettes and designing displays are steps, they are not the first steps. Recruiting providers is only important if there is research being conducted.

2. What are some provider preferences for display features using PGHD?

   • Inhaler use, medications, and environmental data.

   • Line graphs, pie charts, and bar graphs.

   • Overview, zoom, and details on demand.

   • Usability, usefulness, and intent to use.

   Correct Answer: The correct answer is option b. line graphs, pie charts, and bar graphs as noted in the Results section. Inhaler use, medications, and environmental data correspond with display content, not features. Overview, zoom, and details on demand refer to visualization principles; and usability, usefulness, and intent to use were constructs measured in the questionnaire.

Appendix A Interview guide

Cycle	Question	Prompts
1	Is this what you expected to see?	Why? Why not?
	Does this display provide the information needed?	Why? Why not?
	What features are most helpful?	Why? How?
	Is there anything else you would prefer to see?	What? How?
2	Are these the changes you expected to see?	Why? Why not?
	Are there additional changes or edits that would be helpful?	What? How?
	Anything else you would like to add regarding PGHD and pediatric asthma?	What else?

Abbreviation: PGHD, patient-generated health data.

Appendix B Display modification requests

PGHD type	Cycle 1 modification request	Cycle 2 modification request
Display 1
ACT score	Add legend, axes, and color to the legend
Symptoms	Add axes, label axes, add types of symptoms	Vary the shades of blue, add numbers to bars, add lines for day versus night
Air quality and pollen count	Add legend, line up with other data using same time interval	Add color to trendline numbers, change black font for legend
Medication	Include information on the last dose of steroids
Display 2
Medication	Add dosage and adherence, differentiate the bars on the graph, separate symptoms and triggers from other data
Timing	Change all displays from daily to weekly
Exposures	Add types, change name to triggers	Change exercise to a trigger
Symptoms	Add types, differentiate day versus night versus composite	Label y-axis for symptoms and triggers
Overall	Parse out all three data types

Abbreviation: ACT, Asthma Control Test.

Conflict of Interest

None declared.

Acknowledgements

V.L.T. is supported by the Jonas Nurse Leaders Scholar Program (Jonas Philanthropies). Content is solely the responsibility of the authors and does not necessarily represent the official views of Jonas Philanthropies. This work was supported in part by a Western Institute of Nursing/Council for the Advancement of Nursing Science Dissertation Award.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed by University of Utah Institutional Review Board.

• References

• 1 Serebrisky D, Wiznia A. Pediatric asthma: a global epidemic. Ann Glob Health 2019; 85 (01) 6
• 2 Backman H, Räisänen P, Hedman L. et al. Increased prevalence of allergic asthma from 1996 to 2006 and further to 2016-results from three population surveys. Clin Exp Allergy 2017; 47 (11) 1426-1435
• 3 Fuhlbrigge AL. Challenges in childhood asthma mortality persist despite advances in care. J Allergy Clin Immunol Pract 2018; 6 (03) 1037-1038
• 4 Anderson K, Burford O, Emmerton L. App chronic disease checklist: protocol to evaluate mobile apps for chronic disease self-management. JMIR Res Protoc 2016; 5 (04) e204-e204
• 5 Chen C, Haddad D, Selsky J. et al. Making sense of mobile health data: an open architecture to improve individual- and population-level health. J Med Internet Res 2012; 14 (04) e112
• 6 Office of the National Coordinator for Health IT. Conceptualizing a data infrastructure for the capture, use, and sharing of patient-generated health data in care delivery and research through 2024. Accessed February 1, 2020 at: https://www.healthit.gov/sites/default/files/onc_pghd_final_white_paper.pdf
• 7 Fox S, Rainie L. The Web at 25 in the US Pew Research Internet Project. Accessed February 1, 2020 at: https://www.pewresearch.org/wp-content/uploads/sites/9/2014/03/PIP_Report_Future_of_the_Internet_Predictions_031114.pdf
• 8 Papadopoulos NG, Čustović A, Cabana MD. et al. Pediatric asthma: an unmet need for more effective, focused treatments. Pediatr Allergy Immunol 2019; 30 (01) 7-16
• 9 Pinnock H, Slack R, Pagliari C, Price D, Sheikh A. Understanding the potential role of mobile phone-based monitoring on asthma self-management: qualitative study. Clin Exp Allergy 2007; 37 (05) 794-802
• 10 Genes N, Violante S, Cetrangol C, Rogers L, Schadt EE, Chan YY. From smartphone to EHR: a case report on integrating patient-generated health data. NPJ Digit Med 2018; 1 (01) 23
• 11 Saripalle RK. Leveraging FHIR to integrate activity data with electronic health record. Health Technol (Berl) 2020; 10 (01) 341-352
• 12 Abdolkhani R, Gray K, Borda A, DeSouza R. Patient-generated health data management and quality challenges in remote patient monitoring. JAMIA Open 2019; 2 (04) 471-478
• 13 Wright MC, Borbolla D, Waller RG. et al. Critical care information display approaches and design frameworks: a systematic review and meta-analysis. J Biomed Inform X 2019; 3: 100041
• 14 Adler-Milstein J, Nong P. Early experiences with patient generated health data: health system and patient perspectives. J Am Med Inform Assoc 2019; 26 (10) 952-959
• 15 Demiris G, Iribarren SJ, Sward K, Lee S, Yang R. Patient generated health data use in clinical practice: a systematic review. Nurs Outlook 2019; 67 (04) 311-330
• 16 Reading MJ, Merrill JA. Converging and diverging needs between patients and providers who are collecting and using patient-generated health data: an integrative review. J Am Med Inform Assoc 2018; 25 (06) 759-771
• 17 Nittas V, Lun P, Ehrler F, Puhan MA, Mütsch M. Electronic patient-generated health data to facilitate disease prevention and health promotion: scoping review. J Med Internet Res 2019; 21 (10) e13320
• 18 Tiase VL, Hull W, McFarland MM. et al. Patient-generated health data and electronic health record integration: a scoping review. JAMIA Open 2020; 3 (04) 619-627
• 19 Evans SC, Roberts MC, Keeley JW. et al. Vignette methodologies for studying clinicians' decision-making: validity, utility, and application in ICD-11 field studies. Int J Clin Health Psychol 2015; 15 (02) 160-170
• 20 Kim J. Scenarios in information seeking and information retrieval research: a methodological application and discussion. Libr Inf Sci Res 2012; 34 (04) 300-307
• 21 Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA 2000; 283 (13) 1715-1722
• 22 Tiase VL, Sward KA, Del Fiol G, Staes C, Weir C, Cummins MR. Patient-generated health data in pediatric asthma: exploratory study of providers' information needs. JMIR Pediatr Parent 2021; 4 (01) e25413
• 23 Gordon IE. Theories of Visual Perception. Vol. 3. New York, NY: Psychology Press; 2004
• 24 Wagemans J, Elder JH, Kubovy M. et al. A century of Gestalt psychology in visual perception: I. perceptual grouping and figure-ground organization. Psychol Bull 2012; 138 (06) 1172-1217
• 25 Ratwani R. Electronic health records and improved patient care: opportunities for applied psychology. Curr Dir Psychol Sci 2017; 26 (04) 359-365
• 26 Bian J, Weir C, Unni P. et al. Interactive visual displays for interpreting the results of clinical trials: formative evaluation with case vignettes. J Med Internet Res 2018; 20 (06) e10507
• 27 Nelson SD, Del Fiol G, Hanseler H, Crouch BI, Cummins MR. Software prototyping: a case report of refining user requirements for a health information exchange dashboard. Appl Clin Inform 2016; 7 (01) 22-32
• 28 Rudin RS, Fanta CH, Predmore Z. et al. Core components for a clinically integrated mHealth app for asthma symptom monitoring. Appl Clin Inform 2017; 8 (04) 1031-1043
• 29 Shneiderman B. The eyes have it: a task by data type taxonomy for information visualizations. Paper presented at: Proceedings 1996 IEEE Symposium on Visual Languages, Boulder, Colorado, United States. ; September 3–6, 1996: 336-343
• 30 Tufte ER. The visual display of quantitative information. Vol. 2. Cheshire, CT: Graphics Press; 2001
• 31 Wickens CD, Andre AD. Proximity compatibility and information display: effects of color, space, and object display on information integration. Hum Factors 1990; 32 (01) 61-77
• 32 International Design Foundation. Gestalt principles. Accessed February 1, 2020 at: https://www.interaction-design.org/literature/topics/gestalt-principles
• 33 Aigner W, Miksch S, Schumann H. et al. Visualization of Time-Oriented Data. London: Springer Science & Business Media; 2011
• 34 Wang TD, Wongsuphasawat K, Plaisant C, Shneiderman B. Extracting insights from electronic health records: case studies, a visual analytics process model, and design recommendations. J Med Syst 2011; 35 (05) 1135-1152
• 35 Tufte ER, Goeler NH, Benson R. Envisioning information. Vol. 126. Cheshire, CT: Graphics Press; 1990
• 36 Nathan RA, Sorkness CA, Kosinski M. et al. Development of the asthma control test: a survey for assessing asthma control. J Allergy Clin Immunol 2004; 113 (01) 59-65
• 37 Aigner W, Miksch S, Müller W. et al. Visualizing time-oriented data—a systematic view. Comput Graph 2007; 31 (03) 401-409
• 38 Lor M, Koleck TA, Bakken S. Information visualizations of symptom information for patients and providers: a systematic review. J Am Med Inform Assoc 2019; 26 (02) 162-171
• 39 Peters D, Davis S, Calvo RA, Sawyer SM, Smith L, Foster JM. Young people's preferences for an asthma self-management app highlight psychological needs: a participatory study. J Med Internet Res 2017; 19 (04) e113
• 40 Spinuzzi C. The methodology of participatory design. Tech Commun (Washington) 2005; 52 (02) 163-174
• 41 Six JM, Macefield R. How to determine the right number of participants for usability studies. UXmatters. Accessed February 1, 2020 at: https://www.uxmatters.com/mt/archives/2016/01/how-to-determine-the-right-number-of-participants-for-usability-studies.php
• 42 Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. Paper presented at: Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, New York, United States. , April 24–29, 1993: 206-213
• 43 Lazar J, Feng JH, Hochheiser H. Research methods in human-computer interaction. Burlington, MA: Morgan Kaufmann; 2017
• 44 Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995; 7 (01) 57-78
• 45 Wang EH, Zhou L, Chen SK, Hill K, Parmanto B. An mHealth platform for supporting clinical data integration into augmentative and alternative communication service delivery: User-centered design and usability evaluation. JMIR Rehabil Assist Technol 2018; 5 (02) e14
• 46 Harris PA, Taylor R, Minor BL. et al; REDCap Consortium. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208
• 47 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
• 48 Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ 2011; 2: 53-55
• 49 Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005; 15 (09) 1277-1288
• 50 Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: SAGE; 1994
• 51 Hochstenbach LMJ, Courtens AM, Zwakhalen SMG, Vermeulen J, van Kleef M, de Witte LP. Co-creative development of an eHealth nursing intervention: self-management support for outpatients with cancer pain. Appl Nurs Res 2017; 36: 1-8
• 52 Voorend-van Bergen S, Vaessen-Verberne AA, Brackel HJ. et al. Monitoring strategies in children with asthma: a randomised controlled trial. Thorax 2015; 70 (06) 543-550

Publikationsverlauf

Eingereicht: 17. April 2021

Angenommen: 13. Juni 2021

Artikel online veröffentlicht:
21. Juli 2021