A Proposal for a Robust Validated Weighted General Data Protection Regulation-Based Scale to Assess the Quality of Privacy Policies of Mobile Health Applications: An eDelphi Study

Background  Health care services are undergoing a digital transformation in which the Participatory Health Informatics field has a key role. Within this field, studies aimed to assess the quality of digital tools, including mHealth apps, are conducted. Privacy is one dimension of the quality of an mHealth app. Privacy consists of several components, including organizational, technical, and legal safeguards. Within legal safeguards, giving transparent information to the users on how their data are handled is crucial. This information is usually disclosed to users through the privacy policy document. Assessing the quality of a privacy policy is a complex task and several scales supporting this process have been proposed in the literature. However, these scales are heterogeneous and even not very objective. In our previous study, we proposed a checklist of items guiding the assessment of the quality of an mHealth app privacy policy, based on the General Data Protection Regulation. Objective  To refine the robustness of our General Data Protection Regulation-based privacy scale to assess the quality of an mHealth app privacy policy, to identify new items, and to assign weights for every item in the scale. Methods  A two-round modified eDelphi study was conducted involving a privacy expert panel. Results  After the Delphi process, all the items in the scale were considered “important” or “very important” (4 and 5 in a 5-point Likert scale, respectively) by most of the experts. One of the original items was suggested to be reworded, while eight tentative items were suggested. Only two of them were finally added after Round 2. Eleven of the 16 items in the scale were considered “very important” (weight of 1), while the other 5 were considered “important” (weight of 0.5). Conclusion  The Benjumea privacy scale is a new robust tool to assess the quality of an mHealth app privacy policy, providing a deeper and complementary analysis to other scales. Also, this robust scale provides a guideline for the development of high-quality privacy policies of mHealth apps.

The purpose of the study should be clearly defined and demonstrate the appropriateness of the use of the Delphi technique as a method to achieve the research aim.A rationale for the choice of the Delphi technique as the most suitable method needs to be provided The purpose is reported in Abstract and the section Introduction, pages 1 and 2. Appropriateness and rationale could be found in the section Study Design, pages 3 and 4.

Expert panel 9
Criteria for the selection of experts and transparent information on recruitment of the expert panel, socio-demographic details including information on expertise regarding the topic in question, (non)response and response rates over the ongoing iterations should be reported Panel expert information is reported in the section Selection Criteria and Recruitment, page 4. Socio-demographic details and response rates are reported in the section Expert Panel, page 5.

Description of methods 10
The methods employed need to be comprehensible; this includes information on preparatory steps, piloting of material and survey instruments, design of the survey instrument(s), the number and design of survey rounds, methods of data analysis, processing and synthesis of experts' responses to inform the subsequent survey round, and methodological decisions taken by the research team throughout the process Information about methods is reported on the section Round 1, pages 4-5, and the section Round 2, page 5.

Procedure 11
Flow chart to illustrate the stages of the Delphi process, including a preparatory phase, the actual "Delphi rounds," interim steps of data processing and analysis, and concluding steps Flow chart is reported on the section Round 1, page 6.

Discussion of limitations 14
Reporting should include a critical reflection of potential limitations and their impact of the resulting guidance Limitations are reported in the section Limitations, pages 8-9.

Adequacy of conclusions 15
The conclusions should adequately reflect the outcomes of the Delphi study with a view to the scope and applicability of the resulting practice guidance

Supplementary Appendix B
Round 1 Questionnaire (Translated from Spanish) Categorization of the relevance of components of privacy policies.Thank you very much for participating in this study, this page shows basic information about the project.However, you can access the Participant Information Sheet at https://uses0-my.sharepoint.com/:b:/g/personal/jaimebm_us_es/EbIFkEg28eZAvXQJFyLaspsB6eeNeKifGSGPZQHIubIH7A?e=qwEt3T.
If you want to see the content of the survey before continuing, you can see it at https://uses0-my.sharepoint.com/:b:/g/personal/jaimebm_us_es/EdJL1oxgG65Lup3whfXif_8BMRiVWLfyfy4h-FLCXW6V9w?e=TClpYm.
Your participation in this study consists of two phases: • In the first one (this questionnaire) you must fill in a questionnaire, in which your opinion will be asked about the importance of the presence of certain items in the privacy policy documents in mobile health applications.These items are indicated in the Article 13 of the General Data Protection Regulation (GDPR).Additionally, you will be asked to point out, if you wish, any other item that, in your opinion, should be used to assess privacy policies.• In the second round (an email containing a link will be sent to you in the coming weeks), you will be shown aggregated statistical data from the answers of other participants in the study, together with a comparison with your previous round answers.You will be asked again to rate the importance of these items together with others that could be identified in the previous round.
Remember that your participation in this study is voluntary and, by sending this form, you give your consent for your personal data to be processed, in accordance with the information clause, available at https://sic.us.es/sites/default/files /pd/cievaluacionpolprivacidad.pdf.
If you need more information, you may contact Alejandro Carrasco Muñoz (acarrasco@us.es) Contact and demographic data Enter your personal data below (all fields are required) Surname: Name: Position: Institution: Email address (it will be used throughout the study): By checking the following box, you agree to participate in the project and give your consent for your data to be processed in accordance with our privacy policy: [ ]

Assessing the Importance of Certain Items in Privacy Policies
Point out the relative importance, in your opinion, of the presence of certain information (items) in the privacy policies of mobile health applications.When answering this questionnaire, keep in mind that, beyond strict compliance with the GDPR (and, specifically, article 13), you must give your opinion on the importance of these items.
Value the importance that the following information appears in the privacy policies of mobile health applications:

Item identifier
Brief A Scale to Assess the Quality of Privacy Policies Benjumea et al.
Is there any other item that you think should appear in the privacy policy documents in mobile health applications?If so, use the space below to describe it, as well as a brief detail of the reasons why you are making your proposal.
Round 2 Questionnaire (Translated from Spanish) Categorization of the relevance of components of privacy policies (Round 2).
Thank you very much for participating in the second round of this study.Remember you can access the Participant Information Sheet at https://uses0-my.sharepoint.com/:b:/g/personal/jaimebm_us_es/EbIFkEg28eZAvXQJFyLaspsB6eeNe-KifGSGPZQHIubIH7A?e=qwEt3T.
Your participation in this study consists of two phases: • In the first one (already completed) you filled in a questionnaire, in which we asked your opinion about the importance of the presence of certain items in the privacy policy documents in mobile health applications.These items are indicated in the Article 13 of the General Data Protection Regulation (GDPR).Additionally, you were asked to point out, if you wished, any other item that, in your opinion, should be used to assess privacy policies.• In the second round (this one), we have sent you an email with aggregated statistical data from the answers of other participants in the study, together with a comparison with your previous round answers.You are now asked to rate again the importance of these items together with others that have been identified in the previous round.
Remember that your participation in this study is voluntary and, by sending this form, you give your consent for your personal data to be processed, in accordance with the information clause, available at https://sic.us.es/sites/default/files /pd/cievaluacionpolprivacidad.pdf.
If you need more information, you may contact Alejandro Carrasco Muñoz (acarrasco@us.es).
Email address (use the same email you used in round 1): Value the importance that the following information appears in the privacy policies of mobile health applications: (Continued) A Scale to Assess the Quality of Privacy Policies Benjumea et al.
Regarding the purposes for the processing (item I4), what characteristics of the purposes for the processing should be included?(One or more options may be selected) [ ] General description of the purposes for the processing.[ ] Specific description of the purposes for the processing.[ ] Potential benefits to the user and to the data controller.A Scale to Assess the Quality of Privacy Policies Benjumea et al.
Scale to Assess the Quality of Privacy Policies Benjumea et al.