Questionnaire for an App

Version: v0.6

Note: For Capitalized Terms consult our Glossary.

Status of the assay

  • What is the status of this assay?

  • This assay is current as of which date?

About the App

Name and description of the App

  • Name of the App

  • Short description of the App

  • Icon of the App

  • Languages supported by the App

  • URL of the App on iTunes

  • URL of the App on Google Play

  • Website of the App in English

  • Website of the App in the local language (if applicable)

  • What’s the region and purpose of the app (for one-line overview)

About the App Creators

About App Users

  • Who are the intended users for the App? Where are they located?

  • Are there technical, geographical, legal or other limits to who can use the App?

  • How many App Users are currently using the App?

Goals and benefits of the App

  • What are the stated goals of the App?

  • Are there any others goals, not stated by the App Creators, that they are known to also accomplish with this App, or that they could also accomplish with this App in the future?

  • Are there any other goals that others (not App Creators) could also accomplish because this App exists, or is used by certain Users?

  • Are there notable side effects in the use of this App?

Societal context of using the App

  • Is usage of the App required under some circumstances? If so, by whom? What are the consequences of not using it?

  • Are there non-trivial incentives (e.g. financial, access) for using the App? Are there social pressures to use the App?

  • Is a minimum penetration of App usage required in some population before the App can start to be effective?

  • Can the App User deactivate and delete the App?

  • Are there social pressures on the App User resulting from the use of the App, or from information shown by the App? (E.g. if the App indicates that the App User has likely been infected.)

  • Are there social pressures on anybody resulting from information shown by the App run by another App User? (e.g. pressures on an App User’s family or friends if the App identifies the App User as likely infected)

  • Is the App available in all languages and localizations most appropriate for the intended App User population?

App features

  • What are the main features of the App? How are these features intended to help with fighting the disease?

Technology

  • What Architecture does the App use?

  • Is source code of the App available?

  • If the source code is available, under which license is it available?

  • If source code is available, where can it be found?

  • Is the App accessible?

  • What kind of Cloud Component does the App use?

  • What approach does the App take to contact tracing? (If it does)

  • What are the key non-standard communication protocols the App uses? Explain. (These are highly dependent on the App’s features.)

  • Is the App based on anonymous, pseudonymous, or fully identified App Users?

  • Can identities of App Users be tied to, or can they be correlated to specific individuals, and if so, by whom?

  • What technical approaches (e.g. cryptography) does the App use to protect all aspects of the App (e.g. confidential information, operational integrity) from Attackers?

  • What data does the App handle? Where in the Architecture is which data stored or processed? Is all data handled by the App strictly required for the stated goals?

  • Is data not required any more for the stated goals promptly deleted?

  • Is the App a standalone system (“stovepipe”) or is it intended to be used in Federation with other Apps created by others? If so, what are the supported Federation technologies (e.g. protocols/standards), operations and governance?

  • Is there an audit trail of what happens in the App? Can it be accessed by the App User or entities on their behalf?

  • Are the user-facing components of the App built in a way that minimizes potential user mistakes that could be detrimental towards effectiveness or avoidance of risks and harms for themselves and others?

  • Describe the principle of operation

  • How do new App Users discover, and obtain access to the App?

  • How is user support handled?

  • How is the user experience, user understanding, and technical performance of the App being monitored in the field?

  • Can App Users request a copy of the data that has been retained about them? Is the process simple and quick? Is the obtained data easy to understand, verify and use?

  • Can previous App Users request a permanent deletion of the data collected about them? Is the process simple and quick?

  • Can App Users request a correction of data about them? Is the process simple and quick?

  • Can parents or guardians act on behalf of their children in all aspects of the App?

  • Is there an effective complaint process by which App Users can raise issues with the App, or issues with the impact of the use of the App has on them? (not bugs, not technical issues; that is handled in the support question)

  • Are App Users being educated about what it means to use the App, and give their informed consent prior to using the App?

  • If the App performs several distinct functions, can the App User opt-in to some and opt-out of others?

Identity and privacy

  • How are new App Users onboarded on the App? What information do they need to provide to be able to use the App?

  • How long is collected data retained, and where?

  • Are any Backups being made whose retention is longer than the declared Data Retention Period? How is it guaranteed that Backups are deleted on time?

  • Has a Privacy Impact Assessment been performed, and if so, where can it be obtained? Which recommendations have been implemented, and which not? If no such assessment has been performed, why not?

  • Is the App compliant with local regulations on privacy, in particular on privacy of health-related information?

  • Is the App consistent with global best practices on privacy, in particular on privacy of health-related information?

  • What assurances exist that the App will be shut down promptly when appropriate (e.g. when the pandemic has passed, or better approaches for combating the disease have been found)?

  • Is any data collected by the App transmitted beyond the App? If so:

    • Who is the receiver of the data?
    • What is the data that is being transmitted?
    • What are the terms under which the data is transmitted, and what are the safeguards that guarantee the terms are not being violated?
    • Can the transmitted data be correlated by the received with other data they may have or may be able to obtain?

  • Is any data imported by the App from other sources? If so:

    • What data is being imported, and from which sources?
    • How does that increase the effectiveness of the App?
    • Does it potentially increase risks or downsides of the App, and if so, how?

  • Is there a Privacy Policy, and if so, what type of privacy policy is it?

  • If there is a privacy policy, where can it be found, and how loose/airtight is it?

Security

  • How is Data At Rest being secured? Discuss all locations at which Data At Rest exists.

  • How is Data In Motion being secured? Discuss all transfers between locations at which Data At Rest exists.

  • If an App User’s unlocked mobile phone is stolen, what is the maximum impact of the breach on the App User, other App Users, third parties including the App system itself, and effectiveness against the disease?

  • How are the operations of the App monitored with respect to attempted, or successful, Attacks?

  • What operational approaches does the App use to protect all aspects of the App (e.g. requiring two-factor authentication, approval of commits by a second person) from Attackers?

  • What are the operational procedures for access to highly privileged credentials (e.g. server or encryption root keys)?

  • Can App Users verify their build of the App User (e.g. using technologies such as Reproducible Builds)?

  • Has a process been defined for reporting and responding to a security breach? If not, why not?

  • Which entities are required to be trusted by App User to not cause or prevent adverse effects against them?

  • Have there been any reports on attempted or successful attacks on the integrity of any aspect of the App? Do the App Creators report on such attempted or successful Attacks? If not, why not?

  • Are there any reports on attempted or successful correlation of any of the data handled by the App with data from outside the App, or any attempted or successful Re-identification of anonymized or pseudonomized data?

  • Are there any reports on attempted or successful Data Poisoning of some of the data handled by the App?

  • Has an independent security review been performed, and if so, where can it be obtained? Which recommendations have been implemented, and which not? If no such independent review has been performed, why not?

  • What are the procedures and requirements for eligibility of any App Creator employees or contractors to participate in any aspect of the development, operations or governance of the App?

Governance

  • How are decisions made about technology and operations of the App?

  • How are decisions made about governance of the App?

  • Is there a public roadmap for the App, and if so, where can it be found?

  • Is there a whistleblower process for people involved in any aspect of the development, operation, or governance of the App? If not, why not?

  • Should assertions by App Creators prove to be false, or their behavior to be negligent, what are the remedies available to App Users?

  • Is the entire process of App development and operations publicly documented?

Effectiveness

  • List information about the effectiveness of the App against the disease.

  • How is the performance of the App with respect to effectiveness against the disease being monitored?

Validation

  • Which third parties have researched the effectiveness of this App against the disease? Are their reports publicly available, and if so, where?

  • Which third parties have researched the potential downsides or risks of this App? Are their reports publicly available, and if so, where?

  • Has any third-party audit been performed of the App? Who performed the audit, are their reports publicly available, and if so, where?

  • Are any major discrepancies known between self-assertions by the App Creators and Inference or Audits by third parties?

  • Are all relevant technologies, processes, governance and their internal and public documentation periodically and timely updated?

Information sources for this research

Disclaimer and other notes on the assay

  • Disclaimer and open issues that do not fit into any of the other questions.

  • Any other notes that may be of interest

Overall rating

  • Ratings by self, third parties and any audit for the effectiveness of the App

  • Explanatory comments for the rating of the effectiveness of the App

  • Ratings by self, third parties and any audits for the avoidance of potential risks and downsides of the App

  • Explanatory comments for the rating of the avoidance of potential risks and downsides of the App

  • Issue a recommendation to App Users

  • Recommendations to the App Creators