In April this year, the Indian Council of Medical Research (ICMR) came out with a series of guidelines to address ethical challenges present in the use of Artificial Intelligence (AI) in biomedical research and healthcare. ICMR’s guidelines apply to technology developers, healthcare professionals, and research institutes who want to utilize health data for biomedical research and healthcare delivery using AI technology. They are intended to be used by experts and ethics committees reviewing research proposals for AI technology for the two sectors.
ICMR Artificial Intelligence Guidelines: A summary of the key ideas
Why it matters:
AI heavily depends on data obtained from people, which leaves room for potential biases and confidentiality issues. It can also be potentially dangerous for the well-being of a patient. For instance, when the digital health startup Nabla used a cloud-hosted instance of GPT-3 in 2020, it told a mock patient to kill themselves, according to a report by The Register. Thus, to address these ethical issues, there is a need for AI-specific guidelines, especially in the healthcare space. These guidelines could also help take accountability decisions in case any errors emerge during the development and deployment of AI technology.
STAY ON TOP OF TECH POLICY: Our daily newsletter with top stories from MediaNama and around the world, delivered to your inbox before 9 AM. Click here to sign up today!
What do the ethical guidelines entail:
- Autonomy: Humans should have complete control over the AI technology used in medical decision-making. In case the clinical decisions of AI and the physician differ, patients should be given both options. In research projects deploying AI, developers must ask patients for informed consent before participation.
- Safety: All stakeholders bear responsibility for the safety of the patients; their dignity, rights, and safety must be the highest priority. There must be control mechanisms in place to prevent deliberate or unintended misuse of AI technology. The data collected by AI should be anonymized and offline/ delinked from global technology.
- Inclusivity: Before deploying AI technologies, the possibilities of biases must be considered, identified, and thoroughly checked. The performance of the algorithm must be evaluated in different races, ethnic groups, age groups, and social classes. It must be ensured that vulnerable populations and marginalized groups are included as a part of training data sets to avoid distorted or poor performance.
- Risk minimization: If any risk of harm from the technology is identified then the researcher/manufacturer should inform the concerned ethics committee or monitoring board. At every stage of the study, AI technologies must constantly be checked for unintended harm and the patients must be informed of said harm. Patients must have the freedom to withdraw or continue treatment.
- Trustworthiness: AI technology must adhere to all applicable laws and regulations. Developers must ensure predictability in the results and outcomes from the use of their technology. The results should be explainable and based on scientific plausibility. Details about the development and deployment of the AI tools must be readily available to all stakeholders. Information about the technology must be published for public consultation.
- Accountability and liability: The legal responsibility of the AI’s usage needs to be properly defined before it is deployed. Responsibility for harm caused by technology depends on the nature of the harm. If the harm is caused by flaws in functionality, the designer/developer/manufacturer may be held liable.
- Accessibility: AI developers and other concerned authorities must make sure that different user groups have equal access to the technology. Special consideration must be given to those who lack the infrastructure to access such technology. The AI may include local languages in its user interface to improve accessibility.
- Collaboration: Motivation for collaborations and potential conflicts of interest should be examined deeply to avoid harm to the stakeholders. All collaborations concerning data collection and sharing of biological samples, and intellectual property must be submitted to the Health Ministry’s Screening Committee (HMSC) for approval. For international collaborations, Indian laws and guidelines (DISHA & PDP guidelines) are to be adhered to.
- Fairness principles: In case of unfortunate incidents, grievance redressal mechanisms need to be available for the victims. There must also be a safe way for people to raise concerns about its technical, functional, or ethical misuse. The whistleblower’s identity should be protected.
Guiding principles for each stage of development and deployment:
- Development: Hospitals and institutes must know how the data collected during the development phase will be used. This data shouldn’t inflict harm or discrimination on the patients. There should be a feedback mechanism through which users/physicians can provide developers with their input. The provision of removing/modifying the data from the databases also must be ensured in case a patient chooses to opt out at some point.
- Validation: This stage verifies whether the technology developed is a valid solution for a clinical condition. To do so, there needs to be a multi-dimensional multi-sectoral team made up of experts from data science, public health, and engineering to name a few. The validation should include user experience results and should also include details of potential risks to healthcare providers and recipients.
- Deployment: Health professionals should have a fair idea of how the AI technology works and its limitations before deployment. Information on cost-effectiveness and operational costs should be mentioned when available to improve adoption of the AI technology. The patient or their legal representative must be informed of the extent to which AI has been used during biomedical research or clinical decision-making.
You can read the full list of guidelines here.
This post is released under a CC-BY-SA 4.0 license. Please feel free to republish on your site, with attribution and a link. Adaptation and rewriting, though allowed, should be true to the original.
Also read:
