[ad_1]
Artificial intelligence (AI) and algorithmic decision-making systems—algorithms that analyze massive amounts of data and predict the future—are increasingly part of Americans’ daily lives. People are forced to include buzzwords In their education to pass AI-driven recruitment software. Algorithms are deciding who gets housing or financial loan opportunities. and partisan Test software It’s forcing students of color and students with disabilities to struggle with the fear of being locked out of exams or being flagged for cheating. But there is another frontier of AI and algorithms that worries us most: the use of these systems in medicine and medicine.
The use of AI and algorithmic decision-making systems in medicine is increasing, although the current regulation may not be sufficient to detect harmful racial biases in these tools. Details of the tools’ development are largely unknown to clinicians and the public—a lack of transparency that risks automating and exacerbating racism in the health care system. Last week The FDA has issued guidance. Expanding the tools that he plans to control. This expansion guide emphasizes that more needs to be done to combat discrimination and promote equity.
In 2019, A Bomb research It found that many hospitals used clinical algorithms to determine which patients needed care and showed racial disparities — black patients had to be considered sicker than white patients to be referred for the same care. That’s because the algorithms are trained on past data on health care spending, which reflects a history of black patients spending less on their health compared to white patients, due to long-standing disparities in wealth and income. While this algorithm’s bias was eventually discovered and corrected, the incident raises the question of how many other clinical and medical devices may be similarly biased.
Another algorithm, created to determine how many hours of assistance Arkansas residents with disabilities receive each week, was later criticized. Overcutting Home care. Some residents said the sudden outage caused major disruptions to their lives and even required them to be hospitalized. As a result, the lawsuit alleges that numerous errors in the algorithm — mistakes in how it identifies the medical needs of certain people with disabilities — are directly responsible for the wrongful injuries. Despite this outcry, the team that developed the flawed algorithm still creates devices that are used in nearly half of US states and healthcare facilities worldwide.
A recent study An AI tool trained on medical images such as X-rays and CT scans has unexpectedly learned to identify patients’ self-reported race. He has learned to do this even when training with the sole goal of helping clinicians analyze patient images. This technology’s ability to tell a patient’s race — even when their doctor can’t — could be under attack in the future or lead to worse care in communities of color without detection or intervention.
Some algorithms used in the clinical setting are highly regulated in the US by the US Department of Health and Human Services (HHS depressors to pacemakers and now, medical AI systems. Some of these medical devices (including AI) and While devices that assist physicians in treatment and diagnosis are regulated, other algorithmic decision-making tools used in clinical, administrative, and public health settings — such as those that predict risk of death, readmission, and home care needs — are not required to be reviewed or regulated by the FDA or any other regulatory body.
This lack of oversight can lead to biased algorithms being widely used by hospitals and government public health systems, contributing to increased discrimination against black and brown patients, people with disabilities, and other marginalized communities. In some cases, this lack of control can lead to wasted money and lost lives. One such AI tool, developed for early detection of sepsis, is used by more than 170 hospitals and health systems. But a A recent study It found that the tool failed to predict life-threatening disease in 67 percent of those who developed the disease, and created false sepsis alarms in thousands of patients who did not. Recognizing that this failure was the result of under-regulation, the FDA’s new guidelines designate these devices as examples of products that are now regulated as medical devices.
The FDA’s approach to drug oversight, which includes publicly shared data, contrasts with its approach to monitoring medical AI and algorithmic devices through review panels for adverse effects and events. Regulating medical AI presents a new issue and requires different considerations than the hardware devices that the FDA uses to regulate. These tools include Pulse oximeters, Temperature thermometersAnd Scalp electrodes-How well each of these functions in subgroups has been found to reflect racial or ethnic bias. News of these biases only underscores how important it is to properly regulate these tools and ensure they do not discriminate against vulnerable racial and ethnic groups.
While the FDA recommends that device manufacturers test their devices for racial and ethnic discrimination before offering them to the general public, this step is not required. Perhaps more important than reviews after a device is built is transparency during development. A STAT+ news survey Many AI devices that have been approved or cleared by the FDA have been found to not include data diversity information on which the AI was trained, and many of these devices are being cleared. increasing rapidly. other Research AI devices were found to be “underserved patients who were not consistently and selectively diagnosed,” and the rate of underdiagnosis was high.
Unaffordable medical care for marginalized communities. This is unacceptable when these tools make decisions that have life or death consequences.
Fair treatment in the health care system is a civil rights issue. The Covid-19 pandemic has revealed the many ways in which existing social inequity can lead to health care inequity – a complex reality that people try to understand, but that is difficult to accurately reflect in an algorithm. The promise of AI in medicine can help eliminate deeply biased institutional bias and improve health care outcomes; Instead, it threatens to automate this bias.
Policy changes and collaboration among key stakeholders, including state and federal regulators, medical, public health, and clinical advocacy groups and organizations, are needed to address these gaps and inefficiencies. To begin with, in detail a New ACLU white paper:
- Public reporting of demographic data is required.
- As part of the approval or disapproval process, the FDA requires an assessment of the impact of any differences in device performance by racial or ethnic subgroup.
- Device labels should reflect the results of this impact assessment.
- The FTC works with HHS and other federal agencies to develop best practices that device manufacturers should follow to reduce the risk of racial or ethnic discrimination in their devices that are not subject to FDA regulation.
HHS has learned from bombshell publications about racial and ethnic bias embedded in clinical and treatment algorithms that reveal the extent of medical and clinical malpractice. Future certainty.
[ad_2]
Source link
