Contents
Researchers from South Africa proved that the introduction of a system of penalties for artificial intelligence for false diagnoses improves the accuracy of subsequent predictions
What’s going on
- A group of scientists from the University of Johannesburg conducted a study on machine learning algorithms, in which they found that the accuracy of artificial intelligence predictions increases with the introduction of a system of penalties for false diagnoses.
- The study analyzed binary algorithms such as logistic regression, random forest, and XGBoost, all of which are trained on yes/no datasets provided to them.
- The scientists used binary training datasets to diagnose diabetes, breast cancer, cervical cancer, and chronic kidney disease, which classify patients as sick or not.
- The penalty system really works: in the case of chronic kidney disease, before the introduction of “punishment” with the “random forest” algorithm, the prediction accuracy was 97,2%, and after – 99%.
- With other datasets, the results vary depending on the algorithm: for example, the accuracy of diagnosing cervical cancer using the random forest method and XGBoost reached the ideal value of 100%, but in the case of the logistic regression algorithm, they improved, but not to unity, that is, all the same there is some error.
- In fact, this works like machine learning “by the contrary”: a certain “error cost” (that is, a penalty) is put into the algorithm, and the neural network, which usually works to improve its accuracy, is programmed to reduce errors.
- According to the authors of the study, the algorithms are more accurate in determining a healthy person than in identifying sick people. That is why the algorithm receives higher penalties for false negative diagnoses than for false positive ones.
What does it mean
In the modern world, artificial intelligence is increasingly being used to diagnose and predict life-threatening diseases. For example, an IBM research team has developed a machine learning model for detecting Parkinson’s disease at an early stage, and Lithuanian scientists have developed a method for diagnosing Alzheimer’s disease before the first symptoms appear. However, the accuracy of AI diagnoses leaves much to be desired, especially in the case of false negative diagnoses.
It is also worth noting that patients are wary of AI-based diagnostics – a Harvard Business School study shows that Americans tend to refuse medical care from AI and are not ready to pay for this service as much as for help from a medical professional. This is mainly due to the fact that each patient considers their problem to be unique, which means that it cannot be solved using algorithms.