Does Two-Class Training Extract Real Features? A COVID-19 Case Study

Diagnosis aid systems that use image analysis are currently very useful due to the large workload of health professionals involved in making diagnoses. In recent years, Convolutional Neural Networks (CNNs) have been used to help in these tasks. For this reason, multiple studies that analyze the detection precision for several diseases have been developed. However, many of these works distinguish between only two classes: healthy and with a specific disease. Based on this premise, in this work, we try to answer the questions: When training an image classification system with only two classes (healthy and sick), does this system extract the specific features of this disease, or does it only obtain the features that differentiate it from a healthy patient? Trying to answer these questions, we analyze the particular case of COVID-19 detection. Many works that classify this disease using X-ray images have been published; some of them use two classes (with and without COVID-19), while others include more classes (pneumonia, SARS, influenza, etc.). In this work, we carry out several classification studies with two classes, using test images that do not belong to those classes, in order to try to answer the previous questions. The first studies indicate problems in these two-class systems when using a third class as a test, being classified inconsistently. Deeper studies show that deep learning systems trained with two classes do not correctly extract the characteristics of pathologies, but rather differentiate the classes based on the physical characteristics of the images. After the discussion, we conclude that these two-class trained deep learning systems are not valid if there are other diseases that cause similar symptoms.