I have a labeled dataset of two classes. Let’s say sick and healthy patients. My features are patient data as well as diagnostic data such as blood-test results. It’s categorical as well as continuous data. In total several thousand features.
I want to identify the features that best differentiate the sick patients from the healthy ones.
It is not really an anomaly detection problem, because the sickness is not directly indicated by anything in my features. Meaning that for none of my features the sick patients would be indicated by outliers. For example a result could be that the risk of sickness is higher for patients over the age of 50. But being over 50 is not generally uncommon.
The obvious approach would be to use a dedicated feature selection technique. For example a chi-square test, variable importance in decision trees or backward elimination.
The problem about all of these approaches is that they are usually preprocessing steps before another learning algorithm is deployed. I on the other hand don’t want to use another learning algorithm. The entire task is to figure out the most important features.
Are feature selection algorithms really the best way to proceed, or are there any better approaches? Maybe some techniques especially designed for my kind of problems?
Many thanks in advance!