Attribute estimation

One of crucial tasks in machine learning is the evaluation of the quality of attributes. For that purpose a number of measures have been developed that estimate the usefulness of the attribute for predicting the target variable. We will describe separately measures for classification (which are appropriate also for relational problems) and for regression. Most of the measures estimate the quality of one attribute independently of the context of other attributes. However, algorithm ReliefF and its regressional version RReliefF take into account also the context of other attributes and are therefore appropriate for problems with strong dependencies between attributes. The following measures will be described: - Measures for guiding the search in classification and relational problems are: information gain, Gain ratio, distance measure, minimum description length (MDL), J-measure, Gini-index and ReliefF. - The quality of attributes in regression can be evaluated using the following measures: expected change of variance, regressional ReliefF, and minimum description length principle (MDL).