Abstract:
In this thesis, we develop three new methods for feature selection with Multi Layer
Perceptron(MLP) neural networks. In each method, we use a two-step approach.
First, we train a MLP network for a given dataset. Second, we introduce feature
selector variables and form an optimization problem based on some penalty on
these feature selector variables and some measure of redundancy. Then, we optimize
the problem using gradient descent method to find nearly optimal values
of the feature selector variables while keeping the weights of the MLP networks
fixed. First method, which we call Feature Selection with MLP using Approximate
L0-norm and Global Redundancy Control (FSMLP-AL-GRC) uses penalty based
on an approximate L0-norm and global redundancy, i.e., redundancy that is calculated
with features values, without considering the class information. For second
method, we first define a new redundancy measure that uses class label information
while calculating redundancy, we call it class-level redundancy. This method
make use of class level redundancy measure along with an approximate L0-norm
based penalty. We call it Feature Selection with MLP using Approximate L0-norm
and Class-level Redundancy Control (FSMLP-AL-CRC). Last method is a variant of
method two. Here, we replace each feature selector variable with some non-linear
bounded function that always lies between 0 and 1, this function act as feature
attenuating gates. We call this method Gated Feature Selection with MLP using
Class-level Redundancy Control (Gated-FSMLP-CRC). We test these methods experimentally
on different data sets. We also present results on Sonar data using
method Gated-FSMLP-CRC without keeping the weights of MLP fixed during the
learning process.
