In this brief, we consider the feature ranking problem, where, given a set of training instances, the task is to associate a score with the features in order to assess their relevance. Feature ranking is a very important tool for decision support systems, and may be used as an auxiliary step of feature selection to reduce the high dimensionality of real-world data. We focus on regression problems by assuming that the process underlying the generated data can be approximated by a continuous function (for instance, a feedforward neural network). We formally state the notion of relevance of a feature by introducing a minimum zero-norm inversion problem of a neural network, which is a nonsmooth, constrained optimization problem. We employ a concave approximation of the zero-norm function, and we define a smooth, global optimization problem to be solved in order to assess the relevance of the features. We present the new feature ranking method based on the solution of instances of the global optimization problem depending on the available training data. Computational experiments on both artificial and real data sets are performed, and point out that the proposed feature ranking method is a valid alternative to existing methods in terms of effectiveness. The obtained results also show that the method is costly in terms of CPU time, and this may be a limitation in the solution of large-dimensional problems.

Bravi, L., Piccialli, V., Sciandrone, M. (2017). An optimization-based method for feature ranking in nonlinear regression problems. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 28(4), 1005-1010 [10.1109/TNNLS.2015.2504957].

An optimization-based method for feature ranking in nonlinear regression problems

Piccialli V.;
2017-01-01

Abstract

In this brief, we consider the feature ranking problem, where, given a set of training instances, the task is to associate a score with the features in order to assess their relevance. Feature ranking is a very important tool for decision support systems, and may be used as an auxiliary step of feature selection to reduce the high dimensionality of real-world data. We focus on regression problems by assuming that the process underlying the generated data can be approximated by a continuous function (for instance, a feedforward neural network). We formally state the notion of relevance of a feature by introducing a minimum zero-norm inversion problem of a neural network, which is a nonsmooth, constrained optimization problem. We employ a concave approximation of the zero-norm function, and we define a smooth, global optimization problem to be solved in order to assess the relevance of the features. We present the new feature ranking method based on the solution of instances of the global optimization problem depending on the available training data. Computational experiments on both artificial and real data sets are performed, and point out that the proposed feature ranking method is a valid alternative to existing methods in terms of effectiveness. The obtained results also show that the method is costly in terms of CPU time, and this may be a limitation in the solution of large-dimensional problems.
2017
Pubblicato
Rilevanza internazionale
Articolo
Esperti anonimi
Settore MAT/09 - RICERCA OPERATIVA
English
Concave approximation of the zero-norm function; feature ranking; global optimization; inversion of a neural network
Bravi, L., Piccialli, V., Sciandrone, M. (2017). An optimization-based method for feature ranking in nonlinear regression problems. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 28(4), 1005-1010 [10.1109/TNNLS.2015.2504957].
Bravi, L; Piccialli, V; Sciandrone, M
Articolo su rivista
File in questo prodotto:
File Dimensione Formato  
BraviPiccialliSciandroneIEEE.pdf

solo utenti autorizzati

Licenza: Copyright dell'editore
Dimensione 482.81 kB
Formato Adobe PDF
482.81 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2108/194869
Citazioni
  • ???jsp.display-item.citation.pmc??? 1
  • Scopus 14
  • ???jsp.display-item.citation.isi??? 9
social impact