The most widely used forms of model selection criteria, the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC), are expressed in terms of synthetic indicators of the residual distribution: the variance and the mean-squared error of the residuals respectively. In many applications in science, the noise aecting the data can be expected to have a Gaussian distribution. Therefore, at the same level of variance and mean-squared error, models, whose residuals are more uniformly distributed, should be favoured. The degree of uniformity of the residuals can be quantified by the Shannon entropy. Including the Shannon entropy in the BIC and AIC expressions improves significantly these criteria. The better performances have been demonstrated empirically with a series of simulations for various classes of functions and for dierent levels and statistics of the noise. In presence of outliers, a better treatment of the errors, using the Geodesic Distance, has proved essential.

Murari, A., Peluso, E., Cianfrani, F., Gaudio, P., Lungaroni, M. (2019). On the use of entropy to improve model selection criteria. ENTROPY, 21(4), 394 [10.3390/e21040394].

On the use of entropy to improve model selection criteria

Peluso E.
Validation
;
Gaudio P.
Funding Acquisition
;
Lungaroni M.
Validation
2019-01-01

Abstract

The most widely used forms of model selection criteria, the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC), are expressed in terms of synthetic indicators of the residual distribution: the variance and the mean-squared error of the residuals respectively. In many applications in science, the noise aecting the data can be expected to have a Gaussian distribution. Therefore, at the same level of variance and mean-squared error, models, whose residuals are more uniformly distributed, should be favoured. The degree of uniformity of the residuals can be quantified by the Shannon entropy. Including the Shannon entropy in the BIC and AIC expressions improves significantly these criteria. The better performances have been demonstrated empirically with a series of simulations for various classes of functions and for dierent levels and statistics of the noise. In presence of outliers, a better treatment of the errors, using the Geodesic Distance, has proved essential.
2019
Pubblicato
Rilevanza internazionale
Articolo
Esperti anonimi
Settore FIS/01 - FISICA SPERIMENTALE
Settore ING-IND/18 - FISICA DEI REATTORI NUCLEARI
English
Con Impact Factor ISI
Murari, A., Peluso, E., Cianfrani, F., Gaudio, P., Lungaroni, M. (2019). On the use of entropy to improve model selection criteria. ENTROPY, 21(4), 394 [10.3390/e21040394].
Murari, A; Peluso, E; Cianfrani, F; Gaudio, P; Lungaroni, M
Articolo su rivista
File in questo prodotto:
File Dimensione Formato  
entropy-21-00394.pdf

accesso aperto

Licenza: Copyright dell'editore
Dimensione 4.28 MB
Formato Adobe PDF
4.28 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2108/232508
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 17
  • ???jsp.display-item.citation.isi??? 9
social impact