Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties. In this paper we discuss how the theory of reproducing kernel Banach spaces can be used to tackle this challenge. In particular, we prove a representer theorem for a wide class of reproducing kernel Banach spaces that admit a suitable integral representation and include one hidden layer neural networks of possibly infinite width. Further, we show that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel Banach space can be characterized in terms of the inverse Radon transform of a bounded real measure, with norm given by the total variation norm of the measure. Our analysis simplifies and extends recent results in [45,36,37].

Bartolucci, F., De Vito, E., Rosasco, L., Vigogna, S. (2023). Understanding neural networks with reproducing kernel Banach spaces. APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS, 62, 194-236 [10.1016/j.acha.2022.08.006].

Understanding neural networks with reproducing kernel Banach spaces

Vigogna, S
2023-01-01

Abstract

Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties. In this paper we discuss how the theory of reproducing kernel Banach spaces can be used to tackle this challenge. In particular, we prove a representer theorem for a wide class of reproducing kernel Banach spaces that admit a suitable integral representation and include one hidden layer neural networks of possibly infinite width. Further, we show that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel Banach space can be characterized in terms of the inverse Radon transform of a bounded real measure, with norm given by the total variation norm of the measure. Our analysis simplifies and extends recent results in [45,36,37].
2023
Pubblicato
Rilevanza internazionale
Articolo
Esperti anonimi
Settore MAT/06 - PROBABILITA' E STATISTICA MATEMATICA
English
Neural networks
Representer theorem
Radon transform
Reproducing kernel Banach spaces
Bartolucci, F., De Vito, E., Rosasco, L., Vigogna, S. (2023). Understanding neural networks with reproducing kernel Banach spaces. APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS, 62, 194-236 [10.1016/j.acha.2022.08.006].
Bartolucci, F; De Vito, E; Rosasco, L; Vigogna, S
Articolo su rivista
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S1063520322000768-main.pdf

solo utenti autorizzati

Tipologia: Versione Editoriale (PDF)
Licenza: Copyright dell'editore
Dimensione 762.45 kB
Formato Adobe PDF
762.45 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2108/312262
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 17
  • ???jsp.display-item.citation.isi??? 14
social impact