Kernel methods enable the direct usage of structured representations of textual data during language learning and inference tasks. Expressive kernels, such as Tree Kernels, achieve excellent performance in NLP. On the other side, deep neural networks have been demonstrated effective in automatically learning feature representations during training. However, their input is tensor data, i.e., they cannot manage rich structured information. In this paper, we show that expressive kernels and deep neural networks can be combined in a common framework in order to (i) explicitly model structured information and (ii) learn non-linear decision functions. We show that the input layer of a deep architecture can be pre-trained through the application of the Nyström low-rank approximation of kernel spaces. The resulting "kernelized" neural network achieves state-of-the-art accuracy in three different tasks.

Croce, D., Filice, S., Castellucci, G., Basili, R. (2017). Deep learning in semantic kernel spaces. In ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers) (pp.345-354). Association for Computational Linguistics (ACL) [10.18653/v1/P17-1032].

Deep learning in semantic kernel spaces

Croce, Danilo;Filice, Simone;Castellucci, Giuseppe;Basili, Roberto
2017-08-02

Abstract

Kernel methods enable the direct usage of structured representations of textual data during language learning and inference tasks. Expressive kernels, such as Tree Kernels, achieve excellent performance in NLP. On the other side, deep neural networks have been demonstrated effective in automatically learning feature representations during training. However, their input is tensor data, i.e., they cannot manage rich structured information. In this paper, we show that expressive kernels and deep neural networks can be combined in a common framework in order to (i) explicitly model structured information and (ii) learn non-linear decision functions. We show that the input layer of a deep architecture can be pre-trained through the application of the Nyström low-rank approximation of kernel spaces. The resulting "kernelized" neural network achieves state-of-the-art accuracy in three different tasks.
55th Annual Meeting of the Association for Computational Linguistics, ACL 2017
can
2017
Amazon
Rilevanza internazionale
2-ago-2017
Settore ING-INF/05 - SISTEMI DI ELABORAZIONE DELLE INFORMAZIONI
Settore INF/01 - INFORMATICA
English
Language and Linguistics; Artificial Intelligence; Software; Linguistics and Language
https://aclweb.org/anthology/P/P17/
Intervento a convegno
Croce, D., Filice, S., Castellucci, G., Basili, R. (2017). Deep learning in semantic kernel spaces. In ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers) (pp.345-354). Association for Computational Linguistics (ACL) [10.18653/v1/P17-1032].
Croce, D; Filice, S; Castellucci, G; Basili, R
File in questo prodotto:
File Dimensione Formato  
acl2017_deeplearning.pdf

accesso aperto

Licenza: Non specificato
Dimensione 315.42 kB
Formato Adobe PDF
315.42 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2108/198399
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 16
  • ???jsp.display-item.citation.isi??? 12
social impact