We propose an architectural blueprint to implement Q-RTS, Q-Learning Real-Time Swarm Reinforcement Learning algorithm, on FPGA. The design solution is built on FPGA-based Centralized RL Processing Units (CRLPU). A CRLPU processes local and global state-action matrices and exchanges information frames with low-power Microcontroller-based Agents. The novel architecture implementation, for up to 32 Agents with up to 512 states, on a Xilinx Ultrascale device shows low resource requirements in terms of CLB (7%) and memory (2% FF and 22% BRAM). Performance metrics show that the required energy per generated action is always lower than 1 mu J.
Cardarilli, G.c., Di Nunzio, L., Fazzolari, R., Giardino, D., Matta, M., Nannarelli, A., et al. (2020). FPGA implementation of Q-RTS for real-time Swarm intelligence systems. In Asilomar conference on signals, systems, and computers (pp.116-120). IEEE [10.1109/IEEECONF51394.2020.9443368].
FPGA implementation of Q-RTS for real-time Swarm intelligence systems
Cardarilli G. C.;Di Nunzio L.;Fazzolari R.;Re M.;Spano S.
2020-01-01
Abstract
We propose an architectural blueprint to implement Q-RTS, Q-Learning Real-Time Swarm Reinforcement Learning algorithm, on FPGA. The design solution is built on FPGA-based Centralized RL Processing Units (CRLPU). A CRLPU processes local and global state-action matrices and exchanges information frames with low-power Microcontroller-based Agents. The novel architecture implementation, for up to 32 Agents with up to 512 states, on a Xilinx Ultrascale device shows low resource requirements in terms of CLB (7%) and memory (2% FF and 22% BRAM). Performance metrics show that the required energy per generated action is always lower than 1 mu J.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
