FPGA Oriented Compression of DNN Using Layer-Targeted Weights and Activations Quantization

Le résumé fourni par la source

In recent years, complex Deep neural networks (DNNs) have achieved state-of-the-art results in a wide range of real-world tasks. However, the gains in their performance lead to an increase in computations and frequent access to external memory slow down the FPGA inference speed. To address this problem, our research proposes an algorithm-hardware co-design, i.e. a layer-targeted quantization combined with an FPGA streaming architecture to reduce the computations, memory bandwidth and circuit size, specifically the bit numbers, Block Random Access Memories (BRAMs) size and the number of Digital Signal Processing (DSP) blocks while keeping a comparable accuracy drop compared to recent state-of-the-art. Our layer-targeted quantization method balances the bitwidth and model's accuracy drop based on the distribution of weights and activations. Particularly, by repeating simulations, the quantization range and step size are determined to minimize the bitwidth while the accuracy decreases slightly. Moreover, for the large-scale network, the activation function CReLU is used complying with quantization to further improve the accuracy drop. While training the quantized network, a backward method using Straight Through Estimator is proposed to help the quantized model to be converged and retain the accuracy. Our method's efficiency is demonstrated by two experimental results on small-scale network VGG16 with CIFAR-10 and large-scale one VGG16-SSD with VOC07+12 benchmark. The first result shows the model size reduction of nearly 32× for weights and more than 5× for activations with a small loss of accuracy 1.26%. While the second one expresses a comparable accuracy drop of 1.8% with most of state-of-the-art results. For the implementation of CNNs on FPGA, a pipelined streaming accelerator for VGG16-SSD is presented. To minimize the number of on-chip buffer of the large-scale network, together with the software quantization, a data-path optimization technique named Input Feature Map Fully Reuse is applied. Preliminary calculations show that the required BRAMs size is 2974 blocks and the DSP blocks are 552 which is significantly smaller than those of recent published works.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.