Robust Binary Encoding for Ternary Neural Networks Toward Deployment on Emerging Memory

Le résumé fourni par la source

Deep neural networks (DNNs) have enabled state-of-the-art performance across various applications. However, their deployment is often hindered by high energy demands. One solution is to deploy DNNs on hardware equipped with emerging non-volatile memory, which does not require energy to maintain memory state. Nonetheless, this approach may introduce bit-flips in DNN parameters, leading to a drop in model performance. To mitigate this issue, ternary neural networks (TNNs), which are known for their robustness against bit-flips, can be utilized. To accelerate TNNs, it is necessary to define binary representations (BRs) for ternary values to enable bit-wise operations. This paper proposes a framework for identifying a set of BRs that minimizes the discrepancy between the Top-1 Accuracy of TNNs before and after bit-flips, thereby enhancing their robustness. The BRs are evaluated using randomly generated data, the CIFAR-10 dataset, and the ImageNet 2012 dataset. The experimental results demonstrate that our BRs can significantly reduce or maintain the Top-1 Accuracy degradation caused by bit-flips, compared to conventional BRs.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.