Our new paper, recently published in IEEE Access, explores exactly this question.
We are happy to share our latest research on efficient low-bit representation of DNN weights for resource-constrained Edge AI systems as part of the AIDA4Edge project.
Our work explores an often overlooked aspect of floating-point representations: the exponent bias. We show that, by appropriately selecting the bias, the high-SQNR region can be aligned with the variance range of the data. This makes it possible to allocate more bits to the mantissa within the same low-bit budget, improving representation precision while compensating for the reduced variance coverage resulting from using fewer exponent bits.
Based on this insight, our BA-FP8-W (Bias-Aware FP8 for Weight Representation) format is designed specifically for efficient and accurate DNN weight representation.
Paper: Bias-Aware FP8 Format for Efficient Weight Representation in Post-Training Quantization
Authors: Aleksandra Ž. Jovanović, Stefan S. Tomić, Jelena R. Nikolić, Zoran H. Perić, Davide Bertozzi, and Michele Favalli
Affiliations: Faculty of Electronic Engineering, University of Niš (Serbia), School of Engineering, University of Manchester (UK), Department of Engineering, University of Ferrara (Italy), and Abu Dhabi Polytechnic (UAE)
Published in: IEEE Access
