Neuromorphic computing: Spiking neural networks and neuromorphic hardware

W. Maass, “Networks of Spiking Neurons: The Third Generation of Neural Network Models”, Neural Networks, vol. 10, no. 9, pp. 1659-1671, December 1997, https://doi.org/10.1016/S0893-6080(97)00011-7.

M.I. Stan, O. Rhodes, “Learning Long Sequences in Spiking Neural Networks”, Scientific Reports, vol. 14, Article No. 21957, September 2024, https://doi.org/10.1038/s41598-024-71678-8.

C. Frenkel, D. Bol, G. Indiveri, “Bottom-Up and Top-Down Approaches for the Design of Neuromorphic Processing Systems: Tradeoffs and Synergies Between Natural and Artificial Intelligence”, in Proceedings of the IEEE, vol. 111, no. 6, pp. 623-652, June 2023, https://doi.org/10.1109/JPROC.2023.3273520.

M. Bouvier, A. Valentian, T. Mesquida, F. Rummens, M. Reyboz, E. Vianello, E. Beigne, “Spiking Neural Networks Hardware Implementations and Challenges: A Survey”, ACM Journal on Emerging Technologies in Computing Systems (JETC), vol. 15, no. 2, Article No. 22, April 2019, https://doi.org/10.1145/3304103.

Z. Su, S. Ramini, D. C. Marcolin, A. Veronesi, M. Krstic, G. Indiveri, D. Bertozzi, S. M. Nowick “An Ultra-Low Cost and Multicast-Enabled Asynchronous NoC for Neuromorphic Edge Computing”, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 14, no. 3, pp. 409-424, September 2024, https://doi.org/10.1109/JETCAS.2024.3433427.

Y. Hu, Q. Zheng, G. Li, H. Tang, G. Pan, “Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions”, August 2024, arXiv preprint arXiv:2409.02111, https://doi.org/10.48550/arXiv.2409.02111.

Neuromorphic vision

G. Gallego et al., “Event-Based Vision: A Survey”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 01, pp. 154-180, January 2022, https://doi.org/10.1109/TPAMI.2020.3008413.

X. Iturbe et al., “Neuromorphic Vision Modalities in the NimbleAI 3D Chip”, in Proceedings of the 61st ACM/IEEE Design Automation Conference (DAC ’24), New York, NY, USA, 23-27 June, 2024, Article No. 358, https://doi.org/10.1145/3649329.3689622.

X. Iturbe et al., “NimbleAI: Towards Neuromorphic Sensing-Processing 3D-integrated Chips”, in Proceedings of the 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), Antwerp, Belgium, 17-19 April, 2023, pp. 1-6, https://doi.org/10.23919/DATE56975.2023.10136952.

Hybrid Artificial and Spiking Neural Network Models

A. K. Kosta, M. P. E. Apolinario, K. Roy, “Live Demonstration: ANN vs SNN vs Hybrid Architectures for Event-based Real-time Gesture Recognition and Optical Flow Estimation”, in Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada, 17-24 June, 2023, pp. 4148-4149, https://doi.org/10.1109/CVPRW59228.2023.00436.

A. Aydin, M. Gehrig, D. Gehrig, D. Scaramuzza, “A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception”, in Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 17-18 June, 2024, pp. 5701-5711, https://doi.org/10.1109/CVPRW63382.2024.00579.

A. Kugele, T. Pfeil, M. Pfeiffer, E. Chicca, “Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision”, in Proceedings of the 43rd DAGM German Conference, DAGM GCPR 2021, Bonn, Germany, September 28 – October 1, 2021, https://doi.org/10.1007/978-3-030-92659-5_19.

S. A. Tumpa, A. Devulapally, M. Brehove, E. Kyubwa, V. Narayanan, “SNN-ANN Hybrid Networks for Embedded Multimodal Monocular Depth Estimation”, in Proceedings of the 2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Knoxville, TN, USA, 01-03 July, 2024, pp. 198-203, https://doi.org/10.1109/ISVLSI61997.2024.00045.

Y. -T. Hsieh, Z. Li, D. Pompili, “A Lightweight Hybrid Analog-Digital Spiking Neural Network for IoT”, in Proceedings of the 2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), Abu Dhabi, United Arab Emirates, 29 April – 01 May, 2024, pp. 249-253, https://doi.org/10.1109/DCOSS-IoT61029.2024.00045.

S. Negi, D. Sharma, A. K. Kosta, K. Roy, “Best of Both Worlds: Hybrid SNN-ANN Architecture for Event-based Optical Flow Estimation”, arXiv preprint arXiv:2306.02960, June2023, https://doi.org/10.48550/arXiv.2306.02960.

Y. Zhong et al., “PAICORE: A 1.9-Million-Neuron 5.181-TSOPS/W Digital Neuromorphic Processor with Unified SNN-ANN and On-Chip Learning Paradigm”, IEEE Journal of Solid-State Circuits, July 2024, https://doi.org/10.1109/JSSC.2024.3426319.

J. Seekings, P. Chandarana, M. Ardakani, M. Mohammadi, R. Zand, “Towards Efficient Deployment of Hybrid SNNs on Neuromorphic and Edge AI Hardware”, in Proceedings of the 2024 International Conference on Neuromorphic Systems (ICONS), Arlington, VA, USA, 30 July – 02 August, 2024, pp. 71-77, https://doi.org/10.1109/ICONS62911.2024.00018.

Artificial Neural Network Tuning and Optimization

E. Liberis, Ł. Dudziak, N. D. Lane, “ΜNAS: Constrained Neural Architecture Search for Microcontrollers”, in Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys ’21), New York, NY, USA, 26 April, 2021, pp. 70-79, https://doi.org/10.1145/3437984.3458836.

M. M. Hossain, D. A. Talbert, S. K. Ghafoor, R. Kannan, “FAWCA: A Flexible-greedy Approach to find Well-tuned CNN Architecture for Image Recognition Problem”, in Proceedings of the 2018 International Conference on Data Science (ICDATA’18), Las Vegas, Nevada, US, 30 July-2 August, 2018, pp. 214-219, https://sghafoor10.github.io/publications/pdfs/2018/FAWCA%20-%20A%20Flexible-greedy%20Approach%20to%20find%20Well-tuned%20CNN%20Architecture%20for%20Image%20Recognition%20Problem.pdf.

A.-C. Cheng, C. H. Lin, D.-C. Juan, W. Wei, M. Sun, “InstaNAS: Instance-Aware Neural Architecture Search”, in Proceedings of the AAAI Conference on Artificial Intelligence, April 2020, vol. 34, no. 4, pp. 3577-3584, https://doi.org/10.1609/aaai.v34i04.5764.

A. Bizzarri, M Fraccaroli, E. Lamma, F. Riguzzi, “Integration Between Constrained Optimization and Deep Networks: A Survey”, Frontiers in Artificial Intelligence, vol. 7, June 2024, https://doi.org/10.3389/frai.2024.1414707.

H.-I Liu, M. Galindo, H. Xie, L.-K. Wong, H.-H. Shuai, Y.-H. Li, W.-H. Cheng. 2024. “Lightweight Deep Learning for Resource-Constrained Environments: A Survey”, ACM Computing Surveys, vol. 56, no. 10, Article No. 267, October 2024, https://doi.org/10.1145/3657282.

C. Banbury et al., “MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers”, arXiv preprint arXiv:2010.11267, October2020, https://doi.org/10.48550/arXiv.2010.11267.

E. Bellodi, D. Bertozzi, A. Bizzarri, M. Favalli, M. Fraccaroli, R. Zese, “Efficient Resource-Aware Neural Architecture Search with a Neuro-Symbolic Approach”, in Proceedings of the 2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), Singapore, 18-21 December, 2023, pp. 171-178, https://doi.org/10.1109/MCSoC60832.2023.00034.

M. Fraccaroli, E. Lamma, F. Riguzzi, “Symbolic DNN-Tuner”, Machine Learning, vol. 111, no. 2, pp. 625–650, February 2022, https://doi.org/10.1007/s10994-021-06097-1.

M. Fraccaroli, E. Lamma, F. Riguzzi, “Symbolic DNN-Tuner: A Python and ProbLog-based System for Optimizing Deep Neural Networks Hyperparameters”, SoftwareX, vol. 17, Article No. 100957, January 2022, https://doi.org/10.1016/j.softx.2021.100957. 

F. Dall’Occo, A. Bueno-Crespo, J. L. Abellán, D. Bertozzi, M. Favalli, “The Challenge of Classification Confidence Estimation in Dynamically-Adaptive Neural Networks”, in Proceedings of the Embedded Computer Systems: Architectures, Modeling, and Simulation: 21st International Conference, SAMOS 2021, Virtual Event, Berlin, Heidelberg, 4-8 July, 2021, pp. 505 – 522, https://doi.org/10.1007/978-3-031-04580-6_34.

K. Maile, E. Rachelson, H. Luga, D.G. Wilson, “When, Where, and How to Add New Neurons to ANNs”, in Proceedings of the First International Conference on Automated Machine Learning, PMLR, Baltimore, US, 25-27 July, 2022, vol. 188, Article No. 18, pp.1-12, https://proceedings.mlr.press/v188/maile22a.html
H. Liu, K. Simonyan and Y. Yang, “DARTS: Differentiable Architecture Search,” arXiv preprint, arXiv:1806.09055, 2019.

A. Zela, T. Elsken, T. Saikia, Y. Marrakchi, T. Brox and F. Hutter, “Understanding and Robustifying Differentiable Architecture Search,” arXiv preprint, arXiv:1909.09656v2, 2020.

X. Chu, T. Zhou, B. Zhang and J. Li, “Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search,” arXiv preprint, arXiv:1911.12126, 2020.

M. S. Iqbal, J. Su, L. Kotthoff and P. Jamshidi, “FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization Approach for Deep Neural Networks,” Journal of Artificial Intelligence Research, vol. 77, pp. 645–682, 2023, https://doi.org/10.1613/jair.1.14139.

C. White, W. Neiswanger and Y. Savani, “BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 10293–10301, 2021, https://doi.org/10.1609/aaai.v35i12.17233.

H. Alibrahim and S. A. Ludwig, “Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization,” in Proceedings of the 2021 IEEE Congress on Evolutionary Computation (CEC 2021), Kraków, Poland, pp. 1551–1559, 2021, https://doi.org/10.1109/CEC45853.2021.9504761.

K. O. Stanley and R. Miikkulainen, “Evolving Neural Networks through Augmenting Topologies,” Evolutionary Computation, vol. 10, no. 2, pp. 99–127, 2002, https://doi.org/10.1162/106365602320169811.

B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, J. Thomas, T. Ullmann, M. Becker, A.-L. Boulesteix, D. Deng and M. Lindauer, “Hyperparameter Optimization: Foundations, Algorithms, Best Practices, and Open Challenges,” WIREs Data Mining and Knowledge Discovery, vol. 13, no. 2, 2023, https://doi.org/10.1002/widm.1484.

AI Accelerators: Hardware for Neural Network Processing

I. Souvatzoglou, A. Papadimitriou, A. Sari, V. Vlagkoulis, M. Psarakis, “Analyzing the Single Event Upset Vulnerability of Binarized Neural Networks on SRAM FPGAs”, in Proceedings of the 2021 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT), Athens, Greece, 6-8 October, 2021, https://doi.org/10.1109/DFT52944.2021.9568280 .

S. I. Venieris, C.-S. Bouganis, N. D. Lane, “Multiple-Deep Neural Network Accelerators for Next-Generation Artificial Intelligence Systems”, Computer, vol. 56, no. 3, pp. 70-79, March 2023, https://doi.org/10.1109/MC.2022.3176845.

R. T. Syed, M. Andjelkovic, M. Ulbricht, M. Krstic, “Towards Reconfigurable CNN Accelerator for FPGA Implementation”, IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 70, no. 3, pp. 1249-1253, March 2023, https://doi.org/10.1109/TCSII.2023.3241154.

F. Vargas, M. Krstic, M. Andjelkovic, M. Ulbricht, J. Chen, “Silicon Lifecycle Management Based on On-Chip Cross-Layer Sensing and Analytics for Space Applications”, in Proceedings of the 2024 IEEE 25th Latin American Test Symposium (LATS), Maceio, Brazil, 9-12 April, 2024, https://doi.org/10.1109/LATS62223.2024.10534623.

A. Veronesi, M. Krstic, D. Bertozzi, “Cross-Layer Hardware/Software Assessment of the Open-Source NVDLA Configurable Deep Learning Accelerator”, in Proceedings of the 2020 IFIP/IEEE 28th International Conference on Very Large Scale Integration (VLSI-SOC), Salt Lake City, UT, USA, 5-7 October, 2020, pp. 58-63, https://doi.org/10.1109/VLSI-SOC46417.2020.9344109.

A. Veronesi et al., “Cross-Layer Reliability Analysis of NVDLA Accelerators: Exploring the Configuration Space”, in Proceedings of the 2024 IEEE European Test Symposium (ETS), The Hague, Netherlands, 20-24 May, 2024, https://doi.org/10.1109/ETS61313.2024.10568018.

Neural Network Quantization

C. Yuan, S. S. Agaian, “A Comprehensive Review of Binary Neural Network”, Artificial Intelligence Review, vol. 56, pp. 12949–13013, March 2023, https://doi.org/10.1007/s10462-023-10464-w.

R. Sayed, H. Azmi, H. Shawkey, A. H. Khalil, M. Refky, “A Systematic Literature Review on Binary Neural Networks”, IEEE Access, vol. 11, pp. 27546-27578, March 2023, https://doi.org/10.1109/ACCESS.2023.3258360.

W. Wei et al., “Q-SNNs: Quantized Spiking Neural Networks”, in Proceedings of the 32nd ACM International Conference on Multimedia (MM ’24), Melbourne, Australia, 28 October – 1 November, 2024, pp. 8441–8450, https://doi.org/10.1145/3664647.3681186.

X. Sui, Q. Lv, C. Ke, M. Li, M. Zhuang, H. Yu, Z. Tan, “Adaptive Global Power-of-Two Ternary Quantization Algorithm Based on Unfixed Boundary Thresholds”, Sensors, vol. 24, no. 1, 181, 2024, https://doi.org/10.3390/s24010181.

C. Li, L. Ma, S. Furber, “Quantization Framework for Fast Spiking Neural Networks”, Frontiers in Neuroscience, vol. 16, July 2022, https://doi.org/10.3389/fnins.2022.918793.

J. Aspman, G. Korpas, J. Marecek, “Taming Binarized Neural Networks and Mixed-Integer Programs”, in Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, Canada, 20-27 February, 2024, vol. 38, no. 10, pp. 10935-10943, https://doi.org/10.1609/aaai.v38i10.28968.

I. Colbert, A. Pappalardo, J. Petri-Koenig, “A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance”, in Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 1-6 October, 2023, pp. 16943-16952, https://doi.org/10.1109/ICCV51070.2023.01558.

I. Colbert, A. Pappalardo, J. Petri-Koenig, Y. Umuroglu, “A2Q+: Improving Accumulator-aware Weight Quantization”, in Proceedings of the 41st International Conference on Machine Learning (ICML’24), Vienna, Austria, 21-27 July, 2024, vol. 235, pp. 9275–9291, https://openreview.net/pdf?id=mbx2pLK5Eq.

K. Zhao, S. Huang, P. Pan, Y. Li, Y. Zhang, Z. Gu, Y. Xu, “Distribution Adaptive INT8 Quantization for Training CNNs”, in Proceedings of the AAAI Conference on Artificial Intelligence, February 2–9, May 2021, held virtually, vol. 35, no.4, pp. 3483-3491, https://doi.org/10.1609/aaai.v35i4.16462.

Z. Yao et al., “HAWQV3: Dyadic Neural Network Quantization”, in Proceedings of the 38th International Conference on Machine Learning, PMRL, 8-24 July, 2021, Virtual, vol. 139, pp. 11 875–11 886, https://proceedings.mlr.press/v139/yao21a/yao21a.pdf

S. K. Esser, J. L. Mckinstry, D. Bablani, R. Appuswamy, D. S. Modha, “Learned Step Size Quantization”, in Proceedings of the 2020 International Conference on Learning Representations, ICLR 2020, 27-30 April, 2020, https://openreview.net/pdf?id=rkgO66VKDS. 

J. Nikolić, Z. Perić, D. Aleksić, S. Tomić, A. Jovanović, “Whether the Support Region of Three-Bit Uniform Quantizer Has a Strong Impact on Post-Training Quantization for MNIST Dataset?”, Entropy, MDPI, vol. 23, no. 12, 1699, December 2021, https://doi.org/10.3390/e23121699.

Z. Perić, B. Denić, A. Jovanović, S. Milosavljević, M. Savić, “Performance Analysis of a 2-bit Dual-Mode Uniform Scalar Quantizer for Laplacian Source”, Information Technology and Control, vol. 51, no. 4, pp. 625-637, December 2022, http://dx.doi.org/10.5755/j01.itc.51.4.30473.

S. Tomić, J. Nikolić, Z. Perić, D. Aleksić, “Performance of Post-Training Two-Bits Uniform and Layer-Wise Uniform Quantization for MNIST Dataset from the Perspective of Support Region Choice”, Mathematical Problems in Engineering, 1463094, 15 pages, April 2022, https://doi.org/10.1155/2022/1463094.

D. Ćirić, Z. Perić, M. Milenković, N. Vučić, “Evaluating Similarity of Spectrogram-like Images of DC Motor Sounds by Pearson Correlation Coefficient”, Elektronika ir Elektrotechnika, vol. 28, no. 3, pp. 37-44, June 2022, https://doi.org/10.5755/j02.eie.31041.

B. Predić, U. Vukić, M. Saračević, D. Karabašević, D. Stanujkić, “The Possibility of Combining and Implementing Deep Neural Network Compression Methods”, Axioms, vol. 11, no. 5, pp. 229, May 2022, https://doi.org/10.3390/axioms11050229.

J. Nikolić, Z. Perić, S. Tomić, A. Jovanović, D. Aleksić, S. Perić, “Comparative Analysis of the Robustness of 3-bit PoTQ and UQ and their Application in Post-Training Quantization”, Advances in Electrical and Computer Engineering, vol. 24, no. 4, pp. 47-56, Nov. 2024, https://doi.org/10.4316/AECE.2024.04005.

Z. Perić, B. Denić, A. Jovanović, M. Savić, N. Vučić and A. Nikolić, “A Dual-Mode 2-bit Uniform Scalar Quantizer for Data with Laplacian Distribution”, in Proceedings of the 2021 29th Telecommunications Forum (TELFOR), Belgrade, Serbia, 23-24 November, 2021, pp. 1-4, https://doi.org/10.1109/TELFOR52709.2021.9653253.

N. Vučić, Z. Perić, A. Jovanović, “Model of Improved Floating Point 32-bits Quantizer”, in Proceedings of the 57th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), Ohrid, North Macedonia, 16-18 June, 2022, pp. 1-2, https://doi.org/10.1109/ICEST55168.2022.9828586.

J. Nikolić, Z. Perić, S. Tomić, D. Aleksić, “On Different Criteria for Optimizing the Two-bit Uniform Quantizer”, in Proceedings of the 21st International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Bosnia and Herzegovina, 16-18 March, 2022, pp. 1-4, https://doi.org/10.1109/INFOTEH53737.2022.9751268.

A. Jovanović, J. Nikolić, Z. Perić, “On the Rate Redundancy of Uniform Scalar Quantization and Golomb-Rice Coding”, in Proceedings of the 16th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), Niš, Serbia, pp. 324-327, 25-27 October, 2023, https://doi.org/10.1109/TELSIKS57806.2023.10316064.

M. Andjelković, J. Chen, R. T. Syed, M. Marjanović, G. Ristić, M. Krstić, “Prediction of Generated Single Event Transient Pulse Width Using Artificial Intelligence Methods”, in Proceedings of the 33rd IEEE International Conference on Microelectronics (MIEL 2023), Niš, Serbia, 16-18 October, 2023, pp. 191-194, https://doi.org/10.1109/MIEL58498.2023.10315809.

M. Andjelkovic, R. T. Syed, M. Pavlovic, F. Vargas, T. Nikolic, G. Ristic, M. Krstic, “Voltage Glitch Filter and Detector with Self-Checking Capability for FPGA Implementation”, in Proceedings of the 33rd IEEE International Conference on Microelectronics (MIEL2023), Niš, Serbia, 16-18 October, 2023, pp. 143-146, https://doi.org/10.1109/MIEL58498.2023.10315811.

Z. Perić, B. Denić, M. Dinčić, “Performance Analysis of the 24-bits Floating-Point Format for a Gaussian Source”, in Proceedings of the 11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024, https://www.etran.rs/2024/E_ZBORNIK_IcETRAN_2024/036_TEI1.3.pdf.
Y. Sakai and Y. Tamiya, “S-DFP: Shifted Dynamic Fixed Point for Quantized Deep Neural Network Training,” Neural Computing and Applications, vol. 37, pp. 535–542, 2025, https://doi.org/10.1007/s00521-021-06821-x.

H. Chen, L. Liu, J. Meng et al., “AFC: An Adaptive Lossless Floating-Point Compression Algorithm in Time Series Database,” Information Sciences, vol. 654, article 119847, 2024, https://doi.org/10.1016/j.ins.2023.119847.

F. He, K. Ding, D. Yan, J. Li, J. Wang and M. Chen, “A Novel Quantization and Model Compression Approach for Hardware Accelerators in Edge Computing,” Computers, Materials & Continua, vol. 80, pp. 3021–3045, 2024, https://doi.org/10.32604/cmc.2024.053632.

Y. Bai-Kui and R. Shanq-Jang, “Area Efficient Compression for Floating-Point Feature Maps in Convolutional Neural Network Accelerators,” IEEE Transactions on Circuits and Systems II, vol. 70, no. 2, pp. 746–750, 2023, https://doi.org/10.1109/TCSII.2022.3213847.

W. Zhao, Q. Dang, T. Xia, J. Zhang, N. Zheng and P. Ren, “Optimizing FPGA-Based DNN Accelerator with Shared Exponential Floating-Point Format,” IEEE Transactions on Circuits and Systems I, vol. 70, no. 11, pp. 4478–4491, 2023, https://doi.org/10.1109/TCSI.2023.3300657.

L. Kummer, K. Sidak, T. Reichmann and W. Gansterer, “Adaptive Precision Training (AdaPT): A Dynamic Fixed Point Quantized Training Approach for DNNs,” in Proceedings of the 2023 SIAM International Conference on Data Mining (SDM23), Minneapolis, MN, USA, 27–29 April 2023, https://doi.org/10.1137/1.9781611977653.ch63.

M. Junaid, S. Arslan, T. Lee and H. Kim, “Optimal Architecture of Floating-Point Arithmetic for Neural Network Training Processor,” Sensors, vol. 22, article 1230, 2022, https://doi.org/10.3390/s22031230.

Y. Yang, X. Chi, L. Deng, T. Yan, F. Gao and G. Li, “Towards Efficient Full 8-Bit Integer DNN Online Training on Resource-Limited Devices Without Batch Normalization,” Neurocomputing, vol. 511, pp. 175–186, 2022, https://doi.org/10.1016/j.neucom.2022.08.045.

S. Kim and H. Kim, “Zero-Centered Fixed-Point Quantization with Iterative Retraining for Deep Convolutional Neural Network Based Object Detectors,” IEEE Access, vol. 9, pp. 20828–20839, 2021, https://doi.org/10.1109/ACCESS.2021.3054879

M. Fasi and M. Mikaitis, “Algorithms for Stochastically Rounded Elementary Arithmetic Operations in IEEE 754 Floating-Point Arithmetic,” IEEE Transactions on Emerging Topics in Computing, vol. 9, no. 3, pp. 1451–1466, 2021, https://doi.org/10.1109/TETC.2021.3069165.

A. Agrawal, S. M. Mueller, B. M. Fleischer, X. Sun, N. Wang, J. Choi and K. Gopalakrishnan, “DLFloat: A 16-b Floating Point Format Designed for Deep Learning Training and Inference,” in Proceedings of the IEEE 26th Symposium on Computer Arithmetic (ARITH 2019), Kyoto, Japan, pp. 92–95, 2019, https://doi.org/10.1109/ARITH.2019.00023.

N. Burgess, J. Milanovic, N. Stephens, K. Monachopoulos and D. Mansell, “Bfloat16 Processing for Neural Networks,” in Proceedings of the IEEE 26th Symposium on Computer Arithmetic (ARITH 2019), Kyoto, Japan, pp. 10–12, 2019, https://doi.org/10.1109/ARITH.2019.00022.

N. Burgess, C. Goodyer, C. N. Hinds and D. R. Lutz, “High-Precision Anchored Accumulators for Reproducible Floating-Point Summation,” IEEE Transactions on Computers, vol. 68, no. 7, pp. 967–978, 2019, https://doi.org/10.1109/TC.2018.2855729.

R. Banner, Y. Nahshan, E. Hoffer and D. Soudry, “ACIQ: Analytical Clipping for Integer Quantization of Neural Networks,” arXiv preprint, arXiv:1810.05723, 2018.

D. Das, N. Mellempudi, D. Mudigere, D. Kalamkar, S. Avancha, K. Banerjee, S. Sridharan, K. Vaidyanathan, B. Kaul and E. Georganas, “Mixed Precision Training of Convolutional Neural Networks Using Integer Operations,” arXiv preprint, arXiv:1802.00930, 2018.

D. Cattaneo, A. Di Bello, S. Cherubin, F. Terraneo and G. Agosta, “Embedded Operating System Optimization Through Floating to Fixed Point Compiler Transformation,” in Proceedings of the 21st Euromicro Conference on Digital System Design (DSD 2018), Prague, Czech Republic, pp. 172–176, 2018, https://doi.org/10.1109/DSD.2018.00042.

N. Wang, J. Choi, D. Brand, C. Y. Chen and K. Gopalakrishnan, “Training Deep Neural Networks with 8-Bit Floating Point Numbers,” in Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada, pp. 7686–7695, 2018, https://proceedings.neurips.cc/paper_files/paper/2018/file/335d3d1cd7ef05ec77714a215134914c-Paper.pdf.