https://blog.sksoumik.com/tags/optimization/ Recent content in optimization on Sadman Kabir Soumik Hugo -- gohugo.io en Copyright © 2022, Sadman Kabir Soumik Mon, 10 Oct 2022 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/machine-learning-model-compression-techniques/ Mon, 10 Oct 2022 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/machine-learning-model-compression-techniques/ There are 4 main approaches you can consider for model compression. They are: Quantization Pruning Knowledge Distillation Low-Rank Factorization Quantization Quantization is the most general and commonly used model compression method. Quantization reduces a model’s size by using fewer bits to represent its parameters. By default, most software packages use 32 bits to represent a float number (single precision floating point). If a model has 100M parameters and each requires 32 bits to store, it’ll take up 400 MB.