Sparsity-aware deep learning inference runtime for CPUs

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、reg…

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

OpenMMLab Model Compression Toolbox and Benchmark.

PaddleSlim is an open-source library for deep model compression and architecture search.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

Neural Network Compression Framework for enhanced OpenVINO™ inference

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

PyTorch Implementation of [1611.06440] Pruning Convolutional Neural Networks for Resource Efficient Inference

TinyNeuralNetwork is an efficient and easy-to-use deep learning model compression framework.

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (CVPR 2019 Oral)

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

YOLO ModelCompression MultidatasetTraining

Pruning and other network surgery for trained Keras models.

Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes