model-pruning

Use Model Pruning when you need to: Key Techniques : Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity Papers : Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774) Source : ICLR 2024 (arXiv 2306.11695) Source : arXiv 2301.00774 Magnitude Pruning (baseline): Wanda (weights × activations): SparseGPT (second-order): Unstructured (fine-grained): Structured (coarse-grained): Semi-structured (N:M) : Pruning methods at 50% sparsity (LLaMA-7B): Source : Wanda paper (ICLR 2024), SparseGPT paper - Reduce model size by 40-60% with <1% accuracy loss - Accelerate inference using hardware-friendly sparsity (2-4× speedup) - Deploy on constrained hardware (mobile, edge devices) - Compress without retraining using one-shot methods - Enable efficient serving with reduced memory footprint - Prune individual weights - Higher quality (better accuracy) - No hardware speedup (irregular sparsity) - Prune entire neurons, heads, or layers - Lower quality (more accuracy loss) - Hardware speedup (regular sparsity) - Best of both worlds - 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs - Minimal accuracy loss - Wanda Paper (ICLR 2024) : - Wanda GitHub : - SparseGPT Paper : - SparseGPT GitHub : - NVIDIA Sparse Tensor Cores :