Download
AutoGPTQ is an implementation of GPTQ (Quantized GPT) that optimizes large language models (LLMs) for faster inference by reducing their computational footprint while maintaining accuracy.
Features
- Efficient quantization for large language models
- Reduces memory usage without major performance loss
- Supports various precision levels (e.g., 4-bit, 8-bit)
- Compatible with Hugging Face Transformers
- Accelerates inference on GPUs and CPUs
- Helps deploy LLMs on resource-constrained hardware
Project Samples
License
MIT LicenseFollow AutoGPTQ
Other Useful Business Software
Network Management Software and Tools for Businesses and Organizations | Auvik Networks
Reduce IT headaches and save time with a proven solution for automated network discovery, documentation, and performance monitoring. Choose Auvik because you'll see value in minutes, and stay with us to improve your IT for years to come.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of AutoGPTQ!
