GreenBitAI / green-bit-llm
A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
☆83Updated last month
Alternatives and similar repositories for green-bit-llm:
Users that are interested in green-bit-llm are comparing it to the libraries listed below
- QuIP quantization☆52Updated last year
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆36Updated last year
- ☆53Updated 11 months ago
- The official repo for "LLoCo: Learning Long Contexts Offline"☆116Updated 10 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆115Updated 5 months ago
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.☆168Updated last month
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆40Updated last year
- Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs☆110Updated last year
- PB-LLM: Partially Binarized Large Language Models☆152Updated last year
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆121Updated last year
- ☆50Updated 6 months ago
- Repository for CPU Kernel Generation for LLM Inference☆26Updated last year
- Data preparation code for Amber 7B LLM☆89Updated 11 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆198Updated 9 months ago
- A repository for research on medium sized language models.☆76Updated 11 months ago
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆205Updated 11 months ago
- RWKV-7: Surpassing GPT☆83Updated 5 months ago
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆148Updated 3 weeks ago
- ☆198Updated 5 months ago
- ☆125Updated last year
- Cascade Speculative Drafting☆29Updated last year
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆45Updated 2 weeks ago
- ☆48Updated 6 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆73Updated 8 months ago
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆55Updated 2 weeks ago
- ☆45Updated this week
- GPTQLoRA: Efficient Finetuning of Quantized LLMs with GPTQ☆103Updated last year
- ☆33Updated 10 months ago
- KV cache compression for high-throughput LLM inference☆124Updated 3 months ago
- Work in progress.☆58Updated 3 weeks ago