facebookresearch / any4Links
Quantize transformers to any learned arbitrary 4-bit numeric format
☆41Updated last month
Alternatives and similar repositories for any4
Users that are interested in any4 are comparing it to the libraries listed below
Sorting:
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- Framework to reduce autotune overhead to zero for well known deployments.☆81Updated last week
- ☆74Updated 5 months ago
- ☆77Updated 3 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆76Updated last year
- FlexAttention w/ FlashAttention3 Support☆27Updated 10 months ago
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated last month
- ☆159Updated last year
- ☆110Updated last year
- Make triton easier☆47Updated last year
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆43Updated last month
- Transformers components but in Triton☆34Updated 3 months ago
- ☆14Updated last month
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- ☆33Updated last year
- A block oriented training approach for inference time optimization.☆34Updated last year
- This repository contains code for the MicroAdam paper.☆19Updated 8 months ago
- Work in progress.☆72Updated last month
- ☆26Updated last year
- QuIP quantization☆57Updated last year
- ☆32Updated last year
- Repository for CPU Kernel Generation for LLM Inference☆26Updated 2 years ago
- Hacks for PyTorch☆19Updated 2 years ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆80Updated 11 months ago
- ☆81Updated 7 months ago
- A safetensors extension to efficiently store sparse quantized tensors on disk☆153Updated this week
- ☆123Updated 2 months ago
- ☆81Updated last year
- ☆30Updated last week
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆47Updated last year