yuzhenmao / IceFormerLinks
Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).
☆25Updated last year
Alternatives and similar repositories for IceFormer
Users that are interested in IceFormer are comparing it to the libraries listed below
Sorting:
- ☆31Updated last year
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆41Updated last month
- IntLLaMA: A fast and light quantization solution for LLaMA☆18Updated last year
- Quantized Attention on GPU☆44Updated 7 months ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆27Updated last year
- ACL 2023☆39Updated 2 years ago
- ☆21Updated 2 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- Implementation for the paper: CMoE: Fast Carving of Mixture-of-Experts for Efficient LLM Inference☆21Updated 3 months ago
- Transformers components but in Triton☆34Updated last month
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆38Updated last year
- ☆49Updated last month
- ☆30Updated 3 weeks ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆31Updated last year
- [EMNLP 2024] RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization☆37Updated 9 months ago
- ☆75Updated 5 months ago
- LLM Inference with Microscaling Format☆23Updated 7 months ago
- ☆71Updated last month
- High Performance FP8 GEMM Kernels for SM89 and later GPUs.☆14Updated 5 months ago
- Benchmark tests supporting the TiledCUDA library.☆16Updated 7 months ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated 2 years ago
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆38Updated 2 weeks ago
- APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…☆23Updated this week
- Official repository for ICML 2024 paper "MoRe Fine-Tuning with 10x Fewer Parameters"☆20Updated last month
- ☆22Updated last year
- FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation☆49Updated 11 months ago
- Using FlexAttention to compute attention with different masking patterns☆44Updated 9 months ago
- Low-Rank Llama Custom Training☆23Updated last year