huggingface / optimum-tpu
Google TPU optimizations for transformers models
☆102Updated last month
Alternatives and similar repositories for optimum-tpu:
Users that are interested in optimum-tpu are comparing it to the libraries listed below
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"☆53Updated last month
- Pytorch/XLA SPMD Test code in Google TPU☆23Updated 11 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆154Updated 4 months ago
- Set of scripts to finetune LLMs☆36Updated 11 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆261Updated 5 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆196Updated 7 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆290Updated this week
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆35Updated 10 months ago
- Multipack distributed sampler for fast padding-free training of LLMs☆186Updated 7 months ago
- ☆67Updated 2 years ago
- Manage scalable open LLM inference endpoints in Slurm clusters☆253Updated 8 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"☆59Updated 5 months ago
- PB-LLM: Partially Binarized Large Language Models☆151Updated last year
- Experiments for efforts to train a new and improved t5☆77Updated 10 months ago
- ☆49Updated 11 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆227Updated last week
- ☆113Updated 5 months ago
- ☆193Updated 3 months ago
- Collection of autoregressive model implementation☆83Updated 3 weeks ago
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆100Updated 3 months ago
- QLoRA with Enhanced Multi GPU Support☆36Updated last year
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆82Updated last year
- some common Huggingface transformers in maximal update parametrization (µP)☆80Updated 3 years ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆123Updated 3 months ago
- ☆94Updated 9 months ago
- A set of Python scripts that makes your experience on TPU better☆50Updated 8 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆270Updated last year
- ☆194Updated this week
- Prune transformer layers☆68Updated 9 months ago