bojone / tigerLinks
A Tight-fisted Optimizer
☆50Updated 2 years ago
Alternatives and similar repositories for tiger
Users that are interested in tiger are comparing it to the libraries listed below
Sorting:
- Lion and Adam optimization comparison☆64Updated 2 years ago
- A Tight-fisted Optimizer (Tiger), implemented in PyTorch.☆12Updated last year
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆123Updated 8 months ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆62Updated 2 years ago
- 基于Gated Attention Unit的Transformer模型(尝鲜版)☆98Updated 2 years ago
- ☆106Updated last year
- Low-bit optimizers for PyTorch☆131Updated last year
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆102Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆58Updated last year
- [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling☆86Updated 2 years ago
- ☆14Updated last year
- Official implementation of TransNormerLLM: A Faster and Better LLM☆246Updated last year
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models☆328Updated 6 months ago
- ☆63Updated last year
- An Experiment on Dynamic NTK Scaling RoPE☆64Updated last year
- A repository for DenseSSMs☆88Updated last year
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆29Updated last year
- Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales☆32Updated 2 years ago
- Implementation of "Attention Is Off By One" by Evan Miller☆197Updated 2 years ago
- ☆83Updated 4 months ago
- ☆197Updated last year
- [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…☆67Updated last year
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆137Updated last year
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated last year
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆49Updated 2 years ago
- [ICML24] Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs☆92Updated 9 months ago
- [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408☆197Updated 2 years ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆78Updated last year
- Code for paper "Patch-Level Training for Large Language Models"☆87Updated 10 months ago
- Converting Mixtral-8x7B to Mixtral-[1~7]x7B☆22Updated last year