Hannibal046 / nanoRWKV
The nanoGPT-style implementation of RWKV Language Model - an RNN with GPT-level LLM performance.
☆193Updated last year
Related projects ⓘ
Alternatives and complementary repositories for nanoRWKV
- This is the official code repository of MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tas…☆63Updated 3 months ago
- [NeurIPS 2024] BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models☆215Updated this week
- [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding☆230Updated 2 months ago
- A recipe for online RLHF and online iterative DPO.☆436Updated 2 weeks ago
- The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models☆106Updated this week
- Evaluating LLMs with Dynamic Data☆72Updated 2 weeks ago
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models☆387Updated 9 months ago
- Official Code for "SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression"☆95Updated last month
- The official implementation of Self-Play Preference Optimization (SPPO)☆498Updated 3 months ago
- This tool(enhance_long) aims to enhance the LlaMa2 long context extrapolation capability in the lowest-cost approach, preferably without …☆47Updated 11 months ago
- Grimoire is All You Need for Enhancing Large Language Models☆117Updated 8 months ago
- MIXQ: Taming Dynamic Outliers in Mixed-Precision Quantization by Online Prediction☆83Updated 3 weeks ago
- Explore concepts like Self-Correct, Self-Refine, Self-Improve, Self-Contradict, Self-Play, and Self-Knowledge, alongside o1-like reasonin…☆162Updated last week
- A Comprehensive Benchmark for Code Information Retrieval.☆63Updated last month
- ☆115Updated last year
- WorldGPT: Empowering LLM as Multimodal World Model☆123Updated 3 months ago
- ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference☆126Updated 3 weeks ago
- Token Omission Via Attention☆121Updated last month
- RWKV, in easy to read code☆55Updated this week
- Support mixed-precsion inference with vllm☆97Updated 2 weeks ago
- This includes the original implementation of CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control.☆66Updated last month
- ☆14Updated 3 weeks ago
- The official implementation of the ICML 2024 paper "MemoryLLM: Towards Self-Updatable Large Language Models"☆91Updated last month
- ☆115Updated 4 months ago
- Griffin MQA + Hawk Linear RNN Hybrid☆85Updated 6 months ago
- We leverage 14 datasets as OOD test data and conduct evaluations on 8 NLU tasks over 21 popularly used models. Our findings confirm that …☆117Updated last year
- SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia☆41Updated last year
- Code and Checkpoints for "Generate rather than Retrieve: Large Language Models are Strong Context Generators" in ICLR 2023.☆278Updated last year
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆104Updated 2 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆61Updated 7 months ago