BlinkDL / LM-Trick-QuestionsLinks
Here we collect trick questions and failed tasks for open source LLMs to improve them.
☆31Updated 2 years ago
Alternatives and similar repositories for LM-Trick-Questions
Users that are interested in LM-Trick-Questions are comparing it to the libraries listed below
Sorting:
- Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business /…☆39Updated 2 years ago
- Here we will test various linear attention designs.☆58Updated last year
- RWKV model implementation☆38Updated last year
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆62Updated last year
- https://x.com/BlinkDL_AI/status/1884768989743882276☆28Updated last month
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated last year
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆37Updated last year
- Implementation of the Mamba SSM with hf_integration.☆56Updated 9 months ago
- Explorations into adversarial losses on top of autoregressive loss for language modeling☆36Updated 3 months ago
- A large-scale RWKV v6, v7(World, PRWKV, Hybrid-RWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to de…☆35Updated this week
- GoldFinch and other hybrid transformer components☆45Updated 10 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆28Updated 8 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆97Updated 8 months ago
- ☆45Updated 3 months ago
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated 11 months ago
- The reproduct of the paper - Aligner: Achieving Efficient Alignment through Weak-to-Strong Correction☆22Updated last year
- Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response format☆27Updated last year
- Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆41Updated 6 months ago
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆47Updated last month
- Griffin MQA + Hawk Linear RNN Hybrid☆86Updated last year
- Tools for content datamining and NLP at scale☆43Updated 11 months ago
- ☆20Updated last year
- ☆31Updated last year
- Structural Pruning for LLaMA☆54Updated 2 years ago
- Repository for "TESS-2: A Large-Scale, Generalist Diffusion Language Model"☆35Updated 3 months ago
- DPO, but faster 🚀☆42Updated 5 months ago
- Un-*** 50 billions multimodality dataset☆22Updated 2 years ago
- My explorations into editing the knowledge and memories of an attention network☆35Updated 2 years ago
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆56Updated last year
- A repository for research on medium sized language models.☆76Updated last year