BlinkDL / LM-Trick-QuestionsLinks
Here we collect trick questions and failed tasks for open source LLMs to improve them.
☆31Updated 2 years ago
Alternatives and similar repositories for LM-Trick-Questions
Users that are interested in LM-Trick-Questions are comparing it to the libraries listed below
Sorting:
- Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business /…☆39Updated 2 years ago
- Structural Pruning for LLaMA☆54Updated 2 years ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated 2 years ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆99Updated last year
- ☆32Updated last year
- ☆34Updated last year
- Explorations into adversarial losses on top of autoregressive loss for language modeling☆38Updated 8 months ago
- ☆105Updated last year
- Here we will test various linear attention designs.☆61Updated last year
- Implementation of the Mamba SSM with hf_integration.☆56Updated last year
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆38Updated 4 months ago
- SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia☆40Updated 2 years ago
- Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs☆110Updated last year
- Tools for content datamining and NLP at scale☆44Updated last year
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆62Updated 2 years ago
- Griffin MQA + Hawk Linear RNN Hybrid☆89Updated last year
- DPO, but faster 🚀☆45Updated 10 months ago
- RWKV model implementation☆38Updated 2 years ago
- Repository for CPU Kernel Generation for LLM Inference☆26Updated 2 years ago
- Utilities for Training Very Large Models☆58Updated last year
- Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆39Updated 11 months ago
- GoldFinch and other hybrid transformer components☆45Updated last year
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆57Updated last year
- Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts☆119Updated last year
- ☆20Updated last year
- [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling☆39Updated last year
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆56Updated last week
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- A large-scale RWKV v7(World, PRWKV, Hybrid-RWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy…☆45Updated last week
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated last year