BlinkDL / LM-Trick-Questions

Related projects ⓘ

Alternatives and complementary repositories for LM-Trick-Questions

• BlinkDL / WorldModel
  Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business /…
  ☆40Updated last year
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆56Updated 7 months ago
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆43Updated last year
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆36Updated last year
• Zyphra / Zyda_processing
  ☆28Updated 5 months ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆74Updated 6 months ago
• BBuf / RWKV-World-HF-Tokenizer
  ☆33Updated 4 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆19Updated 2 months ago
• lucasjinreal / ImageTokenizer
  imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…
  ☆29Updated 5 months ago
• codekansas / rwkv
  RWKV model implementation
  ☆38Updated last year
• nanowell / Q-Sparse-LLM
  My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
  ☆30Updated 3 months ago
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆53Updated 6 months ago
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆40Updated 4 months ago
• yikangshen / megablocks
  ☆18Updated 5 months ago
• proger / hippogriff
  Griffin MQA + Hawk Linear RNN Hybrid
  ☆85Updated 6 months ago
• xhan77 / in-context-alignment
  In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning
  ☆33Updated last year
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆109Updated last month
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆63Updated last year
• NolanoOrg / sparse_quant_llms
  SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia
  ☆41Updated last year
• BorealisAI / neuzip
  Official repository for the paper "NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks". This rep…
  ☆33Updated 3 weeks ago
• BlinkDL / modded-nanogpt-rwkv
  RWKV-7: Surpassing GPT
  ☆47Updated last week
• lucaslingle / mu_transformer
  Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.
  ☆29Updated 3 weeks ago
• huggingface / peft-pytorch-conference
  Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…
  ☆14Updated last year
• renll / SeqBoat
  [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling
  ☆35Updated 11 months ago
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆93Updated last month
• lucidrains / autoregressive-linear-attention-cuda
  CUDA implementation of autoregressive linear attention, with all the latest research findings
  ☆43Updated last year
• lucidrains / taylor-series-linear-attention
  Explorations into the recently proposed Taylor Series Linear Attention
  ☆90Updated 3 months ago
• kyegomez / FastFF
  Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"
  ☆15Updated 2 weeks ago