BlinkDL / LM-Trick-Questions

Related projects: ⓘ

• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆55Updated 4 months ago
• BlinkDL / WorldModel
  Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business /…
  ☆40Updated last year
• codekansas / rwkv
  RWKV model implementation
  ☆38Updated last year
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆34Updated 10 months ago
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆38Updated 2 months ago
• horseee / LLaMA-Pruning
  Structural Pruning for LLaMA
  ☆55Updated last year
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆87Updated 8 months ago
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆53Updated 4 months ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆48Updated last week
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆101Updated last year
• Zyphra / Zyda_processing
  ☆22Updated 3 months ago
• lucidrains / ReST-EM-pytorch
  ☆42Updated this week
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆37Updated last year
• facebookresearch / RAM
  A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).
  ☆15Updated this week
• TheDuckAI / DUCK-datasets
  ☆19Updated this week
• Zyphra / Zamba2
  PyTorch implementation of models from the Zamba2 series.
  ☆63Updated last month
• BBuf / flash-rwkv
  ☆28Updated 3 months ago
• xhan77 / in-context-alignment
  In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning
  ☆33Updated last year
• renll / SeqBoat
  [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling
  ☆34Updated 9 months ago
• kyegomez / FastFF
  Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"
  ☆15Updated last week
• huyphan168 / PEER
  Mixture of A Million Experts
  ☆29Updated last month
• kyegomez / Paper-Implementation-Template
  A simple reproducible template to implement AI research papers
  ☆21Updated last week
• lucidrains / coordinate-descent-hierarchical-memory
  ☆14Updated this week
• lucidrains / memory-editable-transformer
  My explorations into editing the knowledge and memories of an attention network
  ☆34Updated last year
• LegallyCoder / mamba-hf
  Implementation of the Mamba SSM with hf_integration.
  ☆55Updated 2 weeks ago
• LAION-AI / riverbed
  Tools for content datamining and NLP at scale
  ☆41Updated 3 months ago
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆22Updated 3 months ago
• AlignInc / aligner-replication
  The reproduct of the paper - Aligner: Achieving Efficient Alignment through Weak-to-Strong Correction
  ☆21Updated 3 months ago
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆62Updated last year