RyokoAI / BigKnow2022

Related projects ⓘ

Alternatives and complementary repositories for BigKnow2022

• renll / SeqBoat
  [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling
  ☆35Updated 11 months ago
• yikangshen / megablocks
  ☆18Updated 5 months ago
• acosharma / elita-transformer
  Official Repository for Efficient Linear-Time Attention Transformers.
  ☆18Updated 5 months ago
• EleutherAI / rnngineering
  Engineering the state of RNN language models (Mamba, RWKV, etc.)
  ☆32Updated 5 months ago
• jenni-ai / T2FW
  Fine-Tuning Pre-trained Transformers into Decaying Fast Weights
  ☆19Updated 2 years ago
• allenai / sso
  Repository for Skill Set Optimization
  ☆12Updated 3 months ago
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆63Updated last year
• Doraemonzzz / nanoTransNormer
  ☆11Updated last year
• scottlogic-alex / prm800k-denorm
  Script for processing OpenAI's PRM800K process supervision dataset into an Alpaca-style instruction-response format
  ☆27Updated last year
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆29Updated last week
• recursal / GoldFinch-paper
  GoldFinch and other hybrid transformer components
  ☆39Updated 4 months ago
• OpenMOSE / RWKV-Infer
  A large-scale RWKV v6 inference with FLA . Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy on docker. Suppo…
  ☆16Updated last week
• COLM-org / Template
  ☆8Updated 3 months ago
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆40Updated last month
• xhan77 / in-context-alignment
  In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning
  ☆33Updated last year
• GSYfate / knnlm-limits
  Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"
  ☆18Updated 2 months ago
• RobertCsordas / moe_layer
  sigma-MoE layer
  ☆18Updated 10 months ago
• AlexWan0 / infini-gram
  An unofficial implementation of the Infini-gram model proposed by Liu et al. (2024)
  ☆24Updated 5 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆56Updated 6 months ago
• EleutherAI / mdl
  Minimum Description Length probing for neural network representations
  ☆16Updated last week
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 7 months ago
• cognitiveailab / BYTESIZED32
  Byte-sized text games for code generation tasks on virtual environments
  ☆17Updated 4 months ago
• amirzandieh / HyperAttention
  Triton Implementation of HyperAttention Algorithm
  ☆46Updated 11 months ago
• IBM / ColPret
  Efficient Scaling laws and collaborative pretraining.
  ☆13Updated this week
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆42Updated last week
• Zyphra / Zyda_processing
  ☆27Updated 5 months ago
• petezh / OpenD5
  Tasks for describing differences between text distributions.
  ☆16Updated 3 months ago
• google-research-datasets / swim-ir
  SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…
  ☆44Updated last year
• vaguenebula / AlpacaDataReflect
  An experiment to see if chatgpt can improve the output of the stanford alpaca dataset
  ☆12Updated last year