00ffcc / chunkRWKV6

Related projects ⓘ

Alternatives and complementary repositories for chunkRWKV6

• BBuf / flash-rwkv
  ☆29Updated 5 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆56Updated 6 months ago
• berlino / gated_linear_attention
  ☆98Updated 8 months ago
• HazyResearch / prefix-linear-attention
  ☆45Updated 4 months ago
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆61Updated 6 months ago
• sustcsonglin / mamba-triton
  ☆45Updated 9 months ago
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆34Updated last week
• habanero-lab / APPy
  APPy (Annotated Parallelism for Python) enables users to annotate loops and tensor expressions in Python with compiler directives akin to…
  ☆20Updated last week
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆22Updated 11 months ago
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆57Updated 5 months ago
• sjelassi / transformers_ssm_copy
  ☆24Updated 8 months ago
• linfeng93 / BiTA
  An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.
  ☆22Updated 9 months ago
• LeC-Z / RWKV-nonogram
  A 20M RWKV v6 can do nonogram
  ☆11Updated last month
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 7 months ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆32Updated last month
• xiayuqing0622 / customized-flash-attention
  Fast and memory-efficient exact attention
  ☆27Updated last week
• Doraemonzzz / Awesome-Triton-Resources
  Awesome Triton Resources
  ☆18Updated last month
• dame-cell / Triformer
  Transformers components but in Triton
  ☆27Updated this week
• glassroom / heinsen_attention
  Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
  ☆24Updated 5 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆49Updated 3 months ago
• FasterDecoding / TEAL
  ☆96Updated last month
• shaochenze / PatchTrain
  Code for paper "Patch-Level Training for Large Language Models"
  ☆71Updated this week
• INT-FlashAttention2024 / INT-FlashAttention
  ☆47Updated 2 months ago
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆98Updated 5 months ago
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆50Updated last week
• Edward-Sun / gpt-accelera
  Simple and efficient pytorch-native transformer training and inference (batched)
  ☆61Updated 7 months ago
• kyegomez / FlashAttention20Triton
  Triton implementation of Flash Attention2.0
  ☆22Updated last year
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)
  ☆76Updated last month
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆19Updated 5 months ago
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆53Updated 3 weeks ago