Alternatives and similar repositories for power-attention

Users that are interested in power-attention are comparing it to the libraries listed below

• LucasPrietoAl / grokking-at-the-edge-of-numerical-stability
  ☆98Updated 5 months ago
• xjdr-alt / muzero_sketch
  ☆38Updated 11 months ago
• PrimeIntellect-ai / pccl
  PCCL (Prime Collective Communications Library) implements fault tolerant collective communications over IP
  ☆95Updated last month
• OpenEvaByte / evabyte
  EvaByte: Efficient Byte-level Language Models at Scale
  ☆102Updated 2 months ago
• SinatrasC / entropix-smollm
  smolLM with Entropix sampler on pytorch
  ☆150Updated 7 months ago
• xjdr-alt / llmri
  look how they massacred my boy
  ☆63Updated 8 months ago
• Amplify-Partners / annotation-reading-list
  A reading list of relevant papers and projects on foundation model annotation
  ☆27Updated 4 months ago
• xjdr-alt / entropix-local
  smol models are fun too
  ☆93Updated 7 months ago
• google-deepmind / mishax
  ☆134Updated 2 months ago
• google-deepmind / neural_networks_solomonoff_induction
  Learning Universal Predictors
  ☆76Updated 10 months ago
• xrsrke / pipegoose
  Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
  ☆84Updated last year
• ScalingIntelligence / Archon
  Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.
  ☆173Updated 3 months ago
• Zyphra / tree_attention
  Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
  ☆126Updated 6 months ago
• doomslide / hyperobject
  Plotting (entropy, varentropy) for small LMs
  ☆97Updated last month
• pyember / ember
  ☆182Updated 2 months ago
• tyler-romero / microR1
  Simple repository for training small reasoning models
  ☆33Updated 4 months ago
• kanishkg / stream-of-search
  Repository for the paper Stream of Search: Learning to Search in Language
  ☆148Updated 4 months ago
• athms / mad-lab
  A MAD laboratory to improve AI architecture designs 🧪
  ☆122Updated 6 months ago
• jerber / arc_agi
  ☆54Updated 4 months ago
• google-deepmind / asyncdiloco
  ☆44Updated last year
• neoneye / ARC-Interactive-History-Dataset
  The history files when recording human interaction while solving ARC tasks
  ☆112Updated 2 weeks ago
• xjdr-alt / entropix-trainer
  train entropix like a champ!
  ☆20Updated 8 months ago
• N8python / n8loom
  A tree-based prefix cache library that allows rapid creation of looms: hierarchal branching pathways of LLM generations.
  ☆70Updated 4 months ago
• yixiaoer / tpux
  A set of Python scripts that makes your experience on TPU better
  ☆55Updated 11 months ago
• jenkspt / gpt-jax
  Jax/Flax rewrite of Karpathy's nanoGPT
  ☆57Updated 2 years ago
• MatX-inc / seqax
  seqax = sequence modeling + JAX
  ☆162Updated 2 weeks ago
• magicproduct / hash-hop
  Long context evaluation for large language models
  ☆217Updated 3 months ago
• young-geng / mintext
  Minimal but scalable implementation of large language models in JAX
  ☆35Updated 7 months ago
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆139Updated this week
• tiiuae / Falcon-H1
  All information and news with respect to Falcon-H1 series
  ☆38Updated this week