conceptofmind / PaLM-flax
Implementation of the SOTA Transformer architecture from PaLM - Scaling Language Modeling with Pathways in JAX/Flax
☆14Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for PaLM-flax
- Implementation of a holodeck, written in Pytorch☆17Updated last year
- Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)☆185Updated 2 years ago
- Serialize JAX, Flax, Haiku, or Objax model params with 🤗`safetensors`☆42Updated 5 months ago
- ☆14Updated last year
- JAX implementations of RWKV☆19Updated last year
- Visual search interface☆11Updated 2 years ago
- Engineering the state of RNN language models (Mamba, RWKV, etc.)☆32Updated 5 months ago
- GPT-jax based on the official huggingface library☆13Updated 3 years ago
- Simple implementation of a GPT (training and inference) in PyTorch.☆10Updated 10 months ago
- LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence☆45Updated 2 years ago
- ☆16Updated 2 years ago
- Implementation of Stable Diffusion from scratch [WORK IN PROGRESS]☆21Updated last year
- ☆29Updated 2 weeks ago
- A client library for LAION's effort to filter CommonCrawl with CLIP, building a large scale image-text dataset.☆31Updated last year
- DiCE: The Infinitely Differentiable Monte-Carlo Estimator☆30Updated last year
- Implementation of SelfExtend from the paper "LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning" from Pytorch and Zeta☆13Updated last week
- An EXA-Scale repository of Multi-Modality AI resources from papers and models, to foundational libraries!☆41Updated 9 months ago
- My own attempt at a long context genomics model, leveraging recent advances in long context attention modeling (Flash Attention + other h…☆51Updated last year
- ☆29Updated 2 years ago
- The 2D discrete wavelet transform for JAX☆38Updated last year
- A simple way to manage and store the data related to all your research papers!☆16Updated last year
- A library for simplifying fine tuning with multi gpu setups in the Huggingface ecosystem.☆15Updated 2 weeks ago
- RWKV model implementation☆38Updated last year
- Implementation of Metaformer, but in an autoregressive manner☆23Updated 2 years ago
- A port of muP to JAX/Haiku☆25Updated 2 years ago