gordicaleksa / OpenGemini
Effort to open-source 10.5 trillion parameter Gemini model.
☆17Updated last year
Alternatives and similar repositories for OpenGemini:
Users that are interested in OpenGemini are comparing it to the libraries listed below
- Exploration into the Firefly algorithm in Pytorch☆38Updated 2 months ago
- Implementation of the LDP module block in PyTorch and Zeta from the paper: "MobileVLM: A Fast, Strong and Open Vision Language Assistant …☆16Updated last year
- JAX Implementation of Black Forest Labs' Flux.1 family of models☆31Updated 6 months ago
- Visualize multi-model embedding spaces. The first goal is to quickly get a lay of the land of any embedding space. Then be able to scroll…☆27Updated 11 months ago
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆29Updated last week
- Implementation of a Light Recurrent Unit in Pytorch☆47Updated 6 months ago
- ☆18Updated last year
- Explorations into adversarial losses on top of autoregressive loss for language modeling☆35Updated 2 months ago
- Collection of autoregressive model implementation☆85Updated this week
- Exploration into the Scaling Value Iteration Networks paper, from Schmidhuber's group☆36Updated 7 months ago
- GoldFinch and other hybrid transformer components☆10Updated 3 weeks ago
- The Gaussian Histogram Loss (HL-Gauss) proposed by Imani et al. with a few convenient wrappers for regression, in Pytorch☆58Updated 2 months ago
- Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"☆57Updated last year
- A library for simplifying fine tuning with multi gpu setups in the Huggingface ecosystem.☆16Updated 6 months ago
- Pytorch implementation of a simple way to enable (Stochastic) Frame Averaging for any network☆49Updated 9 months ago
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.☆17Updated last month
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆55Updated 11 months ago
- JAX Scalify: end-to-end scaled arithmetics☆16Updated 5 months ago
- Efficient encoder-decoder architecture for small language models (≤1B parameters) with cross-architecture knowledge distillation and visi…☆23Updated 2 months ago
- https://x.com/BlinkDL_AI/status/1884768989743882276☆27Updated 2 months ago
- Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public☆82Updated 2 months ago
- implementation of https://arxiv.org/pdf/2312.09299☆20Updated 9 months ago
- Exploring an idea where one forgets about efficiency and carries out attention across each edge of the nodes (tokens)☆50Updated last month
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.☆55Updated 3 weeks ago
- Training hybrid models for dummies.☆20Updated 3 months ago
- Toy genetic algorithm in Pytorch☆39Updated this week
- RWKV model implementation☆37Updated last year
- Pixel Parsing. A reproduction of OCR-free end-to-end document understanding models with open data☆21Updated 8 months ago
- Implementation of Spectral State Space Models☆16Updated last year
- Implementation of the proposed Spline-Based Transformer from Disney Research☆88Updated 5 months ago