Pretraining codebase for Apertus models, based on Megatron-LM
☆23Oct 2, 2026Updated last week
Alternatives and similar repositories for pretrain-code
Users that are interested in pretrain-code are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- nanoGPT-like codebase for LLM training☆120Nov 7, 2025Updated 11 months ago
- Pretraining data reconstruction scripts for Apertus☆163Oct 27, 2025Updated 11 months ago
- Compressed sparse matrices☆15Jun 12, 2024Updated 2 years ago
- 249M-param MoE transformer built from scratch in PyTorch. GQA, RoPE, SwiGLU, sparse MoE with 3 aux losses, AMP training loop no Trainer a…☆19May 22, 2026Updated 4 months ago
- MIMIR Artificial Intelligence Platform is a framework for ingesting and processing your data sources to AI models as well as generating o…☆18Jul 9, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Open-source AI agent that reasons by blending many AI models, does real work on its own, and keeps learning — safe, self-hostable, and fu…☆26Updated this week
- MoMo: Momentum Models for Adaptive Learning Rates☆19Jun 12, 2024Updated 2 years ago
- Architected a distributed notebook execution environment simulating Kaggle Kernels. Utilized the Kubernetes K8s API for dynamic pod orche…☆20May 3, 2026Updated 5 months ago
- Source code for my PhD thesis: Backpropagation Beyond the Gradient☆21Feb 25, 2023Updated 3 years ago
- Distributed pretraining of large language models (LLMs) on cloud TPU slices, with Jax and Equinox.☆27Sep 29, 2024Updated 2 years ago
- Open CC BY 4.0 dataset: which local LLMs (Ollama) fit which hardware — params, quantization, RAM/VRAM. Powered by modelfit.io☆24Oct 2, 2026Updated last week
- VPN-over-WebSocket☆14Dec 31, 2018Updated 7 years ago
- Cycles agroecosystem model☆12Jun 11, 2026Updated 3 months ago
- Eurostat Big Data Hackathon 2021☆15Jan 15, 2026Updated 8 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- lets build claude code from scratch☆35May 13, 2026Updated 4 months ago
- Pytorch optimizers implementing Hilbert Constrained Gradient Descent☆19May 9, 2019Updated 7 years ago
- collection of example documents for use within cocalc's library☆17Sep 11, 2025Updated last year
- CVE-2023-28121 - WooCommerce Payments < 5.6.2 - Unauthenticated Privilege Escalation [ Mass Add Admin User ]☆11Jul 14, 2023Updated 3 years ago
- Minimal pretraining script for language modeling in PyTorch. Supporting torch compilation and DDP. It includes a model implementation and…☆50Apr 27, 2026Updated 5 months ago
- Official code for "Algorithmic Capabilities of Random Transformers" (NeurIPS 2024)☆16Sep 28, 2024Updated 2 years ago
- Notebooks for managing NeurIPS 2014 and analysing the NeurIPS experiment.☆13May 22, 2024Updated 2 years ago
- INWX Domrobot Ruby Client☆12Oct 17, 2025Updated 11 months ago
- My website/blog source code. Hosted on this repository/GitHub Pages.☆16Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Analyzing the Voynich Manuscript with computer vision☆12Sep 4, 2024Updated 2 years ago
- Jina VDR is a multilingual, multi-domain benchmark for visual document retrieval☆38Aug 4, 2025Updated last year
- [ICLR 2022] "Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How" by Yuning You, Yue Cao, Tianl…☆14Aug 19, 2022Updated 4 years ago
- NOMU: Neural Optimization-based Model Uncertainty☆10Feb 17, 2023Updated 3 years ago
- A PyTorch Implementation of the Sparsemax operator (https://arxiv.org/pdf/1803.09820.pdf)☆36Dec 26, 2022Updated 3 years ago
- Multi-Word Probabilistic based supertokenizer☆15May 15, 2025Updated last year
- A library for converting IP range to CIDRs or CIDR to IP range☆20Sep 15, 2026Updated 3 weeks ago
- This library contains various Apache Flink connectors to connect to AWS data sources and sinks.☆16Dec 5, 2023Updated 2 years ago
- Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks☆11Oct 21, 2022Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Installation Setup☆19Apr 15, 2026Updated 5 months ago
- Docker Compose based system for running remote browsers (including Flash and Java support) connected to web archives☆16Jun 10, 2021Updated 5 years ago
- A zero-dependency Python module for inspecting and converting coding-agent session files (.jsonl) — Claude Code, Codex, and Pi — into the…☆48Jul 10, 2026Updated 3 months ago
- ☆16Sep 24, 2026Updated 2 weeks ago
- A Web Platform API that allows websites to notify the browser about their users login status☆18Sep 18, 2024Updated 2 years ago
- Unleash the true power of scheduling☆39Sep 9, 2026Updated last month
- An environment based on JSBSIM aimed at one-to-one close air combat.☆14May 15, 2023Updated 3 years ago