Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch
☆45May 20, 2025Updated last year
Alternatives and similar repositories for pretraining-BERT
Users that are interested in pretraining-BERT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Implementation of VQ-VAE with a GPT-style sampler in the JAX and Haiku ecosystem.☆11Nov 23, 2023Updated 2 years ago
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Jan 25, 2023Updated 3 years ago
- ☆14Mar 28, 2025Updated last year
- Repository of the RANLP 2023 paper "Exploring the Landscape of Natural Language Processing Research".☆13Oct 20, 2024Updated last year
- Code for Augment & Reduce, a scalable stochastic algorithm for large categorical distributions☆10May 16, 2018Updated 8 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Learning to Skip the Middle Layers of Transformers☆17Aug 7, 2025Updated 11 months ago
- JAX implementation of the T5 model: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer☆24Jun 10, 2023Updated 3 years ago
- Jupyter notebooks from our weekly (or so) hackathons☆11Dec 3, 2024Updated last year
- Библиотека-обертка, которая позволяет получить доступ к функционалу Quik из Python☆12Feb 16, 2024Updated 2 years ago
- Code for "Exponential Family Estimation via Adversarial Dynamics Embedding" (NeurIPS 2019)☆14Nov 26, 2019Updated 6 years ago
- Code for Semi-crowdsourced Clustering with Deep Generative Models☆12Dec 9, 2022Updated 3 years ago
- Author's implementation of ReBRAC, a minimalist improvement upon TD3+BC☆19Oct 22, 2023Updated 2 years ago
- A large-image collection explorer and fast classification tool☆24Jul 12, 2022Updated 3 years ago
- Code and data for EMNLP2016 article "What makes a convincing argument? Empirical analysis and detecting attributes of convincingness in W…☆13Nov 9, 2016Updated 9 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆12Oct 21, 2017Updated 8 years ago
- A framework for steering MoE models by detecting and controlling behavior-linked experts.☆35Sep 12, 2025Updated 9 months ago
- ☆11Jul 25, 2021Updated 4 years ago
- Posterior with interesting shapes from actually used models☆13Feb 10, 2025Updated last year
- LimeSoup is a package to parse HTML or XML papers from different publishers.☆20Jan 4, 2021Updated 5 years ago
- Some examples and tests with LicheeRV Nano☆37Aug 9, 2025Updated 10 months ago
- PyTorch utilities for ML, specifically speech☆13Jan 30, 2024Updated 2 years ago
- Tree-Invent: A novel molecular generative model constrained with topological tree☆14Jul 26, 2023Updated 2 years ago
- The deribit_historical_trades repository gathers cryptocurrency (BTC, ETH, SOL, USDC) derivatives traded on the cryptocurrency derivative…☆24Mar 1, 2023Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆12Jun 13, 2021Updated 5 years ago
- An automatic Gaussian process classifier.☆13May 28, 2016Updated 10 years ago
- An attempt to create a free PROFINET daemon☆15Oct 24, 2018Updated 7 years ago
- An open source in memory Graph Database for Social Networks☆10Sep 20, 2022Updated 3 years ago
- Resources and documentation for UK Biobank to OMOP CDM v5.3.1 conversion☆10Oct 20, 2020Updated 5 years ago
- ☆12Apr 20, 2023Updated 3 years ago
- A simple to use and powerful web-interface to mess around with Meta's LLaMA LLM.☆16Mar 30, 2023Updated 3 years ago
- Unofficial Implementation of Selective Attention Transformer☆20Oct 31, 2024Updated last year
- code supplement for variational boosting (https://arxiv.org/abs/1611.06585)☆11Jul 24, 2017Updated 8 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Approximate Bayesian Inference Toolkit (Python, C++)☆14Apr 16, 2014Updated 12 years ago
- Efficient 3bit/4bit quantization of LLaMA models☆18May 18, 2023Updated 3 years ago
- ☆13Mar 10, 2026Updated 3 months ago
- ☆13Aug 15, 2024Updated last year
- This is the repository for the single-cell transcriptomics application of the Deep Generative Decoder (DGD), developed by the Krogh group…☆15Aug 8, 2024Updated last year
- Jax like function transformation engine but micro, microjax☆34Oct 25, 2024Updated last year
- Proxy server with bol-van/zapret DPI bypass☆35Apr 18, 2025Updated last year