Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch
☆45May 20, 2025Updated last year
Alternatives and similar repositories for pretraining-BERT
Users that are interested in pretraining-BERT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆26Sep 3, 2024Updated 2 years ago
- Code for Augment & Reduce, a scalable stochastic algorithm for large categorical distributions☆10May 16, 2018Updated 8 years ago
- codes and plots for "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"☆11Dec 30, 2024Updated last year
- Building language models to predict more than one token ahead to enable further ahead predictions.☆12May 22, 2025Updated last year
- ☆10Aug 26, 2022Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- My PhD Thesis (HTML)☆14Jan 29, 2021Updated 5 years ago
- Jupyter notebooks from our weekly (or so) hackathons☆11Dec 3, 2024Updated last year
- Source code repository for our EMNLP paper on cross-domain claim identification☆14Oct 24, 2018Updated 7 years ago
- JAX implementation of the T5 model: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer☆24Jun 10, 2023Updated 3 years ago
- ☆16Apr 26, 2023Updated 3 years ago
- Code accompanying VarGrad: A Low-Variance Gradient Estimator for Variational Inference☆12Oct 12, 2020Updated 6 years ago
- Code and data for EMNLP2016 article "What makes a convincing argument? Empirical analysis and detecting attributes of convincingness in W…☆13Nov 9, 2016Updated 9 years ago
- A large-image collection explorer and fast classification tool☆24Jul 12, 2022Updated 4 years ago
- [COLM 2026] An efficient 3D sampling method for long-CoT LLM.☆16May 25, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Posterior with interesting shapes from actually used models☆13Feb 10, 2025Updated last year
- Tree-Invent: A novel molecular generative model constrained with topological tree☆14Jul 26, 2023Updated 3 years ago
- ☆13Mar 10, 2026Updated 7 months ago
- The very easy BERT pretrain process by using tokenizers and transformers repos☆32Feb 27, 2020Updated 6 years ago
- [Nat. Commun.] Code for paper 'Attention-based multi-label neural networks for integrated prediction and interpretation of twelve widely …☆33Oct 23, 2021Updated 4 years ago
- Userspace USB driver for CAN to USB adapters - based on the Kvaser canlib API☆19Feb 22, 2020Updated 6 years ago
- An automatic Gaussian process classifier.☆13May 28, 2016Updated 10 years ago
- ☆12Apr 20, 2023Updated 3 years ago
- ☆19Jun 10, 2024Updated 2 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Documentation and guidelines for the Alan GPU cluster at the University of Liège.☆22Jul 19, 2023Updated 3 years ago
- Unofficial Implementation of Selective Attention Transformer☆20Oct 31, 2024Updated last year
- Approximate Bayesian Inference Toolkit (Python, C++)☆14Apr 16, 2014Updated 12 years ago
- ☆13Aug 15, 2024Updated 2 years ago
- ☆11Sep 28, 2021Updated 5 years ago
- ☆20Sep 6, 2025Updated last year
- Python validation, summary, and analysis tools for HED (Hierarchical Event Descriptors).☆18Updated this week
- Python interface for Enhanced Monte Carlo (EMC)☆26Oct 5, 2026Updated last week
- Development of High-Throughput Polymer Network Atomistic Simulation☆35Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Jax like function transformation engine but micro, microjax☆34Oct 25, 2024Updated last year
- ☆28Apr 14, 2024Updated 2 years ago
- Fast string distance computation in Python☆16Dec 21, 2022Updated 3 years ago
- A C-based checksec without readelf or grep dependance.☆11Apr 20, 2021Updated 5 years ago
- Official Implementation of TMLR's paper: "TabCBM: Concept-based Interpretable Neural Networks for Tabular Data"☆16Apr 4, 2024Updated 2 years ago
- Implementations of Normalizing Flows in Pytorch/Pyro☆17Apr 20, 2020Updated 6 years ago
- ☆12Nov 6, 2023Updated 2 years ago