Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch
☆45May 20, 2025Updated last year
Alternatives and similar repositories for pretraining-BERT
Users that are interested in pretraining-BERT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Code for 'Answer Matching Outperforms Multiple Choice for Language Model Evaluation' paper☆18Jul 4, 2025Updated last year
- ☆25Sep 3, 2024Updated last year
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Jan 25, 2023Updated 3 years ago
- Repository of the RANLP 2023 paper "Exploring the Landscape of Natural Language Processing Research".☆13Oct 20, 2024Updated last year
- Probe how GPT-n performs on statutory reasoning☆10Sep 17, 2024Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Code for Augment & Reduce, a scalable stochastic algorithm for large categorical distributions☆10May 16, 2018Updated 8 years ago
- Building language models to predict more than one token ahead to enable further ahead predictions.☆12May 22, 2025Updated last year
- Jupyter notebooks from our weekly (or so) hackathons☆11Dec 3, 2024Updated last year
- Code for "Exponential Family Estimation via Adversarial Dynamics Embedding" (NeurIPS 2019)☆14Nov 26, 2019Updated 6 years ago
- Conversion script adapting vicuna dataset into alpaca format for use with oobabooga's trainer☆12Jun 21, 2023Updated 3 years ago
- Code accompanying VarGrad: A Low-Variance Gradient Estimator for Variational Inference☆12Oct 12, 2020Updated 5 years ago
- Code and data for EMNLP2016 article "What makes a convincing argument? Empirical analysis and detecting attributes of convincingness in W…☆13Nov 9, 2016Updated 9 years ago
- [COLM 2026] An efficient 3D sampling method for long-CoT LLM.☆16May 25, 2025Updated last year
- LimeSoup is a package to parse HTML or XML papers from different publishers.☆20Jan 4, 2021Updated 5 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Posterior with interesting shapes from actually used models☆13Feb 10, 2025Updated last year
- Tree-Invent: A novel molecular generative model constrained with topological tree☆14Jul 26, 2023Updated 3 years ago
- PyTorch utilities for ML, specifically speech☆13Jan 30, 2024Updated 2 years ago
- ☆10Dec 4, 2018Updated 7 years ago
- Small repository for my video on LoRA☆16May 14, 2023Updated 3 years ago
- An automatic Gaussian process classifier.☆13May 28, 2016Updated 10 years ago
- Userspace USB driver for CAN to USB adapters - based on the Kvaser canlib API☆19Feb 22, 2020Updated 6 years ago
- Resources and documentation for UK Biobank to OMOP CDM v5.3.1 conversion☆10Oct 20, 2020Updated 5 years ago
- using AI model to infer patient phenotypes from identified named entities (instances of biomedical concepts)☆10Jan 13, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- An implementation of Squared Earth-Mover's Distance loss for Neural Networks.☆14Mar 25, 2023Updated 3 years ago
- An attempt to create a free PROFINET daemon☆15Oct 24, 2018Updated 7 years ago
- ☆12Apr 20, 2023Updated 3 years ago
- A simple to use and powerful web-interface to mess around with Meta's LLaMA LLM.☆16Mar 30, 2023Updated 3 years ago
- ☆19Jun 10, 2024Updated 2 years ago
- Unofficial Implementation of Selective Attention Transformer☆20Oct 31, 2024Updated last year
- Python interface for Enhanced Monte Carlo (EMC)☆24Jul 13, 2026Updated 2 weeks ago
- code supplement for variational boosting (https://arxiv.org/abs/1611.06585)☆11Jul 24, 2017Updated 9 years ago
- Scalable In-Memory Acceleration With Mesh: Device, Circuits, Architecture, and Algorithm☆15Oct 11, 2020Updated 5 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆17Sep 6, 2025Updated 10 months ago
- ☆33Apr 23, 2023Updated 3 years ago
- C++ implementation for Sequence Pair fixed-outline chip floorplanner☆12Dec 27, 2022Updated 3 years ago
- ☆28Apr 14, 2024Updated 2 years ago
- ☆19Apr 25, 2023Updated 3 years ago
- Learning to Skip the Middle Layers of Transformers☆17Aug 7, 2025Updated 11 months ago
- Fork of Enzyme to work on Reverse-Mode Differentiation at the MLIR-level.☆11Apr 23, 2023Updated 3 years ago