Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch
☆45May 20, 2025Updated last year
Alternatives and similar repositories for pretraining-BERT
Users that are interested in pretraining-BERT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆34Apr 23, 2023Updated 3 years ago
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Jan 25, 2023Updated 3 years ago
- Repository of the RANLP 2023 paper "Exploring the Landscape of Natural Language Processing Research".☆13Oct 20, 2024Updated last year
- Probe how GPT-n performs on statutory reasoning☆10Sep 17, 2024Updated 2 years ago
- Code for Augment & Reduce, a scalable stochastic algorithm for large categorical distributions☆10May 16, 2018Updated 8 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- codes and plots for "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"☆11Dec 30, 2024Updated last year
- Building language models to predict more than one token ahead to enable further ahead predictions.☆12May 22, 2025Updated last year
- Source code repository for our EMNLP paper on cross-domain claim identification☆14Oct 24, 2018Updated 7 years ago
- Patch for MPT-7B which allows using and training a LoRA☆57May 20, 2023Updated 3 years ago
- Библиотека-обертка, которая позволяет получить доступ к функционалу Quik из Python☆12Feb 16, 2024Updated 2 years ago
- JAX implementation of the T5 model: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer☆24Jun 10, 2023Updated 3 years ago
- Conversion script adapting vicuna dataset into alpaca format for use with oobabooga's trainer☆12Jun 21, 2023Updated 3 years ago
- ☆12Sep 5, 2021Updated 5 years ago
- EdX course from MIT on machine learning 6.86x☆11Dec 16, 2020Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Code accompanying VarGrad: A Low-Variance Gradient Estimator for Variational Inference☆12Oct 12, 2020Updated 5 years ago
- Use Huggingface Transformer and Tokenizers as Tensorflow Reusable SavedModels☆10Mar 29, 2022Updated 4 years ago
- [COLM 2026] An efficient 3D sampling method for long-CoT LLM.☆16May 25, 2025Updated last year
- Posterior with interesting shapes from actually used models☆13Feb 10, 2025Updated last year
- ☆12Oct 21, 2017Updated 8 years ago
- ☆11Jul 25, 2021Updated 5 years ago
- ☆13Mar 10, 2026Updated 6 months ago
- Tree-Invent: A novel molecular generative model constrained with topological tree☆14Jul 26, 2023Updated 3 years ago
- ☆10Jan 23, 2019Updated 7 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- PyTorch utilities for ML, specifically speech☆13Jan 30, 2024Updated 2 years ago
- ☆19Jun 12, 2026Updated 3 months ago
- Efficient 3bit/4bit quantization of LLaMA models☆18May 18, 2023Updated 3 years ago
- A Supabase MCP server compatible with cursor☆20Feb 13, 2025Updated last year
- ☆10Dec 4, 2018Updated 7 years ago
- Userspace USB driver for CAN to USB adapters - based on the Kvaser canlib API☆19Feb 22, 2020Updated 6 years ago
- Resources and documentation for UK Biobank to OMOP CDM v5.3.1 conversion☆10Oct 20, 2020Updated 5 years ago
- ☆18Nov 23, 2023Updated 2 years ago
- ☆11Dec 30, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- An implementation of Squared Earth-Mover's Distance loss for Neural Networks.☆14Mar 25, 2023Updated 3 years ago
- ☆12Jun 13, 2021Updated 5 years ago
- An attempt to create a free PROFINET daemon☆15Oct 24, 2018Updated 7 years ago
- The deribit_historical_trades repository gathers cryptocurrency (BTC, ETH, SOL, USDC) derivatives traded on the cryptocurrency derivative…☆24Mar 1, 2023Updated 3 years ago
- A simple to use and powerful web-interface to mess around with Meta's LLaMA LLM.☆16Mar 30, 2023Updated 3 years ago
- ☆19Jun 10, 2024Updated 2 years ago
- A fork of textgen that kept some things like Exllama and old GPTQ.☆22Aug 20, 2024Updated 2 years ago