Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post: sidsite.com/posts/bert-from-scratch
☆43May 20, 2025Updated 10 months ago
Alternatives and similar repositories for pretraining-BERT
Users that are interested in pretraining-BERT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆34Apr 23, 2023Updated 2 years ago
- Implementation of VQ-VAE with a GPT-style sampler in the JAX and Haiku ecosystem.☆12Nov 23, 2023Updated 2 years ago
- A collection of various discourse segmenters☆10Jun 30, 2017Updated 8 years ago
- Бэктестинг торговых стратегий с помощью библиотек на Python☆15May 26, 2025Updated 10 months ago
- Code for Augment & Reduce, a scalable stochastic algorithm for large categorical distributions☆10May 16, 2018Updated 7 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- codes and plots for "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"☆10Dec 30, 2024Updated last year
- My PhD Thesis (HTML)☆14Jan 29, 2021Updated 5 years ago
- ☆10Aug 26, 2022Updated 3 years ago
- Code for "Exponential Family Estimation via Adversarial Dynamics Embedding" (NeurIPS 2019)☆14Nov 26, 2019Updated 6 years ago
- Patch for MPT-7B which allows using and training a LoRA☆58May 20, 2023Updated 2 years ago
- Code for Semi-crowdsourced Clustering with Deep Generative Models☆12Dec 9, 2022Updated 3 years ago
- Code accompanying VarGrad: A Low-Variance Gradient Estimator for Variational Inference☆12Oct 12, 2020Updated 5 years ago
- A large-image collection explorer and fast classification tool☆24Jul 12, 2022Updated 3 years ago
- PyTorch utilities for ML, specifically speech☆13Jan 30, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Small repository for my video on LoRA☆16May 14, 2023Updated 2 years ago
- using AI model to infer patient phenotypes from identified named entities (instances of biomedical concepts)☆10Jan 13, 2023Updated 3 years ago
- readthedocs.org documentation for Inkplate boards☆10Aug 25, 2025Updated 7 months ago
- ☆12Apr 20, 2023Updated 2 years ago
- ☆19Jun 10, 2024Updated last year
- Unofficial Implementation of Selective Attention Transformer☆21Oct 31, 2024Updated last year
- code supplement for variational boosting (https://arxiv.org/abs/1611.06585)☆11Jul 24, 2017Updated 8 years ago
- ☆13Mar 10, 2026Updated last month
- Efficient 3bit/4bit quantization of LLaMA models☆18May 18, 2023Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆13Aug 15, 2024Updated last year
- Jax like function transformation engine but micro, microjax☆34Oct 25, 2024Updated last year
- Development of High-Throughput Polymer Network Atomistic Simulation☆25Updated this week
- Python interface for Enhanced Monte Carlo (EMC)☆23Mar 2, 2026Updated last month
- ☆26Apr 14, 2024Updated 2 years ago
- ☆33Apr 23, 2023Updated 2 years ago
- Analyze the waveform shape of neural oscillations.☆12Nov 14, 2018Updated 7 years ago
- A C-based checksec without readelf or grep dependance.☆11Apr 20, 2021Updated 4 years ago
- The non-user-of-rawdraw-facing side of rawdraw.☆12Jan 12, 2021Updated 5 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Tiny Encryption Algorithm implementation in C. It is a terminal program used to encrypt/decrypt files using 128 bit key.☆18Dec 2, 2021Updated 4 years ago
- Network Etiquette (Netiquette) -- Written with 2020 technology in mind☆10Nov 19, 2021Updated 4 years ago
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Mar 1, 2024Updated 2 years ago
- Training a BERT model from scratch.☆11Oct 15, 2023Updated 2 years ago
- ☆19Feb 11, 2022Updated 4 years ago
- GEMM and Winograd based convolutions using CUTLASS☆28Jul 15, 2020Updated 5 years ago
- A library for implementing radio astronomical Measurement Equations☆10Feb 16, 2016Updated 10 years ago