Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆69Jul 20, 2023Updated 3 years ago
Alternatives and similar repositories for LLaMA-Megatron-DeepSpeed
Users that are interested in LLaMA-Megatron-DeepSpeed are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆84Sep 9, 2023Updated 3 years ago
- Best practice for training LLaMA models in Megatron-LM☆666Jan 2, 2024Updated 2 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆2,268Aug 14, 2025Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,450Mar 20, 2024Updated 2 years ago
- 2023 ABCI Llama-2 継続学習プロジェクト☆14Jan 22, 2024Updated 2 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- distributed trainer for LLMs☆591May 20, 2024Updated 2 years ago
- ☆19May 11, 2024Updated 2 years ago
- Official Code for 'EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification' - NAACL 2022☆23May 9, 2022Updated 4 years ago
- ☆16Jan 30, 2022Updated 4 years ago
- Ner with Bert☆16Apr 24, 2019Updated 7 years ago
- 实体链接过程中的一些相关代码☆10Jun 13, 2018Updated 8 years ago
- A crowd-powered database system, with SQL-like query interface, multi-goal optimization☆11Sep 4, 2017Updated 9 years ago
- Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat☆117Jun 5, 2023Updated 3 years ago
- MNVBC项目裁判文书的清洗代码☆12Sep 11, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Code and data for "An Accurate Unsupervised Method for Joint Entity Alignment and Dangling Entity Detection".☆15Mar 26, 2022Updated 4 years ago
- ☆26May 30, 2023Updated 3 years ago
- A self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters…☆215May 28, 2024Updated 2 years ago
- NTK scaled version of ALiBi position encoding in Transformer.☆69Aug 16, 2023Updated 3 years ago
- Official code repository for the main conference paper in EMNLP 2022: SubeventWriter: Iterative Sub-event Sequence Generation with Cohere…☆11Oct 16, 2022Updated 4 years ago
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆98Feb 5, 2024Updated 2 years ago
- The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.☆1,602Dec 15, 2025Updated 11 months ago
- Dataset for Findings of ACL 23 "VCSum: A Versatile Chinese Meeting Summarization Dataset"☆53Jul 25, 2023Updated 3 years ago
- Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.☆1,021Jul 29, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Baseline models for the paper: "Modeling Naive Psychology of Characters in Simple Commonsense Stories" by Hannah Rashkin, Antoine Bosselu…☆16Feb 23, 2021Updated 5 years ago
- CLIPPER: Compression enables long-context synthetic data generation [COLM '25]☆11Apr 16, 2025Updated last year
- Code for ACL2024 paper - Adversarial Preference Optimization (APO).☆55Jun 3, 2024Updated 2 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆19Jul 20, 2023Updated 3 years ago
- Bridge Megatron-Core to Hugging Face/Reinforcement Learning☆236Oct 31, 2026Updated 2 weeks ago
- Data preparation code for CrystalCoder 7B LLM☆46May 10, 2024Updated 2 years ago
- A LLaMA1/LLaMA12 Megatron implement.☆28Dec 13, 2023Updated 2 years ago
- Implementation for https://arxiv.org/abs/2005.00652☆27Dec 8, 2022Updated 3 years ago
- Started as a Team Project for CS690D at UMass Amherst, now turning into pytorch implementation of hyperbolic neural networks using Poinca…☆12Dec 8, 2022Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆49Feb 2, 2024Updated 2 years ago
- WideMLP for Text Classification☆29Apr 21, 2026Updated 6 months ago
- The "GPT-API-Accelerate" project provides a set of Python classes for accelerating the process of generating responses to prompts using t…☆23Oct 12, 2024Updated 2 years ago
- WebApp1k benchmark☆14Nov 21, 2025Updated 11 months ago
- InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning☆293Aug 20, 2023Updated 3 years ago
- ☆43Mar 29, 2023Updated 3 years ago
- Domain-specific preference (DSP) data and customized RM fine-tuning.☆26Mar 7, 2024Updated 2 years ago
Are these results useful?