SparkJiao / llama-pipeline-parallelView external linksLinks
A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to copy code and launch discussions about the problems you have encoured.
☆57Jul 4, 2023Updated 2 years ago
Alternatives and similar repositories for llama-pipeline-parallel
Users that are interested in llama-pipeline-parallel are comparing it to the libraries listed below
Sorting:
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆224Nov 21, 2023Updated 2 years ago
- ☆14Dec 28, 2022Updated 3 years ago
- [EMNLP 2024] Source code for the paper "Learning Planning-based Reasoning with Trajectory Collection and Process Rewards Synthesizing".☆83Jan 14, 2025Updated last year
- 微调阿里开源的文字检测模型，利用合合识别返回的OCR结果作为初始训练数据，对模型进行优化训练，使其更加适应1万张图片的具体场景，提高文字识别的精度。☆10Dec 9, 2024Updated last year
- Mosaic Representation Learning for Self-supervised Visual Pre-training (ICLR2023, Spotlight)☆15Apr 7, 2023Updated 2 years ago
- ☆19Jul 24, 2025Updated 6 months ago
- Simhash and near-duplicate detection☆17Dec 6, 2013Updated 12 years ago
- An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).☆59Aug 13, 2024Updated last year
- [ICLR 2022] Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators☆26Jul 26, 2023Updated 2 years ago
- ☆23May 30, 2022Updated 3 years ago
- Solana Airdrop Faucet: A simple web application that allows users to receive free SOL tokens on the Solana Devnet. Built with Next.js, th…☆11Sep 22, 2024Updated last year
- Source code for ICLR 2021 paper : Pre-training Text-to-Text Transformers for Concept-Centric Common Sense☆27Sep 16, 2021Updated 4 years ago
- [Findings of ACL 2022] Meta-Path Guided Contrastive Learning for Logical Reasoning of Text☆28Mar 21, 2022Updated 3 years ago
- The Web Conference 2020: Structure-Feature based Graph Self-adaptive Pooling☆30Apr 21, 2020Updated 5 years ago
- 高性能文本 Tokenizer 库☆32Feb 2, 2024Updated 2 years ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆40Jan 4, 2024Updated 2 years ago
- Chaucha functions for usage with Github Actions☆11Sep 18, 2020Updated 5 years ago
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆752Sep 27, 2024Updated last year
- This is the pytorch implementation of the long paper on ACL 2020: A Self-Training Method for Machine Reading Comprehension with Soft Evid…☆34Aug 14, 2020Updated 5 years ago
- Official pytorch implementation of the IrwGAN for unaligned image-to-image translation☆34Dec 15, 2021Updated 4 years ago
- Run very cheap game servers (Minecraft, LinuxGSM, etc) in AWS EC2 Spot instances controlled by Discord slash commands. Manage using Terra…☆13Aug 6, 2025Updated 6 months ago
- Website for www.ambitionfund.org, micro-grant program to provide support for underrepresented people who need financial assistance pursui…☆13Jan 6, 2023Updated 3 years ago
- ☆10Feb 10, 2022Updated 4 years ago
- ☆10Dec 24, 2021Updated 4 years ago
- Experimental framework taking inspiration from biological systems, combining compression-based architectures, group theory, and symmetry …☆14Nov 13, 2025Updated 3 months ago
- ☆10Oct 18, 2021Updated 4 years ago
- プログラミング de 落書きで公開しているコードを共有します☆24Updated this week
- ☆37Jan 27, 2022Updated 4 years ago
- Create immutable infrastructure with IaC technologies at AWS with Terraform and Serverless Framework ☁️ The main services used are Dynamo…☆10Jan 3, 2021Updated 5 years ago
- Self service portal for aws workspace☆10Dec 10, 2023Updated 2 years ago
- Tracking Of Agent (actions and belief) and Spatio-TEmporal Reasoning☆14Feb 7, 2020Updated 6 years ago
- A framework for few-shot evaluation of autoregressive language models.☆12Jul 14, 2025Updated 7 months ago
- ☆84Sep 9, 2023Updated 2 years ago
- 定时检索 arXiv（按学科/关键词），自动抽取标题/作者/会议/时间/链接，生成 JSON/Markdown/网页，支持邮件推送与可选 LLM 中英双语摘要。Scheduled arXiv tracker (by categories/keywords) that ext…☆25Updated this week
- Experiment for lsat☆49Jan 20, 2023Updated 3 years ago
- NeurIPS 2021, "Fine Samples for Learning with Noisy Labels"☆41Nov 29, 2021Updated 4 years ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆48Mar 7, 2024Updated last year
- Code for the paper "Query-Key Normalization for Transformers"☆51Mar 6, 2021Updated 4 years ago
- Ring attention implementation with flash attention☆980Sep 10, 2025Updated 5 months ago