Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆19Jul 20, 2023Updated 3 years ago
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆84Sep 9, 2023Updated 3 years ago
- ☆17Jul 10, 2023Updated 3 years ago
- train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism☆224Nov 21, 2023Updated 2 years ago
- ☆16Jan 30, 2022Updated 4 years ago
- A comparison of pretraining framework for LLM☆22Feb 6, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Code for the paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers" with GPT-J implementation.☆15Mar 22, 2023Updated 3 years ago
- chinese-laya☆17Oct 13, 2026Updated 3 weeks ago
- BLOOM 模型的指令微调☆24Jun 15, 2023Updated 3 years ago
- Debug DeepSpeed-Chat step by step in IDE (在IDE里一步一步调试DeepSpeed-Chat)☆10Apr 17, 2023Updated 3 years ago
- ☆11Dec 1, 2020Updated 5 years ago
- code scripts for blog posts I published☆12Jun 17, 2020Updated 6 years ago
- Data from "Crowdsourcing of Parallel Corpora: the Case of Style Transfer for Detoxification" paper☆14Apr 3, 2025Updated last year
- ☆11Mar 6, 2024Updated 2 years ago
- 持续追踪ChatGPT相关的技术资料和行业进展。☆12Apr 24, 2023Updated 3 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.☆68May 9, 2023Updated 3 years ago
- 收录实现中文版ChatGPT的各种技术路线，数据及其他资料☆35Jul 12, 2023Updated 3 years ago
- Text as a Universal User Representation for Transferable Personalization☆13Mar 4, 2026Updated 8 months ago
- ☆307Apr 6, 2023Updated 3 years ago
- chatglm-6b微调/LORA/PPO/推理, 样本为自动生成的整数/小数加减乘除运算, 可gpu/cpu☆165Aug 24, 2023Updated 3 years ago
- Retinal lesions segmentation using CNNs and adversarial training: A Degree Thesis Submitted to the Faculty of Escola Tècnica d’Enginyeria…☆10Oct 27, 2019Updated 7 years ago
- Automated detection of exudates from fundus images plays an important role in diabetic retinopathy (DR) screening and evaluation, for whi…☆11Dec 11, 2020Updated 5 years ago
- llms related stuff , including code, docs☆13Feb 25, 2025Updated last year
- Retrieves parquet files from Hugging Face, identifies and quantifies junky data, duplication, contamination, and biased content in datase…☆53Jul 6, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 基于PaddleNLP开源的抽取式UIE进行医学命名实体识别（torch实现）☆44Aug 5, 2022Updated 4 years ago
- chinese wwm masking and ngram masking based on jieba☆11Jul 25, 2019Updated 7 years ago
- A list of advice on doing research that is useful for me :)☆13Aug 17, 2019Updated 7 years ago
- This repository contains 2 tools: - A py3 Lib for NLP & image-caption metrics - Code for a two-tailed t-test with paired samples. It wil…☆18Apr 4, 2021Updated 5 years ago
- (NBCE)Naive Bayes-based Context Extension on ChatGLM-6b☆15Jun 7, 2023Updated 3 years ago
- EMNLP'22, CEM improves MHCH performance by correcting prediction bias and training an auxiliary cost simulator based on user state and la…☆11Oct 9, 2022Updated 4 years ago
- Implementation of Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems☆15Nov 11, 2023Updated 2 years ago
- DUDE: Document UnderstanDing of Everything Benchmark☆12Mar 27, 2023Updated 3 years ago
- ☆11Oct 8, 2022Updated 4 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- NTK scaled version of ALiBi position encoding in Transformer.☆69Aug 16, 2023Updated 3 years ago
- Resources for our AAAI 2022 paper: "Unsupervised Editing for Counterfactual Stories".☆13Oct 25, 2022Updated 4 years ago
- Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking☆13Feb 5, 2023Updated 3 years ago
- Control 3f robotiq gripper using python and modbus client☆15Jun 27, 2024Updated 2 years ago
- [ICDAR 2023] SelfDocSeg: A self-supervised vision-based approach towards Document Segmentation (Oral)☆43Oct 6, 2023Updated 3 years ago
- DataFountain第五届达观杯第4名方案☆11Dec 3, 2021Updated 4 years ago
- 怎么训练一个LLM分词器☆152Jul 13, 2023Updated 3 years ago