mlfoundations / dclm

Related projects: ⓘ

• XueFuzhao / OpenMoE
  A family of open-sourced Mixture-of-Experts (MoE) Large Language Models
  ☆1,352Updated 6 months ago
• huggingface / datatrove
  Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
  ☆1,935Updated last week
• pytorch / torchtune
  A Native-PyTorch Library for LLM Fine-tuning
  ☆3,942Updated this week
• S-LoRA / S-LoRA
  S-LoRA: Serving Thousands of Concurrent LoRA Adapters
  ☆1,698Updated 7 months ago
• gkamradt / LLMTest_NeedleInAHaystack
  Doing simple retrieval from LLM models at various context lengths to measure accuracy
  ☆1,451Updated last month
• deepseek-ai / DeepSeek-MoE
  DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
  ☆970Updated 8 months ago
• huggingface / nanotron
  Minimalistic large language model 3D-parallelism training
  ☆1,111Updated this week
• myshell-ai / JetMoE
  Reaching LLaMA2 Performance with 0.1M Dollars
  ☆955Updated last month
• casper-hansen / AutoAWQ
  AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:
  ☆1,624Updated this week
• deepseek-ai / DeepSeek-LLM
  DeepSeek LLM: Let there be answers
  ☆1,396Updated 7 months ago
• databricks / megablocks
  ☆1,164Updated last week
• karpathy / nano-llama31
  nanoGPT style version of Llama 3.1
  ☆1,162Updated last month
• microsoft / MInference
  To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up t…
  ☆696Updated last week
• arcee-ai / mergekit
  Tools for merging pretrained large language models.
  ☆4,501Updated this week
• sgl-project / sglang
  SGLang is a fast serving framework for large language models and vision language models.
  ☆5,121Updated this week
• predibase / lorax
  Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
  ☆2,080Updated this week
• mlabonne / llm-datasets
  High-quality datasets, tools, and concepts for LLM fine-tuning.
  ☆1,664Updated last month
• microsoft / Samba
  Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"
  ☆774Updated 3 weeks ago
• lucidrains / self-rewarding-lm-pytorch
  Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI
  ☆1,309Updated 5 months ago
• jiaweizzhao / GaLore
  GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
  ☆1,354Updated last week
• mit-han-lab / llm-awq
  [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
  ☆2,333Updated 2 months ago
• deepseek-ai / DeepSeek-Math
  DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
  ☆782Updated 5 months ago
• multimodal-art-projection / MAP-NEO
  ☆835Updated 2 months ago
• argilla-io / distilabel
  Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…
  ☆1,396Updated this week
• facebookresearch / chameleon
  Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.
  ☆1,756Updated last month
• zou-group / textgrad
  TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients.
  ☆1,542Updated last week
• facebookresearch / MobileLLM
  MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
  ☆929Updated 2 weeks ago
• stanfordnlp / pyreft
  ReFT: Representation Finetuning for Language Models
  ☆1,076Updated 2 weeks ago
• tencent-ailab / persona-hub
  Official repo for the paper "Scaling Synthetic Data Creation with 1,000,000,000 Personas"
  ☆769Updated 2 months ago
• AnswerDotAI / fsdp_qlora
  Training LLMs with QLoRA + FSDP
  ☆1,382Updated last week