modelscope / data-juicer
A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！
☆2,957Updated this week
Related projects ⓘ
Alternatives and complementary repositories for data-juicer
- Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, D…☆4,289Updated this week
- An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention)☆2,666Updated this week
- OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, …☆4,141Updated this week
- 该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题☆1,502Updated 7 months ago
- ModelScope-Agent: An agent framework connecting models in ModelScope with the world☆2,722Updated last week
- Agent framework and applications built upon Qwen>=2.0, featuring Function Calling, Code Interpreter, RAG, and Chrome extension.☆3,505Updated last month
- We unified the interfaces of instruction-tuning data (e.g., CoT data), multiple LLMs and parameter-efficient methods (e.g., lora, p-tunin…☆2,619Updated 11 months ago
- 总结Prompt&LLM论文，开源数据&模型，AIGC应用☆2,685Updated this week
- A series of large language models developed by Baichuan Intelligent Technology☆4,092Updated last week
- LLMs interview notes and answers:该仓库主要记录大模型（LLMs）算法工程师相关的面试题和参考答案☆1,169Updated 11 months ago
- The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.☆721Updated this week
- LMDeploy is a toolkit for compressing, deploying, and serving LLMs.☆4,669Updated this week
- Netease Youdao's open-source embedding and reranker models for RAG products.☆1,487Updated 2 months ago
- An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)☆3,977Updated last week
- ⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SF…☆2,161Updated last year
- Daily updated LLM papers. 每日更新 LLM 相关的论文，欢迎订阅 👏 喜欢的话动动你的小手 🌟 一个☆979Updated 3 months ago
- 基于ChatGLM-6B、ChatGLM2-6B、ChatGLM3-6B模型，进行下游具体任务微调，涉及Freeze、Lora、P-tuning、全参微调等☆2,665Updated 11 months ago
- O1 Replication Journey: A Strategic Progress Report – Part I☆1,318Updated 3 weeks ago
- Best practice for training LLaMA models in Megatron-LM☆628Updated 10 months ago
- GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型☆5,294Updated last week
- ☆506Updated 2 weeks ago
- ☆878Updated 5 months ago
- 开源SFT数据集整理,随时补充☆445Updated last year
- 主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题☆3,778Updated 3 weeks ago
- 【LLMs九层妖塔】分享 LLMs在自然语言处理（ChatGLM、Chinese-LLaMA-Alpaca、小羊驼 Vicuna、LLaMA、GPT4ALL等）、信息检索（langchain）、语言合成、语言识别、多模态等领域（Stable Diffusion、MiniGP…☆1,798Updated 7 months ago
- Firefly: 大模型训练工具，支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、…☆5,865Updated 3 weeks ago
- Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]☆1,637Updated last year
- 利用HuggingFace的官方下载工具从镜像网站进行高速下载。☆831Updated last month
- 🩹Editing large language models within 10 seconds⚡☆1,284Updated last year
- 📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batc…☆2,845Updated this week