本项目从零开始构建并优化了一个千万参数级别的大规模预训练语言模型，涵盖预训练、有监督微调（SFT）和R1推理蒸馏三个阶段。项目采用自定义Transformer架构（包括RMSNorm、分组注意力、多Query机制、SwiGLU激活和RoPE位置编码），实现高效的长文本处理和自回归生成。同时，开发了基于BBPE编码的分词器，针对中文进行了优化，构建了轻量级词表。预训练阶段通过混合精度训练、梯度累积和Cosine Annealing学习率调度，实现全流程训练；微调阶段则利用开源SFT数据集，通过特殊损失掩码优化指令遵循；R1推理蒸馏阶段采用Deepseek-R1数据及损失加权策略，赋予模型逐步推理（慢思考）的能力，显著提升复杂任务的回答质量。
☆24Mar 10, 2025Updated last year
Alternatives and similar repositories for WeakWater-30M
Users that are interested in WeakWater-30M are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- hrtem lattice fringe image analysis algorithms - developed for carbon nanostructure analysis☆10Jan 9, 2026Updated 9 months ago
- ☆11Feb 25, 2022Updated 4 years ago
- A comprehensive evaluation framework for Backboard's memory system using the LoCoMo (Long Conversation Memory) benchmark.☆19Nov 3, 2025Updated last year
- Official implementation for "Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Sampling"☆10May 21, 2024Updated 2 years ago
- 本项目对Deepseek-R1-Distill-Qwen-7B进行心理咨询CoT数据的LoRA微调，以进一步提升Deepseek-R1-Distill-Qwen-7B在心理咨询领域的慢思考能力。☆12Mar 11, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- C++ implementation for 《"GrabCut" — Interactive Foreground Extraction using Iterated Graph Cuts》☆11Jul 25, 2023Updated 3 years ago
- ☆14Jun 30, 2026Updated 4 months ago
- 校园音乐征集投票系统 A system for electing annual school music☆10Updated this week
- ☆15Aug 31, 2025Updated last year
- 使用django对情感分析功能进行封装，里面包含使用情感词典和Bert模型进行情感分类，最后可以使用tensorFlow serving将模型部署在docker中运行。☆13Sep 23, 2019Updated 7 years ago
- [ECCV'24] UNIT: Backdoor Mitigation via Automated Neural Distribution Tightening☆10Dec 18, 2025Updated 10 months ago
- A Real-Time Network Intrusion Detection System based on Transformer-LSTM model (Major Project)☆16May 27, 2024Updated 2 years ago
- 使用分布式技术结合前端Vue框架，后端mycat+mybatis+springboot框架实现金融分布式交易系统，包括：开户，登录，信息查询，明细查询，一对一转账，计息，对账等金融交易的核心功能。☆13Sep 3, 2020Updated 6 years ago
- An android app which converts text/voice input to American Sign Language(ASL)☆13Sep 8, 2016Updated 10 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 反洗钱使用黑名单数据爬取☆16Jul 7, 2015Updated 11 years ago
- 依据最小宽度限定符方案(SmallestWidth 适配)，自动生成资源文件的Android Studio插件☆10Apr 27, 2026Updated 6 months ago
- 基于Open-AI的chatgpt-3.5turbo和Whisper-1实现的语音交互辩论机器人，利用Gradio部署，体验网址如下☆10Sep 15, 2023Updated 3 years ago
- 量化交易网站，软工三大作业迭代三，团队项目☆11Mar 8, 2018Updated 8 years ago
- Taurix OS kernel. Taurix 系统内核，操作系统原理实(xjb)践(写)☆12Dec 20, 2020Updated 5 years ago
- ☆15Oct 19, 2025Updated last year
- 本数据集旨在通过微调大语言模型来提升用户的批判性思维和推理能力。数据集包括结构化的问答环节、角色扮演场景和辩论，旨在引导用户进行分析、评估和构建有逻辑的论证。该数据集鼓励深度思考和反思性学习，特别适用于教育、辅导以及技能开发等领域的应用☆14Sep 15, 2024Updated 2 years ago
- 个人开发的敏捷开发平台☆10Jan 4, 2017Updated 9 years ago
- 基于大模型ChatGLM，微调方式为LORA，集SFT、RM、PPO算法为一体项目☆14Jun 20, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ETL调度管理平台☆12Nov 21, 2022Updated 3 years ago
- NewsApp包含客户端源码、服务端源码、数据库文件。 基于Miscrosoft人工智能项目ProjectOxford中的Recognition Emotion做的， 主要是基于用户的面部表情来推送不同类别的新闻。 Emotion API可以参考：https://www.p…☆10Mar 2, 2016Updated 10 years ago
- Classification and aggregation of russian news articles. University coursework.☆18Jan 21, 2019Updated 7 years ago
- Official implementation for "Instruction Tuning with Retrieval-based Examples Ranking for Aspect-based Sentiment Analysis"☆13Mar 23, 2026Updated 7 months ago
- [ICCV'25] Sparfels: Fast Reconstruction from Sparse Unposed Imagery☆16Jan 18, 2026Updated 9 months ago
- 本项目利用医学领域的 CoT 数据对 Deepseek-R1-Distill-Qwen-7B 进行微调，通过 QLoRA 量化和 Unsloth 加速训练，显著提升模型在复杂医学推理任务中的慢思考能力。知识蒸馏技术使轻量级模型获得大模型的推理优势，实现高效、准确且具有解释性…☆48Mar 10, 2025Updated last year
- ☆19Sep 7, 2025Updated last year
- 这是一个可通过网页远程登录管理、可接入讯飞星火、ChatGPT等大语言模型的微信聊天机器人，使用微信网页版协议。☆16Feb 20, 2024Updated 2 years ago
- 基于CNN的新浪新闻文本分类☆11Jul 22, 2019Updated 7 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆19Oct 9, 2026Updated 3 weeks ago
- Identify the type of news based on headlines and short descriptions☆17Mar 25, 2019Updated 7 years ago
- Testing the performance of CNN and BERT embeddings on GLUE tasks☆15Mar 24, 2023Updated 3 years ago
- 使用Sentencepiece对中文语料进行分词☆13Nov 30, 2023Updated 2 years ago
- ☆12Dec 18, 2024Updated last year
- 自动将GitHub用户批量渲染生成为README中的用户名&&头像墙 | Automatic batch rendering of GitHub users into usernames && avatar wall in README☆17Jun 17, 2022Updated 4 years ago
- 手势识别玩俄罗斯方块小游戏☆12Jul 20, 2021Updated 5 years ago