zhaochenyang20 / ModelServer
Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
☆13Updated last week
Related projects: ⓘ
- Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.☆14Updated last week
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆32Updated 8 months ago
- ☆57Updated 3 weeks ago
- aigc evals☆10Updated 9 months ago
- Manages vllm-nccl dependency☆17Updated 3 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆29Updated 6 months ago
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper☆21Updated 3 months ago
- We aim to provide the best references to search, select, and synthesize high-quality and large-quantity data for post-training your LLMs.☆38Updated 2 weeks ago
- ☆23Updated this week
- Automatic prompt optimization framework for multi-step agent tasks.☆15Updated 2 months ago
- ☆32Updated 3 months ago
- Representing Rule-based Chatbots with Transformers☆17Updated 2 months ago
- FuseAI Project☆75Updated last month
- Code Implementation, Evaluations, Documentation, Links and Resources for Min P paper☆15Updated 3 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆51Updated 5 months ago
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆48Updated last week
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆18Updated 3 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆121Updated 3 months ago
- ☆45Updated 7 months ago
- Large-scale exact string matching tool☆15Updated 11 months ago
- ☆58Updated 3 months ago
- ☆14Updated 11 months ago
- Recursive Abstractive Processing for Tree-Organized Retrieval☆11Updated 3 months ago
- the newest version of llama3,source code explained line by line using Chinese☆21Updated 5 months ago
- [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.☆65Updated 6 months ago
- ☆31Updated 4 months ago
- code for paper 《RankingGPT: Empowering Large Language Models in Text Ranking with Progressive Enhancement》☆28Updated 8 months ago
- 1.4B sLLM for Chinese and English - HammerLLM🔨☆43Updated 5 months ago
- ☆34Updated 2 weeks ago
- An Experiment on Dynamic NTK Scaling RoPE☆59Updated 9 months ago