TylunasLi / fastllmLinks
纯c++的全平台llm加速库，支持python调用，支持chatglm-6B, llama, baichuan, moss基座，x86 / ARM
☆12Updated 3 months ago
Alternatives and similar repositories for fastllm
Users that are interested in fastllm are comparing it to the libraries listed below
Sorting:
- llms related stuff , including code, docs☆13Updated 9 months ago
- accelerate generating vector by using onnx model☆18Updated last year
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆28Updated last month
- 大模型推理框架加速，让 LLM 飞起来☆24Updated last year
- 收集优质的角色扮演聊天数据 | Collection of roleplay conversations of high quality☆15Updated last year
- gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。☆239Updated last week
- 部署你自己的OpenAI api🤩, 基于flask, transformers (使用 Baichuan2-13B-Chat-4bits 模型, 可以运行在单张Tesla T4显卡) ，实现了OpenAI中Chat, Models和Completions接口，包含流式响…☆96Updated 2 years ago
- 基于langchain设计的智能体任务， 包含规划会话场景资源，构建子任务，任务执行器包含（MCTS）☆31Updated last month
- A high-throughput and memory-efficient inference and serving engine for LLMs☆139Updated last year
- 一起来养一只拥有专属记忆的AI猫猫吧！☆10Updated last year
- ✅4g GPU可用 | 简易实现ChatGLM单机调用多个计算设备（GPU、CPU）进行推理☆34Updated 2 years ago
- ☆169Updated last year
- （1）弹性区间标准化的旋转位置词嵌入编码器+peft LORA量化训练，提高万级tokens性能支持。（2）证据理论解释学习，提升模型的复杂逻辑推理能力（3）兼容alpaca数据格式。☆45Updated 2 years ago
- pre-training llama3 using chinese☆13Updated last year
- ☆15Updated last year
- 解决知识库搜索需要手动编排流和意图识别的问题☆19Updated 5 months ago
- Alpaca Chinese Dataset -- 中文指令微调数据集☆217Updated last year
- 介绍docker、docker compose的使用。☆21Updated last year
- A fluent, scalable, and easy-to-use LLM data processing framework.☆26Updated last week
- ChatGPT WebUI using gradio. 给 LLM 对话和检索知识问答RAG提供一个简单好用的Web UI界面☆138Updated last year
- 利用LLM+敏感词库，来自动判别是否涉及敏感词。☆135Updated 2 years ago
- 打造人人都会的NLP，开源不易，记得star哦☆101Updated 2 years ago
- ☆58Updated last year
- bge推理优化相关脚本☆29Updated last year
- rwkv finetuning☆37Updated last year
- qwen models finetuning☆105Updated 9 months ago
- 实现Blip2RWKV+QFormer的多模态图文对话大模型，使用Two-Step Cognitive Psychology Prompt方法，仅3B参数的模型便能够出现类人因果思维链。对标MiniGPT-4，ImageBind等图文对话大语言模型，力求以更小的算力和资源实…☆40Updated 2 years ago
- SearchGPT: Building a quick conversation-based search engine with LLMs.☆47Updated 11 months ago
- You can play any API server that compatible with OpenAI API☆24Updated last year
- A high-throughput and memory-efficient inference and serving engine for LLMs☆132Updated last year