Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"
☆665Feb 24, 2025Updated last year
Alternatives and similar repositories for Craw4LLM
Users that are interested in Craw4LLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- This repository contains the resource introduced in the paper: "Truth or Mirage? Towards End-to-End Factuality Evaluation with LLM-Oasis"…☆25Oct 15, 2025Updated 11 months ago
- NeuraPress 是一个现代化的 Markdown 编辑器，专注于提供优质的微信公众号排版体验。响应式设计，支持移动设备。搭配 DeepSeek和微信公众号助手使用，碎片时间也能用手机发有排版的文章了。☆1,838Apr 21, 2026Updated 5 months ago
- Summarize and query from a lot of heterogeneous documents. Any LLM provider, any filetype, advanced RAG, advanced summaries, scriptable, …☆545Aug 24, 2026Updated last month
- [ACL 2025 Demo] Repository for the demo and paper: ReasonGraph: Visualisation of Reasoning Paths☆516Mar 9, 2026Updated 7 months ago
- The first open-source agent skills builder. Define skills by vibe workflow, run on Claude Code, Cursor, Codex & more. Build Clawdbot 🦞· …☆7,533Jul 29, 2026Updated 2 months ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Speech to Text but with all the bells and whistles and most importantly AI! AI will clean up your filler words, edit and will refine what…☆332Feb 9, 2025Updated last year
- OpenSource Production ready Customer service with built in Evals and monitoring☆1,546Jun 18, 2026Updated 3 months ago
- ☆311Aug 23, 2024Updated 2 years ago
- TrendPublish: 全自动 AI 内容生成与发布系统 | 微信公众号自动化 | 多源数据抓取 (Twitter/X、网站) | DeepseekAI、千问、讯飞模型 | 智能内容分析排序 | 定时发布 | 多模板支持 | Node.js | TypeScript |…☆3,205Aug 4, 2026Updated 2 months ago
- A tool to assist in the interpretation of learned features in sparse autoencoders (in particular the four SAE's trained by Joseph Bloom o…☆19Oct 4, 2024Updated 2 years ago
- The Level-Navi Agent, a framework that requires no training and utilizes large language models for deep query understanding and precise s…☆81Aug 17, 2026Updated last month
- Query and Summarize your chat messages.☆1,028Dec 4, 2024Updated last year
- Web archiving utility library☆12Oct 1, 2026Updated last week
- [AAAI 2025] StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization☆227Jul 18, 2026Updated 2 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- AI ContentCraft is an all-in-one content creation suite that helps creators generate stories, podcast scripts, and multimedia content usi…☆401Jul 4, 2025Updated last year
- Code for ACL25-findings. An LLM-based agent simulation framework that simulates human behavior and generates dynamic, text-based social g…☆97Mar 15, 2026Updated 6 months ago
- [ICLR 2026] A Training-free Iterative Framework for Long Story Visualization☆957Apr 2, 2026Updated 6 months ago
- RAG Web UI is an intelligent dialogue system based on RAG (Retrieval-Augmented Generation) technology.☆3,298Apr 6, 2026Updated 6 months ago
- 为OPC/中小微企业量身打造的自媒体获客智能体☆8,600Updated this week
- PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation☆2,478Sep 10, 2025Updated last year
- 🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)☆6,937Jul 4, 2025Updated last year
- ☆43Mar 16, 2026Updated 6 months ago
- Perplexity style AI Search engine clone built with Gemini 2.0 Flash and Grounding☆2,074Jan 4, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.☆8,329Sep 22, 2026Updated 3 weeks ago
- 将所有AI 产品接入你的微信，打造你个人AI 助理，帮助你解决更多生活日常。☆421Mar 11, 2026Updated 7 months ago
- Toolkit for linearizing PDFs for LLM datasets/training☆19,735Mar 25, 2026Updated 6 months ago
- ☆2,792May 2, 2025Updated last year
- Keep searching, reading webpages, reasoning until it finds the answer (or exceeding the token budget)☆5,240May 1, 2026Updated 5 months ago
- [COLM 2025] "C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing"☆21Apr 9, 2025Updated last year
- Transform PDFs into AI podcasts for engaging on-the-go audio content.☆880Jun 26, 2026Updated 3 months ago
- Netflix-level subtitle cutting, translation, alignment, and even dubbing - one-click fully automated AI video subtitle team | Netflix级字幕切…☆18,768Sep 30, 2026Updated last week
- LINEBot☆13Apr 7, 2025Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- 口袋AI，将世界知识装进口袋。pocketpal-ai 中文版☆595Jul 2, 2026Updated 3 months ago
- [EMNLP 2025] OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking☆485Aug 23, 2025Updated last year
- 一个现代化的全栈 AI Chatbot 应用，使用 React 和 Cloudflare Workers 结合 Connect RPC 构建，通过 Tauri 支持 Web、移动 App 和桌面端☆564Oct 6, 2026Updated last week
- A simple screen parsing tool towards pure vision based GUI agent☆25,516Jul 20, 2026Updated 2 months ago
- [NeurIPS 2024] The official implementation of "Image Copy Detection for Diffusion Models"☆18Oct 1, 2024Updated 2 years ago
- An AI chat bot for small and medium-sized teams, supporting models such as Deepseek, Open AI, Claude, and Gemini. 专为中小团队设计的 AI 聊天应用，支持 De…☆1,188Sep 16, 2025Updated last year
- A visual playground for agentic workflows: Iterate over your agents 10x faster☆5,796Jun 29, 2026Updated 3 months ago