vishwajeetdabholkar / eGet-Crawler-for-ai
Web scraping framework built for AI applications. Extract clean, structured content from any website with dynamic content handling, markdown conversion, and intelligent crawling capabilities. Perfect for RAG applications and AI training data pipelines. Features async processing, browser management, and Prometheus monitoring.
☆33Updated this week
Alternatives and similar repositories for eGet-Crawler-for-ai:
Users that are interested in eGet-Crawler-for-ai are comparing it to the libraries listed below
- ☆26Updated 5 months ago
- GenAI Experimentation☆57Updated last month
- Agentic RAG with Langchain, Qdrant and CrewAI☆42Updated 7 months ago
- This is a repository which uses LangChain LangGraph and DuckduckGo to create a Perplexity Clone☆71Updated 3 months ago
- ☆68Updated 3 months ago
- Tutorial on how to create a ReAct agent without a LLM framework☆40Updated 5 months ago
- ☆57Updated 3 months ago
- ☆29Updated 9 months ago
- ☆20Updated 2 months ago
- Automate web research way beyond the first page of search results; curate knowledge bases to chat with.☆45Updated 3 months ago
- RAG with postgreSQL(nebius) and pgvector☆23Updated last month
- ☆45Updated 8 months ago
- Framework for building, orchestrating and deploying multi-agent systems. Managed by OpenAI Solutions team. Experimental framework.☆83Updated 2 months ago
- Rag Chatbot React And Tyepscript base boilerplate☆32Updated 8 months ago
- ☆75Updated last month
- All code related to medium articles☆12Updated this week
- ☆29Updated 5 months ago
- ☆60Updated 3 months ago
- ☆75Updated this week
- ☆46Updated 3 weeks ago
- Generate Tools and Toolkits from any Python SDK -- no extra code required☆49Updated 2 months ago
- ☆62Updated last month
- This project involves using llamaindex Multi Agents concierge system and Qdrant vector database to customize the RAG application with use…☆44Updated 4 months ago
- ☆35Updated last week
- A few examples of how Dendrite's SDK can be used to automate web processes and build AI agents.☆32Updated 2 months ago
- 🤖 An open-source, AI agent-native research canvas application that performs real-time search with HITL (Human in The Loop) capabilities,…☆34Updated this week
- Many tutorials on LangGraph☆34Updated 8 months ago
- ☆14Updated 4 months ago
- AI real estate agent☆33Updated 11 months ago
- This tool allows you to search ArXiv for scientific papers, extract their content, embed and chunk the text, and ask questions about them…☆25Updated 5 months ago