vishwajeetdabholkar / eGet-Crawler-for-ai
Web scraping framework built for AI applications. Extract clean, structured content from any website with dynamic content handling, markdown conversion, and intelligent crawling capabilities. Perfect for RAG applications and AI training data pipelines. Features async processing, browser management, and Prometheus monitoring.
☆45Updated 2 months ago
Alternatives and similar repositories for eGet-Crawler-for-ai:
Users that are interested in eGet-Crawler-for-ai are comparing it to the libraries listed below
- Demo showcasing how to output CrewAI agent task outputs on the Streamlit UI.☆51Updated 2 months ago
- ☆43Updated 5 months ago
- ☆34Updated this week
- PDF intelligence platform combining IBM Docling for document processing, LlamaIndex for data structuring, and Streamlit for a powerful UI…☆42Updated 4 months ago
- Automated agent using LangChain and Gmail API to classify and respond to incoming emails based on their content.☆12Updated 6 months ago
- An implementation of an iterative knowledge base search agent using Agno agent framework, inspired by Ashpreet DeepKnowledge concept usin…☆14Updated 3 months ago
- AI agent with RAG+ReAct on Indian Constitution & BNS☆64Updated 6 months ago
- All code related to medium articles☆17Updated last week
- ☆21Updated 6 months ago
- GenAI Experimentation☆58Updated last week
- Agentic RAG using Crew AI.☆27Updated 10 months ago
- This is a repository which uses LangChain LangGraph and DuckduckGo to create a Perplexity Clone☆79Updated 6 months ago
- This repository is a combination of llama workflows and agents together which is a powerful concept.☆17Updated 8 months ago
- RAG with postgreSQL(nebius) and pgvector☆24Updated 5 months ago
- ☆13Updated 4 months ago
- ☆55Updated last month
- Local Agentic RAG using Langchain and Agno☆20Updated last month
- A Model Context Protocol (MCP) for analyzing and querying GitHub repositories using the GitHub Chat API.☆35Updated 3 weeks ago
- This tool allows you to search ArXiv for scientific papers, extract their content, embed and chunk the text, and ask questions about them…☆29Updated 9 months ago
- ☆12Updated 5 months ago
- ☆36Updated 4 months ago
- ☆53Updated 6 months ago
- Comprehensive metrics, insights, and visualization for Phidata and Crew AI applications☆25Updated 3 weeks ago
- Building AI Research Assistant: Multi-Agent RAG System Reading From Multiple Unstructured Sources☆21Updated 9 months ago
- Perplexity Lite using Langgraph, Tavily, and GPT-4.☆14Updated last year
- A few examples of how Dendrite's SDK can be used to automate web processes and build AI agents.☆37Updated 6 months ago
- Aient: The Awakening of Agent.☆34Updated this week
- ☆48Updated 2 weeks ago
- Widest collection of generative ai usecases in enterprise & startups☆19Updated last year
- This repository will contain projects on multi-agent applications using frameworks such as crewai, langchain, gradio, hugging face etc.☆23Updated 8 months ago