Aavache / LLMWebCrawlerLinks

A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.
93Updated 2 years ago

Alternatives and similar repositories for LLMWebCrawler

Users that are interested in LLMWebCrawler are comparing it to the libraries listed below

Sorting: