Alternatives and similar repositories for scaling-to-distributed-crawling

Users that are interested in scaling-to-distributed-crawling are comparing it to the libraries listed below

• daveshap / JobMatching

  View on GitHub
  Experiment to match job applications with job descriptions using GPT-3
  ☆14Jul 17, 2022Updated 3 years ago
• Rsweater / Awesome-Lane-Detection

  View on GitHub
  This repository contains content related to 2D and 3D lane detection, as well as video lane detection. There are not only papers here, bu…
  ☆13Sep 1, 2024Updated last year
• mattia-decao / hiero-transformer

  View on GitHub
  ☆12Nov 3, 2024Updated last year
• jdechalendar / why100renewable-suppl

  View on GitHub
  Supplemental code and data for the paper: Turning the spotlight on California’s (dirty) nighttime emissions
  ☆10May 3, 2019Updated 6 years ago
• sof-emu / sof-emu

  View on GitHub
  Experimental Game Server Development
  ☆11Oct 15, 2022Updated 3 years ago
• edureka-git / devops

  View on GitHub
  ☆15Jul 18, 2023Updated 2 years ago
• Vincent-PRO-AI / french-llm-from-scratch

  View on GitHub
  Training a French GPT model from scratch: 260M params, 130M tokens, with fine-tuning on conversations. Full pipeline with dashboard, chec…
  ☆22Jan 18, 2026Updated 3 weeks ago
• TUDB-Labs / TuDB

  View on GitHub
  A distributed graph database system (GDBMS)
  ☆11Feb 20, 2023Updated 2 years ago
• tizerk / hal9000

  View on GitHub
  A local, voice-controlled AI assistant with the personality of HAL 9000 from 2001: A Space Odyssey.
  ☆20Aug 16, 2025Updated 5 months ago
• SonicExE404 / FastOS

  View on GitHub
  ☆29Dec 20, 2025Updated last month
• suralmasha / RuTranscript

  View on GitHub
  Russian phonetical transcription
  ☆11Nov 19, 2025Updated 2 months ago
• liliumbosniacum / redis

  View on GitHub
  ☆12May 30, 2021Updated 4 years ago
• jiani-huang / RecBenchPlus

  View on GitHub
  Benchmark dataset for the paper "Towards Next-Generation Recommender Systems: A Benchmark for Personalized Recommendation Assistant with …
  ☆23May 20, 2025Updated 8 months ago
• mentix02 / instagram

  View on GitHub
  An Instagram clone built using Python and Javascript
  ☆13Apr 14, 2021Updated 4 years ago
• nasa / gesdisc-cloud-tutorials

  View on GitHub
  ☆11Jun 13, 2024Updated last year
• HIRO-MicroDataCenters-BV / rhio

  View on GitHub
  Peer-to-peer NATS message routing and S3 object sync solution
  ☆18Feb 5, 2026Updated last week
• Lumen-Labs / cpp-chunker

  View on GitHub
  Implementation of a fast semantic chunker in C++, installable in python 3.7+ projects.
  ☆22Sep 20, 2025Updated 4 months ago
• nanovms / gpu-nvidia

  View on GitHub
  Nanos klib for NVIDIA GPUs
  ☆14Mar 25, 2025Updated 10 months ago
• paultag / go-dictd

  View on GitHub
  dictd server bindings in go
  ☆10Oct 1, 2016Updated 9 years ago
• zorkow / MaxTract

  View on GitHub
  Old implementation of the MaxTract system for re-engineering mathematical PDF documents.
  ☆12Jan 25, 2016Updated 10 years ago
• SapienzaNLP / conception

  View on GitHub
  Code and experiments for the COLING2020 paper "Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations".
  ☆11Dec 9, 2020Updated 5 years ago
• tuva-health / medicare_cclf_connector

  View on GitHub
  This connector is a dbt project that maps Medicare CCLF claims data to the Tuva Input Layer.
  ☆14Jan 31, 2026Updated 2 weeks ago
• Andrews2017 / KINNEWS-and-KIRNEWS-Corpus

  View on GitHub
  Data, Embeddings, Stopword lists, code, and baselines for COLING 2020 paper titled "KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text …
  ☆13Apr 26, 2024Updated last year
• markizano / ytffmpeg

  View on GitHub
  Scripts I use to convert my videos into stuff that can be posted on YouTube, TikTok, Instagram and other social media sites.
  ☆12Nov 12, 2025Updated 3 months ago
• qh-8 / skeleton

  View on GitHub
  A PHP framework for web artisans.
  ☆10Aug 7, 2024Updated last year
• schmidtbri / using-ml-model-abc

  View on GitHub
  Code showing how to use a model based on the ML model base class.
  ☆10Sep 30, 2022Updated 3 years ago
• Saluana / EZAI-Web-Scraper

  View on GitHub
  An API that allows you to scrape blog posts and articles and get a list of notes or a summary back.
  ☆10Mar 31, 2023Updated 2 years ago
• winston0410 / mkNodePackage

  View on GitHub
  A helper flake for building Node.js package easily with Nix.
  ☆10Oct 9, 2021Updated 4 years ago
• Lelapa-AI / zindi-inkuba-notebook

  View on GitHub
  ☆12Mar 7, 2025Updated 11 months ago
• kylegallatin / components-of-an-ml-system

  View on GitHub
  ☆11Dec 30, 2022Updated 3 years ago
• haraldng / omnipaxos-rocksdb-tutorial

  View on GitHub
  This is a small demo of how to transform a simple single-server RocksDB service written in Rust into a distributed version using OmniPaxo…
  ☆16Feb 5, 2025Updated last year
• weljoan / qdlink-android

  View on GitHub
  ☆11Mar 24, 2021Updated 4 years ago
• IlyaGusev / TaleStudio

  View on GitHub
  Fork of RecurrentGPT with modifications
  ☆10Sep 18, 2024Updated last year
• aws-samples / amazon-ai-with-slack-bot

  View on GitHub
  ☆11Jul 20, 2023Updated 2 years ago
• pinecone-io / VSB

  View on GitHub
  Vector Search Benchmarking suite
  ☆12Nov 20, 2025Updated 2 months ago
• uniwue-zpd / PAGETools

  View on GitHub
  Small collection of PAGE XML related scripts used at the ZPD Würzburg
  ☆12Aug 2, 2024Updated last year
• Ziheng-Liang / wechat_web_scraper

  View on GitHub
  A web scraper and content recommendation engine based on wechat articles.
  ☆11Jul 7, 2016Updated 9 years ago
• KillrVideo / killrvideo-web

  View on GitHub
  The web server and browser single page app for KillrVideo
  ☆11Sep 15, 2022Updated 3 years ago
• dinoperovic / django-salesman-stripe

  View on GitHub
  Stripe payment integration for Salesman.
  ☆12Feb 23, 2023Updated 2 years ago