项目整体分为scrapy-redis分布式爬虫爬取数据、基于ElasticSearch数据检索和前端界面展示三大模块。做此项目是为了熟悉scrapy-redis的基本流程，以及其背后的原理，同时熟悉ElasticSearch的使用。本项目可以作为一个基于ES存储的简单但是相对全面的全栈开发的Demo。项目中所采用的组件均在win10本地环境搭建（伪分布），旨在演示项目流程。你可以参考该项目，并将其扩展到多个主机上，实现分布式ES以及分布式Scrapy。
☆24Dec 8, 2022Updated 3 years ago
Alternatives and similar repositories for JobNews-ElasticSearch-Scrapy_redis
Users that are interested in JobNews-ElasticSearch-Scrapy_redis are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 基于Scrapy+Elasticsearch+Django搭建的分布式电影搜索☆31Jul 25, 2018Updated 8 years ago
- Scrapy+Selenium+Django政府网站爬虫☆13Jul 22, 2019Updated 7 years ago
- 基于Scrapy和Django的二手房爬虫及可视化☆10Nov 22, 2022Updated 3 years ago
- 用scrapy框架爬虫爬取小说导入数据库并用django框架搭建自己的小说网站☆10Jan 10, 2019Updated 7 years ago
- 通过django将scrapy爬取存储到mongodb的数据展示到web页面，增删改查等功能☆13Aug 16, 2018Updated 7 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- scrapy抓取，mysql储存，django展示☆12Feb 6, 2016Updated 10 years ago
- python搭建搜索引擎☆30May 5, 2022Updated 4 years ago
- 网易严选爬虫☆19Aug 30, 2017Updated 8 years ago
- 1. 本项目模块化鲜明、代码简洁、功能小巧、配置灵活，是一个易于上手进行二次开发和快速构建企业内部统一用户认证中心服务的组件。 2. 本项目对服务端session的缓存提供了多种实现方式，如redis的实现方式就多达4种，可基于灵活的配置适配redis的各种部署方式。 3.…☆20Aug 24, 2023Updated 2 years ago
- 基于谷歌大规模网页去重simhash算法，对海量文章（长文本）进行去重。☆11Dec 8, 2022Updated 3 years ago
- 基于SG2300X的视频检索【使用自然语言搜索视频内容，定位到符合描述的具体时间段】☆13Feb 29, 2024Updated 2 years ago
- Springboot + ElasticSearch 构建博客检索系统☆12Mar 5, 2020Updated 6 years ago
- Scrapy, tianya, 天涯; scrapy django增量抓取天涯莲蓬鬼话全部帖子☆26Mar 20, 2025Updated last year
- Enable to share the same TCP port for different applications, for example, http and ssh.☆11Oct 2, 2020Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Performing Latent Semantic Analysis with Python on large datasets.☆13Jun 21, 2022Updated 4 years ago
- 餐厅管理系统 - 练习JDBC、MySQL数据库、德鲁伊连接池的使用；用户登录、订座、点餐、结账、人事管理☆12Feb 22, 2022Updated 4 years ago
- 一个基于elasticsearch开发的搜索引擎网站☆14Nov 22, 2022Updated 3 years ago
- ☆15Oct 10, 2020Updated 5 years ago
- 主要使用python+Scrapy框架去抓取新闻网站☆25Mar 2, 2017Updated 9 years ago
- Implementation of MIT 6.824: Distributed Systems☆16Nov 19, 2017Updated 8 years ago
- 租房网站信息分析项目（Vue+Django+Scrapy）☆26Jun 19, 2020Updated 6 years ago
- 一个简易的正则表达式引擎!☆10Apr 9, 2017Updated 9 years ago
- My homework and projects about learning CMU15-445☆12Feb 23, 2022Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Working example to integrate Midtrans payment gateway with Django☆10Apr 27, 2021Updated 5 years ago
- LAN file transfer daemon & utility for Linux systems written in C.☆10May 3, 2024Updated 2 years ago
- ☆21Jan 9, 2023Updated 3 years ago
- 毕业设计：《基于CLIP模型的视频文本检索设计与实现》☆18Jul 21, 2024Updated 2 years ago
- 长文本相似度模型☆21Nov 24, 2023Updated 2 years ago
- MLflow App Using React, Hooks, RabbitMQ, FastAPI Server, Celery, Microservices☆11Sep 25, 2022Updated 3 years ago
- 8 bit CPU implemented on top of a Digital Circuit Simulator in TypeScript☆12May 15, 2025Updated last year
- Manufacturing process management or Manage inventory statistics, production and orders is a collection of technologies and methods used t…☆12Mar 7, 2024Updated 2 years ago
- A fullstack ecommerce website using Django and React.js☆12Jul 21, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- SSM_CRUD（ssmcrud）,基于ssm+bootstrap的简单员工管理系统。SSM项目☆31Jun 30, 2026Updated last month
- feapder的管道扩展☆15Mar 6, 2023Updated 3 years ago
- 面向证券信息类专业搜索引擎，基于WEB信息挖掘技术的专业搜索引擎设计与实现并着重分析基于特定主题的爬取方法，通过下载Internet上WEB文档，进行过滤、分词、转换等处理工作，并建立索引数据库，最终可由检索器通过用户输入查询关键字，搜索器支持微博客、短信等内容短小而又不规…☆25Dec 3, 2018Updated 7 years ago
- CPU simulation framework for CS520 (Binghamton University, Graduate Computer Architecture)☆10May 10, 2018Updated 8 years ago
- 一个基于分布式爬虫的信安文章搜索引擎☆27May 22, 2023Updated 3 years ago
- ⌨️ RISC-V NS16550A UART driver☆11Mar 24, 2021Updated 5 years ago
- Datatable with websockets through Django Channels and Ajax Rest API☆10Mar 22, 2024Updated 2 years ago