项目整体分为scrapy-redis分布式爬虫爬取数据、基于ElasticSearch数据检索和前端界面展示三大模块。做此项目是为了熟悉scrapy-redis的基本流程,以及其背后的原理,同时熟悉ElasticSearch的使用。本项目可以作为一个基于ES存储的简单但是相对全面的全栈开发的Demo。项目中所采用的组件均在win10本地环境搭建(伪分布),旨在演示项目流程。你可以参考该项目,并将其扩展到多个主机上,实现分布式ES以及分布式Scrapy。
☆24Dec 8, 2022Updated 3 years ago
Alternatives and similar repositories for JobNews-ElasticSearch-Scrapy_redis
Users that are interested in JobNews-ElasticSearch-Scrapy_redis are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 基于Scrapy+Elasticsearch+Django搭建的分布式电影搜索☆31Jul 25, 2018Updated 8 years ago
- 🕷️ [Graduation Project] Scrapy-Redis distributed crawler + Elasticsearch search engine + Django full-stack application; 论文搜索引擎(含Scrapy-R…☆43Feb 18, 2023Updated 3 years ago
- Scrapy+Selenium+Django政府网站爬虫☆13Jul 22, 2019Updated 7 years ago
- python搭建搜索引擎☆30May 5, 2022Updated 4 years ago
- 爬虫项目源码整理,使用redis进行url缓存,hbase进行详细信息的存储。使用zookeeper进行爬虫线程的状态监控。☆17Oct 7, 2015Updated 11 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- 📖 🕛 ✔️ 💯 编码规范集☆10Feb 18, 2021Updated 5 years ago
- Springboot + ElasticSearch 构建博客检索系统☆12Mar 5, 2020Updated 6 years ago
- Enable to share the same TCP port for different applications, for example, http and ssh.☆11Oct 2, 2020Updated 6 years ago
- 实现功能:新输入一段文本,与已有数据进行相似度进行比较,返回TOP10的文本。主要实现方法:jieba中文分词、gensim、TF-IDF词汇重要性、cosine余弦相似度。☆11Jul 30, 2020Updated 6 years ago
- 本项目包含几种常用 NLP算法的实现:关键词(keyword)、命名实体(named entity)、自动摘要(abstract)、文本相似度比较(text similarity)等☆16Jan 16, 2022Updated 4 years ago
- 基于SG2300X的视频检索【使用自然语言搜索视频内容,定位到符合描述的具体时间段】☆14Feb 29, 2024Updated 2 years ago
- 基于 ElasticSearch 的文件检索系统。☆15Jan 8, 2020Updated 6 years ago
- 静态站 用vue-element-admin框架搭建☆12Dec 4, 2018Updated 7 years ago
- 网站监控☆11Nov 9, 2019Updated 7 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- some example plots that are maybe useful?☆11Feb 10, 2026Updated 9 months ago
- ☆15Oct 10, 2020Updated 6 years ago
- Demo SoC☆10Oct 17, 2023Updated 3 years ago
- MySQL storage for OAuth 2.0☆13Mar 13, 2022Updated 4 years ago
- 租房网站信息分析项目(Vue+Django+Scrapy)☆26Jun 19, 2020Updated 6 years ago
- My homework and projects about learning CMU15-445☆12Feb 23, 2022Updated 4 years ago
- 之前写过qq音乐的爬虫,如果想下载无损音质的音乐比较困难,如果没有会员权限始终无法获取到,最后退而求其次,只能下载普通音质。近期看了下酷狗音乐,发现其无损音质和高品质音乐的链接都可以分析出来。全网搜索了一下,找到几个大神做的项目,重新进行复刻和改善,形成了这个项目。主要是搭…☆24Dec 8, 2022Updated 3 years ago
- 8 bit CPU implemented on top of a Digital Circuit Simulator in TypeScript☆12May 15, 2025Updated last year
- ☆10Dec 23, 2020Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- My Personal Blog☆12Updated this week
- SSM_CRUD(ssmcrud),基于ssm+bootstrap的简单员工管理系统。SSM项目☆31Jun 30, 2026Updated 4 months ago
- 一本关于高考志愿填报的书☆12Jun 12, 2019Updated 7 years ago
- 国家税务总局全国增值税发票查验平台☆15Dec 16, 2020Updated 5 years ago
- # redis statefulset☆19Nov 13, 2019Updated 7 years ago
- ☆10Apr 3, 2013Updated 13 years ago
- 一个基于分布式爬虫的信安文章搜索引擎☆27May 22, 2023Updated 3 years ago
- MCP over MQTT, an transport layer for the Model Context Protocol.☆12Jan 7, 2026Updated 10 months ago
- hadoop-3.1.2z在win10上编译的winUtils☆10Jun 24, 2019Updated 7 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- A WeApp project template in TypeScript using VS Code☆15Jul 13, 2017Updated 9 years ago
- 长文本相似度模型☆21Nov 24, 2023Updated 2 years ago
- ⌨️ RISC-V NS16550A UART driver☆11Mar 24, 2021Updated 5 years ago
- A Cross-Platform Lightweight 2D Tank Multiplayer Game in Python 2/3☆10Oct 24, 2020Updated 6 years ago
- 手把手教你ShardingSphere入门☆14Nov 13, 2020Updated 6 years ago
- A x86-16 bit assembly-based operating system tutorial.☆12Nov 21, 2019Updated 6 years ago
- A list of operating systems and stuff☆15Jun 6, 2025Updated last year
Are these results useful?