项目整体分为scrapy-redis分布式爬虫爬取数据、基于ElasticSearch数据检索和前端界面展示三大模块。做此项目是为了熟悉scrapy-redis的基本流程，以及其背后的原理，同时熟悉ElasticSearch的使用。本项目可以作为一个基于ES存储的简单但是相对全面的全栈开发的Demo。项目中所采用的组件均在win10本地环境搭建（伪分布），旨在演示项目流程。你可以参考该项目，并将其扩展到多个主机上，实现分布式ES以及分布式Scrapy。
☆24Dec 8, 2022Updated 3 years ago
Alternatives and similar repositories for JobNews-ElasticSearch-Scrapy_redis
Users that are interested in JobNews-ElasticSearch-Scrapy_redis are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 基于Scrapy+Elasticsearch+Django搭建的分布式电影搜索☆31Jul 25, 2018Updated 7 years ago
- 基于Scrapy-Redis框架与Mongodb的分布式爬虫-elasticsearch搜索引擎打造☆18Apr 21, 2020Updated 6 years ago
- scrapy抓取，mysql储存，django展示☆12Feb 6, 2016Updated 10 years ago
- Unity 结合AIGC 虚拟仿真人项目☆11Feb 21, 2025Updated last year
- 基于谷歌大规模网页去重simhash算法，对海量文章（长文本）进行去重。☆11Dec 8, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Scrapy, tianya, 天涯; scrapy django增量抓取天涯莲蓬鬼话全部帖子☆21Mar 20, 2025Updated last year
- elasticsearch7.9 cdh-ext-parcels and single machine multi instance☆10Jul 12, 2021Updated 5 years ago
- 实现功能：新输入一段文本，与 已有数据进行相似度进行比较，返回TOP10的文本。主要实现方法：jieba中文分词、gensim、TF-IDF词汇重要性、cosine余弦相似度。☆11Jul 30, 2020Updated 5 years ago
- 史上最全编程语言xmind思维导图总结 包含python/php/flask/爬虫/javascript/mysql/nosql/git☆25Jun 29, 2021Updated 5 years ago
- 餐厅管理系统 - 练习JDBC、MySQL数据库、德鲁伊连接池的使用；用户登录、订座、点餐、结账、人事管理☆12Feb 22, 2022Updated 4 years ago
- 一个基于elasticsearch开发的搜索引擎网站☆14Nov 22, 2022Updated 3 years ago
- 🧟Zombioo is a 2D game, created using Pygame and Python🧟 …☆10Jan 19, 2022Updated 4 years ago
- 【Demo】对新闻标题使用TF-IDF向量化和cosine相似度计算完成相似标题推荐☆14Mar 2, 2020Updated 6 years ago
- 基于simhash的文本去重算法☆20Jun 18, 2021Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- 租房网站信息 分析项目（Vue+Django+Scrapy）☆26Jun 19, 2020Updated 6 years ago
- 静态站 用vue-element-admin框架搭建☆12Dec 4, 2018Updated 7 years ago
- 网站监控☆11Nov 9, 2019Updated 6 years ago
- some example plots that are maybe useful?☆11Feb 10, 2026Updated 5 months ago
- 批量下载抖音用户视频☆21Jan 19, 2024Updated 2 years ago
- 一个简易的正则表达式引擎!☆10Apr 9, 2017Updated 9 years ago
- Demo SoC☆10Oct 17, 2023Updated 2 years ago
- Working example to integrate Midtrans payment gateway with Django☆10Apr 27, 2021Updated 5 years ago
- LAN file transfer daemon & utility for Linux systems written in C.☆10May 3, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆21Jan 9, 2023Updated 3 years ago
- 毕业设计：《基于CLIP模型的视频文本检索设计与实现》☆18Jul 21, 2024Updated 2 years ago
- A web application based on Python framework Django , that brings together farmers and retailers on common platform for commodity bidding.☆10Oct 21, 2019Updated 6 years ago
- 长文本相似度模型☆21Nov 24, 2023Updated 2 years ago
- ☆10Dec 23, 2020Updated 5 years ago
- 之前写过qq音乐的爬虫，如果想下载无损音质的音乐比较困难，如果没有会员权限始终无法获取到，最后退而求其次，只能下载普通音质。近期看了下酷狗音乐，发现其无损音质和高品质音乐的链接都可以分析出来。全网搜索了一下，找到几个大神做的项目，重新进行复刻和改善，形成了这个项目。主要是搭…☆22Dec 8, 2022Updated 3 years ago
- 国家税务总局全国增值税发票查验平台☆15Dec 16, 2020Updated 5 years ago
- The DSDT and SSDTs of Lenovo G470 for hackintosh.☆12Dec 23, 2017Updated 8 years ago
- A Cross-Platform Lightweight 2D Tank Multiplayer Game in Python 2/3☆10Oct 24, 2020Updated 5 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A fullstack ecommerce website using Django and React.js☆12Jul 21, 2024Updated 2 years ago
- SSM_CRUD（ssmcrud）,基于ssm+bootstrap的简单员工管理系统。SSM项目☆31Jun 30, 2026Updated 3 weeks ago
- hadoop-3.1.2z在win10上编译的winUtils☆10Jun 24, 2019Updated 7 years ago
- My Personal Blog☆12Updated this week
- 一个基于分布式爬虫的信安文章搜索引擎☆27May 22, 2023Updated 3 years ago
- ⌨️ RISC-V NS16550A UART driver☆11Mar 24, 2021Updated 5 years ago
- Our platform is “E-learning platform” provide some features by using computer vision and artificial intelligence technologies as Real Tim…☆10Jul 31, 2022Updated 3 years ago