itiki/PythonTo-repeat-the-text-Bigdata external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

itiki/PythonTo-repeat-the-text-Bigdata

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/itiki/PythonTo-repeat-the-text-Bigdata)

Close

itiki / PythonTo-repeat-the-text-BigdataView on GitHubMore

• External links
• Readme badge

Python脚本实现千万级文本数据快速去重

☆19Mar 14, 2016Updated 9 years ago

Alternatives and similar repositories for PythonTo-repeat-the-text-Bigdata

Users that are interested in PythonTo-repeat-the-text-Bigdata are comparing it to the libraries listed below

• GuohuaZhuang / deduplication-detecting

  View on GitHub
  文档去重功能是为了解决搜索引擎的文档语义重复的问题，方法是多重哈希下的语义指纹算法。
  ☆12Aug 17, 2013Updated 12 years ago
• israel-dryer / Animated-Popup

  View on GitHub
  Animated PySimpleGUI popup screen for running processes
  ☆16Nov 20, 2019Updated 6 years ago
• lenciel / wechat2mp3

  View on GitHub
  convert audio message extracted from wechat to mp3
  ☆22May 5, 2019Updated 6 years ago
• Yohann-Codes / Pudding

  View on GitHub
  Pudding 是一款迷你级分布式服务框架
  ☆24Apr 26, 2017Updated 8 years ago
• linian365boy / rule-core

  View on GitHub
  规则引擎
  ☆22Feb 28, 2018Updated 8 years ago
• johnnybi8608 / Velora-docker

  View on GitHub
  ☆49Nov 8, 2025Updated 3 months ago
• chucheng92 / HadoopDedup

  View on GitHub
  基于Hadoop和HBase的大规模海量数据去重
  ☆30Apr 8, 2018Updated 7 years ago
• kushao1267 / CommonScripts

  View on GitHub
  平时记录的一些Python常用脚本
  ☆26Aug 7, 2019Updated 6 years ago
• longmenwaideyu / DMHWCF

  View on GitHub
  数据挖掘大作业基于协同过滤推荐算法的电影推荐
  ☆31Oct 21, 2014Updated 11 years ago
• xjs1919 / miaosha

  View on GitHub
  电商网站的秒杀
  ☆28May 15, 2023Updated 2 years ago
• THEGOLDENPRO / fastapi-tailwind

  View on GitHub
  ✨ TailwindCSS v4 support for 🔥 FastAPI without NodeJS and made incredibly simple.
  ☆12Jul 10, 2025Updated 7 months ago
• FannyChung / Sentiment-Analysis

  View on GitHub
  爬取商品评论并对商品评论进行情感分类
  ☆10Aug 14, 2017Updated 8 years ago
• wulizhou / ct-log

  View on GitHub
  基于spring boot的日志系统组件，提供操作日志输出及记录功能
  ☆10Mar 19, 2019Updated 6 years ago
• hello-shf / spring-boot-2.0.4.RELEASE

  View on GitHub
  SpringBoot-2.0.4源码阅读
  ☆28Dec 27, 2022Updated 3 years ago
• yh3434 / vertx-turtorial

  View on GitHub
  a tutorial for vert.x
  ☆17Oct 9, 2018Updated 7 years ago
• iwhgao / PyTextScan

  View on GitHub
  利用python脚本对文本内容进行敏感信息的识别与过滤
  ☆38Jan 12, 2016Updated 10 years ago
• longhuasishen / tddef-toolkit-mybatis

  View on GitHub
  mybatis-generator插件，生成基础dao和常见crud方法，支持mysql和oracle分页，实体类带注释
  ☆10Aug 30, 2016Updated 9 years ago
• falost / avatar-box

  View on GitHub
  圣诞帽、国庆节等节日头像边框生成小程序，已添加红包封面领取提醒的功能
  ☆10Jan 22, 2022Updated 4 years ago
• Zteve / SmallCollections

  View on GitHub
  Memory and GC cheap collections
  ☆12Jun 29, 2015Updated 10 years ago
• baileykm / web-lighter

  View on GitHub
  一个小型的 Java Web 服务端封装
  ☆14May 18, 2019Updated 6 years ago
• dgo2dance / jdpor

  View on GitHub
  基于Django的京东商品比价系统+基于request京东爬虫
  ☆12Jun 19, 2020Updated 5 years ago
• sevengivings / subtitle-extractor

  View on GitHub
  A Python script for AI speech recognition of video or audio file using whisper, stable-ts or faster-whisper and translation of subtitle u…
  ☆10Feb 17, 2025Updated last year
• AntiBro / JaxWay

  View on GitHub
  基于SpringCloud-Gateway的动态网关,可视化的路由配置,请多多 star
  ☆13Dec 10, 2022Updated 3 years ago
• hry8310 / derbymi

  View on GitHub
  这是居于 derby 源代码，通过删减的方式，从里面抽取出sql解析功能。并在此基础上开发出跨库连接查询器。通过该工具可以将连接查询分割成多个单表查询，再将单表结果集进行连接，即将数据库的连接功能上移到工具执行。详情可以查看wiki：readme
  ☆10Feb 14, 2017Updated 9 years ago
• timks-wonder / try_moonshot

  View on GitHub
  本意是想做一个直接调用kimi的API帮我读论文的程序，然后发现API太贵了，但kimi网页版免费，就结合chrome和python写了这么个东西
  ☆12Mar 7, 2024Updated last year
• hkuc / tvbox-config

  View on GitHub
  自用 tvbox 配置,使用tv目录下的数字目录内的 urls.json 可以批量导入源。
  ☆21Feb 20, 2026Updated last week
• 15673312611 / AI_NovelCraft-

  View on GitHub
  ai小说 一键生成多章节的长篇小说,一键生成详细大纲,一键拆分多卷,自然语言消痕,自动衔接之前章节，ai一键去痕 支持deepseek等大模型
  ☆40Feb 5, 2026Updated 3 weeks ago
• MusicXi / demo-swagger2

  View on GitHub
  基于springboot 的swagger2动态接口文档在线生成，集成导出html/markdown/confluence 等静态文档 。 及接口操作AOP日志自动记录
  ☆11Aug 26, 2024Updated last year
• ex34toway / uw-log-es

  View on GitHub
  一个基于ElasticSearch的业务日志记录工具
  ☆10Nov 5, 2018Updated 7 years ago
• Romanluo1990 / spring-cache-redis-expire

  View on GitHub
  spring redis框架过期时间注解支持
  ☆10Oct 17, 2018Updated 7 years ago
• iamzhongyong / qlexpress

  View on GitHub
  规则引擎测试
  ☆10Feb 27, 2014Updated 12 years ago
• chgl16 / chatroom

  View on GitHub
  基于Spring Boot+WebSocket(STOMP) web聊天系统。支持点对点和广播发送消息，Spring Security 单点登录验证
  ☆13Feb 25, 2019Updated 7 years ago
• VXenomac / wechat-chat-history-analyser

  View on GitHub
  微信聊天记录分析器
  ☆12Apr 22, 2022Updated 3 years ago
• ACE-Caedmon / NG-Socket

  View on GitHub
  网络游戏服务器通信框架
  ☆13Apr 29, 2020Updated 5 years ago
• cn27001 / lua-resty-waf

  View on GitHub
  开源waf web 防火墙
  ☆10Nov 30, 2021Updated 4 years ago
• lusson-luo / stock_tool

  View on GitHub
  股票实时查看并告警工具
  ☆12Mar 12, 2023Updated 2 years ago
• wangyan9110 / micro-file

  View on GitHub
  文件微服务，实现基于云服务和本地文件存储的微服务
  ☆10Sep 8, 2016Updated 9 years ago
• yangxb2010000 / delay-schedule

  View on GitHub
  分布式、高可用的延迟调度系统、可以配合消息队列实现延迟任务队列
  ☆12Dec 5, 2023Updated 2 years ago
• Kaperstone / BerkleyDB

  View on GitHub
  Oracle Berkeley DB sourcecode
  ☆12May 5, 2014Updated 11 years ago