medialab / ural
A helper library full of URL-related heuristics.
☆64Updated 4 months ago
Alternatives and similar repositories for ural:
Users that are interested in ural are comparing it to the libraries listed below
- Make it easier to compare and cross-reference the names of companies and people by applying strong normalisation.☆148Updated 3 weeks ago
- Extract networks of entities from journalistic reporting☆48Updated last year
- etl pipeline, graphical explorer and general toolbox for investigations with follow the money data☆15Updated last year
- Trying to generate name synonyms from wikidata☆32Updated 4 years ago
- API client for Aleph, supports bulk entity and document upload.☆28Updated 4 months ago
- Web scraping Page Objects core library☆96Updated last week
- A library to extract a publication date from a web page, along with a measure of the accuracy.☆41Updated 5 years ago
- scraper for facebook, gab, google and tiktok☆22Updated 7 months ago
- Ingestors extract the contents of mixed unstructured documents into structured (followthemoney) data.☆58Updated this week
- Web interface for network analysis.☆21Updated 2 years ago
- Python based Wikidata framework for easy dataframe extraction☆41Updated last year
- python functions for applied use of schema.org☆35Updated 3 years ago
- Alternative robots parser module for Python☆17Updated 2 months ago
- how hard is it to get a list of all local news sites in the United States (LOL)☆8Updated 4 years ago
- Specialized & performant CSV readers, writers and enrichers for python.☆13Updated last year
- A Python scraping module, that extracts text from articles found in RSS feeds. Uses SQLite as database.☆18Updated 7 months ago
- Extract text from HTML☆133Updated 4 years ago
- Sort-friendly URI Reordering Transform (SURT) python module☆41Updated 6 months ago
- Extract place names from a URL or text, and add context to those names -- for example distinguishing between a country, region or city.☆126Updated 10 months ago
- Link Wikidata items to large catalogs☆95Updated 2 weeks ago
- Browser version of Hyphe (WIP)☆30Updated 4 months ago
- Extract city and country mentions from Text like GeoText without regex, but FlashText, a Aho-Corasick implementation.☆60Updated this week
- Inspect a URL and estimate if it contains a news story☆39Updated 2 months ago
- Data cleaning and validation functions for names, languages, identifiers, etc.☆16Updated last month
- How can we improve name matching in screening tools?☆12Updated 3 weeks ago
- A pure-Python robots.txt parser with support for modern conventions.☆58Updated 2 weeks ago
- Page Object pattern for Scrapy☆118Updated last week
- Scrapy schema validation pipeline and Item builder using JSON Schema☆45Updated 3 years ago
- Save an RSS or ATOM feed to a SQLite database☆47Updated 2 years ago
- ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of diff…☆88Updated 3 years ago