mmatlacz / locationextractor Goto Github PK

Library for automatic location extraction from text

Python 98.66% Makefile 1.34%

locationextractor's Introduction

LocationExtractor

Extract place names from a URL or text, and add context to those names -- for example distinguishing between a country, region or city.

Install & Setup

Grab the package using pip (this will take a few minutes)

package not yet availabe on pypi

Basic Usage

Import the module, give some text or a URL, and presto.

TODO

Credits

LocationExtractor is based on:

geograpy

LocationExtractor uses the following excellent libraries:

NLTK for entity recognition
newspaper for text extraction from HTML
jellyfish for fuzzy text match
pycountry for country/region lookups

LocationExtractor uses the following data sources:

This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com
ISO3166ErrorDictionary for common country mispellings via Sara-Jayne Terp

locationextractor's People

Contributors

Stargazers

Watchers

locationextractor's Issues

We should't remove sublocations

Sublocations might be a part of a location name, e.g.:

North America
North Ayrshire
Southern Leyte

Allow users to use their own database

We should allow library users to use their own database

Syntax error when location name contains '

continent_name =='St-theodore-d'acton '

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Creating virtualenv location-extractor-_qvnlsxS-py3.9 in /home/dependabot/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

  PackageNotFound

  Package wily (1.19.0) not found.

  at /usr/local/.pyenv/versions/3.9.0/lib/python3.9/site-packages/poetry/repositories/pool.py:144 in package
      140│                     self._packages.append(package)
      141│ 
      142│                     return package
      143│ 
    → 144│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      145│ 
      146│     def find_packages(
      147│         self, dependency,
      148│     ):

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Investigate speedup from using numba and replacing pandas with numpy;

Make changes if there is at least 10% speedup;

Allow sorting cities by population size

Investigate dataclasses impact on code speed

introduce mypy to the project

We already have provided some type hints, but not for a whole codebase, so we need to cover all codebase with type hintings and add mypy to the project. We needed to add:

proper configuration of mypy in setup.cfg
stage to check type hints in Makefile (or add mypy command call to lint stage)
run mypy in CI