ncdulo / word_tools

Utilities for performing actions on words, or collections of words.

word_tools

Utilities for performing actions on words, or collections of words. This is a slowly expanding library of functions, conversions, and lookups added to as I see fit. Mostly intended for fun, or convenience. Not sure what it will grow into, but it's a nice laid back project.

We are most in need of ideas for new providers! What can word_tools do for you?

Note that many of the things in this repo, and in the code are a little out of proportion compared to what the code actually does. That is perfectly fine. I set up a lot of these extra features, simply to try them out. Having had an interest in them for some time but never actually using them. Bringing us to basically the same purpose behind this project. To expand my knowledge, and grow further as a developer. Also, I just really like the badges.

Requirements

Direct dependencies of this project are requests, beautifulsoup4, click, pymediawiki, and their dependencies. If installing through Pip, or setup.py, they will be installed automatically. A requirements{,_dev}.txt files have been included for a more exact listing of the full requirements, and their version numbers. However, there may be some issues with version numbers on a couple packages that I have not fully worked out yet. The pip and setup.py installation should not encounter these version mismatches.

Installation & Use

Recommended to install into a virtual environment of your desired flavor. I currently recommend virtualenvwrapper, but am using the stock venv module in the example below for simplicity. Save the user on configuring a dependency if they don't have to. If using another method to handle your venvs, modify the below to suit.

# Clone the repo
git clone https://github.com/ncdulo/word_tools.git
cd word_tools

# Create a new virtual environment, and enable it
python -m venv .env
source .env/bin/activate

# Update Pip & friends. Optional, but recommended
pip install --upgrade pip

# Install dependencies and create executable
pip install .
# If you receive errors about "No module named 'word_tools'", try this
pip install --editable .

# With venv active, the executable location will be included in your $PATH
# Otherwise, the path to executable will be
# .env/bin/wt
wt [OPTIONS] PROVIDER WORD
wt --help
wt urbandictionary python --limit 2
wt -l 2 wikipedia python

# Deactivate the virtual environment when finished
deactivate

Contributing

Contributions to word_tools are always very welcome, and very much appreciated. Thank you for the interest in helping out! There are a few guidelines that I try to follow when working on this project. Laid out in the CONTRIBUTING.md file. I admit that I got a little carried away when writing it, and it looks for worse than it actually is. Simply, they are just guidelines. I tried to include the relevant information to get a development environment set up, a breakdown of the project structure, and some basic conventions for coding and submitting your content.

tldr: If you are only submitting an issue of some sort, or a small change, you only need look into the very first section of the contributing guidelines.

Development Goals

This project does not have a strict end-game in sight. It's just something I have been putting together during my free time, in sporadic bursts of interst. There are some additions that I have in mind, but in general, it is very open-ended. Some of these features, or things I would like to learn with this project include:

• Additional Providers
  • Thesaurus Lookup.
  • StackOverflow Search.
  • GitHub Search.
• Testing
  • Integration tests.
  • More testing specific to core classes (GenericProvider, ObjectFactory, etc).
  • Parametrize current tests. Eliminate some of those loops.
  • Are fixtures applicable anywhere?
• Command Line Interface
  • Finalize argument specification.
  • Tidy up, verbose-ify --help text.
• Documentation
  • Write API docs. Probably Sphinx.
  • We're gonna need a man page too.
  • Go all in with readthedocs.io support as well?

In the end, this is just something that I intend to grow over time. Trying out new things as I learn them. Or adding other features as I think them up, or find a need for them. So far, it's been a lot of fun, and I have had the opportunity to try many new things on my own. 10/10, would wt again.

word_tools's Issues

[BUG] Periodic Labeler does not seem to work

Describe the bug
The 'Periodic Labeler' GitHub action does not seem to work in it's current configuration. I have allowed several days for it to repeatedly run, which it does. But my pull requests remain unlabelled.

To Reproduce
Steps to reproduce the behavior:

1. Create a Pull Request
2. Don't add labels
3. Wait

Expected behavior
After this action runs, unlabelled pull requests that match the patterns in .github/labeler.yml should have labels matching the rules applied. This should include pull requests of my own, and external origin.

Additional context
We may just remove this specific action, and replace with one of the other pull request label solutions. However, this will mean pull requests from other users will not be labelled against. Just a limitation of GitHub, I have read.

[DOCS] Update README

The current README is a little.. dated. Originally kind of thrown together fairly quickly. It has not been updated as the project grows. Need to get that changed. Writing an issue to keep it in memory, and to lay out some sections I would like to include.

• Summary
• Requirements
• Installation
• Usage
• Contributing/Bug Reporting
• Development Goals/Long-term

[Feature Request] Redesign CLI

Is your feature request related to a problem? Please describe.
Currently, we have a slightly.. different method of handling our CLI entry-points. They are click commands that are called directly by setting entry-points in the setup.py file. The original intent of this was a set of short and easy to remember aliases such as wt.stoopid or wt.urban. As the project grows, this system is becoming more difficult to maintain. And has been slowly running into errors on occasion.

Describe the solution you'd like
Utilize a proper Python entry point -- through __main__.py, and branching out into individual commands from there. I would like to use a single entry-point, with a minimum of required arguments. Various providers should use similar sets of arguments, where possible.

Fully writing the main function as above, we will very likely be able to do away with cli/cli.py. That file was becoming the source of several sporadic errors. Import errors, runtime errors, strange things were happening. The code was nearly untested and most users will not want an executable style as laid out by it. The new entry-point will do away with all of that. Should provide functional, tested code with a smooth user experience.

Some potential terminal commands may include:

$ word_tools -p/--provider [prov_name] word [limit]
$ word_tools -p urban python 3
$ wt.lookup -p/--provider [prov_name] word [limit]
$ wt -p wiki 'apollo program'

Describe alternatives you've considered
The main alternative to the above of a single entry-point would be dual entry-points. One for transform providers, one for lookup providers. I don't think I like that because it feels like a re-iteration of the original system that got us in trouble to begin with. Using a single entry-point will be more extensible in that we will require less modification to include new providers.

Additional context
None required at this time.

Certain issue label colors are too similar

What's the deal?
After getting the new issue labels set up and working, I feel several of them are far too similar to each other for easy identification. The pairs of similar colors are the repo/enhancement, and invalid/packaging labels.

Why does it matter?
Even for me, with normal color vision, the colors are hard to discern from each other. I imagine this may be worse for people with vision problems, colorblindness, or similar. I don't like struggling to read, and labels are meat for easy identification and categorization. If they're all the same, what do they matter?

What are you going to do about it?
Update the colors! This is a very small, and simple fix that will just mean clicking the 'random' button over and over until I'm happy with the colors it spits out.

Why bother writing an issue?
This issue is solely to note so that I can keep it on my task list. Just needs a few minutes to preview some colors, check them out, and update .github/labels.yml. No need to update the labels directly on GitHub -- our push actions will handle that once pushed to this repo.

[BUG] `test_wikipedia_disambiguation` fails intermittently.

Describe the bug
Once again, the Wikipedia does not seem to play nicely. The dynamic nature of results seems to be tripping me up. Again. This failure seems specific to the Disambiguation pages. test_wikipedia_disambiguation is intended to ensure we receive a DisambiguationError when we search for a known dis-ambiguous page. The test results seem to indicate that sometimes we receive that error, other times Wikipedia finds the closest matching page and returns that instead.

To Reproduce

1. Run pytest
2. Is not failing? Return to step 1.

Expected behavior
The pymediawiki module should throw a DisambiguationError for certain search terms. It does that, but it seems Wikipedia switches things around now and then. Sometimes we get it, sometimes we don't.

Additional context
This is bringing me back to Issue #2. A very similar test failure that was thought to be limited to my previous implementation. Looks like it's not totally my fault this time.

[Feature Request] Refactor `lookup` to use a factory pattern

Is your feature request related to a problem? Please describe.
This refactor was decided upon based on issue #2 and determining what it might take to fix. Giving that problem a good long look at, I think our current lookup module could use some love.

Describe the solution you'd like
Implement a generic factory pattern, which returns 'lookup providers' based on the API source, or data providers (Wikipedia, Urban Dictionary, etc). This should allow a greater flexibility for the end-user in how they query data from our providers, allow for a more fine-grained control over some of the common elements to each provider, and facilitate easier addition of new providers in the future.

Implementation details are not fully decided upon yet, just needed to get the base issue written up and logged here.

Describe alternatives you've considered
No alternatives considered. Only the solution we had used up until this point, which is proving harder to work with by the day.

Additional context
The decision to refactor lookup was arrived at due to issue #2. Finalizing this refactor will help in coming to a valid solution to that issue. They are slightly dependent on each other in that sense.

[Feature Request] Migrate CI from Travis to GitHub

Is your feature request related to a problem? Please describe.
Having zero experience with Continuous Integrating/Testing until a few days ago, I feel there is a lot for me to catch up. This means I may miss some more fitting solutions in favor of a solution I find first. I feel that Travis CI may have been one of those initial solutions.

GitHub Actions provides essentially the same toolset and features, at least for my basic FOSS/personal project use. It does not entirely make sense to me that we should rely on an external service when I can handle it directly from within GitHub. Handling CI with GitHub Actions will also allow it to reside within the same scope as my other actions, under a unified interface. Sounds like a good thing to me!

Describe the solution you'd like
The most fitting solution as of now seems to be migrating our .travis.yml file over to .github/workflows/ci.yml. There may be some syntax updates, or other changes made to the file for it to work. Though I anticipate it will be fairly straightforward, possibly requiring no modifications at all to that new action.

We will also need to consider Codecov.io and it's role in this pipeline. I really enjoy the Codecov web interface, and statistics tracking. Is there an option for that on GitHub?

Describe alternatives you've considered
No alternatives considered for Travis except migration to GitHub Actions. Code coverage should be given thought. Does it remain on Codecov.io? Does it migrate to GitHub Actions as well?

Additional context
None required at this time.

`test_wikipedia` fails intermittently.

Automated testing has introduced, rather quickly, a bug I feared I might encounter. Intermittent failure of some lookup tests. Specifically, this has been due to lookup.wikipedia returning results in a potentially changing order. While the issue has only presented with Wikipedia, I believe it has the potential to effect any of the related functions.

This simply comes down to the dynamic nature of the web, and search results changing over time. One possible way to mitigate this effect may be to check the entire list of fragments, instead of checking them by their order in code. This would mean adding more fragments to certain areas, such as the 'top 5 result fragments', and then checking each one of them to each result. There may be a better way to do that. Look into it.

Until resolved, expect potential intermittent failures in test_lookup.py tests. Restarting the build from Travis CI seems to be a valid workaround.