yelp / beans Goto Github PK

The workday data provider its really just a restful data provider, with some specific workday information, such as url, and fields. These could be factored out, and we could let the user specify their own mapping of db <-> returned json fields.

Something that was mentioned internally that we haven't updated yet.

The Travis CI for this PR failed because of a daylight saving time bug. Might be cool to have daily builds set up for master to catch these things earlier. Travis CI seems to have some support for Cron Jobs.

If a company doesn't have a publicly accessible url for their user photos, or just can't provide a photo url, then we will show a default photo page. We could allow the user to authorize beans to use their Google+ Photo. Until they do so, or the photo url is provided, we will use the default darwin photo

Certain commands in the Makefile are yelp-specific. We may want to consider making things more generic in the future.

Specifically: pip-faster, venv-update, pip-custom platform

For creating meetings outside of work you would need to find a local destination. This is probably going to be a cafe that meets the two participants needs in a set area. We can potentially leverage Yelp's API to facilitate a search and provide either a recommendation or set the meeting location.

Fusion Api: Business Search

We would be interested in search based off of:

• # of people the business can handle (don't overflow cafes)
• the type of location (coffee shop or cafe?)
• set a radius to search for the location in

Looks like some of the files are lowercase, which is why they don't match expectations. On Mac OS X, which is a case insensitive file system, this makes things a tad annoying to deal with.

/Users/bryanmarty/beans/js/routes.jsx
  6:18  error  Casing of ./containers/User does not match the underlying filesystem  import/no-unresolved
  7:17  error  Casing of ./App does not match the underlying filesystem              import/no-unresolved

✖ 2 problems (2 errors, 0 warnings)

 @ ./js/index.jsx 25:14-33

ERROR in ./js/App.jsx

/Users/bryanmarty/beans/js/App.jsx
  2:20  error  Casing of ./components/Header does not match the underlying filesystem  import/no-unresolved
  3:20  error  Casing of ./components/Footer does not match the underlying filesystem  import/no-unresolved

✖ 2 problems (2 errors, 0 warnings)

 @ ./js/routes.jsx 25:11-27
 @ ./js/index.jsx
make: *** [js] Error 2

When a user participates in multiple subscriptions they may not be able to differentiate between the subscription emails. This is because the title of the subscription is not included in the email.

Something like istanbul to the project.

We currently have support for sendgrid, we should create a generalized framework similar to data providers to support multiple email providers.

There are a few config values we have littered throughout the code (such as MEETING_COOLDOWN_WEEKS), that we could instead extra info a simple config.yaml/json.

To upgrade and run the platform on the flexible environment in AppEngine, there are some concrete changes that need to happen.

• Upgrade to Python3
• Upgrade to appengine Flexible environment config
• Switch datastore off of NDB

Right now the match emails provide static text on things that you could discuss. We should explore ways of creating dynamic suggestions for what to talk about during the meeting. This could be based on recent company announcements or fun questions to ask the person.

Relates to #34.

When certain metadata fields are not provided, their use in the frontend JS causes errors.

For example, if metadata is missing, the following JS error appears in the console.

Uncaught (in promise) TypeError: Cannot read property 'company_profile_url' of undefined
    at User.render (User.jsx:28)
    at ReactCompositeComponent.js?d2b3:796
    at measureLifeCyclePerf (ReactCompositeComponent.js?d2b3:75)
    at ReactCompositeComponentWrapper._renderValidatedComponentWithoutOwnerOrContext (ReactCompositeComponent.js?d2b3:795)
    at ReactCompositeComponentWrapper._renderValidatedComponent (ReactCompositeComponent.js?d2b3:822)
    at ReactCompositeComponentWrapper._updateRenderedComponent (ReactCompositeComponent.js?d2b3:746)
    at ReactCompositeComponentWrapper._performComponentUpdate (ReactCompositeComponent.js?d2b3:724)
    at ReactCompositeComponentWrapper.updateComponent (ReactCompositeComponent.js?d2b3:645)
    at ReactCompositeComponentWrapper.receiveComponent (ReactCompositeComponent.js?d2b3:547)
    at Object.receiveComponent (ReactReconciler.js?399b:125)

Add group matching capability, >2 people, to Yelp Beans.

Requirements:

• retain history of previous matches and reduce the chance of repeat matches
• support group sizes 3 to k, where k can be the size of the entire population
• both 2-person matches and >2 person matches are able to make use of weighted inputs (match.py#L160, networkx weighted graph docs.

Using a genetic algorithm here may be useful. All development would be in Python.

Currently the matching algorithm in yelp beans can support preferences in two ways:

1. One is a binary option:
   If the preferences don't match then don't match the people together.
   + Guaranteed match based on preference
   - Increased buckets for matching, if a bucket is odd one person doesn't get matched. Number of buckets == the total of potential people that don't get matched. In the extreme case this is very bad for the population.

2. The other is a softer option:
   If two people have preferences toward something in particular, increase weighted edges between the two of them.
   + Smaller number of buckets, less people that don't get matched.
   - won't guarantee a match based on interests
   - you have to source interests or generate a basic list for people to choose from

Some of our logic requires data from "metadata", which is yelp specific, we should probably refactor this to make those fields optional or configurable somehow.

For those unfamiliar with Google App Engine, there is a bit of a hurdle to go from the current project to something that can be used quickly. All database manipulations happen through the App Engine Datastore portal. We should either give a quick tutorial for initial state or provide a script to populate initial state.

In order to allow users to quickly bootstrap a beans implementation, lets create a new JSON file data provider that looks at a file on disk

Mentioned in #77. We should have an automated way to ensure the database state is clean between tests.

Group matching doesn't support 0,1, or 2 users.

yelp_beans/matching/group_match.py:74 in generate_meetings
yelp_beans/matching/group_match.py:122 in simulated_annealing
yelp_beans/matching/group_match.py:156 in get_mutated_state
ValueError: empty range for randrange()(0,0,0)

If beans were to support matching in a specific city and support neighborhood matching in that city, we would want an expandable way to match people based on their location. There are a few ways to do this.

1. Set up individual instances of the service for each sub region

• + simplifies logic
• - doesn't allow you to save history from meetings across regions (ie if someone met before and they switch regions, the datastore would be different and wouldn't know if they met anyone before in the new region)

2. Use the metadata field for the User entity to specify the User's "location". From there we could specify a matching filter to only look at people in the same area (we do the exact opposite for keeping people from matching in the same department) This would explicitly bucket you based on your area.

• - increases complexity
• + allows for management of an entire area in one app
• - this increases the number of matching bins, meaning low activity areas would be more prone to

Our tests require secrets to run, it probably would be better if they didn't.

Webpack needs to be installed for make development to succeed

npm install webpack -g

Looks like make test will fail with ImportError: No module named google.appengine.ext
Seems like the google app-engine sdk stuff is typically installed outside of the virtualenv at /usr/local/google_appengine (Mac).

I copied the files into the venv manually
cp -fr /usr/local/google_appengine/google venv/lib/python2.7/site-packages

The current experience for an unmatched user is not great. They get an email saying that they could not be matched for the week.

Some have suggested that we could create a meeting with 3 people if 1 was unmatched. This, however, poses some technical challenges.

• We store history for meetings, so we would need to update the meeting to reflect a 1:1 meeting amongst all three individuals. This would ensure no re-match.

• We need to support group emails. Recently this functionality was added, but there may be formatting issues.

• We need to find some unobtrusive way of modifying the current algorithm or making it algorithm unaware. IMO the best bet is to have the algorithm spit out the matches and unmatched and process that list after the fact.

We try to provide some support for other timezones. Timezones need to be supported in both location and in time. For instance, certain times of the year SF is in PST and in other times it is in PDT. When calculating a meeting spec from a meeting subscription, this calculation is incorrect if it is done across timezones. pytest tests/logic/meeting_spec_test.py::test_get_meeting_datetime hits this bug and is failing on master.

Could we have this send slack notifications instead of emails?

These are issues we experienced recently while setting up development environment:

• Minimum node version is required, works for 7.2 (6.1.0 does not have --inspect argument).
• After google app engine is installed and symlinks created, $PYTHONPATH needs to be exported

Add development tip to run make serve and npm run webpack -- --watch in background. This will run webserver and also have webpack watch directory for changes and recompile as necessary.

We can use uglify to compress the overall bundle size and toggle this on and off if we are in prod or dev.

It is sometimes hard to remember the meeting you have accepted on Friday if the last email was on Wednesday. We may want to explore sending out a reminder email the morning of the meeting to reduce the amount of missed meetings.

Make test is slow because it re-runs webpack, even when it doesn't need to. It would be nice for developers if we could improve this.

Can there be a way for people (ie ML engineers) to specify they would like to prefer networking with other ML engineers?

Hi,
This might sound quite a silly request but I am not able to run the program on my local machine.
These are the steps I have followed:

1. Installed Google App Engine.
2. Created a new project on Google cloud platform.
3. Ran 'gcloud components install app-engine-python' on local machine. After running this in command prompt it says 'All components are upto date'
4. Cloned the repo 'git clone [email protected]:Yelp/beans.git' and activated virtual environment in the /beans folder.
5. Run make development. However, on running this it gives the error 'make is not recognized as an internal or external command'

I am running the above on windows machine running Windows 7.

Please help.

Regards,
Amol

[2017-04-24 18:00:09,940]
ERROR in app: Exception on /tasks/email_users_for_weekly_opt_in [GET]

11:00:09.981 Traceback (most recent call last):
11:00:09.981 File "/flask/app.py", line 1982, in wsgi_app
11:00:09.982 response = self.full_dispatch_request()
11:00:09.982 File "/flask/app.py", line 1614, in full_dispatch_request
11:00:09.982 rv = self.handle_user_exception(e)
11:00:09.982 File "/flask/app.py", line 1517, in handle_user_exception
11:00:09.982 reraise(exc_type, exc_value, tb)
11:00:09.982 File "/flask/app.py", line 1612, in full_dispatch_request
11:00:09.982 rv = self.dispatch_request()
11:00:09.982 File "/flask/app.py", line 1598, in dispatch_request
11:00:09.982 return self.view_functionsrule.endpoint
11:00:09.982 File "/yelp_beans/routes/tasks.py", line 41, in weekly_opt_in
11:00:09.982 send_batch_weekly_opt_in_email(spec)
11:00:09.982 File "/yelp_beans/send_email.py", line 103, in send_batch_weekly_opt_in_email
11:00:09.982 'link_to_change_pref': 'https://yelp-beans.appspot.com/'
11:00:09.982 File "/yelp_beans/send_email.py", line 59, in send_single_email
11:00:09.982 return send_grid_client.client.mail.send.post(request_body=message.get())
11:00:09.982 File "/python_http_client/client.py", line 204, in http_request
11:00:09.982 return Response(self._make_request(opener, request))
11:00:09.982 File "/python_http_client/client.py", line 138, in _make_request
11:00:09.982 return opener.open(request)
11:00:09.982 File "/urllib2.py", line 410, in open
11:00:09.982 response = meth(req, response)
11:00:09.982 File "/urllib2.py", line 523, in http_response
11:00:09.982 'http', request, response, code, msg, hdrs)
11:00:09.982 File "/urllib2.py", line 448, in error
11:00:09.982 return self._call_chain(*args)
11:00:09.982 File "/urllib2.py", line 382, in _call_chain
11:00:09.982 result = func(*args)
11:00:09.982 File "/urllib2.py", line 531, in http_error_default
11:00:09.982 raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
11:00:09.982 HTTPError: HTTP Error 401: Unauthorized

When generating new meeting specs for the week, the batch stores the meeting times without taking into consideration the timezone for the subscription.

ie
A meeting subscription with the time 3:00PM in America/Los Angeles is stored instead as 3:00PM UTC, which ends up being 7AM PST during the conversion.

This requires a larger change of storing the timezone in the database per user and defaulting to a timezone if a user doesn't have one. By setting timezone per user, you can have user's across timezones participate in the meetings without being confused as to when they will participate.

The current metrics page is actually wrong. It will count the number of subscriptions, where a user can subscribe to multiple times for a single subscription. This does not give the number of users that have signed up for the subscription. In addition, we are missing some basic metrics that would be useful for anyone using this project.

Related to this github issue: evcohen/eslint-plugin-jsx-a11y#232

Output Example:

ERROR in ./js/components/UserPreferenceForm.jsx

/Users/workman/beans/js/components/UserPreferenceForm.jsx
  1:1  error  Definition for rule 'jsx-a11y/href-no-hash' was not found  jsx-a11y/href-no-hash

✖ 1 problem (1 error, 0 warnings)

npm test creates the following messages from node:

(node:6243) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: connect ECONNREFUSED 127.0.0.1:80
(node:6243) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Travis:

py27 runtests: commands[5] | pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
autopep8 wrapper.........................................................Failed
hookid: autopep8-wrapper
Files were modified by this hook. Additional output:
Fixing yelp_beans/matching/group_match.py

Local:

[Wed 08-30 11:48:35AM]$ pre-commit run --all-files
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
autopep8 wrapper.........................................................Passed
Check JSON...............................................................Passed
Check Yaml...............................................................Passed
Debug Statements (Python)................................................Passed
Tests should end in _test.py.............................................Passed
Flake8...................................................................Passed
Fix requirements.txt.....................................................Passed
Fix python encoding pragma...............................................Passed
Check for byte-order marker..............................................Passed
Check for merge conflicts................................................Passed
Reorder python imports...................................................Passed

[Wed 08-30 11:55:52AM]$ pre-commit run --files yelp_beans/matching/group_match.py
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
autopep8 wrapper.........................................................Passed
Check JSON...........................................(no files to check)Skipped
Check Yaml...........................................(no files to check)Skipped
Debug Statements (Python)................................................Passed
Tests should end in _test.py.........................(no files to check)Skipped
Flake8...................................................................Passed
Fix requirements.txt.................................(no files to check)Skipped
Fix python encoding pragma...............................................Passed
Check for byte-order marker..............................................Passed
Check for merge conflicts................................................Passed
Reorder python imports...................................................Passed

We should require an approved review before being able to merge pull requests

Currently the only way to populate the database is through the google app engine datastore portal. This is prohibitive for users that don't have experience working with this technology. There should, instead, be a simple UI to create new subscriptions.

We have this data provider with a lot of Yelp specific details. Lets factor out those details into a YelpDataProvider that only Yelp uses.

Relates to: #11
Blocks #26