GithubHelp home page GithubHelp logo

mltshp_migration's Introduction

MLTSHP

Status

Build status Coverage Status

Development Environment

MLTSHP is a Dockerized application. This greatly simplifies running the application for local development. We also deploy the application using Docker, so it makes for a more consistent environment to build and test against.

With Docker and a git client installed on your computer, clone the MLTSHP code from Github. If you intend to develop features for MLTSHP, you should clone from your own fork of the code. Once you have a copy checked out locally, use this command to create a settings.py and celeryconfig.py file suitable for local development (edit these as needed, but the defaults should be okay):

$ make init-dev

You should be able to start the app itself using:

$ make run

This will do a lot of things. But ultimately, you should end up with a copy of MLTSHP running locally. It expects to be accessed via a hostname of mltshp.localhost and s.mltshp.localhost. Add these entries to your /etc/hosts file:

127.0.0.1   mltshp.localhost s.mltshp.localhost

The web app itself runs on port 8000. You should be able to reach it via:

http://mltshp.localhost:8000/

Subsequent invocations of make run should be faster, once you have the dependency images downloaded.

You can login as the admin user using the password password. You can also register new user accounts.

While running the service, you can open an editor to the git checkout and make updates to the code. Changes should be reflected as you save your files (no need to restart the service).

The MySQL instance that is launched will be accessible on 127.0.0.1:3306 if you want to look at the database directly (since this is using the default MySQL port, you will probably need to shutdown any existing MySQL server you may have running). The login for the database is root with no password. If you want to mark any of your user accounts as paid users, find them in the user table and set their is_paid value to 1 and their stripe_plan_id column value to mltshp-double.

Logs and Data

When you run the application, it launches it into a background process. But if you want to watch the realtime logs emitted by each service, just use this command:

$ docker-compose logs -f

In addition to that, the web app produces some log files that are captured under the "mounts/logs" folder of your git repository. The directory structure looks like this:

mounts/
    uploaded/
        (transient uploaded file storage)
    logs/
        access.log - nginx access log file
        error.log - nginx error log file
        main-8000.log - python app log file
        celeryd-01.log - celery worker log file
    fakes3/
        (local S3 storage)
    mysql/
        (mysql data files)

AWS S3 Storage

MLTSHP utilizes AWS S3 for storing uploaded images. The development environment provides a dummy S3 server for local operation. But it requires a license key in order to use it. Visit this page to obtain a license key. For individual developers and small organizations, there is no cost. Add the following to a local .env file in the root of the project:

FAKES3_LICENSE_KEY=your-license-key-here

You will find any uploaded files under the `mounts/fakes3' directory.

Note: As of this time, the Docker image for the fake S3 server is incompatible with Apple Silicon CPU architectures. If you are using a computer with an Apple ARM CPU, you will need to use a real S3 bucket (see below).

If you would rather use a real S3 bucket, you can do that too. Create one and then assign these in your local settings.py file:

"aws_bucket": "your-mltshp-bucket-name",
"aws_key": "your-aws-key",
"aws_secret": "your-aws-secret",
## Comment these entries out:
##"aws_host": "fakes3",
##"aws_port": 8000,

Database Migrations

Occassionally, a database migration will need to be performed to bring an older database schema up to date. This will be necessary if you have a MySQL database you use locally for development and testing and keep it versus using the destroy and init-dev commands to make a new one. To update your database, just do this:

$ make shell
docker-shell$ cd /srv/mltshp.com/mltshp; python migrate.py

That should do it.

Tests

With your copy of MLTSHP running, you may want to run unit tests.

Some of the unit tests actually test against Twitter itself, so you'll want to generate a custom Twitter app with your own set of keys. Configure the test_settings in your settings.py file appropriately:

"twitter_consumer_key" : "twitter_consumer_key_here",
"twitter_consumer_secret" : "twitter_consumer_secret_key_here",
"twitter_access_key" : "twitter_access_key_here",
"twitter_access_secret" : "twitter_access_secret_here",

Then, just run:

$ make test

Which will invoke a Docker process to run the unit test suite.

Connecting to the MLTSHP shell

If you ever need to access the Docker image running the application, you can use this command to create a shell (specifically, a shell to the Python web application container):

$ make shell

This should place you in the /srv/mltshp.com/mltshp directory as the root user. You can use apt-get commands to install utilities you may need.

Cleanup

If you ever want to wipe your local data and rebuild your Docker containers, just use this command:

$ make destroy

If you just wish to rebuild the Docker container, use the Docker compose command:

$ docker-compose down

Then, run another make run.

Relationship with MLTSHP-Patterns

The CSS in this repo is just the compiled version of the styles from the MLTSHP pattern library, which can be found in the mltshp-patterns repo. Please do not edit the CSS in this repo, since any changes will be lost the next time we update from the pattern library.

About

MLTSHP is open-source software, ©2017 the MLTSHP team and released to the public under the terms of the Mozilla Public License. A copy of the MPL can be found in the LICENSE file.

Fastly logo MLTSHP is proudly powered by Fastly.

mltshp_migration's People

Contributors

vosechu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

vai

mltshp_migration's Issues

New default shk/malt

Along the lines of Popular and Incoming, add New Users shake that shows posts from people that joined in the last x days? (maybe new prime one as well for people that subscribed in last x days?)

Intermittent post stream problem

It seems like posts sometimes don't get included into the post streams correctly. For instance, I am a follower of the Dance Party shake. Most of the time posts posted in Dance Party show up in my main feed, but sometimes they don't. In that case they do show up in Incoming, as well as the main Dance Party stream. Dance Party is just an example, this happens with individual users as well. I don't know how to reproduce this issue.

[captured from the initial bugs sheet, from user bjrn]

Portrait posts from mobile are incorrectly rotated

When posting directly from a new portrait mode photo on iOS, the image is posted to Mltshp as rotated 90 deg. This is a carryover issue from Mlkshk.

There's an effective workaround to edit (crop, levels, any change) within the iOS Photos app then repost, but it would be much nicer if Mltshp could detect the proper orientation with the default image.

stats issue

I turned my stats off (or set them to hide) way back when and then when I turned them back on, they never really started up again

[captured from the initial bugs sheet, from user jessamyn]

Sanitise the initial import

We'll need to go through the code and assets provided by @bradchoate to determine if there's any personally identifying information, or secrets (ala API tokens, amazon keys, etc) and remove them.

Create a user import process

Import process possibilities

  1. Import all the users/comments/preferences/images, but exclude user content until they opt-in

Pros: We have a copy of all the comments/images so people can sign up a year from now and find all their images repopulated.
Cons: Lots of spam images/comments, expensive queries, possible weirdness in comment threads where one comment says "user not migrated" or something
Questions: What does a comment look like for a user that hasn't opted-in?

  1. Import all the users/comments/preferences/images

Pros: Hella easy
Cons: Possibly illegal/immoral depending on how you read the privacy policy

  1. Import data when a user signs up

Pros: No spam, every user on the site is certainly engaged
Cons: Import process could take hours or days depending on how many images need to be ported over, need to store a copy of all images/comments just in case someone signs up later, super complicated queries to find the just the rows that need to be imported, then off of that all the images that we need to download.
Questions: What does the site look like for the user while the import is running? How do we let them know when it's done? What happens if it fails? What happens if they don't opt-in for a year?

post visibility problem

in chrome i have to load unsafe scripts to see 1 out of 10 posts on mlkshk. usually it's videos. sometimes stills.
[captured from the initial bugs sheet, from user jimbo]

Allow for gift subscriptions

Folks should be able to purchase a membership/subscription to Mltshp for another person. Gift subscriptions should have the same options as regular subscriptions, just a different member name listed than the person paying for it.

Link after I uploaded picture via bookmarklet

Almost always after I upload a pic via the bookmarklet I want to check out if everything went okay, if I have new messages etc. So I click on the MLKSH / MLTSHP logo on the top left of the 'your image has been saved' page. (http://mlkshk.com/tools/p)

BUT THE LOGO HAS NO LINK!

It drives me nuts, and I asked @andre a couple of times, to no avail.

Seems like an easy fix/enhancement.

Browser Extensions Need to be Updated

Currently we do not have the source code for the browser extensions, which need to be updated and signed to work with mlthsp.

Ideally at launch we should have the Safari, Firefox, and Chrome extensions setup and running.

If possible we should try to get an extension for Edge built, but this can be put off until after launch.

Instagram import

Pony request for the ability to post Instagram images and video.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.