paulproteus / ampache-sandstorm-prototype Goto Github PK

Goal:

• After this issue is resolved, it should be possible to make an HTTP request via a command-line client (e.g. curl) to the Subsonic API within Ampache. One good URL to test is the /rest/ping.view endpoint of that API.
• Therefore, I'd like to see a screenshot of a terminal with a successful curl interaction with the /rest/ping.view HTTP API endpoint from an Ampache grain running inside Sandstorm, either as part of a pull request, or as a comment on this issue with a request to close the issue.
• Note that API requests are going to be made to a different subdomain than the main domain of your Sandstorm server. This is normal.

Implementation notes:

• For the purpose of this issue, you should click on the 🔑 ("WebKey") icon in the Sandstorm top bar to generate an API token that you can use. Later issues will integrate this more tightly with the Sandstorm API access control system.
• See #5 for references on Sandstorm's support for apps publishing a HTTP API.

When I try to pack using Sandstorm, I got this error:

D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype>vagrant-spk pack ~/projects/package.spk
~/projects/package.spk
Calling 'vagrant' 'ssh' '-c' 'cd /opt/app/.sandstorm/ && spk pack --keyring=/host-dot-sandstorm/sandstorm-keyring --pkg-def=/opt/app/.sandstorm/sandstorm-pkgdef.capnp:pkgdef /home/vagrant/sandstorm-package.spk && spk verify --details /home/vagrant/sandstorm-package.spk && mv /home/vagrant/sandstorm-package.spk /opt/app/sandstorm-package.spk' in D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype\.sandstorm
qdfxn82a2r6sjgxd075p3d3gzyyekndnajs3uq9ynfjwkf5uu30h
{ "appId": "qdfxn82a2r6sjgxd075p3d3gzyyekndnajs3uq9ynfjwkf5uu30h",
  "packageId": "70543ff8c9383a3b9114d1f9e0599827",
  "title": {"defaultText": "Sandstorm Showcase"},
  "version": 0,
  "marketingVersion": {"defaultText": "0.0.0"},
  "metadata": {
    "icons": {},
    "website": "http:\/\/example.com",
    "codeUrl": "http:\/\/example.com",
    "license": {"none": null},
    "categories": [],
    "author": {"contactEmail": "[email protected]", "upstreamAuthor": "Example App Team"},
    "shortDescription": {"defaultText": "one-to-three words"},
    "screenshots": [] } }
Connection to 127.0.0.1 closed.
Traceback (most recent call last):
  File "<string>", line 1022, in <module>
  File "<string>", line 1019, in main
  File "<string>", line 821, in pack
WindowsError: [Error 3] The system cannot find the path specified
vagrant-spk returned -1

To work at all on Sandstorm, this app will need:

• A .sandstorm/ directory created by vagrant-spk

A pull request to fix this would include:

• A screenshot of vagrant-spk dev showing that the app works

By default (on non-Sandstorm), the administrator of an Ampache install usually has to manually work through a configuration wizard. In my opinion, the fix for this particular issue should assume the user has to manually work through a configuration wizard. Therefore the PR (GitHub Pull Request) that resolves this issue should also say what manual configuration steps are needed. A follow-up pull request should then remove the need for manual configuration.

If you need a remote Sandstorm server for testing, you are very welcome to have an account on https://rose.sandcats.io/ - that's my personal server. For me to create an account you must:

• Log into that server with your GitHub username or Google account.
• File an issue on this repository (ampache-sandstorm-prototype) requesting that I upgrade your user account from "Visitor" status to "User" status.

References to get this issue resolved:

• https://docs.sandstorm.io/en/latest/vagrant-spk/packaging-tutorial/
• https://github.com/ampache/ampache/wiki/Installation

References for music that you can use for testing, that is legal to make copies of:

• http://bakethebreak.com/8-best-free-netlabel-albums/
• https://www.google.com/search?q=best%20creative%20commons%20music

Reference:

• https://docs.sandstorm.io/en/latest/developing/auth/
• https://docs.sandstorm.io/en/latest/search.html?q=roles

I'll make a note to improve the docs.

@paulproteus, do you prefer me to ask a lot in Github so people can learn from my experience, or do you prefer I ask just moderately? It seems you are interested more in contributing to the community than meeting a target.

Goal:

• Demonstrate with a screenshot that an app that uses the Subsonic API can speak to an Ampache grain within Sandstorm.

Implementation notes:

• There are lots of apps that can use the Subsonic API. I am OK with you testing whichever one you find interesting. Here is a list: http://www.subsonic.org/pages/apps.jsp
• Many people say that DSub is really good: http://www.subsonic.org/pages/apps.jsp#dsub
• Since DSub is probably the app that I will use to interact with Subsonic in the long run,I'm happy to pay you back if you buy you a copy of it.
• Sometimes apps within Sandstorm seem to work properly, but Sandstorm blocks some HTTP headers that prevent API clients from working. This problem typically requires changing Sandstorm itself, which is a significant obstacle. Therefore if you find that the Subsonic API client can't be used by any Subsonic API apps, then please leave a comment here rather than waste lots and lots of time debugging it.
• My general rule is that if you are stuck, take 15 minutes to experiment further and document it, and then ask for help. See also https://blogs.akamai.com/2013/10/you-must-try-and-then-you-must-ask.html
• See #5 for references on Sandstorm's support for apps publishing a HTTP API.

Context:

• Typically, in Sandstorm, the Sandstorm shell surrounds the app, displaying the app in an IFRAME. This allows Sandstorm to handle login, etc.

For this issue, when I say "can be accessed via a grain URL", I mean that it should be possible to create a URL like https://local.sandstorm.io/grain/{{someGrainId}}/rest/ping.view and see a response to that API request within the grain-frame and the Sandstorm shell around it. See later issues in the milestone for the ability to access the API without seeing the Sandstorm shell around it.

Please either submit a pull request with packaging changes as-needed to make this work, or show a screenshot. The ping.view API endpoint is a good endpoint to use for taking a screenshot.

This issue might require modifying nginx configuration, and/or it might require modifying Ampache configuration.

Technical references:

• https://docs.sandstorm.io/en/latest/developing/path/
• https://docs.sandstorm.io/en/latest/developing/http-apis/
• https://github.com/ampache/ampache/wiki/API#subsonic-api

When I run vagrant-spk vm up, I got this:

D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype>vagrant-spk vm up
Calling 'vagrant' 'up' in D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype\.sandstorm
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'sandstorm/debian-jessie64' is up to date...
==> default: Machine already provisioned. Run `vagrant provision` or use the `--provision`
==> default: flag to force provisioning. Provisioners marked to run always will still run.

Then when I try to access http://local.sandstorm.io:6080/ in my browser, it can't load.

Previously when I run vagrant-spk vm up, I got different error but changed to the previous log when I run it again. The previous error is:

    default: /opt/app => D:/xampp/htdocs/PHP projects/Asheesh Laroia/ampache-sandstorm-prototype
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

chown `id -u vagrant`:`id -g vagrant` /opt/app

Stdout from the command:



Stderr from the command:

stdin: is not a tty
chown: changing ownership of ‘/opt/app’: Not a directory

FYI, I am using Windows 10, Vagrant 1.8.5, and VirtualBox 5.0.

Is there anything I did wrong? Or is there any compatibility issue?

I think it'd be nice if this were part of the main Ampache project.

I see the database config in the config file, but I am curious whether you already have a database that I can just import or is there no database at all and I should make a fresh new one.

This is Ampache when run on XAMPP:

vagrant-spk dev can compile just okay, but there is something wrong. I can create an instance successfully but instead of showing Ampache, it shows blank. It says loading 15MB which should be right, and the log shows no error.

The log:

...** SANDSTORM SUPERVISOR: Starting up grain.
Installing MySQL system tables...
160808 18:32:42 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
160808 18:32:42 [Note] /usr/sbin/mysqld (mysqld 5.5.50-0+deb8u1) starting as process 29 ...
OK
Filling help tables...
160808 18:32:43 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
160808 18:32:43 [Note] /usr/sbin/mysqld (mysqld 5.5.50-0+deb8u1) starting as process 35 ...
OK

To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:

/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h sandbox password 'new-password'

Alternatively you can run:
/usr/bin/mysql_secure_installation

which will also give you the option of removing the test
databases and anonymous user created by default.  This is
strongly recommended for production servers.

See the manual for more instructions.

You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with mysql-test-run.pl
cd /usr/mysql-test ; perl mysql-test-run.pl

Please report any problems at http://bugs.mysql.com/

waiting for mysql to be available at /var/run/mysqld/mysqld.sock
160808 18:32:43 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
160808 18:32:43 [Note] /usr/sbin/mysqld (mysqld 5.5.50-0+deb8u1) starting as process 39 ...
waiting for mysql to be available at /var/run/mysqld/mysqld.sock
waiting for mysql to be available at /var/run/mysqld/mysqld.sock
waiting for mysql to be available at /var/run/mysqld/mysqld.sock
waiting for mysql to be available at /var/run/mysqld/mysqld.sock
waiting for mysql to be available at /var/run/mysqld/mysqld.sock
waiting for mysql to be available at /var/run/mysqld/mysqld.sock
** SANDSTORM SUPERVISOR: Grain still in use; staying up for now.

@paulproteus do you know anything about this? There are not enough trace of Q&A of Sandstorm in the internet yet.

I tried to pack using Sandstorm, but then it shows me this error instead.

D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype>vagrant-spk pack ~/projects/package.spk
~/projects/package.spk
Calling 'vagrant' 'ssh' '-c' 'cd /opt/app/.sandstorm/ && spk pack --keyring=/host-dot-sandstorm/sandstorm-keyring --pkg-def=/opt/app/.sandstorm/sandstorm-pkgdef.capnp:pkgdef /home/vagrant/sandstorm-package.spk && spk verify --details /home/vagrant/sandstorm-package.spk && mv /home/vagrant/sandstorm-package.spk /opt/app/sandstorm-package.spk' in D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype\.sandstorm
VM must be running to open SSH connection. Run `vagrant up`
to start the virtual machine.
Traceback (most recent call last):
  File "<string>", line 1022, in <module>
  File "<string>", line 1019, in main
  File "<string>", line 819, in pack
  File "<string>", line 288, in call_vagrant_command
  File "subprocess.py", line 540, in check_call
subprocess.CalledProcessError: Command '['vagrant', 'ssh', '-c', 'cd /opt/app/.sandstorm/ && spk pack --keyring=/host-dot-sandstorm/sandstorm-keyring --pkg-def=/opt/app/.sandstorm/sandstorm-pkgdef.capnp:pkgdef /home/vagrant/sandstorm-package.spk && spk verify --details /home/vagrant/sandstorm-package.spk && mv /home/vagrant/sandstorm-package.spk /opt/app/sandstorm-package.spk']' returned non-zero exit status 1
vagrant-spk returned -1

But the problem is, the machine is already running. Prove:

D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype>vagrant global-status
id       name    provider   state    directory                                                                          
------------------------------------------------------------------------------------------------------------------------
0191611  default virtualbox running  D:/xampp/htdocs/PHP projects/Asheesh Laroia/ampache-sandstorm-prototype   

Reference:

• https://docs.sandstorm.io/en/latest/developing/http-apis/#creating-an-offer-template

That way at least we can stop relying on the WebKey button.

It's still suboptimal for onboarding mobile users. We'll see if Sandstorm can improve its mobile onboarding soon, but let's not make that a blocker.

This issue tracks the creation of an SPK file that, when uploaded to a Sandstorm server, results in a working Ampache installation without any configuration wizard. (By contrast, in #1, the only important aspect is that the app work in vagrant-spk dev mode.)

The PR for this should fix any issues (e.g. missing files in sandstorm-files.list). To verify that this issue is resolved, I will test on my own system that I can build a SPK out of this git repository.

The Sandstorm SPK should launch properly whether or not the server uses HTTPS for its main URL. You should be able to verify this (1) making sure the SPK works properly when uploaded to a Sandstorm at local.sandstorm.io which vagrant-spk creates for you -- this is HTTP, and (2) by uploading the SPK to https://rose.sandcats.io/ and making sure it works properly on rose.sandcats.io -- this is HTTPS, since rose.sandcats.io uses HTTPS. Hopefully it Just Works, but you might have to fix some issues. I'm available for help as needed.

Here are some technical references on that issue:

• Explanation of HTTP/HTTPS mixed content: https://developer.mozilla.org/en-US/docs/Web/Security/Mixed_content/How_to_fix_website_with_mixed_content
• Discussion of HTTP/HTTPS configuration for an Apache-based Sandstorm app: https://botbot.me/freenode/sandstorm/2016-07-31/?msg=70527911&page=1
• Code in the vagrant-spk lemp stack to hopefully make sure that the app knows if it's behind HTTPS or not: https://github.com/sandstorm-io/vagrant-spk/blob/master/stacks/lemp/service-config/nginx.conf#L31

This would be a follow-up PR that removes the need to manually work through the Ampache configuration wizard.

The most basic integration with Sandstorm user accounts is that:

• When the user visits an Ampache grain, they are automatically logged in to Ampache with full admin rights.
• Therefore the initial owner of the Ampache grain should be careful with whom they share Ampache.

(By contrast, before this issue gets resolved, it's OK if the user of the Ampache app within Sandstorm has to log in to Ampache using a username like admin and a password like admin.)

In this issue, I don't think it's important to support more than one Sandstorm "role."

How to verify this issue is resolved

When creating a new grain of Ampache, on e.g. my laptop using vagrant-spk dev, if I am logged into Sandstorm as "Alice Dev Admin", then I will make sure that the Ampache grain:

• Automatically logs me in, without prompting for a username and password.
• The user that is automatically logged-in has the ability to upload music and play music.

References

Implementing this will require an understanding of how Sandstorm handles authentication and passes the information to apps in HTTP headers. Here are some references on that:

• Technical overview of HTTP headers that Sandstorm sends to apps: https://docs.sandstorm.io/en/latest/developing/auth/
• Example sandstorm-pkgdef.capnp that does not use roles: https://github.com/jacksingleton/hacker-slides/blob/master/.sandstorm/sandstorm-pkgdef.capnp

It will also require creating a new authentication plugin for Ampache, which means writing PHP code, as well as configuring Ampache in Sandstorm to use that authentication plugin. I don't know a whole lot about how to write an authentication module for Ampache, but I have some ideas and one reference. You will probably start by copying the LDAP module, and then modifying it so that it looks at HTTP headers. The authentication module should be part of the pull request that resolves this issue, or alternatively it can be in a separate pull request.

• https://github.com/ampache/ampache/wiki/LDAP

@paulproteus when I try to start vagrant, I got an error requiring login detail. Can you provide me with one?

More detail:

D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype>vagrant-spk vm up
Calling 'vagrant' 'up' in D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype\.sandstorm
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'sandstorm/debian-jessie64' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
The box 'sandstorm/debian-jessie64' could not be found or
could not be accessed in the remote catalog. If this is a private
box on HashiCorp's Atlas, please verify you're logged in via
`vagrant login`. Also, please double-check the name. The expanded
URL and error message are shown below:

URL: ["https://atlas.hashicorp.com/sandstorm/debian-jessie64"]
Error:
Traceback (most recent call last):
  File "<string>", line 1022, in <module>
  File "<string>", line 1019, in main
  File "<string>", line 925, in vm_subcommand
  File "<string>", line 288, in call_vagrant_command
  File "subprocess.py", line 540, in check_call
subprocess.CalledProcessError: Command '['vagrant', 'up']' returned non-zero exit status 1
vagrant-spk returned -1

I have authorized. Please upgrade

Note that we probably only want upstream to allow it to be enabled if the SANDSTORM=1 environment variable is set. Otherwise it creates a security issue for upstream.

When I run vagrant-spk dev with image file set in sandstorm-pkgdef.capnp I got this error:

D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype>vagrant-spk dev
Calling 'vagrant' 'ssh' '-c' '/opt/app/.sandstorm/build.sh && cd /opt/app/.sandstorm && spk dev --pkg-def=/opt/app/.sandstorm/sandstorm-pkgdef.capnp:pkgdef' in D:\xampp\htdocs\PHP projects\Asheesh Laroia\ampache-sandstorm-prototype\.sandstorm
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Nothing to install or update
Package guzzle/common is abandoned, you should avoid using it. Use guzzle/guzzle instead.
Package guzzle/stream is abandoned, you should avoid using it. Use guzzle/guzzle instead.
Package guzzle/parser is abandoned, you should avoid using it. Use guzzle/guzzle instead.
Package guzzle/http is abandoned, you should avoid using it. Use guzzle/guzzle instead.
Generating autoload files
> ComponentInstaller\Installer::postAutoloadDump
Compiling component files
*** Uncaught exception ***
/opt/app/.sandstorm/sandstorm-pkgdef.capnp:130: failed: Couldn't read file for embed: /opt/app/images/preview/catalog.jpg
Connection to 127.0.0.1 closed.
Traceback (most recent call last):
  File "<string>", line 1022, in <module>
  File "<string>", line 1019, in main
  File "<string>", line 674, in dev
  File "<string>", line 288, in call_vagrant_command
  File "subprocess.py", line 540, in check_call
subprocess.CalledProcessError: Command '['vagrant', 'ssh', '-c', '/opt/app/.sandstorm/build.sh && cd /opt/app/.sandstorm && spk dev --pkg-def=/opt/app/.sandstorm/sandstorm-pkgdef.capnp:pkgdef']' returned non-zero exit status 1
vagrant-spk returned -1

The problem is, the file really exist. I even checked from ssh.