GithubHelp home page GithubHelp logo

aaronpk / websub.rocks Goto Github PK

View Code? Open in Web Editor NEW
26.0 5.0 7.0 3.15 MB

Test suite and debug utilities for W3C WebSub

Home Page: https://websub.rocks

License: Apache License 2.0

PHP 92.12% CSS 1.88% JavaScript 0.31% Shell 0.32% XSLT 5.38%
w3c indieweb websub pubsubhubbub validator test-suite

websub.rocks's Introduction

websub.rocks

WebSub test suite and debugging utility.

You can test your Publisher, Subscriber and Hubs using websub.rocks.

Configuring websub.rocks to run locally can be a bit of a challenge, so here are some resources you may find useful to test your local services against the public websub.rocks.

Installation Instructions

Dependencies

Instructions

You can either install websub.rocks from git, or download a zip file from the list of releases.

Installation from Git

Requires installing dependencies via Composer

git clone [email protected]:aaronpk/websub.rocks.git
cd websub.rocks
composer install

Installation from zip release

Contains all dependencies already packaged.

  • Download latest release
  • Extract to a folder on your computer

Configure websub.rocks

Copy the lib/config.template.php file to lib/config.php and fill it out. You'll need to set the following:

  • $base - the full base URL where you've installed websub.rocks, e.g. http://websubrocks.example.com/
  • $redis - configure the host and port to your Redis instance, e.g. tcp://127.0.0.1:6379
  • $dbhost and others - configure the name, host, username and password for your MySQL database
  • $skipauth - set to true to bypass emailing login links
  • $secret - set to a random string, used for signing tokens
  • $mailtun - if you want websub.rocks to email login URLs, configure your Mailgun account info here

Install Redis

Install Redis however is appropriate for your platform. The default configuration is fine.

Install MySQL

Install MySQL, and create a new database called websubrocks.

Create the Database

The database schema is in the database/ folder, so you can set up the database with the following command:

mysql -u root websubrocks < database/schema.sql

Install Nginx

Websub.rocks requires nginx compiled with the nginx push stream module to enable realtime features of the tool.

You will need to build nginx from source in order to include the module. Feel free to follow any nginx tutorial to install it on your platform. The instructions should look more or less like the below.

  • download nginx source from http://nginx.org
  • git clone [email protected]:wandenberg/nginx-push-stream-module.git
  • ./configure --prefix=/usr/local/nginx --add-module=../nginx-push-stream-module --with-http_v2_module
  • make -j 4
  • sudo make install

Nginx Configuration

You'll need to modify your nginx configuration to include the following.

http {
  # your existing defaults are probably fine

  # Set the php pool to where you have configured php-fpm to run.
  # Note this may also be a socket instead of a port.
  upstream php-pool {
    server 127.0.0.1:9000;
  }

  push_stream_shared_memory_size 32M;
}

server {
  listen       80;
  server_name  websubrocks.dev;

  root /path/to/websub.rocks/public;

  # index.php handles all requests that aren't static files
  location / {
    try_files $uri /index.php?$args;
  }

  location ~ \.php$ {
    fastcgi_pass    php-pool;
    fastcgi_index   index.php;
    include fastcgi_params;
    fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
  }

  # These are the push-stream streaming endpoints for publishing and subscribing

  location /streaming/pub {
    push_stream_publisher admin;
    push_stream_channels_path    $arg_id;
  }

  location /streaming/sub {
    add_header 'Access-Control-Allow-Origin' '*';
    push_stream_subscriber eventsource;
    push_stream_channels_path    $arg_id;
    push_stream_message_template                "{\"id\":~id~,\"channel\":\"~channel~\",\"text\":~text~}";
    push_stream_ping_message_interval           10s;
  }
}

websub.rocks's People

Contributors

aaronpk avatar ajorg avatar cweiske avatar da2x avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

websub.rocks's Issues

Hub test 100: server returns HTTP 500 during verification phase

Hub test #100 returns HTTP 500 Internal Server Error during verification phase. For example:

https://websub.rocks/hub/100/sub/G1efInC7xJM2VLBpHove?hub.mode=subscribe&hub.topic=https%3A%2F%2Fwebsub.rocks%2Fhub%2F100%2Fpub%2FG1efInC7xJM2VLBpHove&hub.challenge=bdmRT4a14eHkaCwEvN0sXnkW8sTcjNPk&hub.lease_seconds=864000

https://websub.rocks/hub/100/sub/KkqDrnsQVt1xJLvBvNMi?hub.mode=subscribe&hub.topic=https%3A%2F%2Fwebsub.rocks%2Fhub%2F100%2Fpub%2FKkqDrnsQVt1xJLvBvNMi&hub.challenge=bDugliQDZNLKs2FrHA9O0YH5RrtF65sV&hub.lease_seconds=864000

https://websub.rocks/hub/100/sub/jf2zl4j0xwcM2cUHfZL5?hub.mode=subscribe&hub.topic=https%3A%2F%2Fwebsub.rocks%2Fhub%2F100%2Fpub%2Fjf2zl4j0xwcM2cUHfZL5&hub.challenge=tEZaZGcQH4QM7ezdQrTKemERqXfRPCrp&hub.lease_seconds=864000

Run websub.rocks locally

Hey!

First off, thanks for this project, it is really helpful!

Second, is it possible to host this locally and run against a websub client implementation on localhost? What would be useful for quick debugging, not having to expose the server on the Internet.

If so, could you perhaps add some instructions on how to do that? I couldn't find any, and I'm not familiar with Composer. I can dig into those docs, but it would be nice with a small summary on how to start it up. If I get it up, I can submit a PR.

Thanks!

Hub test 100: The notification is missing the HTTP Link header with rel=self indicating the topic URL of this notification.

I'm testing my WebSub phubb running at http://phubb.cweiske.de/hub.php, and test 100 says after I clicked on "Create post":

Error
The notification is missing the HTTP Link header with rel=self indicating the topic URL of this notification.

I'm pretty sure that I send this header. This is my debug output (JSON-escaped):

"header": [
    "content-type: text/html; charset=UTF-8",
    "content-length: 2700",
    "link: <https://websub.rocks/hub/100/pub/rXgPGgsHoABiRx8WFDUD>; rel=\"self\"",
    "link: <http://phubb.cweiske.de/hub.php>; rel=\"hub\""
],

subscriber validators 201, 202, 300, 301, 302 do not work as expected

The validation tests 201, 202, 300, 301, 302 for the subscriber do not work as I expect them to do:

  • 201/202: They publisher does not respond with a http status code of 301/302 and a http location header. Instead the publisher responds with a http status code 200.

  • 300/301/302: The test ends after a succesful subscription. No content is being delivered by the hub. There is no content notification (http post) being sent.

Idea: Test distribution of ATOM content instead of HTML

Hello,
At the moment, it looks like to the content distributed by a distribution test is only HTML:
https://websub.rocks/subscriber/300

However, there are clients supporting only ATOM (which also seems to be a relatively more common scenario), for which the proposed test does not work.
What do you think about proposing a distribution test of ATOM content?

If there is interest, I could try to submit a PR.
Let me know :-)

verification and validation of intent should 'logically begin' after 202 is returned

I encounter this (among others) at https://websub.rocks/subscriber/100. My original POST subscription request (as defined in # 5.1 Subscriber Sends Subscription Request) gets answered with a 400 code. The body includes the following explanation:
{"error":"verification_failed","error_description":"The callback URL did not confirm the verification request.","code":0,"callback_response":false,"type":"error"}

This seems in violation of the spec (specifically the last quoted MUST NOT):

If a hub finds any errors in the subscription request, an appropriate HTTP [RFC7231] error response code (4xx or 5xx) MUST be returned. In the event of an error, hubs SHOULD return a description of the error in the response body as plain text, used to assist the client developer in understanding the error. This is not meant to be shown to the end user. Hubs MAY decide to reject some callback URLs or topic URLs based on their own policies (e.g., domain authorization, topic URL port numbers). However, since verification and validation of intent are asynchronous steps that logically begin after the HTTP response has been returned, the HTTP response MUST NOT depend on the process or outcome of verification or validation.

It seems like the test should return 202, and only then hit my callback. I discovered this as my test procedure actually only exposed the callbacks after a 202 response is received. I've fixed that as it depends on the (localhost) subscriber outracing the (external) hub in getting them up (which is not guaranteed, even though it's likely), but that doesn't mean there isn't a bug on the test suite side too.

(Moved from w3c/websub#122)

Publisher test: server returns HTTP 500 during intent verification phase

I am investigating why notifications are not getting through in this test:

image

In my hub logs, I see that the request to /publisher/callback fails, with a HTTP 500:

https://websub.rocks:443 "GET /publisher/callback?token=ClIQLYdYwbJWypHye2Md&hub.mode=subscribe&hub.topic=https%3A%2F%2Fwww.example.com%2F&hub.challenge=da7e7403-bf5c-4d3e-bd15-f33305d5453b&hub.lease_seconds=7200 HTTP/1.1" 500 None

(I swapped out my hub's URL with www.example.com)

This causes a failure of the subscriber intent verification at the hub-side, as in "5.3 Hub Verifies Intent of the Subscriber."

Publisher Subscription Test - subscription denied is ignored

After a subscription verify I was testing my code for denial and forced all verifies to fail on my side, but websub.rocks still continued with the test as if it had been approved.

Sequence:

websub.rocks sends subscription POST

hub.callback --> https://websub.rocks/publisher/callback?token=Eonf5DmLMcnwiMbGvumJ
hub.topic --> https://bear.im
hub.mode --> subscribe
hub.lease_seconds --> 7200

kaku then responds with a verify GET

https://websub.rocks/publisher/callback?hub.challenge=089f5374-c579-11e6-a7bb-0cc47a205c0a&token=%5Bu%27Eonf5DmLMcnwiMbGvumJ%27%5D&hub.topic=https%3A%2F%2Fbear.im&hub.mode=subscribe&hub.lease_seconds=7200

kaku then forces a subscribtion denied

https://websub.rocks/publisher/callback?token=%5Bu%27Eonf5DmLMcnwiMbGvumJ%27%5D&hub.topic=https%3A%2F%2Fbear.im&hub.mode=denied&hub.reason=Verification+Failed

The hub test fails because of missing spaces in the link header of the notification

The notification with the following header fails -
"Link: <https://websub.rocks/hub/100/sub/HdyJyGpboUHLlf3FON1k>;rel=self,<http://pubsubhubbub.appspot.com/>;rel=hub

When a space is added after the semicolons and commas, the test passes. AFAICT, the grammar in RFC5988 spec doesn't enforce spaces.

I see that the test uses IndieWeb\http_rels to extract the links which contains the following lines
$links = explode(', ', trim(substr($f, 6)));
and
$hrefandrel = explode('; ', $link);

lease_seconds should be optional during unsubscribing

While testing my implementation with https://websub.rocks/hub/104, after clicking the 'Unsubscribe' button (using a public hub) while 'Waiting for unsubscription to be confirmed' I get the following error:

Error!

The verification request was missing the hub.lease_seconds parameter

The spec has the following to say about lease_seconds (5.3 Hub Verifies Intent of the Subscriber):

hub.lease_seconds
REQUIRED/OPTIONAL. The hub-determined number of seconds that the subscription will stay active before expiring, measured from the time the verification request was made from the hub to the subscriber. Hubs MUST supply this parameter for subscription requests. This parameter MAY be present for unsubscribe requests and MUST be ignored by subscribers during unsubscription.

In short, I think the test suite, acting as a subscriber here, violates the last MUST.

(Moved from w3c/websub#122)

Invalid mode

With the below CURL command Websub.Rocks is responding with an 'invalid mode' error. I cannot determine how I am not passing 'subscribe' correctly. I tried with or without quotes. Is the request malformed? Or, is there some erroneous subtly in Websub.rocks?

$ curl --request POST --url 'https://websub.rocks/blog/100/dtAV4vYXWvpQjYum6wXT/hub?hub.mode=subscribe&hub.topic=https%3A%2F%2Fwebsub.rocks%2Fblog%2F100%2FdtAV4vYXWvpQjYum6wXT&hub.callback=http%3A%2F%2Fwww.example.com%2F12'

$ {"error":"invalid_mode","type":"error"}

Test Plan

Testing your Subscriber

Discovery

  • GET or HEAD, finding the Link header with rel=hub and rel=self
  • GET and look in XML contents
  • GET and look in HTML contents
  • Must prioritize Link header over HTML link tag

Subscription

  • Request a subscription to a hub that sends a 307/308 redirect to another hub
  • Subscribe to a URL that returns a 301 redirect to another URL (see #98)
  • Subscribe to a URL that reports a different rel=self from the input URL

Tracking Features

  • Request a subscription with callback, mode, topic parameters
  • Request a subscription with a secret
  • Request a subscription with lease_seconds
  • Unsubscribe

Subscription Verification

  • Reject a verification request with an invalid topic URL

Tracking Features

  • Acknowledge a valid pending subscription

Distribution

  • Accept a valid signature for authenticated distribution
  • Reject an invalid signature
  • Reject delivery with no signature if subscription was made with a secret

Tracking Features

  • Return 2xx when the notification payload is delivered

Testing your Publisher

The spec only describes how the publisher advertise the hub and topic URLs.

  • Publisher advertises the URLs in HTTP headers
  • Publisher advertises the URLs in HTML tags
  • Publisher advertises the URLs in XML tags
  • Publisher advertises the hub URL in the .host-meta file

These tests are not for compliance, but help publishers test whether their feeds are working properly.

  • A notification is received when the publisher has updated the content

Testing your Hub

Subscription

  • Allows subscription requests with callback, mode and topic parameters
  • Allows subscriptions with a secret
  • Must ignore additional parameters
  • Must allow subscribers to re-request active subscriptions
  • Supports unsubscribing

Tracking Features

  • Allows subscribers to request a lease duration
  • Ignores a lease duration request

Distribution

  • Sends a notification with matching content type
  • Sends only a diff in the notification
  • Sends a notification with a valid signature

Email delivery failing

Lightly anonymized log entry from my mail server:

Nov 19 20:07:20 MACHINENAME postfix/smtpd[14486]: NOQUEUE: reject: RCPT from mail-s68.mailgun.info[184.173.153.196]: 554 5.1.8 [email protected]: Sender address rejected: Domain not found; from=[email protected] to= proto=ESMTP helo=<mail-s68.mailgun.info>

The problem appears to be a missing DNS record (of some kind) for mail.pubsub.rocks?

Create new post on 103 responds with a 500

After successfully subscribing to a 103 feed, creating a new post results in an http 500.
The item is actually created but the publish action to the sub seems to fail as I don't receive any callback.

Screenshots

  1. The feed contains 3 items
    image

  2. The publication fails
    image

  3. A new item appears on the feed
    image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.