No test backend about mod_push

[royneary]

The existing backends could simply log the notifications before sending them.
But I suppose your idea is to prevent connections to the vendor's push service before having configured credentials (e.g. the APNS certificate), am I right?

[basak]

Yes - not just the credentials, but not even so far as getting a token. There are many steps involved in getting everything working correctly as a whole, and it's useful to be able to build things up one step at a time. So to start with, I'm just trying to register with a fake token in order to see if a notification attempt even goes out at the time I expect. It fails - I get a not implemented error for a pubsub push - but it isn't clear to me if this is because I have done something wrong or if that part has yet to be implemented in your implementation here.

BTW, excellent job so far - thank you. My motivation is to get XMPP working well on my Ubuntu phone, although I'm not sure how much time I can commit to this. So I'm trying to write a PoC client that works, though I think I'm caught up with where you're up to now and just need to wait for you to finish the ejabberd mod_push implementation before I can continue. My approach is to make a client work on my desktop machine first, since it is easier for me this way and I know I can transfer that stack to the Ubuntu phone easily enough later. So it is convenient to not have to integrate with the Ubuntu SDK until I have integration with ejabberd and mod_push working.

[royneary]

a mod_push_dummy backend shouldn't be much work, I put it on my todo list. Currently invalid credentials or tokens don't have any side effects (in the future I'm planning to unregister clients automatically in such cases), the vendor's server response will just be logged. You must set "loglevel: 5" for that.

Do you have two mod_push instances running and one of them is trying to publish at the other's push pubsub host? That should work. Can you post the mod_pubsub and mod_push parts of your configuration?

[basak]

I have a single ejabberd instance running with mod_push enabled. I have two accounts on it. I'm connecting to one account with my experimental client that registers for push notifications. Then I'm sending a message to it from a regular client, and expecting to see an attempt to connect to Ubuntu's push server to deliver the push (or, if I had a test push backend, then just a log entry).

I'm not sure exactly what logic mod_push uses to decide what should prompt a push notification to be sent (IMHO this should be part of the XEP). I tried sending a message when my experimental client is still online, and again after a disconnect without closing the session.

I don't see the pubsub error any more when I try this, but I do see an error logged in ejabberd:

2015-07-07 19:40:30.705 [info] <0.2362.0>@ejabberd_c2s:fsm_next_state:2491 Waiting for resumption of stream for [email protected]/pushclient
2015-07-07 19:40:30.705 [error] <0.2362.0>@ejabberd_hooks:run_fold1:371 {undef,[{mod_push,on_wait_for_resume,[300000,{jid,<<"ubuntu">>,<<"example.com">>,<<"pushclient">>,<<"ubuntu">>,<<"example.com">>,<<"pushclient">>}],[]},{ejabberd_hooks,safe_apply,3,[{file,"src/ejabberd_hooks.erl"},{line,385}]},{ejabberd_hooks,run_fold1,4,[{file,"src/ejabberd_hooks.erl"},{line,368}]},{ejabberd_c2s,fsm_next_state,2,[{file,"src/ejabberd_c2s.erl"},{line,2500}]},{p1_fsm,handle_msg,10,[{file,"src/p1_fsm.erl"},{line,582}]},{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,247}]}]}
running hook: {mgmt_wait_for_resume_hook,[{jid,<<"ubuntu">>,<<"example.com">>,<<"pushclient">>,<<"ubuntu">>,<<"example.com">>,<<"pushclient">>}]}

but I don't see any attempt to send out a push notification in tcpdump. My experimental client logs all the XML communication: http://paste.ubuntu.com/11837340/

The ejabberd configuration for this test is:

 mod_pubsub: 
  host: "push.example.com"
  nodetree: "virtual"
  plugins:
    - "push"
mod_push:
  backends:
    -
      type: ubuntu
      register_host: "example.com"
      pubsub_host: "push.example.com"

Apart from that the only change from default is to bump the loglevel to 5 and to add "example.com" to hosts. I don't have example.com registered in DNS obviously - I presume this isn't required for testing as its only ejabberd that needs to know (I've overridden both my experimental and standard client to connect to localhost).

[royneary]

You need to set jid='push.example.com' in the enable request.

Push notifications are sent only when a client is in stream management's 'waiting for resumption' state (aka zombie state). Mobile clients are expected to close their TCP connection (without sending </stream>) when they want to receive push notifications. Perhaps I should write that more clearly into the documentation.

But that undef error is something else. I recently changed both hooks in my ejabberd fork [1] and mod_push [2]. Did you get those changes in both ejabberd and mod_push when you installed?

[1] royneary/ejabberd@af49a9b
[2] f250de2

[basak]

Did you get those changes in both ejabberd and mod_push when you installed?

I started a few days ago, so no, but I did pick up the mod_push update yesterday. But I hadn't realised that you'd made a corresponding change in the ejabberd fork so that will be what I was missing. Updating this and correcting the registration jid as you described made it work - thanks! Now I see a request going out to push.ubuntu.com that then fails (as expected).

I can experiment with getting the client code going now. Thank you!

Mobile clients are expected to close their TCP connection (without sending ) when they want to receive push notifications.

This might be awkward as I'm not sure that the Ubuntu app environment has any opportunity to run code before it is suspended. I will look into this.

[royneary]

Thank you too for testing mod_push!

A while ago I read about the about_to_suspend and about_to_stop handlers in [1]. But apparently the application model is not ready yet. According to [2] apps currently get killed with SIGSTOP when they leave foreground. Thats not nice :-/ I'm very interested in your findings. My motivation before starting mod_push was to make an Ubuntu phone client, too. Maybe we can cooperate after mod_push is in a reasonable state.

[1] https://docs.google.com/document/d/1ij8RtPsR_eYMW3mys8Gu1Y2CVFZpjXdMpdIjIGZ1SCA/edit?pli=1
[2] https://lists.launchpad.net/ubuntu-phone/msg13602.html

[basak]

OK, so I've got my prototype to the stage where it signs in with XMPP, gets a push token, registers it, and the notifications arrive back in correctly.

If I kill the app, then the TCP connection is terminated and mod_push sends notifications that arrive. This was useful for testing that the whole thing is working, so mod_push has been functional enough to get this far - thank you!

However if I just switch the app to the background, then you're right in that SIGSTOP is used to stop the app from consuming battery (and nefarious user-hostile communication, etc). At this stage:

• The TCP XMPP connection is still alive.
• If I send a message, it arrives into the phone's TCP queue (as seen by netstat).
• The server's TCP send queue is empty, so I presume this means that the kernel has sent a TCP ack even though the client process is stopped. So I further presume that the server will never see a TCP timeout in this state no matter how long it waits, except maybe if it sends so much data that the client's kernel doesn't want to buffer it for userspace any more.
• The client's TCP receive queue contains bytes, so I presume this is the incoming message stanza waiting for the client process to be resumed.
• Currently, this means that mod_push doesn't send a notification, which means that the app won't really work.

I could try to see if I can get the app to detect that it is being backgrounded and kill the TCP connection. But even if the environment allows me to do this I'm not sure this is ideal - it'd be race between the app being suspended and being able to execute code in time.

I think a better way might be for the server to detect if the XEP-0198 ack hasn't come back within a fairly short timeout (eg. 30 seconds). This would make for quite a good user experience too I think - no matter what, if the app hasn't actively (in userspace) acked the message, then there will be a notification to the user. The client (and/or notification helper) can always de-dupe any resends later (both for notification and message stanza retransmissions).

Would this be difficult to implement in mod_push/ejabberd?

I also wonder to what extent the required behaviour (when to push) needs to be configurable dynamically by the client rather than being hardcoded in the server configuration, as different clients may need different behaviours.

Maybe we can cooperate after mod_push is in a reasonable state.

Sure!