Messages sent out through simcard (via task checking) don't appear in sent items list. about smssync

[Nhorning]

OK. This is a severe issue. I just had a report from the field where someone didn't use my filter word. I clicked on the icon to import messages from my in box, held down on the message I wanted to send to Ushahidi and clicked the check mark. Nothing happened. So I clicked on the "sync" icon. Suddenly 30 or so messages popped up on sms sync saying "Smssync message sent" and "sms delivered" and this kept happening even after I stopped the service and closed the app. There are about 25 messages in my other sim card's in-box saying "thank you for reporting" and I have no idea what I just sent out to my user base. Did I just resend out all of the alerts I've been sending? Did they even send in the first place? I don't know.

What happened may have been the result of my incompetent use of the interface. It may have been a bug. I have no way of going back and piecing together what happened because there is no record of it.

[eyedol]

@Nhorning try to reproduce the issue. It's would be the easiest way for me to figure out where the issue might be coming from.

[Nhorning]

Hi Henry,

Sorry if I wasn't clear above. This issue I'm reporting here isn't what happened to me today. The issue I'm reporting is that I have no record of what items are being sent out via sms through my phone by SMSsync. I need to to have that, or I don't really know what I have been sending to people. What happened today served to illustrate that.

Case in point: While I could likely easily reproduce the issue I had today by following the same steps above. I will under no circumstances do that, because I don't know what happened and what messages went to who. I can't reproduce something for you that might result in all the people I've signed up for this deployment receiving multiple duplicate messages on their phone. They would likely very quickly unsubscribe from alerts. We need to have a record of what is being sent out so we can

A: Check to see that the task checking feature is actually working and that messages are being distributed.

B: Be able to tell you what happened when there was an error, and know whether or not it is safe to try and reproduce something, without having to risk creating a negative user experience.

[eyedol]

@Nhorning This is more a feature then. I'll add a label to indicate which messages have been sent via SMS. At the moment it just for HTTP.

You can also request a feature on Ushahidi so the alert feature can have an interface to manage the alerts being sent.

[Nhorning]

I'm pretty sure I already did that. I know that I meant to, but I'll
check to make sure. That would help, but this issue is different,
however. I think it's more important to see that the messages ushahidi
thinks are sending are actually being sent. I can check in the Alerts
tab in the Ushahidi interface to see how many alerts Ushahidi says have
gone to each recipient. What I can't do is check on the phone to see
that those were actually sent.

On 11/01/2012 02:36 PM, Henry Addo wrote:

@Nhorning https://github.com/Nhorning This is more a feature then.
I'll add a label to indicate which messages have been sent via SMS. At
the moment it just for HTTP.

You can also request a feature on Ushahidi so the alert feature can
have an interface to manage the alerts being sent.

—
Reply to this email directly or view it on GitHub
#54 (comment).

Web Bug from
https://github.com/notifications/beacon/J6T91GIPIyhU-8ti4GCGP2jqgtYm9FMvKhd1la_Hg5VAAduJ0R1dOgFd-OC8pGRa.gif

Neil Horning
Information and Communications Technology
PBI Nepal
Mobile: [phone]
Office: [phone]

[mandric]

Can't you also save the response you generate for SMSSync on the server side? I dont' really think it's the responsibility of SMSSync to track all your outgoing/incoming messages, it merely shuffles them along so once data hits your web server, it's up to you to track it, what data was sent/received to/from SMSSync?

[Nhorning]

Henry, do you have any update on this? I haven't received any messages from the area's that I've signed myself up for, and I highly suspect that task checking is not working consistently. I know it works some of the time because the balance on the sim card drops, but I need to know when it is and when it isn't so I can adjust accordingly.

@mandric: As long as SMSSync doesn't pass delivery reports back onto the server, the server has no way of knowing which messages have actually been sent.

A while ago I did some very simple testing because I noticed that at least a few messages that should have been sent out from SMSSync never got through.

The test was very simple; I manually inserted timestamped rows into the sending queue table in our web server's database and waited for the messages to appear on my other phone. At first it looked like some messages got dropped, but they came through after some minutes, out of order. Fine, that's not so serious in our use case and I don't even know if the relevant GSM network standards promise to deliver messages in order. What's more interesting, to me, anyway, is that some of them arrived on the target phone twice.

If message delivery is not completely 100 % trustworthy, then it would be a very nice option for diagnostic purposes to have SMSSync keep a list of outgoing SMS messages for which delivery confirmation has not been received(It shouldn't automatically try to resend them until receipt has been confirmed(to avoid spamming due to mere delays), but it could be an option.)

[mandric]

@agenttihirvinen I think there is also some inherent fallibility we have to live with but we can minimize that. I also don't think we want to put more complexity into smssync, like an outgoing message tracker. Once smssync has queued up a message in the android sms plumbing and replies with a positive response, it is done. We might want to add that last part, a callback request smssync makes to finalize a queued outgoing transmission.

[mandric]

Looking at the code, right now I don't think the sendResponseFromServer method returns anything. We could have it return a timestamp or null on failure then we can construct another http request that sends that data back to your server like:

PUT /v1/your-api/
Host: 192.168.2.1:80

{ "uuid": "7cb8cfb7267c2296d3f4ff9e08c4f80e","sms_queued_on":1353004747307}

I'm using the idea of HTTP Callbacks I've been using on my installation, not for this purpose, but it could provide that, https://github.com/mandric/SMSSync/wiki/HTTP-Callbacks. I'm in favor of minimizing the amount of "data management" we do on smssync and keep it as simple as possible.

[mandric]

Here is basically where the sms data ends up going: http://developer.android.com/reference/android/telephony/gsm/SmsManager.html#sendMultipartTextMessage(java.lang.String,java.lang.String,java.util.ArrayList<java.lang.String>,java.util.ArrayList<android.app.PendingIntent>,java.util.ArrayList<android.app.PendingIntent>)

So if there is no exception thrown here then we have success at this point?

[Nhorning]

It is possible that I don't completely understand your approach, but until your reasoning is explained fully I will remain emphatically apposed to it. We are not all developers here. From the way it looks, I have no way of doing anything with what you are proposing besides wait for someone on Ushahidi/Web to write a patch that tracks whether SMSSync is doing it's job with outgoing messages. From a user stand point that is far from ideal. SMSsync should tell me whether it is working. I shouldn't have to write (or ask someone to write and then wait multiple months for) a separate interface to tell me whether it is working.

If SMSsync can maintain a list of messages that have been sent from the mobile network to my server, it should also maintain a list of messages that have been sent from my server to the mobile network. Full stop.

[mandric]

SMSSync does tell you it's working. When a message goes into the Sent box, it has been processed successfully. If you have Get Reply From Server checked then a success message was received from the server and outgoing messages have been queued. If there is a failure, SMSSync should display a message and keep the message in the pending box. If this is not the case then we should fix that bug. If you have additional processing needs then you should build them into your application, not SMSSync. Your features/needs will get much more mileage in your application then in SMSSync. SMSSync should remain simple, limited and stable doing the least amount of work as possible. Do one job, and do it well.

It's better to put your efforts into your application than in SMSSync. You don't want to be checking SMSSync for outgoing messages. You don't want to be checking SMSSync at all, actually. It should quietly be working and interfacing with your application, running tasks. We should put more control in the HTTP API layers, not in SMSSync itself. So your application can drive SMSSync better and communicate with it better.

Maybe adding an outbox is not out of scope, it's not my call, might even be a good idea, I just dont' really care if my data is in SMSSync, I want it in my application. I see more value in adding a log view or something so if we need to inspect some history or error reporting, it's available, for example.

SMSSync does flash a notification "message sent" and "message delivered". Instead of the notification, it could put the message in a queue and then flush this queue to the server on next task check/auto sync attempt. That shouldn't be too complicated, and would allow much better tracking on the server side.

[Nhorning]

1. "When a message goes into the Sent box, it has been processed successfully. If you have Get Reply From Server checked then a success message was received from the server and outgoing messages have been queued. If there is a failure, SMSSync should display a message and keep the message in the pending box. If this is not the case then we should fix that bug" ----- This is not the case. My outgoing messages do not appear in my sent box, and when they fail, they don't appear in my pending box. That is why I filed this BUG 18 days ago.
2. I am not building an application. I am using Ushahidi to send alerts of human rights abuses and security incidents to ~150 human rights defenders in the field. At the top of this page it says "Ushahdi/SMSsync." That indicates to me that this application should function together well with Ushahidi. When it is pointed out that it does not do this, developers should work to fully understand the issue and then work towards solving it.

[Nhorning]

@agenttihirvinen Yes. That would be good, however it should also include the outgoing sms messages in the sent items list. To do otherwise would require waiting for the ushahidi team to write a better alert tracking interface into ushahid/web. It would be fine if they did this as well, but I assume that would take a development cycle or two. In the meantime I have 150 HRDs in the field, and I don't know whether my messages are going to them or not. I need this application fixed. Then lets worry about feature requests for Ushahidi/Web.

[mandric]

@Nhorning Have you thought about reporting this bug on the web app issue tracker? I think it would be good to link these two issues.

[Nhorning]

Nice!