What might cause the delay of the subscription callback? about aws-mobile-appsync-sdk-ios

[fans3210]

Here is some new feedbacks. When I just subscribe, I won't miss msgs, but after a while eg: 5-10 mins. Although the mqtt connection state is still connected, msgs are missing. I have to use a temporary solution to force query one days msgs to make sure no msg is missing for my chat app. I don't think it's a solution for long time.

[undefobj]

Hi @fans3210 could you share your GraphQL schema (including the mutations and subscriptions you're running) as well as the client iOS code that you're running?

[fans3210]

@undefobj , here is the latest update of this issue:
https://forums.aws.amazon.com/thread.jspa?threadID=285375&tstart=0

For the code part, I can share some logics related to appsync part. Our app is a chatting app.
graphql url: https://vcxs6xpb6jftfcuwpgl3ldexs4.appsync-api.ap-southeast-1.amazonaws.com/graphql

client side mutation and subscriptions:

subscription NewMsgSubscription($conversationId: String!) {
    subscribeToNewMsg(conversationId: $conversationId) {
        conversationId
        messageId
        messageType
        text
        senderId
        mediaFileUrlStr
        dateSent
        isFromMobile
    }
}

subscription NewConversationSub {
    subscribeToNewConversation
}


mutation NewMessage($conversationId: String!, $messageId: ID!, $messageType: Int!, $text: String, $senderId: String!, $mediaFileUrlStr: String, $dateSent: String!, $isFromMobile: Boolean!) {
    newMessage(conversationId: $conversationId, messageId: $messageId, messageType: $messageType, text: $text, senderId: $senderId, mediaFileUrlStr: $mediaFileUrlStr, dateSent: $dateSent, isFromMobile: $isFromMobile) {
        conversationId
        messageId
        messageType
        text
        senderId
        mediaFileUrlStr
        dateSent
        isFromMobile
    }
}

mutation NewConversation($conversationId: String!) {
    newConversation(conversationId: $conversationId)
}

query GetMessages($conversationId: String!, $after: String!, $limit: Int!, $offset: Int!) {
getMessages(consersationId: $conversationId, after: $after, limit: $limit, offset: $offset) {
        items {
            conversationId
            messageId
            messageType
            text
            senderId
            mediaFileUrlStr
            dateSent
            isFromMobile
        }
    }
}

For the ios code, it's nothing special, I'm just using the code generator to generate the appsync api code, and for appsynclient, I'm using my private fork which contains the changes of the latest appsync release:

https://github.com/fans3210/aws-mobile-appsync-sdk-ios

[fans3210]

There is no issue with mutation, only subscriptions have the missing/delay issue, so I will share a bit of the code about subscription, which is an extension of AppSyncClient class:

Pls ignore the 'NonDisconnectionMQTTStateHandler', I modified source code in my private fork as mentioned above and expose the detailed connection state for each conversation(conversationid as topic) to the user such as: non-observerble, connecting, connected, disconnected etc. That is to guarantee on my side that the msg delay/missing happened when I'm in 'connected' state

public func subscribe(convervationId: String, otherMQTTConnectionStateHandler: @escaping NonDisconnectionMQTTStateHandler, simpleResultHandler handler: @escaping SimpleSubResultHandler<Message>) -> AWSAppSyncSubscriptionWatcher<NewMsgSubscriptionSubscription>? {
      let sub = NewMsgSubscriptionSubscription(conversationId: convervationId)
      do {
          
          let watcher = try self.subscribe(subscription: sub, otherMQTTConnectionStateHandler: otherMQTTConnectionStateHandler) { result, _, err in
              
              if let err = err as? AWSAppSyncClientError {
                  
                  //handle 403 case here
                  //handle 403 case here
                  if let statusCode = err.response?.statusCode, statusCode == 403 {
                      handler(.fail(OnePipAppSyncErr.sessionExpired))
                      return
                  }
                  
                  handler(.fail(OnePipAppSyncErr.common(err)))
                  return
              }
              
              if let err = err as? AWSAppSyncSubscriptionError {
                  Log.error("subscription callbackfunc ERR  client disconnected for dialogId: \(convervationId), err: \(err)")
                  handler(.fail(OnePipAppSyncErr.clientDisconnected))
                  return
              }
              
              
              //graphql errs
              if let errs = result?.errors, errs.count > 0 {
                  
                  handler(.fail(OnePipAppSyncErr.common(errs.first!)))
                  return
              }
              
              
              Log.verbose("subscribe to new msg result handler called without err")
              if let result = result, let newMsg = result.data?.subscribeToNewMsg {
                  
                  let df = Formatter.iso8601
                  let senderId = newMsg.senderId
                  ,messageId = newMsg.messageId
                  ,dialogId = newMsg.conversationId
                  ,dateSent = df.date(from: newMsg.dateSent)!
                  ,type = AppSyncMsgType(rawValue: newMsg.messageType)!
                  
                  var msgData: MessageData
                  switch type {
                  case .text:
                      //text
                      let jsonContent = newMsg.text!
                      ,data = jsonContent.data(using: .utf8)!
                      ,jsonDecoder = JSONDecoder()
                      ,wrapped = try! jsonDecoder.decode(AppSyncMsgContentWrapper.self, from: data)
                      ,content = wrapped.content
                      
                      msgData = .text(content)
                  case .audio:
                      //audio
                      let jsonContent = newMsg.mediaFileUrlStr!
                      ,data = jsonContent.data(using: .utf8)!
                      ,jsonDecoder = JSONDecoder()
                      ,wrapped = try! jsonDecoder.decode(AppSyncMsgContentWrapper.self, from: data)
                      ,fileKey = wrapped.content
                      
                      let remoteAudioFileMeta = FileMeta(mediaType: .audio, fileKey: fileKey, moduleSource: .chat, localDirectory: nil, thumbnailLocalDirectory: nil, localId: UUID().uuidString)
                      
                      msgData = .media(attachment: remoteAudioFileMeta)
                  case .image:
                      //image
                      let jsonContent = newMsg.mediaFileUrlStr!
                      ,data = jsonContent.data(using: .utf8)!
                      ,jsonDecoder = JSONDecoder()
                      ,wrapped = try! jsonDecoder.decode(AppSyncMsgContentWrapper.self, from: data)
                      ,fileKey = wrapped.content
                      
                      let remoteImageFileMeta = FileMeta(mediaType: .image, fileKey: fileKey, moduleSource: .chat, localDirectory: nil, thumbnailLocalDirectory: nil, localId: UUID().uuidString)
                      msgData = .media(attachment: remoteImageFileMeta)
                  case .doc:
                      //doc
                      let jsonContent = newMsg.mediaFileUrlStr!
                      ,data = jsonContent.data(using: .utf8)!
                      ,jsonDecoder = JSONDecoder()
                      ,wrapped = try! jsonDecoder.decode(AppSyncMsgContentWrapper.self, from: data)
                      ,fileKey = wrapped.content
                      ,metaData = wrapped.metaData
                      
                      var remoteDocFileMeta = FileMeta(mediaType: .doc, fileKey: fileKey, moduleSource: .chat, localDirectory: nil, thumbnailLocalDirectory: nil, localId: UUID().uuidString)
                      remoteDocFileMeta.metaData = metaData
                      
                      msgData = .media(attachment: remoteDocFileMeta)
                  }
                  
                  //by default set the read to true for query msgs, force for subscription
                  let msg = Message(messageId: messageId, dialogId: dialogId, senderId: senderId, data: msgData, dateSent: dateSent, isRead: false)
                  handler(.result(msg))
                  
              } else {
                  handler(.fail(OnePipAppSyncErr.unknown(nil)))
              }
          }
          
          return watcher
      } catch {
          guard let _ = error as? AWSAppSyncSubscriptionError else {
              handler(.fail(OnePipAppSyncErr.common(error)))
              return nil
          }
          
          handler(.fail(OnePipAppSyncErr.clientDisconnected))
          return nil
      }
      
  }

[fans3210]

And one more thing, I received "subscription terminated" error on my ios client very frequently these days. Is it due to my network? Although I still have re-subscription logics, I didn't face it that often before.

[fans3210]

Please feel free to ask me if the code provided is not enough for you and I will try to make a mini-project if needed

[undefobj]

What kind of a resolver are you using? DynamoDB or Lambda?

[fans3210]

@undefobj lambda which inserts data to the RDS. Every time I faced subscription error I would check the log and db and no issue with the lambda function and the data insertion.

[undefobj]

Subscriptions in AppSync don't fire until the data source returns results to AppSync. In the case of Lambda, you control the execution so it's possible that even though the write to the database is happening your function isn't exiting (or isn't exiting properly) hence the delay between when subscription notifications are triggered. If you have a higher Lambda timeout and also don't return values from the function immediately after doing the write, you'll see delays.

[fans3210]

@undefobj , for some delay, it's not like a few seconds, but is missed. As mentioned in https://forums.aws.amazon.com/thread.jspa?threadID=285375&tstart=0, I received all the missing msgs from other topics immediately after receive one in my subscribed topic

[undefobj]

It's possible that the Lambda times out and a well formed response isn't being sent back to the AppSync resolver. Again, with Lambda it's in your control to execute and return context to AppSync or unexpected things can happen. What I might suggest is since you have the forum thread open, posting your Lambda code and config in that repo along with request IDs and timestamps so that one of the service engineers could look at the code & tracing logs to help pin down what is happening.

[fans3210]

@undefobj will do and will append the lambda functions here and in the forum too. I've also opened a case in the support center and one engineer asked me to attempt to open a few tabs in my browser and use appsync console to make the mutation and subscriptions. I received each message in my subscriptions. No message was lost. I was also curious whether it's due to my office network because I used swiftybeaver to check the logs of the usage of the ios app and find a lot of subscription errors recently from the call back:

subscription callbackfunc ERR client disconnected for dialogId: 10069, err: AWSAppSyncSubscriptionError(additionalInfo: Optional("Subscription Terminated."), errorDetails: Optional(["recoverySuggestion": "Restart subscription request.", "failureReason": "Disconnected from service."]))

Although I can guarantee that no session is expired and network is not down...

[fans3210]

Attached is the lambda func of the mutation, timeout is 10 seconds:

'use strict';

const mssql = require('mssql')
const moment = require('moment')

const config = {
  user: process.env.DB_USER,
  password: process.env.DB_PWD,
  server: process.env.DB_SERVER,
  database: process.env.DB_DBNAME,

  options: {
    encrypt: false // Use this if you're on Windows Azure
  }
}

exports.handler = (event, context, callback) => {
  
  mssql.connect(config)
    .then(pool => {
      const parsedDateTime = moment(event.dateSent).utc().format('YYYY-MM-DD HH:mm:ss')
      console.log('pool from callback, datetime = ' + parsedDateTime)
      
      //FIXME: note, mediafileurl str is filekey, will be renamed
      const content = event.messageType == 0 ? event.text : event.mediaFileUrlStr
      console.log('content of msg is' + content + 'isFromMobile value is ' + event.is)
      
      return pool.request()
      .input("msgId", mssql.VarChar(50), event.messageId)
      .input("userId", mssql.BigInt, event.senderId)
      .input("chatId", mssql.BigInt, event.conversationId)
      .input("text", mssql.NVarChar(mssql.MAX), content)
      .input("dateSent", mssql.VarChar(50), parsedDateTime)
      .input("messageType", mssql.Int, event.messageType)
      .input("IsFromMobile", mssql.Bit, event.isFromMobile)
      .query(
        'insert into Messages (MessageID, UserID, ChatID, CreatedTimeStamp, MessageType, Content, IsFromMobile) \
        values (@msgId, @userId, @chatId, @dateSent, @messageType, @text, @isFromMobile)'
      )
      
    }).then(result => {
      console.log('insert result')
      mssql.close()
      console.log(result)
      return callback(null, {
        'conversationId': event.conversationId,
        'messageId': event.messageId,
        'messageType': event.messageType,
        'text': event.text,
        'senderId': event.senderId,
        'mediaFileUrlStr': event.mediaFileUrlStr,
        'dateSent': event.dateSent,
        'isFromMobile': event.isFromMobile
    })
    }).catch(err => {
      console.log('err is ' + err)
      callback(err)
    })

  mssql.on('error', err => {
    console.log('mssql on error:' + err)
    return callback(err)
  })

}

[rohandubal]

@fans3210 we have released a bunch of improvements for subscriptions in SDK version 2.6.21 Could you please try and see if you are seeing improvements?

[fans3210]

@rohandubal . Sorry totally forgot about this issue. Will close it. Thanks.