parser cannot handle numbers about qjson

[operatornormal]

It seems to me that this issue might be due to usage of global variable errno in multithreaded environment. Yet this problem has occurred systematically and reliably with me. I'm not sure. But anyway, in json_scanner.yy/cc there is usage of method strtoull() and afterwards there is test about global error variable errno. This is prone to break if any method anywhere sets errno, including everything in stdio.h etc. Fortunately there is also easy fix as according to documentation strtoull will return ULLONG_MAX if number is too large and this return value naturally is not affected by MT environment. So I fixed the problem this way:

diff -u -r q1/qjson-master/src/json_scanner.cc q2/qjson-master/src/json_scanner.cc
--- q1/qjson-master/src/json_scanner.cc 2014-12-12 17:46:47.000000000 +0200
+++ q2/qjson-master/src/json_scanner.cc 2014-12-21 18:39:44.000000000 +0200
@@ -3393,8 +3393,10 @@
 #line 82 "json_scanner.yy"
 {
                 m_yylloc->columns(yyleng);
-                *m_yylval = QVariant(strtoull(yytext, NULL, 10));
-                if (errno == ERANGE) {
+               unsigned long long converted = strtoull(yytext, NULL, 10);
+               fprintf(stderr, "Converted number %lu errno = %d\n", converted, errno) ;   
+                *m_yylval = QVariant(converted);
+                if (errno == ERANGE && converted == ULONG_LONG_MAX) {
                     qCritical() << "Number is out of range: " << yytext;
                     return yy::json_parser::token::INVALID;
                 }
diff -u -r q1/qjson-master/src/json_scanner.yy q2/qjson-master/src/json_scanner.yy
--- q1/qjson-master/src/json_scanner.yy 2014-12-12 17:46:47.000000000 +0200
+++ q2/qjson-master/src/json_scanner.yy 2014-12-21 18:39:18.000000000 +0200
@@ -81,8 +81,10 @@
 [0-9]         |
 [1-9][0-9]+   {
                 m_yylloc->columns(yyleng);
-                *m_yylval = QVariant(strtoull(yytext, NULL, 10));
-                if (errno == ERANGE) {
+               unsigned long long converted = strtoull(yytext, NULL, 10);
+               fprintf(stderr, "Converted number %lu errno = %d\n", converted, errno) ;   
+                *m_yylval = QVariant(converted);
+                if (errno == ERANGE && converted == ULONG_LONG_MAX) {
                     qCritical() << "Number is out of range: " << yytext;
                     return yy::json_parser::token::INVALID;
                 }

Here 2 additional things might be considered if anyone is willing to commit this into version control too: 1: removal of the printf() as it was for me only and 2: removal of the conditioning with errno as it really is not thread safe. I have been using qjson in a program that happily parses in multiple threads and also does IO so .. errno just might come up for whatever reason.

Also it might be worth mentioning that the problem occurred only in MinGW build. In linux I have never seen this.

[flavio]

That's strange. Can you provide more details? QJson has an extensive test suite which would have spotted this issue.

[jrcole45]

My function is:

ErrorCode FormatJson::deSerializeJson(const QByteArray &request, QVariantMap &result)
{
    bool canParseFromJson;

    QJson::Parser parser;
    result = parser.parse(request, &canParseFromJson).toMap();

    if(canParseFromJson == false)
    {
        result = QVariantMap();
        return ERR_INPUT_PARAMETER;
    }

    return SUCCESS;
}

#############################################
This is what I am serializing (serialization works fine) with serializer

QVariantMap request;
qint64 channelNumber = 4;
    request.insert(""parameters, channelNumber);
    request.insert(command, "form_network");
 request.insert("type" "zigbee_manage");

JSON from serializing I am using is:
{ "command" : "form_network", "parameters" : 4, "type" : "zigbee_manage" }

Number is out of range: 4
json_parser - syntax error found, forcing abort, Line 1 Column 1

It works if I use request.insert(""parameters, QString::number(channelNumber));

[flavio]

I'm a bit confused, can you just write down the json data you are trying to convert to a QVariant?

[flavio]

BTW, I tried to do something like that:

void TestSerializer::testFoo()
{
  Serializer serializer;
  Parser parser;
  QVariantMap request;
  bool ok;

  quint64 num = 4;
  request.insert(QLatin1String("parameters"), num);
  request.insert(QLatin1String("command"), QLatin1String("foo"));

  const QByteArray json = serializer.serialize( request, &ok);
  QVERIFY( ok );
  qDebug() << json;

  QByteArray expected = "null";
  QVariant writtenThenRead = parser.parse( json, &ok );
  QVERIFY(ok);
  qDebug() << writtenThenRead;
}

But everything works fine when using latest version of QJson from git:

QDEBUG : TestSerializer::testFoo() "{ "command" : "foo", "parameters" : 4 }" 
QDEBUG : TestSerializer::testFoo() QVariant(QVariantMap, QMap(("command", QVariant(QString, "foo") ) ( "parameters" ,  QVariant(qulonglong, 4) ) )  )

[Drusy]

Hello flavio,

I am reproducing this issue with on the dev/master with your code example.
Serialization seems fine, but the parsing still fails :

true "{ "command" : "foo", "parameters" : 4 }" 
Number is out of range:  4 
json_parser - syntax error found,  forcing abort, Line 1 Column 1 
false "{ "command" : "foo", "parameters" : 4 }" 
QVariant(, ) 

Do you have any clue ?
Regards.

[mugiseyebrows]

Reseting errno before calling strtoull also works. I got this issue only when I build application (not qjson itself) in release mode, and it probably has something to do with signals and slots (multithreading?). I use MinGW gcc version 4.6.2 (GCC).

[chiffonqian]

yeh,i have the same problem.
"{"state":1,"obj":"OK","partnerKey":null}"
Number is out of range: 1
the problem only happen in only some pc(win7),and when using QNetworkReply.
the reply is not empty,but the parser cannot work .