abdallah-elshamy / octave-gsoc-json Goto Github PK

To make it easier for newcomers to this fork to see what is important, I suggest to change the default branch to json

https://docs.github.com/en/github/administering-a-repository/setting-the-default-branch

Just as a reminder in extension to our discussion on the Octave maintainers mailing-list I finally figured out a working configure.ac check for RapidJSON, especially for a newer version by checking for the rapidjson/cursorstreamwrapper.h header.

diff -r 652a675c0916 configure.ac
--- a/configure.ac	Mon Aug 03 11:13:26 2020 -0400
+++ b/configure.ac	Wed Aug 12 12:49:26 2020 +0900
@@ -1331,6 +1331,22 @@ OCTAVE_CHECK_LIB(pcre, PCRE,
   ],
   [libpcre], [REQUIRED])

+### Check for RapidJSON header only library.
+
+AC_LANG_PUSH([C++])
+AC_CHECK_HEADER([rapidjson/rapidjson.h], [have_rapidjson=yes], [have_rapidjson=no
+  rapid_json_warning="RapidJSON library not found.  Octave will not be able to read or write JSON files."])
+if test "$have_rapidjson" = yes; then
+  AC_CHECK_HEADER([rapidjson/cursorstreamwrapper.h], [have_rapidjson=yes], [have_rapidjson=no
+    rapid_json_warning="RapidJSON 1.1.0 or older found, but latest development version needed.  Octave will not be able to read or write JSON files."])
+fi
+if test "$have_rapidjson" = yes; then
+  AC_DEFINE(HAVE_RAPIDJSON, 1, [Define to 1 if RapidJSON is available.])
+else
+  OCTAVE_CONFIGURE_WARNING([rapid_json_warning])
+fi
+AC_LANG_POP([C++])
+
 ### Check for readline library.

 OCTAVE_ENABLE_READLINE

Using this patch for merging with the Octave sources, can you wrap the code in the main function of jsondecode.cc and jsonencode.cc with something like this for HAVE_RAPIDJSON

https://hg.savannah.gnu.org/hgweb/octave/file/652a675c0916/libinterp/dldfcn/audioread.cc#l82
https://hg.savannah.gnu.org/hgweb/octave/file/652a675c0916/libinterp/dldfcn/audioread.cc#l192

Calling the Octave interpreter is expensive. Please avoid calls like these
https://github.com/Abdallah-Elshamy/octave/blob/7b96f4464082091fbfcaa6f8d4b686c31a054bee/libinterp/corefcn/jsondecode.cc#L87

and use the octave_NaN macro defined in the liboctave/util/lo-ieee.h header.

Two small fixes, making the code more readable:

and

Abdallah,

Sorry for coming late at your code, but I assume, you are also busy with your exams. You did not contribute that much code yet, but there will be more to come after your exams 😉

You introduced the folder "test/json-encode-decode", I would suggest "test/json" to be sufficient.

In the header section of your file:

https://github.com/Abdallah-Elshamy/octave/blob/1538ec5f09f0d78a6b7654238af5a2dd33c711c2/test/json-encode-decode/jsondecodetest.m#L1-L3

I suggest you note, that it should not conform to Octave coding convention to be able to be tested in Matlab, for example see

https://github.com/Abdallah-Elshamy/octave/blob/1538ec5f09f0d78a6b7654238af5a2dd33c711c2/test/classdef/classdef.tst#L29-L30

Maybe you can copy your header from there.

https://github.com/Abdallah-Elshamy/octave/blob/7b96f4464082091fbfcaa6f8d4b686c31a054bee/libinterp/corefcn/jsondecode.cc#L34-L45

This function clearly belongs to the string_vector class implemented in liboctave/util/str-vec.h liboctave/util/str-vec.cc. The functionality is not related to JSON only.

Please add a comment for now, that we do not forget to get this right before preparing the final cset.

As we discussed last Friday, we should consider proper treatment of all ASCII characters in both JSON conversion directions. According to Wikipedia this should be all:

\0\a\b\t\n\v\f\r !"#$%&''()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

Own experiments with Matlab showed me, that basically escaping with \ happends, e.g. \n becomes \\n, etc.

Running your latest test code in Matlab, I get

>> jsonencodetest
exp = '[[[1,Nan],[3,Infinity]],[[5,6],[7,-Infinity]]]'
act = '[[[1,2],[NaN,Infinity]],[[5,6],[7,-Infinity]]]'

Error using jsonencodetest (line 54)

Maybe there is a flaw in

https://github.com/Abdallah-Elshamy/octave/blob/3c69f16d6cc3eb12fadd3a1990f6d693608390f1/test/json/jsonencodetest.m#L50-L54

@Abdallah-Elshamy Thank you for your continued work on the JSON project. I think the number of branches in your repo should not exceed one. How can I run your test files

• test/json/jsonXXcodetest.m

together with your work on

• libinterp/corefcn/jsondecode.cc

My idea for the end of GSoC is, that all your changes are accumulated in a feature branch "json", that is converted into a pull request, that can be exported to be merged into Octave.

@Abdallah-Elshamy Thank you for your great work on jsondecode.cc. According to your blog you are more than ahead of time "06/7 – 18/7 | 12 days | Implementing jsondecode" 👍

On the maintainers mailing-list Markus added a remark:

Maybe as a feedback I can see some compiler warning. But that should be easy to fix.

jsondecode.cc: In function ‘octave_value decode_number(const Value&)’:
jsondecode.cc:60:1: warning: control reaches end of non-void function
[-Wreturn-type]
   60 | }
      | ^
jsondecode.cc: In function ‘octave_value decode_array(const Value&)’:
jsondecode.cc:218:1: warning: control reaches end of non-void function
[-Wreturn-type]
  218 | }
      | ^
jsondecode.cc: In function ‘octave_value decode(const Value&)’:
jsondecode.cc:235:1: warning: control reaches end of non-void function
[-Wreturn-type]
  235 | }
      | ^

All these instances look like this

https://github.com/Abdallah-Elshamy/octave/blob/933e6ff0b47fb85e7cbe72b30bcd1660eb86deff/libinterp/corefcn/jsondecode.cc#L47-L60

My compiler, gcc 7, did not complain, but maybe Markus uses a newer one. Indeed it is safer to implement some default else-branch with something like

else
  error ("jsondecode.cc: Unidentified type.");

to avoid undefined states.

Otherwise, please take your time document (comment) your work, extend jsondecodetest.m, work on Abdallah-Elshamy/octave#5 or start with jsonencode.cc.

Hi @Abdallah-Elshamy

Thanks for the comprehensive list of unit tests. They highlighted one difference between JSONio and jsonencode/jsondecode (test 22: b = instead of b = ) that I was not aware of.

Concerning the other points mentioned on the maintainers mailing list, for JSONio, I have added some extra options, Indent and ReplacementStyle, defaulting to Matlab. Perhaps a similar approach could be taken for Octave?

@Abdallah-Elshamy Thank you for your implementation so far. How is the progress this week? Please use the time for adding nice Doxygen comments to your source code. https://wiki.octave.org/C%2B%2B_style_guide#Doxygen_Style_Guide