Doesn't emit subdir. about compiledb

[lyonlh]

@ryosa, file path is at "file" field. I guess the reason scan-build does nothing is the project-root path is not absolute. I've opened a pull request to solve this problem.

[nickdiego]

@ryosa Yeah, currently directory is always set to PROJ_DIR cmd line option. directory is expected to be the build working dir at the point that compilation unit was compiled [1]

BTW, What was the exact command you're trying to run? Didn't understand the point about using compiledb-generator with scan-build

The right way to solve this problem actually would be to process the the make messages about "directory changing", and use that to set "directory" property.
I'll to fix that soon.

Thanks for reporting.

[1] https://clang.llvm.org/docs/JSONCompilationDatabase.html#format

[nickdiego]

@ryosa, file path is at "file" field. I guess the reason scan-build does nothing is the project-root path is not absolute. I've opened a pull request to solve this problem.

@lyonlh Not sure if that would solve this problem. Probably this is happening because the build system is using relative paths (relative to the current building dir) to the source files.

@ryosa can you please provide some log of your build and maybe some other infos (build system used, etc.) ??

[ryosa]

@nickdiego I use analyze-build of clang 3.4, not scan-build. Sorry to confuse you.
I can provide you with a part of build log, but the server's default language is Japanese and the messages of GNU make are not English. Is that alright with you?

[nickdiego]

@ryosa Ok. I think you can use LANG=C before your make command to force the output to be in english. Actually, I think compiledb-parser expects it to be in english, not sure.

example command:

LANG=C make all

Thanks

[lyonlh]

@nickdiego You're right. I've created a new pull request which leverages make --print-directory option to get the subdir and put it into "directory" field.
@ryosa Maybe this is the solution.

[nickdiego]

@ryosa As #8 has been merged, please check if it solves this issue.

[ryosa]

@nickdiego @lyonlh Thanks. But #8 didn't solve the issue. Below is a part of build log.

 make[4]: Entering directory `/root/ymph3/src/syssrc/drv'
 cd ../../.. && make  am--refresh
 make[5]: Entering directory `/root/ymph3'
 make[5]: Nothing to be done for `am--refresh'.
 make[5]: Leaving directory `/root/ymph3'
 make[5]: Entering directory `/root/ymph3'
 make[5]: Nothing to be done for `am--refresh'.
 make[5]: Leaving directory `/root/ymph3'
 cd ../../.. && make  am--refresh
 make[5]: Entering directory `/root/ymph3'
 make[5]: Nothing to be done for `am--refresh'.
 make[5]: Leaving directory `/root/ymph3'
 cd ../../.. && make  am--refresh
 make[5]: Entering directory `/root/ymph3'
 make[5]: Nothing to be done for `am--refresh'.
 make[5]: Leaving directory `/root/ymph3'
 make[5]: Entering directory `/root/ymph3'
 make[5]: Nothing to be done for `am--refresh'.
 make[5]: Leaving directory `/root/ymph3'

After emitted the above, make begins to compile the source files in /root/ymph3/src/syssrc/drv, then directory in the output file is /root/ymph3 only, and that's exactly the same as before.

BTW, I altered the last line of compiledb-gen-make like below.

LANG=en_US.UTF-8 make -Bnkw "${make_opts[@]}" | $parser_cmd "${parser_opts[@]}"

[nickdiego]

make[5]: Nothing to be done for `am--refresh'.

Looks like there's some checking (maybe from automake, looks like you're using autotools as build system) which overrides the behavior of "make -B", what should force all the targets to be executed unconditionally.
I think you have 2 options to workaround this:

1. Try clean build (you can use a separate build dir if your build system supports shadow building, e.g: mkdir build2 && cd build2 && configure .. && compiledb-make -o compile_commands.json
2. Check if automake provide some option similar to make --always-make, and use that to force it to show the exact compile commands for all source files.

[ryosa]

@nickdiego Here is a command line to compile /root/ymph3/src/syssrc/drv/DevInit.cpp

/opt/timesys/toolchains/x86_64-linux-463/core2duo/toolchain/bin/ccache /opt/timesys/toolchains/x86_64-linux-463/core2duo/toolchain/bin/x86_64-timesys-linux-gnu-g++ -DHAVE_CONFIG_H -I. -I../../.. -I. -I../inc -I../..//inc -I../..//config -I../..//../OSWrapper/h -I/LinuxX86_64-Haswell/usr/src/linux-3.10.5/usr/include -I/LinuxX86_64-Haswell/usr/src/linux-3.10.5/include -I/LinuxX86_64-Haswell/usr/src/linux-3.10.5/arch/x86/include -I/LinuxX86_64-Haswell/usr/local/libcont/5th/20170601/nicfum -I/LinuxX86_64-Haswell/usr/local/libcont/5th/20170601/nicfum/MES_include -I/LinuxX86_64-Haswell/usr/local/libcont/5th/20170601/nicfum/MES_include/mes -I/LinuxX86_64-Haswell/usr/local/include -I/LinuxX86_64-Haswell/usr/local/include/intel -I../..//../modules/3.10.5 -I../..//ipsrc/IPparameters -I../..//ipsrc/IPparameters/forPMS -I../..//ipsrc/inc -I../..//syssrc/inc -I../..//syssrc/commonapi/lib/rsa/MES_include/soc/mes -I../..//syssrc/commonapi/system/dbg -I../..//syssrc/commonapi/ -I../..//syssrc/application/mfp/ -I../extdrv -I../..//../IpLib/IpCommonComponent/inc -I../..//../IpLib/IpDeviceComponent/inc -I../..//../IpLib/IpTaskComponent/inc -I ../../ipsrc/PMS/inc -I ../../ipsrc/PMS/Log -g -O2 -Wall -Wno-deprecated-declarations -Wno-trigraphs -Wno-unused-but-set-variable -fomit-frame-pointer -fno-strict-aliasing -fno-common -fsigned-char -pipe -Wno-uninitialized -Wno-conversion-null -Werror -D_REENTRANT -D_PTHREADS -DLYNXOS -DLINUX -DSSLC_SMALL_CODE -DNO_SOFTWARE_CRYPTO -DTORNADO -DDEF_PPSystem -DDEF_ENGINE -DDEF_SupportSSD -DDEF_IWSCOPYJOB -DDEF_Soc1200 -DPMS_PP -g -O2 -MT DevInit.o -MD -MP -MF .deps/DevInit.Tpo -c -o DevInit.o DevInit.cpp

Yes, the project utilizes automake. I would have to make clean then build to generate Makefiles, which takes 6 hours...

[lyonlh]

@ryosa @nickdiego
Sorry, it's my bug introduced in #8.
After investigated the build log and parsed it by the parser, I figured out that the regex mismatched cause of head white-space. I fixed it in pull request #9.
Please try again.

Thanks.

[ryosa]

@lyonlh #9 looks good. Thanks a lot.

But checking the output json, I had a question, that is, relative path is included in 'file'. Here is an example.

  "directory": "/root/ymph3/src/ipsrc/IPparameters/forCps2010", 
  "arguments": [
   "c++", 
   "-DHAVE_CONFIG_H", 
   "-I.", 
   "-I../../../..", 
   "-I.", 
   "-I../inc", 
   "-I../../..//inc", 
   "-I../../..//config", 
   "-I../../..//../OSWrapper/h", 
   "-I/LinuxX86_64-Haswell/usr/src/linux-3.10.5/usr/include", 
   "-I/LinuxX86_64-Haswell/usr/src/linux-3.10.5/include", 
   "-I/LinuxX86_64-Haswell/usr/src/linux-3.10.5/arch/x86/include", 
   "-I/LinuxX86_64-Haswell/usr/local/libcont/5th/20170601/nicfum", 
   "-I/LinuxX86_64-Haswell/usr/local/libcont/5th/20170601/nicfum/MES_include", 
   "-I/LinuxX86_64-Haswell/usr/local/libcont/5th/20170601/nicfum/MES_include/mes", 
   "-I/LinuxX86_64-Haswell/usr/local/include", 
   "-I/LinuxX86_64-Haswell/usr/local/include/intel", 
   "-I../../..//../modules/3.10.5", 
   "-I../../..//ipsrc/IPparameters", 
   "-I../../..//ipsrc/IPparameters/forPMS", 
   "-I../../..//ipsrc/inc", 
   "-I../../..//syssrc/inc", 
   "-I../../..//syssrc/commonapi/lib/rsa/MES_include/soc/mes", 
   "-I../../..//syssrc/commonapi/system/dbg", 
   "-I../../..//syssrc/commonapi/", 
   "-I../../..//syssrc/application/mfp/", 
   "-I../../..//../IpLib/IpCommonComponent/inc", 
   "-I../../..//../IpLib/IpDeviceComponent/inc", 
   "-I../../..//../IpLib/IpTaskComponent/inc", 
   "-I..", 
   "-I", 
   "../../PMS/inc", 
   "-I", 
   "../../PMS/Log", 
   "-Wall", 
   "-Wno-deprecated-declarations", 
   "-Wno-trigraphs", 
   "-Wno-unused-but-set-variable", 
   "-Wno-uninitialized", 
   "-Wno-conversion-null", 
   "-Werror", 
   "-D_REENTRANT", 
   "-D_PTHREADS", 
   "-DLYNXOS", 
   "-DLINUX", 
   "-DSSLC_SMALL_CODE", 
   "-DNO_SOFTWARE_CRYPTO", 
   "-DTORNADO", 
   "-DDEF_PPSystem", 
   "-DDEF_ENGINE", 
   "-DDEF_SupportSSD", 
   "-DDEF_IWSCOPYJOB", 
   "-DDEF_Soc1200", 
   "-DPMS_PP", 
   "-c", 
   "'./'`HardwareDriver/ddVirtualVD.cpp"
  ], 
  "file": "'./'`HardwareDriver/ddVirtualVD.cpp"

Does this affect clang's behavior?

[ryosa]

Clang seems to understand relative path in 'file'. It didn't output error for the 'command' and 'file' in question.

[lyonlh]

@ryosa But it seems weird, there are some quotes in the file field, the file path is retrieved from build log and should not add additional characters. Could you please provide some build logs? So I can check this issue.
Thanks.

[nickdiego]

"file": "'./'`HardwareDriver/ddVirtualVD.cpp"

Yeah, as @lyonlh said, thia one for example is pretty weird. We need to see the exact log line thelat generated this entry to try to figure it out why this is happaning.

thanks

[nickdiego]

Also, i think this particular issue is already solved. Thanks to @lyonlh and @ryosa por reporting it. So I'll close this and this escaping related issue can be open and we can move the discussion to it.

[ryosa]

The quotes in 'arguments' and 'file' were due to description in Makefile.

ddVirtualVD.o: HardwareDriver/ddVirtualVD.cpp
$(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ddVirtualVD.o -MD -MP -MF $(DEPDIR)/ddVirtualVD.Tpo -c -o ddVirtualVD.o test -f 'HardwareDriver/ddVirtualVD.cpp' || echo '$(srcdir)/'HardwareDriver/ddVirtualVD.cpp

As long as clang properly deals with this, there's no reason to open a new issue.
Thanks guys!