Use control flow info from IDA about retdec-idaplugin

[Alexey-Danilchenko]

I think it needs to go one step further. Currently retdec is only useful as IDA plugin if it can decode file itself - IDA however has a lot of loaders that can take care of complicated binary formats and process something like firmwares even. Ideally retdec should be used what it is designed for - decompilation and use as mush as possible from IDA. I.e. for function decompilation it should be possible for IDA to pass on entire disassembly of the function with resolved symbols (if any) that should aid decompilation. There should be no need for retdec to decode the binary.

[PeterMatula]

@Alexey-Danilchenko

1. It is true, that it would be the best if RetDec could do a high quality decompilation of a given binary data chunk. That would allow IDA plugin to send any data + metadata to RetDec - it would no longer matter what was the underlying source and whether RetDec can handle this source on its own. This is something to aim for. Exporting control flow metadata from IDA is a good first step.
2. If IDA plugin passes binary data (selected range in binary) to decompile + metadata about basic blocks and their relations, it should be as good as if it passed IDA's assembly itself - given that RetDec's disassembler interprets the same data into the same instructions as IDA's disassembler would. However, RetDec needs to disassemble the data on its own, IDA's disassembly is useless for us. It is the disassembling/decoding phase that creates LLVM intermediate representation that we work with. RetDec does not know how to create LLVM IR from IDA's disassembly, and it would not be an easy thing to do.

[Alexey-Danilchenko]

@PeterMatula

I see (2) is certainly a blocker for this. Perhaps the approach to explore here is for plugin to pass a binary block to decompile with the metadata? For example if the function is selected for decompilation, pass the function body as bytecodes (that is accessible from IDA API and since it was loaded is not file format specific) together with any symbolic information (variables, types etc). I got particularly interested in this since RetDec includes PowerPC an I have a custom loader for firmware that I am maintaining which uses PowerPC code.

Generally speaking though the main strength of IDA is that disassembly of the code is happening interactively and in the process the data gets refined, variables could be named and typed etc - so just using the binaries and decoding them in RetDec is sort of loosing all that information.