arizvisa / ida-minsc

A plugin based on IDAPython for a functional DWIM interface. Current development against most recent IDA is in the "persistence-refactor" branch, ancient (but stable) work is in "master", so... create an issue if you want/need something backported. Use "Wiki" or "Discussions" for examples, and smash that "Star" button if you like this.

IDA-minsc

Website: https://github.com/arizvisa/ida-minsc Documentation: https://arizvisa.github.io/ida-minsc Mantras: https://github.com/arizvisa/ida-minsc/wiki/Mantras Wiki: https://github.com/arizvisa/ida-minsc/wiki Changelog: https://github.com/arizvisa/ida-minsc/wiki/Changelog IRC: #eof on EFnet

General

IDA-minsc is a plugin for IDA Pro that assists a user with scripting the IDAPython plugin that is bundled with the disassembler. This plugin groups the different aspects of the IDAPython API into a simpler format which allows a reverse engineer to script different aspects of their work with very little investment.

A number of concepts are introduced such as a tagging system, support for multicased functions, and filtering with the intention that most search and annotation issues can be performed with just a few lines of code. This should enable a user to write quick, hacky, temporary code that can be used to augment their reversing endeavors without distraction.

Installation

Installation should be pretty simple and requires simply cloning the repository directly into the user's IDA user directory. On the Windows platform, this is typically located at %APPDATA%/Hex-Rays/IDA Pro. Whereas on the Linux platform this can be found at $HOME/.idapro. This contents of this repository should actually replace that directory. If you have any files that presently reside there, simply move them into the repository's directory. After installation, IDA Pro should load its IDAPython plugin which should result in the idapythonrc.py belonging to IDA-minsc being executed which will then replace IDAPython's default namespace with the one belonging to the plugin's.

To clone the repository in a directory $TARGET, one can simply do:

$ git clone https://github.com/arizvisa/ida-minsc "$TARGET"

After cloning the repository, the user will need to install its required Python dependencies into their site-packages. This can be done using pip which is a tool that is bundled with Python. The file that contains the user's requirements is in the root of the repository as requirements.txt.

To install the required Python dependencies, one can run pip as so:

$ pip install -r "requirements.txt"

At this point when the user starts IDA Pro, IDA-minsc will replace IDAPython's namespace with its own at which point can be used immediately. To verify that IDA-minsc was installed properly, one can simply type in the following at the IDAPython prompt:

> database.config.version()

This should then return the number 0 since no database has been loaded.

Quick Start

After installing the python dependencies, you can do something like the following to list all the functions in your database:

> database.functions.list()

Or to iterate through all the functions in the database, you can try:

> for ea in database.functions():
      print(hex(ea))

Please refer to the documentation for more details on what this plugin makes available to you.

Documentation

Comprehensive documentation is available at the project page on github.io, or can be built locally via the "docs" branch.

If the user wishes to build documentation for local use, they will first need to install the Sphinx package. Afterwards, the entirety of the documentation resides within in the "docs" branch. Simply checkout the branch, change the directory to "docs", and then run GNU make as:

$ make html

This will result in the build system parsing the available modules and then rendering all of the documentation into the _build directory relative to the docs/Makefile. Documentation can be generated for a number of different formats. To list all of the available formats, type in make help at the command prompt.

Contributing

See CONTRIBUTING.md for best practices on reporting issues or for adding functionality to this project.

Thanks

Thanks to a number of anonymous and non-anonymous people whom have helped with the development of this plugin over all of these years.

ida-minsc's Issues

How check current code of function ?

Hi ! arizvisa ,
How to judge whether the current code is a function?
Using db.functions.xxx() will fail, because I have a section of Code that is not defined as a function, it is Red, when it has not been defined as a function,
Is there a corresponding method to detect whether it is a function? I use db.type.has_name(xx), but the result is not ideal.
Is there such a function type.is_func()?

Add another case to functions which act on a single address to allow using the user's current selection or operand number with those functions.

Some of the functions like database.set.undefined assume the currently selected address when provided no parameters. There's a bunch of them that actually perform this logic. It would actually be pretty cool to also implement cases for these functions which can take an interface.bounds_t. This is so the current selection could also be used to apply a singular operation to a range of addresses when the function is given no parameters.

This would require adding another case to these functions which can take an interface.bounds_t. This new case can then use database.address.iterate to apply the function to each address within the range. Then the default case for the function which uses the current address would need to check to see if a selection or just a single address is selected, then we can dispatch to the correct function case for whatever the user has selected.

The netnode for "$ tagcache" seems to be being reused between separate databases.

The "$ tagcache" capability of ida-minsc is based around the contents of a blob started in a netnode. The idea is that the netnode is created when the database is created and then just referenced everytime it is needed. It seems that the logic for creating the netnode's identifier is scoped according to the internal.comment.tagging namespace as opposed to being scope according to the database itself. This netnode is instantiated by either the "idp.ev_init" or "idp.init" hooks.

The issue is because the identifier for this netnode is started within the class' definition and then only updated if the attribute is missing. This means that we're essentially trashing any blob that has the same netnode id when we open up another database which can also result in some weird values being displayed within the cache.

The entirety of the busted logic is in the implementation of the tagging base class within base/_comment.py:

    @classmethod
    def __init_tagcache__(cls, idp_modname):
        cls.node()
        logging.fatal(u"{:s}.init_tagcache('{:s}') : Initialized tagcache with netnode \"{:s}\" and node id {:#x}.".format('.'.join(('internal', __name__, cls.__name__)), internal.utils.string.escape(idp_modname, '\''), internal.utils.string.escape(cls.__node__, '"'), cls.__nodeid__))

    @classmethod
    def __nw_init_tagcache__(cls, nw_code, is_old_database):
        idp_modname = idaapi.get_idp_name()
        return cls.__init_tagcache__(idp_modname)

    @classmethod
    def node(cls):
        if hasattr(cls, '__nodeid__'):
            return cls.__nodeid__
        node = internal.netnode.get(cls.__node__)
        if node == idaapi.BADADDR:
            node = internal.netnode.new(cls.__node__)
        cls.__nodeid__ = node
        return node

function.within() returns true on externals within the "extern" segment for elf files

For some strange reason, function.within() is saying that externals in the "extern" segment is within a function. This appears to be IDA's doing because if you move your cursor to any of them, you can hit Alt+P and it'll let you edit its bounds as a function. Despite IDA being crazy here, minsc should detect this and return false since it's not actually a function.

Feature: Real decorator support for namespace aliases

Right now only function aliases are supported via internal.utils.alias. Aliases for namespaces (database.integer aliased as database.i, database.entries aliased as database.exports, database.address as database.a, or database.type as database.t, etc.) are implemented by (straight-up) assigning the namespace to the target alias variable. This has negative effects with things such as help() since due to more than one variable being assigned for the same class, the documentation for the class is listed multiple times.

It'd be nice to do something similar to internal.utils.alias, but rather for namespaces so that way the help() for the namespace alias can simply be a reference to the real namespace. Then the reStructuredText generator in docs/docparse.py within the docs branch can be modified to reference the decorator to determine what aliases exist for a particular namespace.

Fix for #20 results in unusable strings due to unicode strings not being handled by IDAPython.

This is fine and all that because everything is essentially being treated as ascii. This "was" the case, however, until the fix for issue #20 was requested. This fixed db.get.string so that it would properly decode the strings in the database instead of transforming them to UTF8 (IDA's solution). This fix resulted in the API always returning a unicode type.

However, unicode types aren't supported by idaapi which means that anything returned by db.get.string will need to be properly encoded before using it with IDAPython.

Feature: Modify custom exceptions to search through the stack frames for the function name and its arguments

Whenever an exception is thrown, or a message is logged (using logging.root) the function that caused it, its arguments, and an error message is manually put together via a format-spec. This is fucking horrible to maintain and there's some cases where they're inconsistent (or I missed some variables in my format-spec). This is because this sort of stuff is a real bitch to test (large project without unit-testing syndrome).

It'd be nice if there's some way to automatically determine these things at the time the exception is raised so that way this would be a little bit more manageable. Maybe this would have to be done by requiring all custom-exceptions include an argument with the function name and its parameters. Then one can look at the types defined by the "multicase" decorator and automatically output that part.

If it's possible to walk through the stack frame (similar to internal.interface.address.pframe), however, then this would be awesome because it could be made to be entirely automatic.

Feature: The "notepad" namespace in the ui module needs methods to modify its contents

The ui.notepad namespace is very primitive and is thus pretty useless unless you want to hide or show the window. It'd be nice if some methods were exposed that would allow a user to programmatically modify its contents. Methods that modify its contents might require either doing some work with the Qt API, or poking around to see if the contents are in a netnode or something related.

ReadOrWriteError exception raised when trying to use comment api in IDA 6.8

This issue manifested after fixing issue #59 with PR #60 (some prior conversation exists in those tickets, so start there if you need context for this).

Python>db.comment(0x006DA9CF, "123")
CRITICAL:root:internal.interface.priorityhook.callback(7186895, False) : Callback for "IDB_Hooks.cmt_changed" with priority (+0) raised an exception while executing <bound method type.old_changed of <class 'hooks.address'>>
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : Traceback ("IDB_Hooks.cmt_changed" was hooked at)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 1059, in ida_is_busy_sucking_cocks
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : make_ida_not_suck_cocks(idaapi.NW_INITIDA)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 993, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : ui.hook.idb.add('cmt_changed', address.old_changed, 0)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) :
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : Traceback (most recent call last):
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 219, in old_changed
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : internal.comment.contents.set_address(ea, 0)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base_comment.py", line 971, in set_address
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : raise internal.exceptions.ReadOrWriteError(u"{:s}.set_address({:#x}, {:d}{:s}) : Unable to write name to address {:#x}.".format('.'.join(('internal', name, cls.name)), address, count, ', {:s}'.format(internal.utils.string.kwargs(target)) if target else '', address))
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : ReadOrWriteError: internal.comment.contents.set_address(0x6da9cf, 0) : Unable to write name to address 0x6da9cf.
WARNING:root:internal.interface.priorityhook.callback(7186895, False) :

Feature: Implement module-specific logging for all of the modules

Right now ida-minsc explicitly uses logging.root to log error messages and status. Due to it explicitly modifying the root logger, this means it doesn't really play nice with people who customize their logging. This originated mostly due to my laziness, and just kind of stuck since it's already a ton of labor to emit all of a function's arguments manually.

The proper way to do this would be to use the root logger to create the initial logger, and then for each module create a sub-logger with the module name. Specifically one would create the root logger for the plugin in __root__.py, stash it somewhere (like maybe in _exceptions.py or something), and then fetch a sub-logger at the beginning of each module.

This would allow a user to customize where (and how) logs are emitted. As an example, like if they want to output them directly to a file such as when using IDA to batch-process a bunch of binaries instead of using IDA's output log parameter ("-L") to capture them.

Bug: Internal modules that transform their names do not get named properly within sys.modules

Due to ida-minsc's use of Python's sys.meta_path in order to allow its different modules to take priority over user-installed libraries in Python's site-package, the sys.modules dictionary might have incorrect module names stashed into it. As an example, internal.utils will exist as two separate modules, internal and _utils, instead of internal and internal.utils.

This can result in a pretty fucked module cache, and really manifests itself when a user needs to reload() a namespace'd module. In order to reload() a module, the user will actually need to reload() the module's namespace which causes the loader in the meta_path to re-parse "all" of the namespace's sub-modules. Normally, though, a user should never need to reload() unless they're doing development on ida-minsc itself. Plus, a "normal" user will never notice this issue because the resulting namespace module will actually lazy-load its sub-modules despite the docstring that's automatically generated not being entirely correct.

If the user is doing development on ida-minsc, however, then this actually manifests itself as a huge pain in the ass. To solve this, the sys.meta_path classes might need to be rewritten a bit, or maybe even dumbed down so that relative importing of modules (packages) are used. This will only work if we can trust/make Python's module loader deal with ida-minsc's weird usage of modules and classes.

Unable to list imports against an elf executable with the imports module name is None

When calling database.imports.list() with an ELF executable, the module name returned from database.imports.iterate() is None which results in the following exception when trying to calculate the length of the module name.

Python>db.imports.list()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 292, in database.list
    return f(*arguments, **keywords)
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 1754, in list
    @utils.multicase()
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 1406, in wrapper
    return F(*res, **kwds)
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 1762, in list
    maxmodule = max(builtins.map(utils.fcompose(utils.second, utils.first, len), res) or [''])
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 47, in <lambda>
    fcompose = lambda *f: builtins.reduce(lambda f1, f2: lambda *a: f1(f2(*a)), builtins.reversed(f))
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 47, in <lambda>
    fcompose = lambda *f: builtins.reduce(lambda f1, f2: lambda *a: f1(f2(*a)), builtins.reversed(f))
TypeError: object of type 'NoneType' has no len()

Somewhat related, when calling database.exports.list() the export address is printed out twice due to the export ordinal for an entrypoint being returned as the export's address. This can probably be cleaned up.

Python> db.exports.list()
...
[7] 0x180611c60 : (0x8) IAEModule_IAEKernel_UnloadModule
[8] 0x180a6dce0 : (0x180a6dce0) DllEntryPoint

Feature: Expose IDA's navigation stack as a tree instead of a list that can be interacted with

This was also something that was experimented with during "toolbag" as a way of showing a user what they've currently visited and what tags are available at different locations. This was scrapped when the plugin was named "idascripts" due to the desire to keep gui-related components out of the repository, but made its way into "toolbag" as the infinitely awesome "history" view. Its only issue is that it required manual input to add locations to it.

It'd be nice, however, if there was a similar (but internal) data structure that could be used to store locations as the user navigates through the database. This way a user can immediately see what tags are available at different positions instead of requiring they manually list them via database.tags or function.tags.

IDA's navigation stack is simply that, a stack. However, code that is compiled is stored as a tree, and when executed is best represented as a graph. Therefore, if we have some kind of tree-like structure ("trie"), then this might simplify the sharing and recalling of navigation history within a database.

This component can then actually be used to replace IDA's concept of "marks". This way instead of marking a specific address in the database, the user can create and use a "trie" for each reversing project. It is suspected that this will vastly improve the way users keep track of things they have reversed within the database. This way when the user opens up a database, they can immediately list the different tries (or projects) they have previously created, select one of the projects they were working on, and then immediately pick up where they left off.

Again, this would allow a user to keep track of the different things they visited while working on a specific reversing problem and more importantly keep track of which tags they created for each node within the "trie". If this automatically updating "trie" is implemented, then maybe at some point we can create another plugin based on it that shows it as a history graph along with the tags that were made at each position in the same way the author of "toolbag" implemented it.

The structure.member_t.refs() function only returns operands that reference the field, and not operands that access a globally defined structure

When using structure.member_t.refs(), only operands that have been converted to a structure-field (with "t"), and does not reference fields that are defined globally. This happens because idaapi.get_opinfo() doesn't actually return a structure type for that given operand, and so minsc is not able to distinguish the type of the operand.

It also appears that structure.structure_t.refs() is skipping over some of its reference types as well. It probably makes more sense for this method to return both the members and the addresses that are referenced by the structure, instead of just addresses.

Feature: database.marks needs a matcher class since it's a filterable list

Most list-like things in ida-minsc expose a matcher class along with a number of keywords that can be used to filter their contents. database.marks is another one of those namespaces but for some reason it's missing a matcher class in order to filter it. The reason for this might be that the number of marks that IDA allows for you to create in a database is limited (I think 64?).

Nonetheless, it doesn't hurt to implement database.marks.list, database.marks.iterate, and database.marks.search like all of the other searchable namespaces. This will require implementing a matcher class for it and maybe a few keywords to filter each individual mark.

Feature: database.stash and exposing a filesystem-like api in an IDA database

This was experimented with during "toolbag" development like forever ago, but never made it into ida-minsc due to the vastly different intentions and mantras between both plugins. In essence, a filesystem-like api exposed via something like database.stash would be very useful for allowing a user to bundle arbitrarily-sized data with their IDA database. This would facilitate higher-level plugin development which needs to cache extra data in the database without storing it externally via the platform's regular filesystem.

The "blob" type for a netnode in an IDA database has some size limitations that need to be abstracted around in order to provide this capability. As a result, to support arbitrarily sized data, we could use a linked list or a tree for searching a file's different chunks. A better way would be to actually implement an embedded filesystem using "blob" types. Actually, we can probably implement something similar to a FAT-based filesystem (or some another filesystem) using blobs as its primitive storage mechanism.

The lower-level components could then be implemented as internal.netnode.nodefs, at which point some higher-level interface could be exposed via the database module. If we have some kind of filesystem like this, then we could begin to consider arbitrarily sized tags that have zero size limitations. We could also modify the semantics of tagging a bit so that anything that's double-underscored (Python name mangling) would result in a hidden tag that is physically stored in the filesystem. This would allow serialization of types other than the basic Python ones that we presently encode within comments.

Earlier versions of IDA are not rendering their output using sys.displayhook which products incorrectly formatted output

It seems that IDAPython isn't using sys.displayhook to write its output to the console. This results in the output wrapper not properly formatting types that need to be re-formatted as strings. As an example, when trying to display the typeinfo of a structure member you'll get the following:

Python>struc.by(0).members[0].typeinfo
<idaapi.tinfo_t; proxy of <Swig Object of type 'tinfo_t *' at 0x04DAE578> >

Python>sys.displayhook(struc.by(0).members[0].typeinfo)
unsigned __int32

IDAPython is likely doing its fiddling with redirecting stdout and stderr inside python/init.py.

Feature: More support for the Hex-Rays decompiler

There's some basic support for interacting with the decompiler within the function module, but it's very minimal and is only used to match instructions to lines of code. I've had a number of issues with instability (crashy-crashy) when calling the different functions that are exposed by IDAPython and so I'm always very hesitant to script it.

Another reason why I'm hesitant to use the decompiler is that although I have a license for it, it's never used because I (straight-up) prefer assembly listings due to the implicit information that is exposed to the user such as function locality (boundaries for object files before the linker gets ahold of them), or the hot-paths as determined by the compiler.

Nonetheless, the decompiler is a significant part of IDA and thus it deserves its place within ida-minsc. Unfortunately, I'm not sure what's useful to expose to a user because of the prior mentioned reasons. If anybody has any suggestions or wants to contribute, please let me know in this thread.

IDA-minsc compatibility with the IDAPython version from IDA 7.5.200619

Minsc's documentation doesn't mention anything about compatibility with IDA 7.5. This is because it hasn't yet been tested on IDA 7.5 due to my personal hatred for IDA 7.x's licensing stupidities which makes it super-fucking expensive for me to test the extension on platforms that I don't use regularly.

Currently IDA 7.5 was related in 2020/5/20 (as per https://www.hex-rays.com/blog/ida-pro-7-5-released/), with SP1 in 2020/6/19 (as per https://www.hex-rays.com/blog/ida-pro-7-5-sp1-released/), and SP2 in 2020/7/28 (as per https://www.hex-rays.com/blog/ida-pro-7-5-sp2-released/).

AttributeError: 'module' object has no attribute 'FF_STRU', 'asciflag', etc.

IDA 7.4 disables 6.x compatibility by default which means a number of symbols are unavailable. Although 6.x compatibility can be re-enabled (https://www.hex-rays.com/products/ida/support/ida74_idapython_no_bc695.shtml), we should definitely transition to using the new 7.x symbols when they're available.

The table for the porting guide can be found at https://www.hex-rays.com/products/ida/support/ida74_idapython_no_bc695_porting_guide.shtml

AArch and AArch64 register support (KeyError raised)?

It's been pointed out that AArch64 register support is lacking. When accessing a register off of the architecture's dictionary a misleading KeyError will be raised. This is actualy due to the AArch64 registers not being completely defined. This includes the X*, the W* registers, and possibly a few more. It was mentioned that there might be a few registers for the (32-bit) AArch architecture that are not defined as well.

Please add these as the architectures are somewhat useless without them.

The `instruction.op_refs` function sucks when trying to fetch the operand references of a global

The instruction.op_refs function is used for identifying instructions and operand numbers that reference a particular operand. This works fine for structure member offsets, or stack variables which reference the function's frame and such. However, when using it against an operand referencing a global, it's pretty much worthless. When it's actually applied to an operand referencing a global, it won't return all of the operand references to the global but only the current one that you used to ask references for.

Ideally there should be another case for instruction.op_refs that returns all of the references to the target of the operand rather than just the operand itself.

I'm classifying this as a bug as well as an enhancement because this makes instruction.op_refs pretty worthless for certain operand types.

Documentation: Include docs for switching IDA 7.4's IDAPython from Python3 to Python2

IDA 7.4 includes the option to enable the Python3 or Python2 IDAPython plugin by default. It would be great to update the docs to reference how to enable Python2 for IDAPython in a user's installation.

A friend pasted me the following filenames that contain the plugin libraries to rename for his platform.

 1994  mv idapython.dylib idapython.3.disabled
 1996  mv idapython64.dylib idapython64.3.disabled
 1997  mv idapython.2.disabled idapython.dylib
 1998  mv idapython64.2.disabled idapython64.dylib
/Applications/IDA Pro 7.4/ida.app/Contents/MacOS/plugins

Feature: Automatically updating the arrow position on the navigation band when using different APIs

It'd be neat if the navigation band is updated everytime a function is called on some particular address. This is typically done with the ui.navigation namespace. Anyways, this way if you write a loop that extracts an operand from an instruction, decodes data from the database, or iterates through tags or imports, etc., then the user can see the general area or address that their code is currently working with. This'll also give users the added benefit of being able to tell if their code is busted and not actually interacting with the database in any way due to the navigation band not being updated.

It'd also be cool to use one of the other navigation band arrow types to mark the location of the last exception on the navigation band, or some other symbols to mark addresses for other purposes.

This could probably be implemented as a decorator of some sort in the internal.utils module that could then be applied to any function which takes an address as one of its parameters. I'm not sure, however, how this might affect performance. Essentially we'd be adding another python function call on top of the already-existing cost of multicased functions...

Feature: Register matching semantics can be kind of weird since it depends on IDA's idea of what the operand is doing

Currently the semantics of register matching is based on what IDA thinks an operand is doing due to register matching's usage of instruction.op_state. In IDA, an operand is either read from, written to, or both. And so, register matching (database.address.nextreg, database.address.prevreg, function.chunk.register, function.block.register, etc.) is based on what IDA thinks the operand is doing.

Unfortunately this is wrong because things such as operand "phrases" are not actually writing to their registers. Actually, the operand as a whole is written to, but the registers themselves are actually "read from". A hack for this was in place originally as the instruction.ir namespace, but this was deprecated and eventually removed because it was a terrible Intel-only hack.

The regmatch helper in internal.interface should be re-implemented so that it properly identifies if a register is actually being read from or modified in some way. This means that phrases (or the symbols within the phrase, really) need to be checked if they're referencing a register, and if so then the regmatch helper should terminate, return true, or whatever it does.

Once this is fixed, then database.address.nextreg and database.address.prevreg can probably be modified to terminate when trying to locate an instruction that reads from a register which was overwritten by a prior instruction (register has gone out of scope). Unfortunately without being 100% certain you're in a function and have a flow chart of how the code is to be executed, there isn't a reliable way to figure this out. Building the control flow graph on each call is obviously out of the question, and caching it is kind of extreme. It'd be nice if we could do this properly without being in a function.

Maybe a better way to determine a register value's scope (rather than changing the semantics of these two functions) would be to expose a general combinator that a user can pass as a predicate. I think I have one in a database somewhere that does this already, but it would need to have a good intuitive name and then tested properly as it's such a weird thing to do when you don't have a flow-chart for what it is you're matching for.

GH-57 make comment has some error repost.

hi, use db.comment function :

Python>db.comment(0x89d117, "123")

CRITICAL:root:internal.interface.priorityhook.callback(9031959, False) : Callback for "IDB_Hooks.cmt_changed" with priority (+0) raised an exception while executing <bound method type.old_changed of <class 'hooks.address'>>
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : Traceback ("IDB_Hooks.cmt_changed" was hooked at)
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 1059, in ida_is_busy_sucking_cocks
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : make_ida_not_suck_cocks(idaapi.NW_INITIDA)
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 993, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : ui.hook.idb.add('cmt_changed', address.old_changed, 0)
WARNING:root:internal.interface.priorityhook.callback(9031959, False) :
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : Traceback (most recent call last):
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 213, in old_changed
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : logging.debug(u"{:s}.old_changed({:#x}, {:d}) : Received comment.changed event for a {:s} comment at {:#x}.".format('.'.join((name, cls.name)), ea, repeatable, 'repeatable' if repeatable else 'non-repeatable', ea))
WARNING:root:internal.interface.priorityhook.callback(9031959, False) : NameError: global name 'repeatable' is not defined
WARNING:root:internal.interface.priorityhook.callback(9031959, False) :
123

Putting files in root of IDA appdir has some issues

Do the ida-minsc files need to be in the root of the IDA appdata folder?
Is it perhaps required to add in the various hooks?
Is there any way of doing it from its own subdir, such as "ida-minsc", inside the appdata dir?

Reasons:

• It makes it a bit hard to differentiate between what files in the appdata dir are ida-minsc files or what are files for other stuff.
• Similar to the above it complicates uninstall, knowing what files to remove
• When attempting to clone the git repo into the appdata dir it complains due to dir having existing files in it
• Once cloned the repo thinks that any other existing files should now be part of the repo, which isnt the case if they aren't ida-minsc specific. I know they can be added to .gitignore, but it seems a bit messy having to do so for any various files you may have. I guess you could use whitelisting style gitignore rules to instead specify all the files that are in the repo, so any others are ignored.
• Does the shortcut.cfg from the repo need to be included? It contains user specific shortcuts and including it will overwrite another users unique settings. Unless there is a specific reason I suggest removing it from being tracked by the repo.

Feature: Proper support for unions within the structure module and any functions that use them

Unions are pretty much the same as a structure except that the field offsets do not change and there's an extra flag that's passed to idaapi.add_struc. Once a union is identified, then its .ptr.has_union() method will return true. Integrating this into the already existing structure.structure_t (or really structure.members_t) class might result in member searches being completely different. This will also mean that any logic that uses structures (instruction.op_structure, or especially database.get.structure) might need to know if a union is being specified.

Implementing this properly might require another class or something so it does not gunk up the structure.structure_t and structure.members_t class. It is due to this reason that unions were not even approached as one of ida-minsc's mantras is to avoid custom objects at all costs as they might require a schema that will require a user to refer to documentation.

Feature: C++ symbol name mangling type extraction

Mangled C++ names contain a load type information in them. Unfortunately all of the C++ demanglers (and the idc.Demangle funtion in IDA) only provide an API for converted a string between its mangled and unmangled form. The internal.declaration module was actually initially split up into its own module in order to support these different mangling formats. So, instead of demangling or mangling a symbol, the idea was to have the capability to extract the different components from a mangled symbol name.

This way if a Borland, GNU, or a Microsoft Visual C++ mangled symbol is known, one would be able to extract type information from it such as what type of method it is, its arguments + types, its calling convention, and its return type. This can then be exposed to the user or used by the function module to eventually group different symbols together.

As mentioned before, the internal.declaration module was originally for this, but (due to laziness on part of the author) it was hacked together with regexes. This, of course, is a completely non-elegant and incorrect solution to this problem. To complete this, a proper parser will need to be implemented for the different mangling formats which can then be integrated into internal.declaration.

custom.tagfix.extracomments ERROR

custom.tagfix.extracomments() ErrorTraceback (most recent call last):
File "", line 1, in
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\custom\tagfix.py", line 233, in extracomments
for ea in db.address.iterate(left, right):
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 1740, in iterate
res = step(res)
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base_utils.py", line 292, in database.next
return f(*arguments, **keywords)
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 1856, in next
return cls.next(ea, 1)
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base_utils.py", line 292, in database.next
return f(*arguments, **keywords)
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 1866, in next
return cls.nextF(ea, utils.fidentity, count)
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base_utils.py", line 292, in database.nextF
return f(*arguments, **keywords)
File "C:\Users\Administrator\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 1925, in nextF
raise E.AddressOutOfBoundsError("{:s}.nextF: Refusing to seek past the bottom of the database ({:#x}). Stopped at address {:#x}.".format('.'.join((name, cls.name)), config.bounds()[1], idaapi.get_item_end(ea)))
AddressOutOfBoundsError: database.address.nextF: Refusing to seek past the bottom of the database (0x41d000). Stopped at address 0x41d000.

IDA 6.8: The newprc priority hook for initializing the typemapper doesn't seem to work

This plugin is really cool !! But he does not support ida6.8,

Ida6.8 required for company work ! Can it be resolved and let it support ? thk

that installed show this error log :

internal.interface.priorityhook.apply('newprc') : Unable to apply the hook for an unknown method.
Traceback (most recent call last):
  File "D:\tools\IDA_Pro_v6.8_and_Hex-Rays_Decompiler_(ARM,x64,x86)_Hskyre\python\idaapi.py", line 601, in IDAPython_ExecScript
    execfile(script, g)
  File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\idapythonrc.py", line 198, in <module>
    __root__ = imp.load_source('__root__', os.path.join(root, '__root__.py'))
  File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\__root__.py", line 108, in <module>
    ui.hook.idp.add('newprc', __import__('internal').interface.typemap.__newprc__, 0)
  File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base\_interface.py", line 298, in add
    res = self.apply(name)
  File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base\_interface.py", line 338, in apply
    raise NameError("{:s}.apply({!r}) : Unable to apply the hook for an unknown method.".format('.'.join(('internal', __name__, cls.__name__)), name))
NameError: internal.interface.priorityhook.apply('newprc') : Unable to apply the hook for an unknown method.

The tag cache can get out of sync when writing comments of the wrong type for a given address

When updating tags, the values in the tagging netnode will occasionally desynchronize. Although this eventually gets fixed the next time a comment is made at the miscached address, this can result in queries fetching an item from the cache that was already removed. This issue is hidden because the results from both database.tag, and function.tag are checked for the keys that the user requests prior to yielding their result, but this check shouldn't be necessary anyways.

Feature: Support for register based calling conventions

There is some support for identifying the different calling conventions as long as they're stack-based and has a prototype (arguments are fetched via internal.declaration.arguments). If there isn't a prototype, then the function's frame is scraped for everything above the saved program counter at which point a tuple for each frame member representing the argument is returned.. This was figured out via some reversing of the "sup" value representing the prototype for a function, but is not implemented when the calling convention uses registers to pass or return arguments.

This is needed for things like function.frame.arguments, or to implement a function.frame.result which will allow a user to fetch the way a function returns its result. The only issue with implementing this is that calling conventions become architecture/platform-specific (and I'm not sure if IDA even exposes this to us either).

All of the architecture/compiler-specific code is stored within the instruction module. If the calling conventions were to be properly implemented, then this code might need to be moved into an internal.architecture, or internal.compiler module. At this point then we could then probably store a table that contains the available calling conventions for each byte in the prototype's "sup" value. We'd also need to store how the standard calling conventions (cdecl and stdcall) influence registers for each architecture, or compiler. This might be the best way as the architecture-specific operand decoders and things aren't something the user needs to interact with and so it doesn't make sense to expose them via the instruction module.

Once this is implemented, then a user will be able to programmatically determine which registers are used to return a result from a function, or what registers are used as input to one. This would be awesome as the way I've seen most people do this is via the Hex-Rays decompiler (and not everybody has a license).

Documentation: IDA 7.4SP1 was released and docs need to be updated

IDA 7.4SP1 was released as noted at https://www.hex-rays.com/products/ida/7.4sp1/index.shtml. This is primarily due to a Python 3.8.0 issue that resulted in a symbol not being exported which broke IDA. It appears that now IDA checks your Python version to inform you about the busted version of Python3.

Anyways, the documentation needs to be updated in order to reference that this SP is also supported by the project.

This should avoid issues like #36.

Feature: Better support for bitfields within the enumeration module

The enumeration module is kind of hacky but still works for querying enumerations and their members. Enumeration members don't really have an identifier to reference them like structure members, and so there's no wrapper class similar to structure.member_t. As a result of this, bitfields for enumerations are very poorly supported. By poorly supported, I really mean it's nearly non-existent.

The author pretty much uses tags for dealing with bitfields, anyways...and so, due to this other solution already working satisfactorily, tje support for bitfields in enumerations is pretty weak. If users really care about this, however, then maybe it'll be worth the time figuring out how to properly integrate this within the already existing enumeration member interface without changing it too much.

Comments don't appear to remain hooked 100% of the time, and some events are missed.

Every once in a while, the hooks for updating the cache when the user changes the comments in a function or the database don't execute. This can result in the cache not being up to date at all times.

Some of the symptoms that appear are that the ValueError exception gets raised mentioning that the generator is already executing:

CRITICAL:root:internal.interface.priorityhook.callback(4199627, False, 'this') : Callback for "IDB_Hooks.changing_cmt" with priority (+0) raised an exception while executing <bound method type.changing of <class 'hooks.address'>>
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') : Traceback ("IDB_Hooks.changing_cmt" was hooked at)
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :   File "/home/user/idapro-7.4/python/2/ida_idp.py", line 3792, in dispatch
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :     cb.fun(slot, *args)
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :   File "/home/user/work/ida-minsc/base/_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :     result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :   File "/home/user/work/ida-minsc/misc/hooks.py", line 842, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :     ui.hook.idb.add('changing_cmt', address.changing, 0)
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') : 
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') : Traceback (most recent call last):
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :   File "/home/user/work/ida-minsc/base/_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :     result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :   File "/home/user/work/ida-minsc/misc/hooks.py", line 186, in changing
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') :     try: cls.event.send((ea, bool(repeatable_cmt), utils.string.of(newcmt)))
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') : ValueError: generator already executing
WARNING:root:internal.interface.priorityhook.callback(4199627, False, 'this') : 

Segfault (occasionally) when calling a custom-wrapped function

For some reason IDA will occasionally crash when calling a decorated/wrapped function that calls a native idaapi function. After some work at narrowing it down, it seems that this happens only when eventually calling a wrapped function that defines a variable kwargs parameter.

The symptom manifests itself during the wrappee trying to access any attributes off of this kwargs parameter as in some cases this parameter has been released and already garbage-collected at the time of its access. In most cases, the crash happens during the implementation of the LOAD_ATTR opcode when evaluating a frame with the wrapped code object.

This symptom is significantly amplified when the user combines a number of functions using fcompose, fpartial, fbox, etc.

Question: Which versions of IDA are supported?

The documentation states that IDA versions: 6.9 up to 7.1.180227 are supported.
https://arizvisa.github.io/ida-minsc/install.html#software-requirements

And the contribution guidelines states that new git tags will be added on new IDA versions.
https://github.com/arizvisa/ida-minsc/blob/master/CONTRIBUTING.md#new-versions-of-ida-tags

As far as I can see, there are no git tags on this repo, and the project is actively developed. So I wonder if newer IDA versions are supported, like 7.4?

Double-labeling issue when applying a name to an address belonging to a jumptable or a switch within a function

When renaming a label for a switch (or a jumptable) with database.name(), both the specified name and the original name are displayed at the given address. This appears to be because both a global and local name are being applied to the same location.

The semantics for minsc warrant that all labels within a function will be converted to local whereas all names not within a function are to be converted to global global. Despite these constraints, the listed option still must be considered for both label types.

Building documentation with Python3

At the moment, the plugin is only compatible with Python2 due to a variety of reasons. Fortunately IDA supports both Py2 and Py3 plugins (albeit not at the same time). However, I imagine at some point IDA is going to switch to Py3-only which will break this plugin entirely.

Py3 3.10.x will introduce a pattern matching syntax into the language and so if we want to leverage this in order to wrap Hex-Rays, we'll need to support Python3 at some point.

MIPS64 architecture support (KeyError and other exceptions raised when decoding an operand)

It was pointed out that only support for the MIPS32 architecture is implemented. When accessing an operand off of the arch MIPS64 architecture's dictionary the value returned from ins.op_bits() does not correlate to the actual operands. This is actually due to MIPS64 architecture not actualy having been implemented yet.

I guess it's now time to do this... :-/

Feature: The "plugin" namespace and support for 3rd-party plugins

In order to allow ida-minsc to fully support other plugins (and maybe do weird stuff with them), the "plugin" namespace was created. As of now, plugins aren't really supported especially because they might hook things and not be polite about their hooks.

To deal with this issue and possibly others, the initial idea was to implement a wrapper around IDA's plugin system and expose it via the "plugin" namespace. This would consist of enumerating all the plugins dropped in ida-minsc's "plugin" directory, and initializing each one's plugin_t individually. This way during ida-minsc's initialization, ida-minsc can monkey patch a few things or check the plugin's structure to figure out what hotkeys and hooks and things it sets up. Afterwards, we can then politely integrate the plugin's hooks into IDA-minsc's hooks and update the current hotkey state.

No progress has been made on this front because honestly...the author doesn't use any other plugins and so he's pretty ignorant to these types of tools. (Maybe I have too much pride or something?). Anyways, this would be super nice to have so that way installing 3rd party plugins is pretty seamless and doesn't require any sort of hackery on part of the user.

Rebasing a database to a new address results in a number of errors being displayed.

When rebasing the database in IDA 7.5, a bunch of exceptions are logged while executing hooks. Although the rebase is successful, the tag cache is screwed up as it's still pointing to the original locations due to the lack of being transformed.

It seems that the hook is being dispatched after the rebase has happened resulting in the following exception while trying to access an address prior to the rebase.

Traceback (most recent call last):
  File "/home/user/work/ida-minsc/base/_interface.py", line 382, in closure
    result = callable(*parameters)
  File "/home/user/work/ida-minsc/misc/hooks.py", line 687, in rename
    fl = database.type.flags(ea)
  File "/home/user/work/ida-minsc/base/_utils.py", line 338, in database.flags
    return f(*arguments, **keywords)
  File "/home/user/work/ida-minsc/base/database.py", line 3132, in flags
    return getflags(interface.address.within(ea))
  File "/home/user/work/ida-minsc/base/_interface.py", line 740, in within
    return cls.__within1__(*args)
  File "/home/user/work/ida-minsc/base/_interface.py", line 720, in __within1__
    raise internal.exceptions.OutOfBoundsError(u"{:s} : The specified address {:#x} is not within the bounds of the database ({:#x}<>{:#x}).".format(entryframe.f_code.co_name, ea, l, r))
OutOfBoundsError: closure : The specified address 0x15008 is not within the bounds of the database (0x400000<>0x419578).

The next exception is raised while trying to access the segment info vector. This looks like an off-by-one when accessing it.

Traceback (most recent call last):
  File "/home/user/work/ida-minsc/base/_interface.py", line 382, in closure
    result = callable(*parameters)
  File "/home/user/work/ida-minsc/misc/hooks.py", line 600, in rebase
    msg = u"Rebasing tagcache for segment {:d} of {:d} : {:#x} ({:+#x}) -> {:#x}".format(si, scount, info[si]._from, info[si].size, info[si].to)
  File "/home/user/idapro-7.5/python/2/ida_moves.py", line 366, in __getitem__
    return _ida_moves.segm_move_info_vec_t___getitem__(self, *args)
IndexError: out of bounds access

Documentation: The semantics of backticks vs double-backticks should be inverted

The semantics of backticks vs double-backticks for docs/docparse.py should be inverted so that variable names are single-backticked and references are double-backticked. This is so the docstrings will be less-cluttered when calling help() on them.

Unable to set a structure member's custom type, it always is 'int'

@arizvisa hi, I have to come and ask you questions, I don't know what went wrong, I can't set the member's custom type , it awaly is 'int':

import idaapi
import idc


st1 =structure.by_name('TTDBTextInfoGroup')
struct_name = 'TTDBTextInfoGroup*'
f1 = st1.members.by(offset=0)

print f1.typeinfo.dstr()
ordinal = f1.typeinfo.get_ordinal()
f1.typeinfo.set_numbered_type(idaapi.cvar.idati,ordinal,BTF_STRUCT, struct_name)
print f1.typeinfo.dstr()

Originally posted by @gool123456 in #61 (comment)

UnsupportedCapability when using function.convention(): Specified prototype declaration is a type forward which is currently unimplemented.

When function.convention() is called on functions with a specific prototype, the following exception is raised:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 374, in fn
    return func(*arguments, **keywords)
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 292, in function.convention
    return f(*arguments, **keywords)
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\function.py", line 208, in convention
    return convention(ui.current.address())
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 292, in function.convention
    return f(*arguments, **keywords)
  File "C:\Users\user\AppData\Roaming\Hex-Rays\IDA Pro\base\function.py", line 222, in convention
    raise E.UnsupportedCapability(u"{:s}.convention({!r}) : Specified prototype declaration is a type forward which is currently unimplemented.".format(__name__, func))
UnsupportedCapability: function.convention(6446198110L) : Specified prototype declaration is a type forward which is currently unimplemented.

This is due to the author not quite understanding how IDA creates this calling convention. One example can likey be found in version version 20,0,0,272 or flash.ocx and is associated with the fptc function at address 0x180392d30. This function has the prototype TIMECALLBACK fptc;.

Its supval for key 0x3000 is formatted like:

Python>internal.netnode.sup.get(h(), 0x3000).encode('hex')
3d0d54494d4543414c4c4241434b00

And the entire netnode has the following supvals defined:

Python>internal.netnode.sup.repr(h())
[0] 1b : '\xff\x81\x12\xc0;\x01\x01'
[1] 1000 : '\x04\x00P\x004\x00Q\x00'
[2] 3000 : '=\rTIMECALLBACK\x00'

The db.get.string() can not run at ida 6.8

Hi @arizvisa , this method Show Error in ida 6.8 :

Python>db.get.string(0x00000001409D9B70) Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base\_utils.py", line 290, in database.string return f(*arguments, **keywords) File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base\database.py", line 4877, in string strtype = idaapi.get_str_type(ea) AttributeError: 'module' object has no attribute 'get_str_type'

---

The code I modified is passed, can you pull it into the project ? :

   @classmethod
    def string(cls, ea, **length):
        """Return the array at the address specified by `ea` as a string.

        If the integer `length` is defined, then use it as the length of the array.
        """

        # Fetch the string type at the given address
        if idaapi.__version__ < 7.0:
            strtype = idc.GetStringType(ea)
        else: 
            strtype = idaapi.get_str_type(ea)

        # If no string was found, then try to treat it as a plain old array
        # XXX: idaapi.get_str_type() seems to return 0xffffffff on failure instead of idaapi.BADADDR
        if strtype in {idaapi.BADADDR, 0xffffffff}:
            res = cls.array(ea, **length)

            # It wasn't an array and was probably a structure, so we'll just complain to the user about it
            if not isinstance(res, _array.array):
                raise E.InvalidTypeOrValueError(u"{:s}.string({:#x}{:s}) : The type at address {:#x} cannot be treated as an unformatted array and as such is not convertible to a string.".format('.'.join((__name__, cls.__name__)), ea, u", {:s}".format(utils.string.kwargs(length)) if length else '', ea))

            # Warn the user and convert it into a string
            logging.warn(u"{:s}.string({:#x}{:s}) : Unable to automatically determine the string type at address {:#x}. Treating as an unformatted array instead.".format('.'.join((__name__, cls.__name__)), ea, u", {:s}".format(utils.string.kwargs(length)) if length else '', ea))
            return res.tostring()

        # Get the string encoding (not used)
        encoding = idaapi.get_str_encoding_idx(strtype)

        # Get the terminal characters that can terminate the string
        sentinels = idaapi.get_str_term1(strtype) + idaapi.get_str_term2(strtype)

        # Extract the fields out of the string type code
        res = idaapi.get_str_type_code(strtype)
        sl, sw = ord(res) & idaapi.STRLYT_MASK, ord(res) & idaapi.STRWIDTH_MASK

        # Figure out the STRLYT field
        if sl == idaapi.STRLYT_TERMCHR << idaapi.STRLYT_SHIFT:
            shift, f1 = 0, operator.methodcaller('rstrip', sentinels)
        elif sl == idaapi.STRLYT_PASCAL1 << idaapi.STRLYT_SHIFT:
            shift, f1 = 1, utils.fpass
        elif sl == idaapi.STRLYT_PASCAL2 << idaapi.STRLYT_SHIFT:
            shift, f1 = 2, utils.fpass
        elif sl == idaapi.STRLYT_PASCAL4 << idaapi.STRLYT_SHIFT:
            shift, f1 = 4, utils.fpass
        else:
            raise E.UnsupportedCapability(u"{:s}.string({:#x}{:s}) : Unsupported STRLYT({:d}) found in string at address {:#x}.".format('.'.join((__name__, cls.__name__)), ea, u", {:s}".format(utils.string.kwargs(length)) if length else '', sl, ea))

        # Figure out the STRWIDTH field
        if sw == idaapi.STRWIDTH_1B:
            f2 = operator.methodcaller('decode', 'utf-8')
        elif sw == idaapi.STRWIDTH_2B:
            f2 = operator.methodcaller('decode', 'utf-16')
        elif sw == idaapi.STRWIDTH_4B:
            f2 = operator.methodcaller('decode', 'utf-32')
        else:
            raise E.UnsupportedCapability(u"{:s}.string({:#x}{:s}) : Unsupported STRWIDTH({:d}) found in string at address {:#x}.".format('.'.join((__name__, cls.__name__)), ea, u", {:s}".format(utils.string.kwargs(length)) if length else '', sw, ea))

        # Read the pascal length if one was specified in the string type code
        if shift:
            res = cls.unsigned(ea, shift)
            length.setdefault('length', res)

        # Now we can read the string..
        res = cls.array(ea + shift, **length).tostring()

        # ..and then process it.
        return f1(f2(res))

database.get.string() incorrectly includes null terminator in string

If a C style string "abc" exists in the idb at address 0, then database.get.string(0) will return the string "abc\0" instead of "abc". It will therefore have a length of 4 instead of 3.

Attempting to use the returned string in python formatting such as: print("foo %s bar" % database.get.string(0)) will print foo abc instead of the expected foo abc bar

I am using the latest ida-minsc code from github with IDA32 v7.1 64bit for windows.

No exception is raised when IDA fails to modify the property of a structure or its members

When accessing a structure (strucure_t) or a frame (func.frame) there are a number of properties that are exposed which can be modified. Each member from the structure (member_t) can also contain more properties which can be modified. Each of these are decorated with the @property decorator.

If IDA fails to modify these properties as per the implementation, nothing is raised or returned indicating success/failure. Although the setters of these methods actually return the result, since these are treated as object properties their result cannot be captured in any way. Hence this can result in confusion when trying to modify a property and nothing appears to actually have happened.

This pattern exists in a number of places within the structure module:

• structure_t.name
• structure_t.comment
• structure_t.size
• structure_t.index
• member_t.name
• member_t.comment
• member_t.type

It also seems like the following properties are mis-decorated. The member_t.typeinfo for example is mis-decorated with the @type.getter decorator.

• member_t.typeinfo

ImportError: No module named builtins

The requirements.txt is missing an entry for the future module which results in the "builtins" module not being able to be imported.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
...
ImportError: No module named builtins