nxp-mcuxpresso / pypemicro

Python PEMicro debug probes support

This is simple package that provides Python interface for PEMicro debug probes precompiled libraries. The package provides most of functionality of the PEMicro libraries and their debug probes.

The package is tested only with Multilink/FX and Cyclone/FX probes on NXP ARM microcontrollers.

The PEMicro company helps with this development, so big Thanks to them (www.pemicro.com).

Author: Petr Gargulak (NXP 2020-2023, www.nxp.com)

Dependencies

There is no dependencies

Installation

Directly from www.pypi.org:

    $ pip install pypemicro

pypemicro's Issues

Error raised on Apple M1 systems

The error below is raised from pyocd list -vv when running on an Apple M1 MacBook Air.

Pypemicro needs to gracefully handle situations where it's run on a system for which it does not have a valid library. Or at least raise a distinct PEMicroUnsupportedArchitecture exception so that pyocd-pemicro can catch and gracefully handle it.

0000177:DEBUG:pemicro:There is no PEMICRO library.
0000177:CRITICAL:__main__:PEMicro Exception with error code:PEMicro Exception with error code:PEMicro Exception with error code:Unable to find any usable library in the system!
Traceback (most recent call last):
  File "/Users/creed/projects/pyocd/venv/mac39/lib/python3.9/site-packages/pypemicro/pemicro.py", line 411, in get_pemicro_lib
    filename = PyPemicro.get_newest_lib_filename(libs_list)
  File "/Users/creed/projects/pyocd/venv/mac39/lib/python3.9/site-packages/pypemicro/pemicro.py", line 389, in get_newest_lib_filename
    raise PEMicroException("Unable to find any usable library in the system!")
pypemicro.pemicro.PEMicroException: PEMicro Exception with error code:Unable to find any usable library in the system!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/creed/projects/pyocd/venv/mac39/lib/python3.9/site-packages/pypemicro/pemicro.py", line 427, in list_ports
    lib = PyPemicro.get_pemicro_lib()
  File "/Users/creed/projects/pyocd/venv/mac39/lib/python3.9/site-packages/pypemicro/pemicro.py", line 415, in get_pemicro_lib
    raise PEMicroException(str(exc))
pypemicro.pemicro.PEMicroException: PEMicro Exception with error code:PEMicro Exception with error code:Unable to find any usable library in the system!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/creed/projects/pyocd-pemicro/pyocd_pemicro/pemicro_probe.py", line 78, in get_all_connected_probes
    port_list = pemicro.list_ports()
  File "/Users/creed/projects/pyocd/venv/mac39/lib/python3.9/site-packages/pypemicro/pemicro.py", line 429, in list_ports
    raise PEMicroException(str(exc))
pypemicro.pemicro.PEMicroException: PEMicro Exception with error code:PEMicro Exception with error code:PEMicro Exception with error code:Unable to find any usable library in the system!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/creed/projects/pyocd/pyocd/__main__.py", line 402, in run
    self._COMMANDS[self._args.cmd](self)
  File "/Users/creed/projects/pyocd/pyocd/__main__.py", line 462, in do_list
    ConnectHelper.list_connected_probes()
  File "/Users/creed/projects/pyocd/pyocd/core/helpers.py", line 109, in list_connected_probes
    allProbes = ConnectHelper.get_all_connected_probes(blocking=False)
  File "/Users/creed/projects/pyocd/pyocd/core/helpers.py", line 82, in get_all_connected_probes
    allProbes = DebugProbeAggregator.get_all_connected_probes(unique_id=unique_id)
  File "/Users/creed/projects/pyocd/pyocd/probe/aggregator.py", line 64, in get_all_connected_probes
    probes += cls.get_all_connected_probes(unique_id, is_explicit)
  File "/Users/creed/projects/pyocd-pemicro/pyocd_pemicro/pemicro_probe.py", line 83, in get_all_connected_probes
    six.raise_from(cls._convert_exception(exc), exc)
  File "<string>", line 3, in raise_from
pyocd.core.exceptions.ProbeError: PEMicro Exception with error code:PEMicro Exception with error code:PEMicro Exception with error code:Unable to find any usable library in the system!

get_pemicro_lib() raises different error depending on get_newest

In PyPemicro.get_pemicro_lib(), when run on a system where no valid library is found, you will get different exceptions depending on the value of the get_newest parameter.

Here's the main code of get_pemicro_lib():
https://github.com/NXPmicro/pypemicro/blob/806c213b13b884650cf0310ba6bb949d2ab1f0b2/pypemicro/pemicro.py#L401-L405

The branch that calls get_newest_lib_filename() checks whether the libs_list parameter is empty, and raises a specific PEMIcroException if so.

The other branch that calls get_lib_filename() doesn't check if libs_list is empty, and grabs index 0. So if libs_list is empty, you'll get an IndexError exception where on the newest branch you'd get a PEMIcroException.

No tags for releases on pypi.org: 0.1.0 to 0.1.9 - excluding 0.1.3 - are affected

Hi! As I'm currently looking into packaging this project as a dependency for other packages (pyocd-pemicro), I am evaluating the upstreams of many packages.

I realized that although this is the upstream for pypemicro, it does not reflect the releases of many versions in this repository.
The pypi history of the project shows releases 0.1.0 up to 0.1.9, however currently there is only a tag for 0.1.3 available in this repository.

@Gargy007 as you seem to maintain this repository and the pypi releases, can you please add tags for the releases? It helps downstreams to switch to them if something is wrong with the sdist tarballs on pypi.org and more importantly it enables anyone to lookup changes more easily and mitigate supply chain attacks.

Generally a workflow in which the release to pypi.org is automated in a CI pipeline from a tag (i.e. tag -> pipeline builds/tests -> release sdist/wheel to pypi.org) is advisable for maximum transparency.

AttributeError: 'PyPemicro' object has no attribute 'lib' for version 0.1.8

This exception occurs when running pyocd list with the latest pyocd-pemicro v1.1.1 and pypemicro 0.1.8.

Traceback (most recent call last):
  File "/Users/creed/projects/pyocd-clean/venv/lib/python3.9/site-packages/pypemicro/pemicro.py", line 688, in __del__
    if self.lib is not None:
AttributeError: 'PyPemicro' object has no attribute 'lib'

This was on macOS 11.6.1 and without the PEMicro drivers installed.

Source code for shared libraries?

Hi! I would like to package this project for Arch Linux (as this is a dependency for spsdk).

Unfortunately this repository carries prebuilt binaries, which I would rather build from source instead, given that the sources of this repository fall under the terms of the BSD 3-clause.

However, there seems to be no source code available for these files. Where do they originate from? How have they been built? The Linux libraries lack full RELRO.
Are the binaries and their sources also covered by the BSD 3-clause license, as the initial commit implies?

FreeBSD/*BSD support missing

Hello world :-)

This pypemicro is now part of the https://github.com/pyocd/pyOCD and became a blocker on FreeBSD platform, probably other *BSD platforms (i.e. OpenBSD, NetBSD, etc), and other platforms not on the list of: Windows, Linux, Darwin.

• I will try to fix the OS detection mechanism.

(venv37zephyr) pyocd list
0001557:CRITICAL:__main__:Unable to determinate running operation system (FreeBSD)
Traceback (most recent call last):
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pypemicro/pemicro.py", line 401, in get_pemicro_lib
    libs_list = PyPemicro.get_pemicro_lib_list(dllpath=dllpath)
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pypemicro/pemicro.py", line 142, in get_pemicro_lib_list
    lib_name = PyPemicro.get_library_name()
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pypemicro/pemicro.py", line 99, in get_library_name
    raise PEMicroException(f"Unable to determinate running operation system ({system_name})")
pypemicro.pemicro.PEMicroException: Unable to determinate running operation system (FreeBSD)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pypemicro/pemicro.py", line 419, in list_ports
    lib = PyPemicro.get_pemicro_lib()
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pypemicro/pemicro.py", line 407, in get_pemicro_lib
    raise PEMicroException(str(exc))
pypemicro.pemicro.PEMicroException: Unable to determinate running operation system (FreeBSD)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd_pemicro-1.0.4-py3.7.egg/pyocd_pemicro/pemicro_probe.py", line 81, in get_all_connected_probes
    port_list = pemicro.list_ports()
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pypemicro/pemicro.py", line 421, in list_ports
    raise PEMicroException(str(exc))
pypemicro.pemicro.PEMicroException: Unable to determinate running operation system (FreeBSD)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd-0.30.4.dev31-py3.7.egg/pyocd/__main__.py", line 150, in run
    status = cmd.invoke()
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd-0.30.4.dev31-py3.7.egg/pyocd/subcommands/list_cmd.py", line 93, in invoke
    ConnectHelper.list_connected_probes()
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd-0.30.4.dev31-py3.7.egg/pyocd/core/helpers.py", line 109, in list_connected_probes
    allProbes = ConnectHelper.get_all_connected_probes(blocking=False)
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd-0.30.4.dev31-py3.7.egg/pyocd/core/helpers.py", line 82, in get_all_connected_probes
    allProbes = DebugProbeAggregator.get_all_connected_probes(unique_id=unique_id)
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd-0.30.4.dev31-py3.7.egg/pyocd/probe/aggregator.py", line 64, in get_all_connected_probes
    probes += cls.get_all_connected_probes(unique_id, is_explicit)
  File "/home/x/usr/local/venv37zephyr/lib/python3.7/site-packages/pyocd_pemicro-1.0.4-py3.7.egg/pyocd_pemicro/pemicro_probe.py", line 90, in get_all_connected_probes
    raise cls._convert_exception(exc) from exc
pyocd.core.exceptions.ProbeError: Unable to determinate running operation system (FreeBSD)

• Another issue is the binary blob library provided with the package. Is it Open-Source? Can I compile it somehow?

Tests fail on non x86_64 machines?

I am packaging this module for openSUSE, as it is a dependency of pynitrokey.

I noticed that some of the tests fail, but only on non-x86_64 machines:

• i586
• armv7l
• s390x
• ppc64
• ppc64le
• aarch64

The tests that fail are the following:

[   16s] FAILED tests/test_basic.py::test_library_load - pypemicro.pemicro.PEMicroExce...
[   16s] FAILED tests/test_basic.py::test_lib_version - pypemicro.pemicro.PEMicroExcep...
[   16s] FAILED tests/test_basic.py::test_lib_dll_version - pypemicro.pemicro.PEMicroE...
[   16s] FAILED tests/test_basic.py::test_list_devices - pypemicro.pemicro.PEMicroExce...
[   16s] FAILED tests/test_basic.py::test_list_probes - pypemicro.pemicro.PEMicroExcep...

And all of them fail due to pypemicro.pemicro.PEMicroException: There is any suitable library for this OS.

I daresay that not many people will configure their nitro key on s390x systems, but aarch64 or ppc64 (or even i586) might be possible.

Does this module simply not work on those architectures? If so, I would try to only build it for x86_64 (where the tests succeed).

Or am I missing some other dependency?

Thanks in advance,
Johannes

Exception messages have multiple copies of prefix

When an exception is raised, the message prefix has PEMicro Exception with error code: added as a prefix multiple times.

When pyocd receives the exception, it ends up looking like PEMicro Exception with error code:PEMicro Exception with error code:PEMicro Exception with error code:Unable to find any usable library in the system!, with three copies of the prefix.

Only log info about PEMicro shared library once

Pypemicro repeatedly outputs the same log message when opening the PEMicro library. This is annoying for pyocd users when it is polling for devices.

The request is that pypemicro track that it has logged info about the library, so it only logs once per process.

We should also consider using the passed-in loggers for all logging. This allows pyocd to redirect the pypemicro logging to a trace log. However, I have mixed feeling about this, since technically using its own logger is correct. Passed-in loggers are used for the JLink probe only because in that case, the logging is being done by the JLink shared lib and being redirected to a Python library by the C callback. So maybe pyocd should just set the pypemicro logger level directly?

Example log:

0001662:DEBUG:pemicro:Opened PEMicro library: c:\python38\lib\site-packages\pypemicro\libs\Windows\unitacmp-64.dll
Waiting for a debug probe to be connected...
0002351:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0002938:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0003535:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0004100:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0004696:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0005251:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0005817:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0006408:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0006981:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0007544:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0008144:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0008712:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0009316:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0009886:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0010453:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0011051:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
0011645:DEBUG:pemicro:Opened PEMicro library: unitacmp-64.dll
…
…

Mach-O ARM64 libraries available to support Apple silicon?

Hi and thanks for creating this!

I was wondering whether you had access to the unitacmp-64.dylib as Mach-O 64-bit executable arm64? I couldn't find it from PEMicro's website unfortunately.

Running a x86_64 Python3 with the x86_64 dylib works (because of Rosetta 2 emulation), but won't recognize my USB Multilink device.

Best,
Christian