olofk / ipyxact

Python-based IP-XACT parser

ipyxact

Python-based IP-XACT parser

ipyxact is a helper library that creates python structures from parsed IP-Xact files. It is intended to make it easier to use IP-Xact files in a python application

ipyxact is also a part of the award-winning Lattice Propel Design Environment

Also distributed with the source are some example applications to show how ipyxact can be used and an example IP-Xact component file (generic_example.xml).

python gen_c_header.py *xml_file* creates a C header file from the memory maps in xml_file

python gen_markdown.py *xml_file* creates markdown documentation of the memory maps in xml_file

python print_filesets.py *xml_file* creates a list of the verilog files found in the fileSets section of xml_file

ipyxact is compatible with Python >= 2.7

ipyxact's Issues

a small buggette in the parser WRT boolean

it appears that the first elif below should simply be an if. This blows on import using either python 2 or 3
class IpxactBool(str):
def new(cls, *args, **kwargs):
if not args:
return None
expr = args[0].strip(' \t\n\r')
elif expr in ['true', 'false']:
return super(IpxactBool, cls).new(cls, expr)
else:
raise Exception

I'm surprised this got passed even the most basic of testing. Makes me think I'm doing something wrong. Sorry if this is a red herring.

how to generate verilog file from my_reg_descr.xml ?

Your code is amazing! But can you tell me if the song turns my reg desrip xml file into verilog file xxx.v ? Thank you very much!

cannot load leon2 registers file

loading leon2 register description file
https://github.com/bat52/cheap_pie/blob/master/devices/leon2_creg.xml
fails with this error

ValueError: invalid literal for int() with base 10: '4 * (2 ** 10)'

Fields not having a reset mask are still parsed with a mask of all zeros

The IP-XACT standard allows for an optional mask element inside a field reset element. This is used to define which bits of the field have a known reset value. The absence of the mask element makes this N/A, i.e. it is equivalent to a mask of the same size as the register field and consisting of all 1's.

When the latter situation occurs, the ipyxact parser (v0.3.2) still evaluates the mask as all 0's. This violates the standard and makes impossible to discriminate between a mask of all 0's and an absent mask (i.e. N/A or all 1's).

For instance, consider the ipxact.xml sample below:

<ipxact:component xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ipxact="http://www.accellera.org/XMLSchema/IPXACT/1685-2014" xsi:schemaLocation="http://www.accellera.org/XMLSchema/IPXACT/1685-2014/index.xsd">
    <ipxact:vendor>some_vendor</ipxact:vendor>
    <ipxact:library>some_library</ipxact:library>
    <ipxact:name>some_component</ipxact:name>
    <ipxact:version>1.0</ipxact:version>
    <ipxact:memoryMaps>
        <ipxact:memoryMap>
            <ipxact:name>some_memory_map</ipxact:name>
            <ipxact:addressBlock>
                <ipxact:name>some_address_block</ipxact:name>
                <ipxact:baseAddress>0x0</ipxact:baseAddress>
                <ipxact:range>1</ipxact:range>
                <ipxact:width>32</ipxact:width>
                <ipxact:register>
                    <ipxact:name>some_register</ipxact:name>
                    <ipxact:addressOffset>0x0</ipxact:addressOffset>
                    <ipxact:size>32</ipxact:size>
                    <ipxact:access>read-write</ipxact:access>
                    <ipxact:field>
                        <ipxact:name>without_mask</ipxact:name>
                        <ipxact:bitOffset>0</ipxact:bitOffset>
                        <ipxact:resets>
                            <ipxact:reset>
                                <ipxact:value>0xff</ipxact:value> <!-- No mask here, i.e. should be N/A or all 1's -->
                            </ipxact:reset>
                        </ipxact:resets>
                        <ipxact:bitWidth>8</ipxact:bitWidth>
                        <ipxact:access>read-write</ipxact:access>
                    </ipxact:field>
                    <ipxact:field>
                        <ipxact:name>with_zero_mask</ipxact:name>
                        <ipxact:bitOffset>8</ipxact:bitOffset>
                        <ipxact:resets>
                            <ipxact:reset>
                                <ipxact:value>0xff</ipxact:value>
                                <ipxact:mask>0x0</ipxact:mask> <!-- This mask is all 0's -->
                            </ipxact:reset>
                        </ipxact:resets>
                        <ipxact:bitWidth>8</ipxact:bitWidth>
                        <ipxact:access>read-write</ipxact:access>
                    </ipxact:field>
                    <ipxact:field>
                        <ipxact:name>with_nonzero_mask</ipxact:name>
                        <ipxact:bitOffset>16</ipxact:bitOffset>
                        <ipxact:resets>
                            <ipxact:reset>
                                <ipxact:value>0xff</ipxact:value>
                                <ipxact:mask>0xff</ipxact:mask> <!-- This mask is all 1's -->
                            </ipxact:reset>
                        </ipxact:resets>
                        <ipxact:bitWidth>8</ipxact:bitWidth>
                        <ipxact:access>read-write</ipxact:access>
                    </ipxact:field>
                </ipxact:register>
            </ipxact:addressBlock>
            <ipxact:addressUnitBits>8</ipxact:addressUnitBits>
        </ipxact:memoryMap>
    </ipxact:memoryMaps>
</ipxact:component>

When parsed:

from ipyxact.ipyxact import Component

component = Component()
component.load("ipxact.xml")

register = component.memoryMaps.memoryMap[0].addressBlock[0].register[0]
fields = register.field
print(f'Printing fields from register {register.name}:')

for f in fields:
    _reset = f.resets.reset.value
    _mask = f.resets.reset.mask
    print(f'    Field {f.name} has reset value of x{_reset:X} and mask of x{_mask:X} --> overall reset value: x{_reset & _mask:X}')

It yields:

Printing fields from register some_register:
    Field without_mask has reset value of xFF and mask of x0 --> overall reset value: x0
    Field with_zero_mask has reset value of xFF and mask of x0 --> overall reset value: x0
    Field with_nonzero_mask has reset value of xFF and mask of xFF --> overall reset value: xFF

As you can see, fields without_mask and with_zero_mask are both evaluated with a mask of all 0's and there's no way to tell the former doesn't have a mask defined in the first place. I believe the above considerations also applies to register resets as per IP-XACT 2009, even though I haven't tested it.

I suspect this behavior is due to:

• The IpxactInt type being used for mask inside ipyxact_yaml.py.
• Line 150 of ipyxact.py setting an attribute value of eval(_type)() if the element is not found when parsing the IP-XACT.

The preferred behavior for me would be not to create a mask attribute inside the reset class if there's no mask in the IP-XACT, but it may involve a re-structuring on how the parser works.

A sub-optimal solution could be creating another integer type that defaults to all 1's and use it for the mask. However, note that in this case there's no easy way to dynamically adjust the number of bits that should be 1 w.r.t. the field/register size, so a worst-case guess is needed (a bit of an hack).

Loading Xilinx File failes (root.tag == S+self._tag)

Hello

I'm completely new to this topic.
I would like to read and modify a Xilinx xci file.

f1 = open('axis_switch_0.xci', 'r')
from ipyxact.ipyxact import Component
c1 = Component()
c1.load(f1)

Traceback (most recent call last):
File "", line 1, in
File "ipyxact.py", line 106, in load
raise Exception

What am I doing wrong?
The parsed file is attached.
axis_switch_0.xci.zip

Thank you for your help

Provide support for vendor specific extensions

This is a feature request for adding support to parse major vendor extensions (Xilinx) to ipyxact.

Below is an example of a fileSet extension that specifies a dependency between a Xilinx core to a 'subCore'

    <spirit:fileSet>
      <spirit:name>xilinx_utilityxitfiles_xilinx_com_ip_xbip_utils_3_0__ref_view_fileset</spirit:name>
      <spirit:vendorExtensions>
        <xilinx:subCoreRef>
          <xilinx:componentRef xilinx:vendor="xilinx.com" xilinx:library="ip" xilinx:name="xbip_utils" xilinx:version="3.0" xilinx:coreRevision="10">
            <xilinx:mode xilinx:name="read_mode"/>
          </xilinx:componentRef>
        </xilinx:subCoreRef>
      </spirit:vendorExtensions>
    </spirit:fileSet>

Ipxact Items can't have their own children

Hello @olofk,

I use your library in my company, and I'm new to it.
I noticed with surprise that if we add an object to another object that contains a children property then each object of that type will have the child in question.

Can an object that contains children have its own objects? Or am I using your library wrong?

Here is an example that illustrates my problem :

import ipyxact.ipyxact

f = open("/Users/saschasalles/Desktop/out.xml", "w")
component = ipyxact.ipyxact.Component()
component.name = "testname"
component.version = "1.0"
component.description = "desc"
component.vendor = "BIG COMPANY"
component.library = "library"

component.memoryMaps = ipyxact.ipyxact.MemoryMaps()
memoryMap = ipyxact.ipyxact.MemoryMap()
memoryMap.name = "RegisterMap"
memoryMap.addressUnitBits = 32
component.memoryMaps.memoryMap.append(memoryMap)

addressBlock = ipyxact.ipyxact.AddressBlock()
addressBlock.name = "FUNC"
addressBlock.width = 32
addressBlock.baseAddress = 2684485632

addressBlock2 = ipyxact.ipyxact.AddressBlock()
addressBlock2.name = "FUNC2"
addressBlock2.width = 24
addressBlock2.baseAddress = 2684485631

register1 = ipyxact.ipyxact.Register()
register1.name = "REG1"
register1.description = "desc"
register1.access = "read-write"
register1.addressOffset = 0
register1.dim = 0

addressBlock.register.append(register1)

memoryMap.addressBlock.append(addressBlock)
memoryMap.addressBlock.append(addressBlock2)
component.write(f)

generate this :

<?xml version='1.0' encoding='UTF-8'?>
<ipxact:component xmlns:ipxact="http://www.accellera.org/XMLSchema/IPXACT/1685-2014">
    <ipxact:description>desc</ipxact:description>
    <ipxact:vendor>BIG COMPANY</ipxact:vendor>
    <ipxact:library>library</ipxact:library>
    <ipxact:name>testname</ipxact:name>
    <ipxact:version>1.0</ipxact:version>
    <ipxact:memoryMaps>
        <ipxact:memoryMap>
            <ipxact:name>RegisterMap</ipxact:name>
            <ipxact:addressUnitBits>32</ipxact:addressUnitBits>
            <ipxact:addressBlock>
                <ipxact:name>FUNC</ipxact:name>
                <ipxact:baseAddress>2684485632</ipxact:baseAddress>
                <ipxact:width>32</ipxact:width>
                <ipxact:register>
                    <ipxact:name>REG1</ipxact:name>
                    <ipxact:description>desc</ipxact:description>
                    <ipxact:access>read-write</ipxact:access>
                    <ipxact:addressOffset>0</ipxact:addressOffset>
                    <ipxact:dim>0</ipxact:dim>
                </ipxact:register>
            </ipxact:addressBlock>
            <ipxact:addressBlock>
                <ipxact:name>FUNC2</ipxact:name>
                <ipxact:baseAddress>2684485631</ipxact:baseAddress>
                <ipxact:width>24</ipxact:width>
                <ipxact:register>
                    <ipxact:name>REG1</ipxact:name>
                    <ipxact:description>desc</ipxact:description>
                    <ipxact:access>read-write</ipxact:access>
                    <ipxact:addressOffset>0</ipxact:addressOffset>
                    <ipxact:dim>0</ipxact:dim>
                </ipxact:register>
            </ipxact:addressBlock>
        </ipxact:memoryMap>
    </ipxact:memoryMaps>
</ipxact:component>

In this case I don't want my addressBlock2 to contain my register1.

Thank you in advance for your help,
Best Regards,
Sascha Salles

Create a new release

Could you create a new release on pypi with the latest changes?
Thanks!

Issue adding AlternateRegister field to yaml des

Hi,
I am using your ipyxact lib to process an XML related to an IP.
In this XML some r"egister" has the sub-field "alternateregisters", which has "alternateregister" and so on..
I have modified the yaml in this way:
.......
alternateRegister:
MEMBERS:
name: str
description: str
access: str
addressOffset: IpxactInt
size: IpxactInt
volatile: IpxactBool
CHILDREN:
- field
- reset
alternateRegisters:
CHILDREN:
- alternateRegister
register:
MEMBERS:
name: str
description: str
access: str
addressOffset: IpxactInt
size: IpxactInt
volatile: IpxactBool
CHILDREN:
- field
- reset
CHILD:
- alternateRegisters

but now what happens is that inside "field" of "register", there are also the "field" coming from "alternateregister".
Seems that due to the presence of "field" inside "register" and "alternateregister", the library append
under "field" of "register" also the "field" of "alternateregister"
What do you think about?
Thanks in advance

Reading multiple components does not work

When reading in 2 components in the same python instance, the second one is incomplete:

In [1]: f1 = open('file1.xml', 'r')
In [2]: f2 = open('file2.xml', 'r')
In [3]: from ipyxact.ipyxact import Component
In [4]: c1 = Component()
In [5]: c2 = Component()
In [6]: c1.load(f1)
In [7]: c2.load(f2)
In [8]: c1.memoryMaps
Out[8]: <ipyxact.ipyxact.MemoryMaps at 0x2ffa290>
In [9]: c1.memoryMaps.memoryMap
Out[9]:
[<ipyxact.ipyxact.MemoryMap at 0x3085dd0>,
 <ipyxact.ipyxact.MemoryMap at 0x30870d0>]

In [10]: c2.memoryMaps

In [11]: c2.memoryMaps.memoryMap
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-4b0e797fcc70> in <module>()
----> 1 c2.memoryMaps.memoryMap

AttributeError: 'NoneType' object has no attribute 'memoryMap'

I need to start a new python interpreter and load everything again to be able to read the second file correctly.

I tried by deleting the references to c1 and by unloading/reloading the module but only a fresh python interpreter will make it work again.

Changing the order of the opening, instance creation or loading does not make a difference.

add github topics (at least `ip-xact`) to this project

I'd love to see this project show up on this list https://github.com/topics/ip-xact

Required Python version

You should mention somewhere that the minimum required Python version is 2.7 (I guess). I tried running it with Python 2.6 (old Linux install) and I got a cryptic error message that made me scratch my head a bit.

Getting ValueError when trying to parse a file path with env var inside <spirit> tags

I ran into an issue like this:

Traceback (most recent call last):
  File "uvmf/my_script.py", line 15, in <module>
    ipxact_reader = IpxactReader(ipxact_file)
  File "/userwork11/tpoikela/git/uvmf/uvmf_tb_gen/uvmf_tb_gen.py", line 107, in __init__
    ipxact_comp.load(f)
  File "/userwork11/tpoikela/git/uvmf/.venv/lib/python3.8/site-packages/ipyxact/ipyxact.py", line 108, in load
    self.parse_tree(root, self.nsmap[self.nsversion])
  File "/userwork11/tpoikela/git/uvmf/.venv/lib/python3.8/site-packages/ipyxact/ipyxact.py", line 144, in parse_tree
    t.parse_tree(f, ns)
  File "/userwork11/tpoikela/git/uvmf/.venv/lib/python3.8/site-packages/ipyxact/ipyxact.py", line 135, in parse_tree
    t.parse_tree(f, ns)
  File "/userwork11/tpoikela/git/uvmf/.venv/lib/python3.8/site-packages/ipyxact/ipyxact.py", line 123, in parse_tree
    setattr(self, _name, eval(_type)(tmp.text))
  File "/userwork11/tpoikela/git/uvmf/.venv/lib/python3.8/site-packages/ipyxact/ipyxact.py", line 62, in __new__
    return int(expr.replace('_', ''), base)
ValueError: invalid literal for int() with base 10: '${MODULESPATH}/module/doc/moduleprogmodel.xlsx'

In the IP-XACT file, I see that it has specified some filepaths like this:

<spirit:parameter>
            <spirit:name>imported_from</spirit:name>
            <spirit:value>${MODULES_PATH}/module/doc/module_prog_model.xlsx</spirit:value>
</spirit:parameter>

Expected result: The parser does not throw ValueError, and parses the structure and filepath with env var correctly. No env var expansion/substitution is expected.

Read multi dimensional array

The specs say that:
dim (optional; type: unsignedLongintExpression (see C.3.8)) assigns an unbounded dimension to the register, so it is repeated as many times as the value of the dim elements

So I tried to have multiple dim tag, and the code compiles:

<ipxact:register>
          <ipxact:name>WDT_CR</ipxact:name>
		  <ipxact:dim>2</ipxact:dim>
		  <ipxact:dim>3</ipxact:dim>

However, I am having trouble accessing the second dim dimension, if I do register.dim I would get 2 only. My yaml struct is:

register:
  MEMBERS:
    name: str
    description: str
    dim: IpxactInt
    access: str
    addressOffset: IpxactInt
    size: IpxactInt
    range: IpxactInt
    volatile: IpxactBool
    isPresent: str

How would I access the multiple dim tag?

<spirit:master/> and <spirit:slave/> info for BusInterfaces cannot be extracted in useful way

Since master/slave is defined as an empty tag like this (see also http://www.accellera.org/XMLSchema/SPIRIT/1685-2009/busInterface.xsd) :

<spirit:busInterface>
  <spirit:slave/>
  <!-- OR -->
  <spirit:master/>
  ...
<spirit:busInterface/>

in the current implementation of ipyxact they are always empty strings, regardless of their presence or absence in the IP-XACT file. Thus, it is not possible to extract master/slave information from the parsed structure.

I don't know if ipyxact can currently handle a case like this, but one option would be to set the property as True if it's defined, and False otherwise. However, I don't think this fits into your current scheme of defining the grammar/types using yaml.

Do release to get YAML warning away from people

We have multiple people confused by the warning from pyyaml fixed in 064467e. Can do you a pypi release to get this commit to people?

additional fields for 'register'

Hi,

i am using the package for registers description and noticed that you don't include the reset field inside the register type like i have in my ipxact file:

so i edited the ipyxact_yaml.py file as follows:

1. added the "reset" field to the "register" CHILDREN
   

2. added a new "reset" field with "value" and "reset" MEMBERS
   

and it is working for me (i am using Python 3.6.3 and ipyxact v0.2.3)

Thanks,
Shay

Cannot obtain both attribute and value for a tag

This is a completely arbitrary example, but take the following:

  <ipxact:tagOne>
    <ipxact:tagTwo name='a name'>0</ipxact:tagTwo>
  <ipxact:tagOne>

There's no way to get both the name attribute and the 0, as tags designated as CHILD or CHILDREN in the yaml override MEMBERS of the same name.

[Q] Looking for a Tool - Register Read / Write UI based on IP-XACT

Hi,

Looking for a tool that provides a UI based on an IP-XACT XML.

Of course such a tool would need read/write hooks for accessing the relevant target (or emulation) platform.

Thanks,

Alon

Additional fields into ipyxact/ipxact_yaml.py

I have problem with adding necessary fields into YAML (in ipyxact/ipxact_yaml.py).
Can't google what mean CHILD, CHILDREN and MEMBERS in apply to YAML-markup.

I need to parse default reset value, in XML that look as (register -> field -> resets -> reset -> value):

				<ipxact:register>
					<ipxact:name>IDM</ipxact:name>
					<ipxact:description>ID Mask</ipxact:description>
					<ipxact:dim>0</ipxact:dim>
					<ipxact:addressOffset>'hA0</ipxact:addressOffset>
					<ipxact:size>32</ipxact:size>
					<ipxact:volatile>false</ipxact:volatile>
					<ipxact:access>read-write</ipxact:access>
					<ipxact:field>
						<ipxact:name>IDM</ipxact:name>
						<ipxact:description>ID Mask</ipxact:description>
						<ipxact:bitOffset>0</ipxact:bitOffset>
						<ipxact:resets>
							<ipxact:reset>
								<ipxact:value>9'h1FF</ipxact:value>
							</ipxact:reset>
						</ipxact:resets>
						<ipxact:bitWidth>9</ipxact:bitWidth>
						<ipxact:volatile>false</ipxact:volatile>
						<ipxact:access>read-write</ipxact:access>
						<ipxact:modifiedWriteValue>modify</ipxact:modifiedWriteValue>
					</ipxact:field>
				</ipxact:register>

My modification of ipyxact/ipxact_yaml.py:

field:
  MEMBERS:
    name: str
    description: str
    bitOffset: IpxactInt
    bitWidth: IpxactInt
    modifiedWriteValue: str
    testable: str
    volatile: IpxactBool
    access: str
    resets:
      MEMBERS:
        reset:
          MEMBERS:
            value: str

some error after modification:

Traceback (most recent call last):
    component.load(args.xml_path[0].name)
  File "ipyxact/ipyxact/ipyxact.py", line 96, in load
    self.parse_tree(root, self.nsmap[self.nsversion])
  File "ipyxact/ipyxact/ipyxact.py", line 128, in parse_tree
    t.parse_tree(f, ns)
  File "ipyxact/ipyxact/ipyxact.py", line 119, in parse_tree
    t.parse_tree(f, ns)
  File "ipyxact/ipyxact/ipyxact.py", line 119, in parse_tree
    t.parse_tree(f, ns)
  File "ipyxact/ipyxact/ipyxact.py", line 119, in parse_tree
    t.parse_tree(f, ns)
  File "ipyxact/ipyxact/ipyxact.py", line 119, in parse_tree
    t.parse_tree(f, ns)
  File "ipyxact/ipyxact/ipyxact.py", line 107, in parse_tree
    setattr(self, _name, eval(_type)(tmp.text))
TypeError: eval() arg 1 must be a string or code object

Avoid yamlload warning

I get the following message:

/home/philipp/.local/lib/python3.7/site-packages/ipyxact/ipyxact.py:178: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  _generate_classes(yaml.load(ipxact_yaml.description))

Question about project roadmap

@olofk, I am very interested in this project for my job as we would like to perform IP-XACT parsing and then make appropriate transformations.
I just wanted to know whether this is actively maintained and we can contribute with pull requests

please add example of how to run program to README

Hi,
I'm interested in parsing IP-XACT, and your tool looks great.
But could you post an example of how to run it?
I get errors when I do:
python gen_c_header.py generic_example.xml

It looks like that's how you want things done.

Use XML schemas to generate API

Instead of hand-writing the yaml file, wouldn't it be better to generate the class API from the XML schemas?

I've started looking into generateDS and pyxb as options for doing this.