pyscience-projects / pyevtk Goto Github PK

Hello all, and thank you for creating such a helpful package - the only one really that doesn't need to pull in a whole libvtk. I would like to have a downstream Python library depend on pyevtk; however, the latest PyPI version is behind the latest GitHub release and conda version which are, in turn, 22 commits behind the master.

The last commits also add quite a few improvements that would be useful for downstream library developers (who publish their package to PyPI too). Would it be possible to update the PyPI version to 1.1.2 and perhaps publish the latest commits in a new release?

I wanted to bring to your attention that @paulo-herrera has moved the official site for PyEVTK to GitHub: https://github.com/paulo-herrera/PyEVTK

Given this move, it might be confusing to users if there is an official GitHub site and also this mirror site with modifications, particularly if this site is used for pypi and conda-forge feedstocks. I suggest you consider making pull requests with whatever changes you would like to make to the official site instead and maybe consider archiving this repo. Perhaps @paulo-herrera would be willing to add you as collaborators if there are significant contributions to be made, but that's obviously up to him.

I discuss this further in my proposed update to the conda-forge feedstock here:
conda-forge/pyevtk-feedstock#1

If cell data has a single slice in one direction, imageToVTK assumed it always to be 3D. But from given spacing of 0.0 one could determine that the cell elements should be 2D (or even 1D).

None of the provided routines can handle field data.

Meshio for example is capable of that, but there only structured meshes are supported.

It would be nice to also export field_data passed as a dictionary in the high-level routines.

Would it be possible with this low-level function to just create a field data section:

Cheers, Sebastian

On pypi the latest release is from june 5, 2024 at release 1.6.0, but on thi repository, release stops at 1.2.0 with latest commit a year old. Is there a fork of this project?

Due to leaving my position in academia, I will no longer have time to spend on this project going forward.
@somada141 @xylar

If he's interested, I would nominate @MuellerSeb to replace me.

I was able to use the pointsToVTK and VtkGroup functions to generate a Multiblock-Data Collection. However, for larger Collection it becomes difficult to know which Block inside the Collecton corresponds to which Data.

After doing a bit of Internet-search and also trying for myself, I was able to find out how I can "name" the Blocks (i.E. Datasets) inside a Collection, just by adding the "name" attribute (in this case by hand):

<?xml version="1.0"?>
<VTKFile type="Collection" version="0.1" byte_order="LittleEndian">
<Collection>
<DataSet timestep="0" group="" part="0" name="2.50mm" file="vtk/SSB_JetABC__XZ_Plane_x_loc=_2.50_mm.vtu"/>
<DataSet timestep="0" group="" part="0" name="" file="vtk/SSB_JetABC__XZ_Plane_x_loc=_15.00_mm.vtu"/>
<DataSet timestep="0" group="" part="0" file="vtk/SSB_JetABC__XZ_Plane_x_loc=_30.00_mm.vtu"/>
</Collection>
</VTKFile>

Resulting in the following Data Hierarchy:

As you can see, that for the first Dataset, the MultiBlock is named "2.50mm", as specified, and the other two are still name "Multi-block Dataset", regardless whether the value for name was blank or not given.

I already tried to insert it into the code, maybe you can then test it and hopefully merge it into a new tag. The changes would be the following:

    def addFile(self, filepath, sim_time, group="", part="0", name=""):

        self.xml.addAttributes(
            timestep=sim_time, group=group, part=part, file=filename, name=name
        )

Would you mind either uploading a source archive for 1.1.1 to pypi or making a github release from the tag, @somada141 ? For conda I'll need an archive online at a static url with a checksum.

I've added infra for automatic deploys to {test,}pypi warehouses, but this fails with non-descriptive error https://github.com/pyscience-projects/pyevtk/runs/1749613193?check_suite_focus=true
Supposedly this is fixed by pypi/warehouse#8565
So it could only be the test instance is lagging. Either way I currently have no further time to debug this.

I am able to generate a perfectly good .vtu file using unstructuredGridToVTK. As a natural next step, I am using VtkGroup to combine multiple .vtu into one .pvd (along the lines of this example).While it is able to generate .pvd file without errors, the file can not be read by Paraview (version 5.11.0). Following is the error:

ERROR: In vtkXMLReader.cxx, line 305
vERROR: In vtkXMLReader.cxx, line 305
vtkXMLUnstructuredGridReader (0000015830D9A8E0): Error opening file C:/Users/vd6558/Downloads/Graphics/Raw_0_0.vtu

ERROR: In vtkExecutive.cxx, line 741
vtkCompositeDataPipeline (0000015841504AF0): Algorithm vtkXMLUnstructuredGridReader (0000015830D9A8E0) returned failure for request: vtkInformation (0000015852D91B30)
  Debug: Off
  Modified Time: 521679
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_INFORMATION
  FORWARD_DIRECTION: 0
  ALGORITHM_AFTER_FORWARD: 1

I'm preparing a conda-forge recipe for pyevtk atm, since I want to use it in a conda package for pymor. Turns out the current tarball does not include the README.md, so setup.py from an sdist fails. I will fix this, add a check in CI and would then create a 1.1.1 tag.

Sounds good, @somada141 ?

Could you upload that to PyPi too, @somada141 ?

When I get around to adding more tests than just importing submodules, we could switch to deploying tagged builds to pypi from travis builds. It's pretty straightforward.

I believe much of pyevtk's functionality is captured in meshio (a project of mine). Perhaps worth checking out.

pyevtk stores the data associated with the given mesh in append data, that is given in a raw binary format.
Setting of raw:

This is producing invalid XML files and other software is may unwilling to read the files (e.g. meshio see: nschloe/meshio#786)

We should use base64 encoded data, where the binary data is encoded with ascii-characters as suggested by VTK. Some references:

• https://vtk.org/Wiki/VTK_XML_Formats
• http://www.earthmodels.org/software/vtk-and-paraview/vtk-file-formats
• https://mathema.tician.de/what-they-dont-tell-you-about-vtk-xml-binary-formats/

When using appended data with base64, we have to keep in mind, that the offsets are changing, since they address the character in the base64 encoded string and not the binary offset like with raw data:

meshio already implemented writing routines using base64 encoding. We should have a look there:
https://github.com/nschloe/meshio/blob/d9c05ae688858b5166630874f3ec875a43b8fd37/meshio/vtu/_vtu.py#L482

I noticed two bugs with the dataset attributes:

• The dataset attributes are written lowercase, which seems wrong. The output results in <CellData scalars="test">, while <CellData Scalars="test"> would be correct. For definition of the VTK names see here: https://gitlab.kitware.com/vtk/vtk/-/blob/v9.0.1/Common/DataModel/vtkDataSetAttributes.cxx#L42
  I do not know how VTK and ParaView (should) handle this internally. I found a few code points where it looks like they at least sometimes do a case insensitive check, but ParaView only works fully as expected when using the uppercase first letter Scalars. Also files exported by ParaView use the uppercase first letter.
  Bug probably is here: https://github.com/pyscience-projects/pyevtk/blob/v1.2.0/pyevtk/vtk.py#L389-L398
• The second bug is in the high-level interface: https://github.com/pyscience-projects/pyevtk/blob/v1.2.0/pyevtk/hl.py#L48-L57
  There, always the first key is set as scalar array, no matter if the data behind this key is scalar or vector data. Also the Vectors attribute is never set at all, even if there is vector data.

I written a structured grid to a vtk file using both the vtk python interface and pyevtk. The structuredgrid, when rendered with an contour in paraview appears dark using the file written with pyevtk. The attached image shows the vtk lib output rendering on the left and the pyevtk output on the right.
Is there an easy fix to this?

How about adding time series capabilities?

I am using your package for writing out simulation data and it works pretty well, but what I had to implement myself was writing files that would enclose a set of simulation results in a time series.
This is supported in Paraview relatively recently (5.6 or so).
I am not sure if there are other ways marking time in a file.

Let's say I have a simulation of 3 seconds with 31 "frames", i.e. one frame every 100 ms.
I write them out into a folder data/ (e.g. frame_000.vtk and so on) and place a file named data.vtr.series in the same folder as the data/ folder.

The code looks something like this:

import os, json
def save_vtr_series(out_dir, fname_prefix, series_dict):
    fname = os.path.join(out_dir, fname_prefix + ".vtr.series")
    with open(fname, "w") as fh:
        json.dump(series_dict, fh, indent=4)

This way, I end up with a file named data.vtr.series which is essentially JSON that looks like this:

{
    "file-series-version": "1.0",
    "files": [
        {
            "name": "data/frame_000.vtr",
            "time": 0.0
        },
        {
            "name": "data/frame_001.vtr",
            "time": 9.999999999999999e-04
        },
... and so on ...
    ]
}

While I came across this vt{r,...}.series format somewhere (maybe on StackExchange), I can't find any usable documentation.

So, this seems to work for me and it might be useful for others. It has, however, a couple of drawbacks: The main one, imho, is that the series file can only be written out at the end of a simulation. Otherwise it needs to be updated, which is a relatively slow process.