computationalradiationphysics / isaac Goto Github PK

Checking every memory to be freed at program exit (kt)

It's a GIF, so why doesn't it move??!!

It would be great if the isaac server could optionally specify the device to open the listening ports on.

Currently, a --web_port 1234 it is listening on all devices, I would make the long syntax of that --web_port *:1234. (If nothing is specified, implicitly *: or localhost: should be assumed).

(just to document our offline results)
Building against libwebsockets (LWS) 1.7.4 leads to a missing libwebsockets-1.7.4/lib/cmake/libwebsockets/LibwebsocketsTargets.cmake file which is not installed by LWS' make install as in previous releases.

This might be an upstream bug or us using the wrong file (should we use LibwebsocketsConfig.cmake instead?)

At the moment ISAAC is "only" suitable to render gas and iso raycastings of 3D volumes described as a couple of disjunct cubiods distributed over the computation nodes. However there are way more ways to render 3D fields, more ways of 3D volume adresses like tetrahedron meshes or even 2D simulation data, not suitable to be converted to a 3D volume with it's own ways of rendering like a 40,000 x 40,000 scalar field per gpu. This needs a whole knew approach of live visualization.

So imo ISAAC should put

• the dimension of the data,
• the access pattern of the data and
• the render algorithms (depending on the dimension)

to own functors, integrated as template parameters as it is now happening with the render controller and compositor especially for stereographic rendering.
For the last one an mpl::list probably makes sense and to choose at runtime which of the preselected rendering algorithms to use - like now with the choice between iso surface and glowing gas rendering. Unfortunately I still doubt, that a rendering selection per source works with the ISAAC approach.
The template parameters can also implement constant atributes for metaprogramming, stating that the volumes don't need to be disjunct at all, which is e.g. not needed for iso surfaces when a z-buffer is used. With this one of the biggest disadvantages of ISAAC would vanish - at least for some renderings - without the need to think about sending data over network until the data is disjunct.

In the very end ISAAC should be some kind of compile time construction kit to get the most fitting visualization for every simulation with an easy way to extened the template implementations for new needs of simulations.

Feature for the future

A paper from 2013 that we all love is: http://arxiv.org/pdf/1301.4546v1.pdf

While this can be realized in PIConGPU with multi-plugins (registering ISAAC multiple times) it might also be interesting for the sake of bullet-time all-the-things to abstract that already in ISAAC so that multiple instances of cameras, working on individual sources with each their individual functor chains and individual transfer functions, create streams.

@psychocoderHPC just noted that for 3D rendering this would also be beneficial since the two cameras have a fixed position dependency (but everything else stays the same, so some resources might want to be shared).

Atm there is the problem with steographic view, that with very specific camera angles a wrong sorting at least for one eye is possible as the projection matrizes used here are not centered.

The distance algorithm for the pre IceT sorting should consider the projection matrix (and not only the modelview).

At the moment every connected client has it's own gstreamer instance running and compressing video data even if two clients watch the very same stream. Obviously this is stupid and wastes a lot of cpu ressources. So the plugin should be improved to compress only once per kind of stream and send it to different clients afterwards.

Kelvin-Helmholtz not Kevin!

For some reasons the isaac desctructor segfaults on Piz Daint.

Possible (very ugly) work around: Not deleting the visualization object after using it for now.

It may sometime be necessary to kill the server or it kills itself or maybe the connection breaks. It should be checked how the visualization reacts at suchs moments (my guess: endless loop in sending data) and the behaviour should be changed to
a) telling the simulation, so that it may pause / exit if it think, that is the right thing to do and
b) if the simulation didn't decid to exit, isaac should try to reconnect to the given ip and port from time to time (every X seconds).

With this the isaac server could be restartet without the need of the simulation to restart, too.

For slow networks the JPEG size should be adjustable.
Less quality, but higher framerate.

It would be nice to have smaller JPEGs, either with

• a smaller resolution or
• a worse quality

especially for small handheld devices like Smartphones.

Styling the HTML5 client GUI with CSS3 is a great task to make the GUI look more professional and to improve usability.

This can be a good issue for students.

Depends on a clean underlying HTML box layout (no tables for layout, clean div and span elements, etc.).

Useful links:

• W3.CSS (just to get some inspiration what is possible without js)

Please let us finish the implementation of the install process so that a make install really installs the library in a sane way (copies the required headers).

It's fine that the server is build at make, too. It can then be simply copied to CMAKE_INSTALL_PREFIX/bin/ alongside the library.

With gcc 4.9.2 on hypnos (HZDR) I get the following warning during build of PIConGPU:

src/picongpu/include/plugins/IsaacPlugin.hpp:386:1561:   required from here
src/isaac/insitu/isaac.hpp:378:567:
  warning: missing initializer for member ‘isaac::clipping_struct::elem’
  [-Wmissing-field-initializers] TScale scale

isaac version: 7e64d00
PIConGPU version: theZiz/picongpu@70c0593

(Not sure if this is actually in the PIConGPU repo/branch I linked.)

A js-based "take the tour"/"walkthrough"/"GUI Tutorial" (e.g., OpenStreetMap's "start the walk-through" (needs login)) would be excellent to improve the usability.

In such, one explains the basic elements of the GUI in a reasonable order by highlighting elements one by one and adding a overlay box to each of it explaining what it does (and maybe invoking some user interaction to continue).

It is nice to steer transfer functions freely, but not-so-nice to redo it every time starting a new simulation. Same for functor chains. So somehow™ these functions should be saved, either

• on the server as templates for ever simulation or
• on the client, which would be very implementation specific. For the reference client a cookie may be a good idea.
• or both. ;)

Some examples for implementation CI:

CPU example: https://github.com/pngwriter/pngwriter/blob/master/.travis.yml
GPU example (CUDA 5.5.): https://github.com/ComputationalRadiationPhysics/mallocMC/blob/dev/.travis.yml
GPU example (CUDA 7.5 & cache): https://github.com/wjakob/pybind11/pull/72/files

At the moment JPEG images come to the server, are converted from base64 to binary, extracted to a bitmap and sent to all ImageConnectors. However the JPEG URI JSON ImageConnector does compress the same image to an JPEG again and also converts it to base64. So a pass through would be nice to save ressouces.

add_definitions such as -march=native -mtune=native should not be set manually but controlled by CMake host and cross-compile chains.

I disabled those to get rid of compile issues on AWS.

It would be nice to have a functors such as

• clip(min, max) (values <min are mapped to min, values >max are mapped to max)
• select(min, max) (values <min or >max are assigned "no value"/kept transparent)
• threshold([<|<=|>=|>], value)

The can be useful when visualizing small detail areas with color bars just focusing on a small fluctuation in a much larger field.

At the moment the interpolation has to be choosen global for the whole visualization, but it would be much cooler to be able to choose this per source instead.

Maybe some high-level tools & script for crumbling over our firewalls can help users in complex scenarios such as Client -> ISP aka Starbucks -> LoginNode -> HeadNode -> Cluster where each of them defines yet-an-other firewall layer.

• wikipedia
• hacky ssh tunnel scripts
• natty

At the moment only RTP ImageConnectors are available, which have some disadvantages. First of all RTP works this way:

• You give an IP address and a port
• RTP send video frames via UDP to this address and port
• That's it.
  There is no way of sending feedback to the server with RTP, especially not, that I don't want to view this stream anymore. Isaac knows, when a client still observes a simulation, so it can stop the stream. However the client cannot view the stream on another device, because RTP uses the IP which connected to the isaac server and told, that it wants to observe the simulation.

In fact isaac should not do any of this anyway. For this RTSP exist, which also implements a TCP feedback channel. So isaac should use this in the future.

The current install docs for, e.g., libwebsockets do not tell an unexperienced user that either third party dependencies such as OpenSSL & zlib are required to be installed or can simply be disabled because we won't use them (yet) anyway.

Please add the CMake flags such as -LWS_WITH_SSL=OFF in the libwebsockets section to your INSTALL.md so users can smoothly go along via copy-&-paste install. You can also document your options to e.g. disable gstreamer via -ISAAC_GST=OFF directly there, too. Don't let the users search for that, not all have ccmake installed.

Atm it is possible to setup arbitrary projection matrizes. However the client doesn't use this feature. So a button for switching between perspective and orthographic projection would be nice.

Since the server and clients share no code with the isaac library (not even headers) we should split them in separate repos.

This will ease packing, e.g., spack recipes for library & server.

• add "protocol_version" attribute to json so server and client can check & abort on protocol mismatch for further development with breaking in-JSON-protocol changes

@erikzenker @theZiz split from #40

Document what you plan to do.

When connecting to an already running simulation with a second HTML client, the initial zoom level of the bounding box is not set until one sets a zoom level again.

Found with interface_presentation.htm in current dev as of de32442

The license headers mention the "GNU General Lesser Public License" in the last paragraph which has the 2nd and 3rd word swapped.

The eye catcher image on the top of the README.md is broken :)

It would be very nice to be able to have some source seen a glowing gas and others as iso surface at the same time.

Atm if a functor chain length > 3 is chosen, the cuda init needs ages (~10 minutes with 4, 2h with 5) for the very first run. However not every of the 4*5^n device functions is used, but which one is decided at runtime and not changed very often. So maybe some laziness options in cuda may help?

We should fix some typos (e.g.,boost version) in the Requirements and also add

• links to projects
• package names (Debian, Arch)
• list maybe some sub-dependencies that break peoples necks during installs

The goal is to have some kind of free defineable (no need to be axis orthogonal!) number of clipping planes for a better view of inner events.

At the moment isaac has 4٠n^m differt functor chain functions for every combination of the n functors with a max length of m and 4 different types of sources (scalar and 2-, 3- and 4-dimension vector field).
However it may be a cool idea to have k٠n^m different functor chain functions instead for every of the k sources.

This should be at least evaluated.

At the moment JSON objects e.g. contain all weights for all sources in an array with the size of the number of sources.

However it makes more sense to have each source as JSON object with it's own attributes. So in fact the whole JSON interaction system should be more OOP like.

So isaac needs to change from

{
    "weights": [1,2,3],
    "sources": ["Source 1", "Source 2", "Source 3"]
}

to

{ "sources":
    "Source 1":
    {
        "weight": 1
    },
    "Source 2":
    {
        "weight": 2
    },
    "Source 2":
    {
        "weight": 2
    },
}

(or similar)

Is it possible to have the data source itself, currently at the beginning of the functor chain, being included inside the functor chain?

E.g., idem(DensityElectrons) | add(DensityProtons)

It would be awesome and quite handy to be able to connect via telnet or similar to the isaac server and to send and receive data with a text based interface. However nobody wants to enter JSON themself, so the most common messages should be easier to send like this

> list⏎
Simulation A, 12 Nodes, 512x512x512
Simulation B, 42 Nodes, 1024x1024x128
> observe "Simulation A"⏎
{
    "metadata": {
        "currentStep": 2342
    }
}
{
    "metadata": {
        "currentStep": 2343
    }
}
> metadata "Simulation A" "pause" true⏎

Furthermore recevied JSON packages should be still JSON, but formated e.g with indentation.

We should write an additional client class that can be used together with widgets in Jupyther (IPython) notebooks.

Allows integration with things like offline-data from openPMD-viewer.

Suggested repo name:

• isaac-python

It may be a good idea to have options like weight and whether a source is choosen or not bundled to one functor chain text field like this

source1(3.5) | length | mul(3.5) | volume
source2(1.5) | length | mul(3.5) | interpolate | iso

With this only source 1 and source 2 would be drawn with the weight 3.5 and 1.5. Every other available source would be deactivated. Furthermore only source 2 with use interpolation. Also both sources would use different render mechanisms. See also #13 and #12.

While idling isaac needs atm ~10% cpu time. This should be improved or at least investigated, why it is this way.

Currently isaac is hard coded for 3 dimensional alpaka devices like it is done here. For visualization of 2 dimensional domains it would be nice to use 2 dimensional devices.

If isaac is used with threading enabled the marked lines can result in a racecondition.
https://github.com/ComputationalRadiationPhysics/isaac/blob/dev/lib/isaac.hpp#L1674-L1675

If the master thread is reading or manipulation the json object while the thread is deleting the old and creates a new one.

In libwebsocket (LWS) 1.7.4 the following function calls create a deprecated warning with isaac 7e64d00:

isaac/server/WebSocketDataConnector.cpp:202:21: warning: ‘const lws_extension* lws_get_internal_extensions()’ is deprecated (declared at ~/lib/libwebsockets-1.7.4/lib/cmake/libwebsockets/../../../include/libwebsockets.h:1890) [-Wdeprecated-declarations]
   info.extensions = lws_get_internal_extensions();
                     ^
isaac/server/WebSocketDataConnector.cpp:202:49: warning: ‘const lws_extension* lws_get_internal_extensions()’ is deprecated (declared at ~/lib/libwebsockets-1.7.4/lib/cmake/libwebsockets/../../../include/libwebsockets.h:1890) [-Wdeprecated-declarations]
   info.extensions = lws_get_internal_extensions();
                                                 ^

When communication via JSON packages, it would be very helpful to plan the access via RESTful interfaces.

Actually not only client <-> server but also the connection server <-> simulation should be done in a RESTful way. The simulation itself is only a resource, too and it might be necessary to not only get a "next"/"current" step but an older (or later) one. If requested, the simulation might choose to stop the current computation, load a checkpoint and iterate to the requested step (resource).

This would be very in-line with asynchronous and parallel work-flows instead of "live" work flows on a single simulation (which don't scale).

The already buillt in min max function is nice, but histograms would be much more helpful to find good transfer functions for a given and maybe unknown (!) simulation.

Documenting what would be necessary for a WebRTC binding. (Split from #40)

Might be too complex for now and we can almost always guarantee a TCP forwarding as in the current workflow.

When building third party applications (in our case this PIConGPU branch) while pointing the isaac "install" to the source directory directly (instead of to the installed header location), the CMakeLists.txt seems to be chosen in find_package(ISAAC) over the ISAACConfig.cmake.

This causes the cmake command to fail as long as the CMakeLists.txt file is present.