esmf-org / esmf-profiler

ESMF Profiler application converts binary traces into a web based, dynamic report.

esmf-profiler

Kanban Board

Status of issues

Discussion Board

Features, bugs, and enhancements.

Description

The ESMF-Profiler project is designed to take trace binary output and display it in a web based GUI.

Getting a Binary Trace

To collect a binary trace, set the environment variables to the values below:

export ESMF_RUNTIME_PROFILE=ON
export ESMF_RUNTIME_PROFILE_OUTPUT="BINARY SUMMARY"

Quickstart

Local Install:

⚠️ At the time of this release IntelPython is not compatible.

1. Clone the latest stable branch from the esmf-profiler repository.

2. cd into the appliction path

cd esmf-profiler

3. Ensure that the install_dependencies.sh and install.sh have executable permissions

chmod +x ./install_dependencies.sh 
chmod +x ./install.sh

4. Execute both scripts as shown below. A venv folder will be created on success.

./install_dependencies.sh && ./install.sh

5. Activate the virtual environment.

source ./venv/bin/activate

6. For pre-release, install the esmf-profiler using pip editable install. Running tests require access to PyPi Public Repositories. This step is optional, but encouraged. Some HPC platforms do not have open internet access, so will not be able to install the tests.

pip install -e . or pip install -e .[test] // to run tests

7. Confirm the installation was successful. If so, you should now be able to type esmf-profiler into your terminal and see the help output.

python -m pytest // optional, to run tests
esmf-profiler

ℹ️ If using C-Shell, deactivate then reactivate the virtual environment to run the applicication.

Install with Docker

A Dockerfile is included in the repository to allow users using Docker to install the application with ease.

1. Clone the latest stable branch from the esmf-profiler repository.

2. cd into the appliction path

cd esmf-profiler

3. Build the image

docker build -t esmf-profiler-image .

4. Run the application in the image. For example, assuming your binary traces are in ./traces

docker run -it -v $(pwd)/traces:/home/traces esmf-profiler-image esmf-profiler -t /home/traces -n 'profilename' -o /home/traces/output

This will spin up an esmf-profiler-image and mount ./traces from client to host.

Here, output is being directed to /home/traces/output.

⚠️ Be sure the output path you pass to the esmf-profiler command has your client folder as it's root (./traces in the example). Otherwise, the output will not persist to your client machine after the application has run.

Generating and Viewing Profiles

You can see the help menu displayed below after installation by running ```esmf-profiler``` with no arguments.

usage: esmf-profiler [-h] -t TRACEDIR -n NAME -o OUTDIR [-p PUSH] [-v] [-s]

ESMF Profiler

optional arguments:
  -h, --help            show this help message and exit
  -t TRACEDIR, --tracedir TRACEDIR
                        directory containing the ESMF trace
  -n NAME, --name NAME  name to use for the generated profile
  -o OUTDIR, --outdir OUTDIR
                        path to output directory
  -p PUSH, --push PUSH  git url of remote repository where to push profile
  -v, --verbose         enable verbose output
  -s, --serve           start a local server to host the profile results

Examples

esmf-profiler -t ./fresh_traces -n "build_abc123" -o "output" -s

This is the most common way to use the profiler. It requires some simple user permissions on the server you're running the application on.

You're telling the application to:

• -t ./freshtraces The traces to run on are contained in fresh_traces, a subdirectory of the current working directory
• -n "build_abc123 Name the output (the profile) "build_abc123". This will be used for titles, directories, links, etc. in the GUI and file structure.
• -o "output" Send the output to output a directory that will be created (if no already exists) in the current working directory. All output will be sent there.
• -s On success, start an ultra-light-weight python native web server at 0.0.0.0:8000

esmf-profiler -t ./fresh_traces -n "build_abc123" -o "output" -p '[email protected]:user_name/repo_name.git'

This example will run the processing work locally as the first example, but then publish the results to the repository at [email protected]:user_name/repo_name.git.

To view the profile on GitHub.io, you need to explicitly turn on "GitHub Pages" within the target repository. Refer to the Quickstart for Github Pages for details on how to turn on "Pages" within your repository.

docker build -t esmf-profiler-image .

docker run -it -v $(pwd)/traces:/home/traces esmf-profiler-image esmf-profiler -t /home/traces -n 'my_profile' -o /home/traces/output

This example utitilizes the [Dockerfile] for those leveraging Docker.

The image is built first, then <current_working_directory>/traces is mounted to /home/traces on the virtual machine.

Commands are the same for local and docker installations.

Here, we're telling the application to:

• -t ./freshtraces The traces to run on are contained in /home/traces (locally, $pwd and traces)
• -n "build_abc123 Name the output (the profile) "my_profile".
• -o "/home/traces/output The target for the output of the applications. **This must be a directory mounted via the docker run command as discussed above. Otherwise, the results will not persists after the docker run command has completed.

Dependencies

ℹ️ All dependencies are installed using any of the installation methods in Quick Start section. This section is primarly for reference and historical purposes.

Install Babeltrace2

Use a Package Manager

There are some package managers that install Babeltrace2.
https://babeltrace.org/#bt2-get However, you need to make sure the Python bindings are included.

Build Babeltrace2 from source

Prereqs:

• glibc-2.0 (Ubuntu: sudo apt-get install libglib2.0)
• swig (Ubuntu: sudo apt-get install swig)

Download: https://www.efficios.com/files/babeltrace/babeltrace2-2.0.4.tar.bz2

Note: Babeltrace2 appears to require the GNU compiler >9.0 - there are failures with Intel.

Linux gcc 9.3

cd /path/to/babeltrace2-2.0.4
./configure --prefix=/path/to/babeltrace2/INSTALL/2.0.4 --enable-python-bindings --enable-python-plugins --disable-debug-info --enable-compile-warnings=no
make -j4
make install

# set up PYTHONPATH so that the bt2 module is available
export PYTHONPATH=/path/to/babeltrace2/INSTALL/2.0.4/lib/python3.5/site-packages

# if installed in non-standard place, you need to set LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=/path/to/babeltrace2/INSTALL/2.0.4/lib

Cheyenne

module load gnu/9.1.0 python/3.7.9

SWIG is needed for the build process and is not in the default Cheyenne PATH. Download: http://prdownloads.sourceforge.net/swig/swig-4.0.2.tar.gz

./configure --prefix=/glade/u/home/dunlap/bt/INSTALL/swig-4.0.2
make
make install

export PATH=/glade/u/home/dunlap/bt/INSTALL/swig-4.0.2/bin:$PATH

Install babeltrace2

cd /path/to/babeltrace2-2.0.4
./configure --prefix=/glade/u/home/dunlap/bt/INSTALL/babeltrace2-2.0.4 --enable-python-bindings --enable-python-plugins --disable-debug-info
make -j6
make install

# set LD_LIBRARY_PATH so Python3 can find the dynamic library
export LD_LIBRARY_PATH=/glade/u/home/dunlap/bt/INSTALL/babeltrace2-2.0.4/lib:$LD_LIBRARY_PATH

Resources

https://docs.pytest.org/en/6.2.x/ https://code.visualstudio.com/docs/python/testing http://www.swig.org/Doc4.0/SWIGDocumentation.html

esmf-profiler's Issues

The performance has dropped a lot when changing charts.

• The last commit seems to bring in a couple of issues. The performance has dropped a lot when changing charts.
  

Implement Accordion style Textual Data Representation

We can use bootstrap in the meantime, but this will need to be dynamic enough to feed the JSON data transport object in the function/instantiation.

https://getbootstrap.com/docs/5.1/components/accordion/

Drill down and then back out, the TOP level chart is no longer correct.

I am also seeing that when I drill down and then back out, the TOP level chart is no longer correct. Maybe the updateLevel() is no longer working correctly? I start here at the TOP:

The drill down a level:

Then I click the "/TOP" in the breadcrumb:

Obviously, the two TOPs should be the same - so something is going wrong when updating the level.

The "Load Balance" menu item is linked to "https://earthsystemmodeling.org/."

It should probably just be "#" so that the user is not redirected.

More robust way to hand root path

We need a more robust way to get a handle on the esmf-profiler root path. Either that or we need to bundle the static site files into the Python install

CLI no longer outputting static web files

esmf-profiler -t <> -n <> -o <> should create json files as well as the web asset files into it's output.

It did, now it doesn't.

It should.

Application dependency exists outside of INSTALL

After a clean install and a verified test run, if you delete the directories the compressed files were uncompressed into, the application fails to start.

gcc requirement is 9.0.0, should be >9

@rsdunlapiv

I think the code is looking for an exact match for gnu 9.0.0, but anything above 9 is fine.

https://github.com/esmf-org/esmf-profiler/blob/development/install_dependencies.sh

Initial load balance plot

Initial design of load balance plot. Each level of the a timing tree is represented as a stacked bar chart.

First pass styles

General:

1. The browser tab says "React App" - please change the title to "{name} - ESMF Profiler" where {name} is the user-supplied name of the trace, or timestamp if not available
2. Please remove "Copyright O Your Website 2021 " from the bottom.

On the Bootstrap card:

3. please make the title bar smaller and the "Load Balance" font smaller
4. if possible, I'd like a flatter look, such as removing the shadow altogether
5. Chart title = "PET Timings"
6. X.Axis title = "PET Number"
7. YAxis title = "Time (s)"
8. Add the highcharts menu in the top right of the chart to download images and/or the data
9. if possible remove 'highcharts.com" from the bottom right of the chart

On the sidebar:

10. Change font so that it's not all caps: "ESMF PROFILER" should be "ESMF Profiler"
11. Initial version can be 0.1

Fix issue with Select All/None.

When using select all and select none buttons, anything user selected is not included in the selection/de-selection.

It should be.

Docker run fails

Describe the bug

After manually fixing the docker build issue (removing the -b argument to git clone), the docker run command fails:

(venv) rocky@rocky-Latitude-E5570:~/esmfdev/esmf-profiler.v1/tests/fixtures/test-traces$ docker run -it -v $(pwd):/home/traces esmf-profiler-image esmf-profiler -t /home/traces/atm-ocn -n 'atm-ocn-test' -o /home/traces/output
Traceback (most recent call last):
  File "/usr/local/bin/esmf-profiler", line 33, in <module>
    sys.exit(load_entry_point('profiler', 'console_scripts', 'esmf-profiler')())
  File "/usr/local/bin/esmf-profiler", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/esmf-profiler/src/profiler/main.py", line 16, in <module>
    from profiler.analyses import LoadBalance
  File "/home/esmf-profiler/src/profiler/analyses.py", line 1, in <module>
    from profiler.event import TraceEvent, DefineRegion, RegionProfile
  File "/home/esmf-profiler/src/profiler/event.py", line 3, in <module>
    import bt2
ModuleNotFoundError: No module named 'bt2'

To Reproduce
Run the "docker run" command in the quick start.

Expected behavior
The "docker run" should succeed.

The issue seems to be that in the Dockerfile this path is hardcoded:

# TODO 
# Set envs for Python and LD_Library
ENV PYTHONPATH="/home/esmf-profiler/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.9/site-packages:$PYTHONPATH"

However, in the container itself, the real path is under python 3.8:

root@8a171f26a81f:/home/esmf-profiler# ls /home/esmf-profiler/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.9/site-packages
ls: cannot access '/home/esmf-profiler/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.9/site-packages': No such file or directory
root@8a171f26a81f:/home/esmf-profiler# ls /home/esmf-profiler/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.8/site-packages    
bt2  bt2-2.0.4-py3.8.egg-info

If the version of python3 is not exactly specified in the docker build, then it could evolve as the Ubuntu base image evolves. Therefore we should probably use the find command to locate the installation directory. Either that or force the container to always install Python3.X so we can use it reliably in the path.

Add name and timestamp to top of the page as a header.

The values are in data/site.json.

First pass at command line arguments

Here is a reasonable first target:

Process a trace and write out static web site to local file system

$ ./esmf-profiler /path/to/traceout --name mytrace --outdir /path/to/output/root

The name should be used in the title of the static site and at the top of the UI.

Process a trace and push to repo on GitHub:

$ ./esmf-profiler /path/to/traceout --name another_trace --push [email protected]:esmf-org/app-profiles.git

In this case we assume the user already has access to the repo at [email protected]:esmf-org/app-profiles.git and has write permissions. If not a friendly error message can be provided. The repo would be cloned in a temporary location, the static site added, and them commit/push.

In both cases (local file system and push to remote repo) we need a structured approach to the output directory structure. Here is a good first target.

Assume the user provides --name bananas01 --outdir /path/to/root. Use a combination of the user name (use whoami) and the trace name so that the static site would be put here:

/path/to/root/<username>/bananas01/

In the case of the remote repo, the same approach can be used. Assume the user provides --name bananas01 --push [email protected]:esmf-org/app-profiles.git Also assume the temporary file space is ./.tmp:

./.tmp/app-profiles/<username>/bananas01/

Re-running the profiler again with the same name would overwrite the existing directory if it is present.

Drill down load balance chart

This seems like a promising option for enabling drill down into the load balance chart.

https://jsfiddle.net/BlackLabel/px1eq8uv/

Review code in main.py

d040d93

See comment lines 141... discuss strategy

Optimize Dockerfile

In Dockerfile

• Verify all dependencies are required
  apt-get install
RUN apt-get install -y -q git build-essential libssl-dev libffi-dev python3 python3-pip python3-dev bison python3.8-venv flex libglib2.0-dev

Create Docker container and implement in GitHub Action

We envision three different ways that people can run the esmf-profiler, from most work to least:

1. Local installation
2. Docker container
3. "No Install" - Traces are processed automatically on a GitHub.io repo

For option 1, the user would install and run the esmf-profiler locally or on an HPC machine. The install_dependencies.sh script and install.sh scripts can be used to build the dependencies and install a Python virtual environment containing the esmf-profiler Python package. Then the user runs the profiler via the CLI, e.g.,

esmf-profiler -t /path/to/trace -n mytrace -o out --push [email protected]:esmf-org/app-profiles.git

For option 2, we would provide a Docker container that the user can pull from DockerHub that contains the profiler and all dependent software installed. The user would issues a docker run command with arguments to execute profiler.

For option 3, the same container create in option 2 would be installed as a GitHub Action on a GitHub.io site, such as app-profiles. To generate a profile the user would:

1. clone the repo
2. create a new empty branch and add a `trace.tar.gz` file containing the application trace
3. commit and push the branch
4. the GitHub action would extract the trace, run the Docker container containing the esmf-profiler, and push the static site to the local repo
5. the pushed branch would be deleted (to reduce repo size)

One question is where do we get the command line parameters above, such as "-n mytrace"? An appealing option here is to use the commit message as a way for the user to provide command line options to the profiler, e.g.:

  git commit -a -m "name=mytrace, another_option=this, third_option=that"

The commit message can be parsed by the GitHub action and the parameter extracted and passed into the esmf-profiler.

To facilitate individual users and organizations in creating their own GitHub.io site to host their own profiles, we should create a GitHub template repository with the Action already set up.

Look into boost module for performance improvement

https://www.highcharts.com/docs/advanced-chart-features/boost-module

Do we need the bt2 Python Library?

Do we need to use the bt2 python module? If all the data you need is spit out the same as just using babeltrace2, I think we could abstract out the python module and just call babeltrace2 from python and collect the results in memory / tmp space etc.

But maybe it's offering something I don't understand.

Add build argument to Dockerfile to select branch of esmf-profiler to clone

In the Dockerfile, change the git clone command to clone from master after initial release.

External Dependency Install Issue

After installing, BT2 and Swig are installed into a sub directory /INSTALL.

However, if you delete the directories containing the uncompressed install files, the app fails to run.

Create Stacked Bar Chart

Create a stacked bar chart for visualizing region times.

• HighCharts (https://www.highcharts.com/demo/column-stacked-and-grouped)
• Must be able to drill down into nested layers

Add username to site.json

In order for links to work when the site is published to github pages, the web app will need the username passed to it in the JSON.

When the user click an option, that <username>/<name> will have to be prefixed to each call. Otherwise, they're take back to root.

Region timing per iteration

Another useful analysis is to see for a single PET the cost of each iteration of a region. For example, assume the atmosphere model executed the phase RunPhase1 200 times. Do each of these iterations cost the same? Is there a pattern to any variation?

For example, in the plot below, there is a regular increase in execution time about every 44 iterations. This indicates that the model is doing some extra work on those timesteps that the user would need to verify is expected.

Note that this is just a plot for a single PET. There could be value in also aggregating over multiple PETs, particularly to show outliers.

Optimize swig tar/install

There looks to be a lot of cruft in the swig tar like examples, etc.

If we don't need them, delete them,

Delete any other cruft you find or create tickets to do so.

Issue installing App with PIP

Installing with pip requires the -e flag for local install in order to find the profiler module.

-e flags are only supposed to be for development.

Fix the issue in the Dockerfile and the rest of the application.

Create JSON data structure for transport layer

Need a standard way of passing results from application to application.

JSON to start, CSV if needed down the road.

The JSON will represent the execution tree, nesting children in the 'children' array of each node.

Load balance link goes to root instead of staying in Load balance chart.

• When I click on "Load Balance" on the menu on the left, the URL changes to: https://ryanlong1004.github.io/ instead of staying on the current profile.

Will need to add user/report name to link.

PET sequence pattern for x-axis

Is your feature request related to a problem? Please describe.
Cases with components that use interleaved petLists, and/or threading.

Describe the solution you'd like
For interleaved petLists, and threaded components, which idle certain PETs, the resulting active PETs are in a specific repeating patter (every other PET, or every 4th PET, etc.). Being able to apply such a pattern to filter along the x-axis of the profile plot would be convenient to display only active PETs of a specific component.

Fails to view static site locally

Describe the bug

After generating a profile, I try to view the static site locally by navigating to the index.html file on my local file system and opening it directly in Chrome.

I see the following error in the Chrome console:

Report.jsx:22 Fetch API cannot load file:///home/rocky/esmfdev/esmf-profiler.v1/tests/fixtures/test-traces/output/data/load_balance.json. URL scheme "file" is not supported.
l @ Report.jsx:22
Report.jsx:39 Fetch API cannot load file:///home/rocky/esmfdev/esmf-profiler.v1/tests/fixtures/test-traces/output/site.json. URL scheme "file" is not supported.
(anonymous) @ Report.jsx:39

To Reproduce
Generate a profile and try to open the static site from the local file system from Chrome.

Expected behavior
The static site should be viewable locally. Not all users will want to push to GitHub.io or a web server.

This issue seems to discuss the problem and offers a solution, albeit one that looks complex:
https://stackoverflow.com/questions/49971575/chrome-fetch-api-cannot-load-file-how-to-workaround

Center the buttons below the chart legend.

Center the select all and select none buttons in the chart menu.

Docker build fails

Describe the bug

The docker build command fails:

Step 7/12 : RUN git clone -b https://github.com/esmf-org/esmf-profiler.git
 ---> Running in ec127b66bf8d
fatal: You must specify a repository to clone.

usage: git clone [<options>] [--] <repo> [<dir>]

To Reproduce
Run the docker build from the quick start:

docker build -t esmf-profiler-image .

Expected behavior
The docker build should succeed.

It appears that the issue is the "-b" argument to git clone is expecting a branch name, but none is provided. From the Dockerfile:

RUN git clone -b https://github.com/esmf-org/esmf-profiler.git

This should either clone the default branch, or be hard-coded to main branch, or potentially be a build argument to docker. (To allow the user to select which branch to build from.)

Remove all console.log statements from GUI

"CLICK" has made it's way in. Remove it, and any others.

Issue Creating Virtual Environment in some OS

Using WSL:

Not sure if this is specific to WSL.

Perform cleanup after install scripts

Delete and directories or files left over from installations.

Make push_to_repo more efficient

main.py -> push_to_repo(...

This deletes/reclones every time which can be inefficient if the repo is large instead we want to check whether it exists already and see if we can git pull

Fix per quote.

Example of memory profiling

Here is an example of an current memory profiling activity and associated plots that the esmf-profiler should support in a future release.
ufs-community/ufs-weather-model#779

"name" CLI argument is alpha-numeric only

We should allow any standard names that would be allowed as file names.

Create "Success" messages for manual installs

Add a "success" message and ensure returning the correct error codes during manual installs for logs or monitoring users.

Close Local Server More Gracefully

Intercept the Ctrl-C and perform cleanup and checks if necessary.

Install.sh reports it cannot find the python bindings leading to later failures

Describe the bug

After running ./install_dependencies.sh && ./install.sh on my system, I see an error message that it failed to find the Python Babeltrace2 module:

/dependencies/babeltrace2-2.0.4/../INSTALL/babeltrace2-2.0.4/share/doc/babeltrace2'
make[2]: Leaving directory '/home/rocky/esmfdev/esmf-profiler.v1/dependencies/babeltrace2-2.0.4'
make[1]: Leaving directory '/home/rocky/esmfdev/esmf-profiler.v1/dependencies/babeltrace2-2.0.4'
/home/rocky/esmfdev/esmf-profiler.v1
Creating virtual environment
Failed to find Python Babeltrace2 module: /home/rocky/esmfdev/esmf-profiler.v1/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.9/site-packages
/home/rocky/esmfdev/esmf-profiler.v1/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.8/site-packages

Strangely, that path does exist:

ls /home/rocky/esmfdev/esmf-profiler.v1/dependencies/INSTALL/babeltrace2-2.0.4/lib/python3.9/site-packages
bt2  bt2-2.0.4-py3.9.egg-info

So it may be that the check itself is wrong.

This results in existing the install.sh script early, so that the LD_LIBRARY_PATH and PYTHONPATH env. variables are not appended to the venv/bin/activate script. The then leads to the test failure:

(venv) rocky@rocky-Latitude-E5570:~/esmfdev/esmf-profiler.v1$ python -m pytest
========================================================================== test session starts ===========================================================================
platform linux -- Python 3.9.5, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 -- /home/rocky/esmfdev/esmf-profiler.v1/venv/bin/python
cachedir: .pytest_cache
rootdir: /home/rocky/esmfdev/esmf-profiler.v1, configfile: pytest.ini, testpaths: tests
plugins: cov-3.0.0
collected 0 items / 1 error                                                                                                                                              

================================================================================= ERRORS =================================================================================
__________________________________________________________________ ERROR collecting tests/test_trace.py __________________________________________________________________
ImportError while importing test module '/home/rocky/esmfdev/esmf-profiler.v1/tests/test_trace.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_trace.py:8: in <module>
    from profiler.main import create_site_file, run_analsysis
src/profiler/main.py:16: in <module>
    from profiler.analyses import LoadBalance
src/profiler/analyses.py:1: in <module>
    from profiler.event import TraceEvent, DefineRegion, RegionProfile
src/profiler/event.py:3: in <module>
    import bt2
E   ModuleNotFoundError: No module named 'bt2'
======================================================================== short test summary info =========================================================================
ERROR tests/test_trace.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================ 1 error in 0.26s ============================================================================

To Reproduce

Run the install script. Not sure if this is an issue specific to my machine.

Expected behavior

The install.sh script should correctly locate the bt2 python bindings and not exit early.

Desktop (please complete the following information):

• OS: Ubuntu
• Version: 20.04

The push option no longer works

Describe the bug

The -p or --push option no longer works. Instead of pushing to the provided repo URL, the profile is pushed back to the esmf-profiler on the main branch!

(venv) rocky@rocky-Latitude-E5570:~/esmfdev/esmf-profiler.v1$ esmf-profiler -t tests/fixtures/test-traces/atm-ocn-concurrent -n v1testb -o v1testb -p [email protected]:esmf-org/app-profiles.git
profiler.main : Processing trace: tests/fixtures/test-traces/atm-ocn-concurrent
profiler.main : Generating profile
hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (4/4), done.
Unpacking objects: 100% (14/14), 1.27 KiB | 325.00 KiB/s, done.
remote: Total 14 (delta 10), reused 14 (delta 10), pack-reused 0
From github.com:esmf-org/esmf-profiler
 * [new branch]      55_dockerfile_bug -> origin/55_dockerfile_bug
Enumerating objects: 1899, done.
Counting objects: 100% (1899/1899), done.
Delta compression using up to 8 threads
Compressing objects: 100% (1003/1003), done.
Writing objects: 100% (1036/1036), 27.66 MiB | 734.00 KiB/s, done.
Total 1036 (delta 573), reused 13 (delta 1), pack-reused 0
remote: Resolving deltas: 100% (573/573), completed with 168 local objects.
To github.com:esmf-org/esmf-profiler.git
   4fd67fe..c6d9b9a  main -> main

To Reproduce
Run the esmf-profiler with the -p option and give it a valid git URL.

Expected behavior
The URL provided to -p should be used. In the case above:

-p [email protected]:esmf-org/app-profiles.git

The prior logic cloned this repo into a temp directory, copied the profile into it, then committed and pushed. It seems that this logic has been lost.

Implement Pytest

Start testing critical functionality with Pytest.

Option to reverse, or arbitrarily order regions along y-axis

Is your feature request related to a problem? Please describe.
To interpret the profile for a complex run sequence can be difficult. One approach that is starting to make sense to me is to activate specific regions and build up the individual columns bottom up. However, I have noticed that the y-axis ordering of the regions is in reverse order. Better said, it is in "top down" order, but the columns of course are being built up "bottom up", and so it makes it difficult to display it the way I have in mind.

Describe the solution you'd like
Simplest is to have a revers y-axis button.

Describe alternatives you've considered
The more powerful approach would be to be able to re-shuffle the order of regions in the legend below the graph, and have the order then be reflected in how each column is built up.

Update subprocess.run commands in Python

Add the key/value check=True to calls of subprocess.run to ensure command failures raise exceptions.

Install version reported from pip should match tag

Describe the bug

Installing version 0.1 of the esmf-profiler shows this:

Installing collected packages: profiler
  Attempting uninstall: profiler
    Found existing installation: profiler 0.0.1
    Uninstalling profiler-0.0.1:
      Successfully uninstalled profiler-0.0.1
  Running setup.py develop for profiler
Successfully installed profiler

To Reproduce
Run the installation script.

Expected behavior

(1) the name of the package should be "esmf-profiler" (not "profiler")
(2) the version reported should match the repo tag (0.1 instead of 0.0.1) and should always stay synced

Allow user to configure the analyses they want to see

Given an application trace, there are many potential analyses possible, so the user needs to be able to specify what analyses they want to see, i.e., what plots should be generated. Due to the size of the traces, we cannot generate a single JSON representation and expect the browser to process it. Instead, each specific analysis will need a custom JSON stream (targeted for a specific javascript plot). For example, the user could specify that they are interested in looking at load balance and would like a set of load balance plots for the first two levels of the timing tree -- and nothing deeper.

An appealing option for configuration is through command line arguments:

$ ./esmf-profiler --load-balance=depth:2

From this you would expect a series of load balance plots (stacked bar charts). One for the first level. And one for each of the second levels in the tree. The depth parameter would allow the user to control the tradeoff between how much time it takes to generate the profile plots and how much detail they see.

The user could know ahead of time they they are interested only in specific regions in the timing tree, such as the region called "[ATM-TO-OCN] RunPhase1". This calls for a filter option:

$ ./esmf-profiler --load-balance=filter:"[ATM-TO-OCN] RunPhase1"

In this case the profiler would generate a bar chart showing execution time for every PET for the region named "[ATM-TO-OCN] RunPhase1". (i.e., PET number on the x-axis, total execution time on the y-axis).

This could be generalized to a regex style filter:

$ ./esmf-profiler --load-balance=filter:"*ATM-TO-OCN*"

In this case you would expect to see a set of bar charts, one for each timed region with a name containing the substring "ATM-TO-OCN".

The user might also want to restrict the set of PETs shown in the load balance plot:

$ ./esmf-profiler --load-balance=pets:0-155,filter:"[ATM-TO-OCN] RunPhase1"

This would show a load balance bar chart for the region named "[ATM-TO-OCN] RunPhase1" with only PETs 0-155 in the x axis.

Longer term, additional analyses will be possible and could also be controlled using command line arguments. One example is showing, for a single PET, the execution time of each iteration of a give region (see #3). This requires some information from the user including which PET and the region name. For example:

$ ./esmf-profiler --execution-time=pet:124,name:"[ATM] RunPhase1"

This would generate a plot as shown in #3. This analysis requires capturing the individual RegionEnter and RegionExit events.

Another example of this is an analysis that shows memory allocations over time. (NOTE: We do not yet have these events in the trace, but it can be extended to support this.)

# example of how to specify a memory plot over time showing Resident Set Size in the y axis for PET 214. 
$ ./esmf-profiler --memory=type:rss,pet=214

The various options could be combined:

# generate a load balance bar chart for depth 1 (includes all PETs) AND a memory usage chart for PET 0 showing the High Water Mark
$ ./esmf-profiler  --load-balance=depth:1 --memory=type:hwm,pet=0

Using a dashboard-like layout, each of the requested analysis could be linked in a menu on the left. Then the user could click on each analysis and be shown the corresponding plot(s).

Need more colors to label profiled regions

Is your feature request related to a problem? Please describe.
Working with a large coupled UFS configuration it took me a while to figure out that different profiled regions were using the same color in the profile output. This was very confusing at first.

Describe the solution you'd like
Use more colors to distinguish regions better.

Describe alternatives you've considered
Alternatively use different fill patterns?

Additional context
N/A

Create chart data class (JS)

Need to create a class to handle the ingestion and parsing of JSON data in order to render the data to HighCharts.

The class should include filtering, reducing and mapping methods, as well as any convenience methods found along the way.