GithubHelp home page GithubHelp logo

llnl / callflow Goto Github PK

View Code? Open in Web Editor NEW
29.0 9.0 4.0 30.93 MB

Visualization tool for analyzing call trees and graphs

License: MIT License

JavaScript 26.51% HTML 0.07% Python 35.91% Vue 36.68% C 0.73% Shell 0.09%
performance-analysis hierarchical-data comparative-analysis data-analytics visual-analytics visualization graph trees hpc performance

callflow's People

Contributors

bhatele avatar bhatiaharsh avatar jarusified avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

pombredanne jarusified huutnguyen

callflow's Issues

Bug: The 'pillow>=6.2.0' distribution was not found and is required by matplotlib

Running callflow_server command throws the following error:

Traceback (most recent call last):
  File "/usr/local/bin/callflow_server", line 33, in <module>
    sys.exit(load_entry_point('CallFlow==1.1.1', 'console_scripts', 'callflow_server')())
  File "/usr/local/bin/callflow_server", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/server/__init__.py", line 6, in <module>
    from .notebook_server import _load_ipython_extension
  File "/usr/local/lib/python3.8/site-packages/server/notebook_server.py", line 27, in <module>
    from callflow.argparser import ArgParser
  File "/Users/jarus/Work/llnl/CallFlow/callflow/__init__.py", line 10, in <module>
    from .datastructures.graphframe import GraphFrame
  File "/Users/jarus/Work/llnl/CallFlow/callflow/datastructures/graphframe.py", line 9, in <module>
    import networkx as nx
  File "/usr/local/lib/python3.8/site-packages/networkx/__init__.py", line 128, in <module>
    import networkx.drawing
  File "/usr/local/lib/python3.8/site-packages/networkx/drawing/__init__.py", line 6, in <module>
    from . import nx_pydot
  File "/usr/local/lib/python3.8/site-packages/networkx/drawing/nx_pydot.py", line 27, in <module>
    from pkg_resources import parse_version
  File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3239, in <module>
    def _initialize_master_working_set():
  File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3222, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3251, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 567, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 884, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", line 770, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pillow>=6.2.0' distribution was not found and is required by matplotlib

v.1.3.0 Stage - hpctoolkit-allgather-database fails to load some VIS due to timeout

-- SuperGraph
Issue: The endpoint request timesout because of some cycle issue I believe.

-- CCT
Issue: callsite index mapping is not matching certain callsites. Could be a bug.

2021-07-26 11:28:23,139 - server.provider_api:log_exception:1892 - ERROR - Exception on /cct [POST]
Traceback (most recent call last):
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/flask_cors/decorator.py", line 128, in wrapped_function
    resp = make_response(f(*args, **kwargs))
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_api.py", line 146, in single_cct
    nxg = self.request_general({"name": "cct", **data})
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_base.py", line 389, in request_general
    nll = NodeLinkLayout(sg=sg, selected_runs=operation["dataset"])
  File "/Users/jarus/Work/llnl/CallFlow/callflow/layout/node_link.py", line 48, in __init__
    self._add_node_attributes()
  File "/Users/jarus/Work/llnl/CallFlow/callflow/layout/node_link.py", line 78, in _add_node_attributes
    datamap[column][callsite] = self.sg.get_module(callsite_idx)
  File "/Users/jarus/Work/llnl/CallFlow/callflow/datastructures/supergraph.py", line 140, in get_module
    return self.callsite2module[callsite_idx]
KeyError: 107

v.1.3.0 Stage - caliper-simple-json errors

The stage branch fails to load the supergraph view for the single-rank case (e.g., caliper-simple-json). The following are the error logs associated failed endpoint fetches.

-- boxplot

2021-07-26 11:19:57,439 - server.provider_api:log_exception:1892 - ERROR - Exception on /single_boxplots [POST]
Traceback (most recent call last):
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/flask_cors/decorator.py", line 128, in wrapped_function
    resp = make_response(f(*args, **kwargs))
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_api.py", line 217, in single_boxplot
    result = self.request_single({"name": "boxplots", **data})
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_base.py", line 493, in request_single
    sg=sg, name=callsite, ntype=ntype, proxy_columns=sg.proxy_columns
  File "/Users/jarus/Work/llnl/CallFlow/callflow/modules/boxplot.py", line 88, in __init__
    self.result["tgt"] = self.compute(df)
  File "/Users/jarus/Work/llnl/CallFlow/callflow/modules/boxplot.py", line 101, in compute
    mask = outliers(df[tv], scale=self.iqr_scale)
  File "/Users/jarus/Work/llnl/CallFlow/callflow/utils/utils.py", line 134, in outliers
    assert isinstance(data, (pd.Series, np.ndarray))
AssertionError

-- single_histogram and single_scatterplot.

07-26 11:19:57,429 - server.provider_api:log_exception:1892 - ERROR - Exception on /single_scatterplot [POST]
Traceback (most recent call last):
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/jarus/Work/llnl/CallFlow/venv/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/flask_cors/decorator.py", line 128, in wrapped_function
    resp = make_response(f(*args, **kwargs))
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_api.py", line 210, in single_scatterplot
    result = self.request_single({"name": "scatterplot", **data})
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_base.py", line 424, in request_single
    for module_idx in sg.module_aux_dict.keys()
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_base.py", line 424, in <dictcomp>
    for module_idx in sg.module_aux_dict.keys()
  File "/Users/jarus/Work/llnl/CallFlow/callflow/datastructures/supergraph.py", line 115, in get_name
    return self.idx2module[idx]
KeyError: (17, 17)

Jupyter Notebook: Can't parse config file

The example Jupyter Notebook can't be run as there seems to be an issue reading the example config file:

singleConfig = ConfigFileReader(singleConfigPath)

Causes the following error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-16-81364d44a7da> in <module>
      1 # Read config file.
----> 2 singleConfig = ConfigFileReader(singleConfigPath)

C:\temp\Anaconda\lib\site-packages\callflow\operations\read_config.py in __init__(self, filepath, config_json)
     31 
     32         self.scheme = self.json["scheme"]
---> 33         self.datasets = self.json["datasets"]
     34         self.runName = self.json["run_name"]
     35         self.save_path = os.path.join(self.data_path, ".callflow")

KeyError: 'datasets'

Config file: hpctoolkit-cpi-database/callflow.config.json
Environment: CallFlow 1.1.1 / Jupyter Notebook 6.0.3 / Anaconda Python 3.7.6 / Win10 64bit

v.1.3.0 Stage - Caliper .cali file missing column - "time"

The sample datasets caliper-simple-cali and caliper-lulesh-cali fail to load the VIS because of the missing "time" column.

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/jarus/miniconda/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/Users/jarus/miniconda/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_base.py", line 133, in mp_dataset_load
    read_parameter=read_param,
  File "/Users/jarus/Work/llnl/CallFlow/callflow/datastructures/supergraph.py", line 443, in load
    apply_func=lambda _: _.mean(),
  File "/Users/jarus/Work/llnl/CallFlow/callflow/utils/df.py", line 124, in df_bi_level_group
    return {_: _df.xs(_)[_cols] for _ in _levels}
  File "/Users/jarus/Work/llnl/CallFlow/callflow/utils/df.py", line 124, in <dictcomp>
    return {_: _df.xs(_)[_cols] for _ in _levels}
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/series.py", line 906, in __getitem__
    return self._get_with(key)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/series.py", line 946, in _get_with
    return self.loc[key]
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/indexing.py", line 879, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1099, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1037, in _getitem_iterable
    keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1254, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "/Users/jarus/miniconda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1316, in _validate_read_indexer
    "Passing list-likes to .loc or [] with any missing labels "
KeyError: "Passing list-likes to .loc or [] with any missing labels is no longer supported. The following labels were missing: Index(['time'], dtype='object'). See https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike"
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "venv/bin/callflow", line 33, in <module>
    sys.exit(load_entry_point('CallFlow', 'console_scripts', 'callflow')())
  File "/Users/jarus/Work/llnl/CallFlow/server/main.py", line 81, in main
    cf.load()
  File "/Users/jarus/Work/llnl/CallFlow/server/provider_base.py", line 103, in load
    partial(self.mp_dataset_load, save_path=load_path), self.datasets
  File "/Users/jarus/miniconda/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Users/jarus/miniconda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
KeyError: "Passing list-likes to .loc or [] with any missing labels is no longer supported. The following labels were missing: Index(['time'], dtype='object'). See https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike"

Improvement: subset datasets upon click "text-guides" on SuperNode

  • Enabling reprocessing when the number of datasets in the ensemble is different.
  • Grouping fails due to the 1st step. This is primarily because the column, "path" is treated as string when callflow.SuperGraph.load() is called.
  • Improvecallflow.operations.group design.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.