http://vmprof.com/static/images/loader.gif is too easily missed when we click somewhere in the page, and it's the only feedback that we actually clicked and have to wait. Worse, it might be completely off the page if we scroll down before clicking.

Intro and motivation

As already noted here vmprof/vmprof-python#90 (comment), we have implemented our own vmprof server, for the following reasons:

• At that time, vmprof-server was very slow on large profiles (multiple hours of runtime) as it stored the profiles in the SQL database. (I'm not sure how the current implementation compares to ours.)
• We wanted to have a good memory profile viewer in the server
• We don't really need user accounts etc.

Features of our implementation

I can't share the source code of our server just yet, for bureaucratic reasons, but I can share some information and a few screenshots here.

Properties and differences to vmprof-server:

• About 1000 LOC
• Uses vmprof-server CPU viewer (no jitlog integration yet)
• Much improved memory viewer based on Plotly:
  • Shows time and date on X axis
  • Data points aren't simply sampled from the complete data, but binned (mean), so that you don't miss spikes due to too-low sampling interval
  • Show memory usage mean, max or both
  • Nice interaction with the graph due to Plotly
  • Shows absolute or relative runtime on X axis
• Allows to search for projects and functions/callables
• Stores profiles as gzipped msgpack files, no decoding done in the server whatsoever: Data is encoded to .msgpack.gz in the client once, and delivered to the browser UI as-is. The only exception to this is for memory profile resampling.

We have also implemented a new client:

• About 100 LOC

• Interface isn't a script runner like python -m vmprof yourscript.py but a decorator that is applied to to-be-profiled callables, like

  @profile
  def somefunc():
      ...

• Allows to tag your submissions with a project name

• Automatically tags your submissions with the top-level function/callable name (somefunc)

• Client can upload normal vmprof profile files

• Client protocol not compatible to the vmprof-python protocol (but very similar)

Screenshots

Landing page with project names and top-level function names (in red). Search filters may be shared using the arrow on the right

Integration of vmprof-server CPU viewer

The memory viewer, showing max memory usage for each bin (no hidden spikes!). On the right: Upper stacktrace shows the largest common denominator of all stack traces of the bin. Lower stacktrace extends the upper one by the most common stack trace of the bin (28% of the stack traces in the bin were equal to the "concatenation" of the two stack trace parts).

Memory viewer showing mean + max of each bin

Future of our server, integration in vmprof-server

I think our server has some nice properties, mainly the memory viewer and the storage system (although I'm not sure how it compares to the current vmprof-server JSON/Gzip storage system in terms of performance). We'd love the contribute most of it back to vmprof-server proper.

Possibility A: Integrate vmprof-server into our server

• Integrate jitlog into our server
• Maybe integrate user accounts into our server
• Make our server the new official server

Possibility B: Integrate our memory viewer into vmprof-server

• Add memory view to vmprof-server
• Change protocol accordingly

What do you guys think?

Do this: click variable p1, it stays orange; click variable p2, it stays green; click p2 again to remove the color; click p2 again to add green again; now the next color is green as well, instead of being purple.

"just red" seems to be not very interesting. We can think about what we want to convey

While uploading a trace. CPython 3.6.1, macOS El Capitan. The trace is much smaller than 100 MB (~28 MB).

> python -m vmprof.upload --web-auth .. output-small.log
Uploading to http://vmprof.com...
 => Uploading the cpu profile...
/Users/ttvrtkovic/workspace/ngs-worker-ct2/.venv/lib/python3.6/site-packages/vmshare/service.py:96: UserWarning: WARNING: vmprof.com limits the compressed profile file to 100 MBs. The upload might fail!

  warnings.warn("WARNING: vmprof.com limits the "
server rejected meta data. status: 400, msg: '["could not extract meta data for feb6a5e5-5a0a-43cd-8069-99daf1fe090e. exception: [Errno 111] Connection refused"]'
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/ttvrtkovic/workspace/ngs-worker-ct2/.venv/lib/python3.6/site-packages/vmprof/upload.py", line 22, in <module>
    main()
  File "/Users/ttvrtkovic/workspace/ngs-worker-ct2/.venv/lib/python3.6/site-packages/vmprof/upload.py", line 19, in main
    'VM': interpname })
  File "/Users/ttvrtkovic/workspace/ngs-worker-ct2/.venv/lib/python3.6/site-packages/vmshare/service.py", line 124, in post
    Service.FILE_CPU_PROFILE, compress=False)
  File "/Users/ttvrtkovic/workspace/ngs-worker-ct2/.venv/lib/python3.6/site-packages/vmshare/service.py", line 106, in post_file
    self.stop_if_error_occured(response)
  File "/Users/ttvrtkovic/workspace/ngs-worker-ct2/.venv/lib/python3.6/site-packages/vmshare/service.py", line 73, in stop_if_error_occured
    raise ServiceException()
vmshare.service.ServiceException

Using a local vmprof-server instance, JIT traces can't be displayed any more after using the server for a while. At that point, all API calls (e.g. to "GET /api/jit/meta/7a53594c-f12b-439f-b5b2-62a6c63e1cef/ HTTP/1.1") seem to return a JSON containing just {"error": ""} and the spinner just keeps spinning with no user-visible indication of what's wrong. Digging down a bit, it appears that vmcache.cache.CacheProtocol._handle() fails with an AssertionError.

It would be good to print a warning if no source code has been found just before uploading.
Example that does not work:

http://vmprof.com/#/a5162b7025b506d6a3989fe99af3d752/traces
src: https://paste.pound-python.org/show/CtaOtb9opQltq2Bg3m1X/

Should we archive the repo? It has not been updated for a few years and is showing bitrot. In particular, the version of django used needs an update. I think these days users are better off with the local displays, using something like https://github.com/Cskorpion/vmprof-firefox-converter to view the data in firefox.

It's down for everyone.

In the meantime, are there any alternative vmprof servers?

Currently the order is imposed by the position in the log file. Would be good to be able to sort by count and by name

Python-3.4
error extract

[25/Jun/2015 12:11:35] "POST /api/user/ HTTP/1.1" 500 106891

<div id="summary">
  <h1>TypeError at /api/user/</h1>
  <pre class="exception_value">Unicode-objects must be encoded before hashing</pre>
  <table class="meta">

Consider following screenie:

Weight of "update" and "chunked" relative to each other is represented correctly by block width.

Weight "update" relative to parent is represented wrong, stats show 19%, but box is >50%.

Is that by design?
Or only because "own" box is minimum allowed width instead of (100-19-11-3)%?

We need a better way to navigate between bridges: when we look at a list of operations, the guards with bridges attached should be marked (and stand out, with a direct link to the bridge); and when we go to a bridge we should know where it comes from, in the sense of which parent guard failed.

I uploaded two performance executions on vmprof.com using my access token. I was naively under the impression that if I used my access token it would be private too me. Honestly I don't know why I thought so since it doesn't say that in the documentation. Sorry.

Would it be possible to remove id d4db0d7f-0a87-4203-a0a3-789d541ce30e and 1274d5cd-cd39-44be-b6ab-e9772833104d?

Sorry for the trouble, it's a great tool 👍

Currently you need to click through a node in order to see percent time info. It's tedious, especially considering that relative time isn't represented by segment sizes (#2).

E.g. below, I'd like to know time info for wrapper when I hover on the segment.

It should probably work like the bridge links in the Rewritten/Asm views, but in all views.

URL : vmprof report
When I click on the the function_name (underlined on the screenshot), the user id is missing on the target url, and it shows an empty page.

Chrome complains about the something that js tries to do:

Sample profile: http://vmprof.com/#/04175aa1-72a7-49de-bd93-894cf04f95c9/traces

This is not tested at all. We already had one issue because of that:
#7

So two separate points:

• disable those completely for CPython
• actually make them reported in PyPy to some extent

Despite 3.4 being in travis (no sure how this got there) Python 2.7 is still default deploy target.

Missing requirements after merging 67bf0d8

(.virtualenv) vagrant@vmprof:/vagrant/vmprof-server$ python manage.py runserver 0.0.0.0:8000
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/vagrant/vmprof-server/.virtualenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/vagrant/vmprof-server/.virtualenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 312, in execute
    django.setup()
  File "/vagrant/vmprof-server/.virtualenv/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/vagrant/vmprof-server/.virtualenv/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/vagrant/vmprof-server/.virtualenv/local/lib/python2.7/site-packages/django/apps/config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/vagrant/vmprof-server/vmlog/models.py", line 4, in <module>
    import lzma
ImportError: No module named lzma

Not sure if this is the correct place to report issues with vmprof.com, apologies if not.

I've been getting the following repeatedly (8 to 8:30ish, Pacific time):

$ venv/dsb/bin/python -m vmprof --web raw2prep.py ...
...
Compiling and uploading to http://vmprof.com...
VMProf log: Server rejected profile. status: 500, msg: '<h1>Server Error (500)</h1>'

Not sure what else I can do to debug.