Dbux is an Integrated Debugging Environment (IDbE) and Omniscient Debugger that makes JavaScript application's run-time behavior come alive, visible and interactive.
This issue is to allow for collecting initial feedback regarding more complex data analysis methods and tools to be used on Dbux data.
WARNING: This issue is only to collect well-thought-out and researched commentary. If you want to discuss or ask questions, please bring them to the Dbux Discord.
As a general hint, you probably want to get started with this task by looking at our two data-related folders:
@dbux/data is the JavaScript module we also use internally to preprocess and manage all the data before visualizing it and making it interactive in the Dbux VSCode Plugin.
analysis contains a few Python notebooks for rudimentary analysis on extracted data for testing and development purposes. We exported the data via the (also rather crude) dbux.exportApplicationData command
Remove "trace" and "error" buttons from top right toolbar
If we can have conditional buttons (if we can hide them dynamically) in the toolbar, keep them, and only show them when relevant
Add "Run with Dbux" button to top right toolbar
Add "Error" button to traceDetails
Add "Toggle Call Graph" button (two buttons that switch with the call graph visible status) to traceDetails (merged from #194)
Add a "Help" button to "Applications". When clicked, open our Github repository (where they will see the README with more and more information available).
Probably also need to fix (if not entirely rethink) DataProvider.resolveCallIds
Fix rendering of TraceType.{Await,Resume} ContextNodes in Call Graph (broken loc and undefineddisplayName). We want to link them against their realContext.
TODO: probably more than just this.
traceDetailsView features might be bugged (not tested recently)
Overall it does not even seem entirely broken, but a lot more testing is needed:
NOTE: This issue only covers basicawait support. The Async Call Graph feature is tracked in #210.
NOTE: The following goes for the two SocketServers that we are running:
RuntimeServer (initRuntimeServer)
TerminalSocketServer
Make sure, the servers don't start unless necessary:
don't start RuntimeServer unless necessary
when activating any bug in projects
when runCommands.js -> runFile function is called
If server fails to launch, logError properly, and also explain to the user why that might be the case (i.e. it might be because another instance of VSCode has already started such server)
register socket server with ExtensionContext (must have dispose method), so it will shut down when extension deactivates
TerminalServer should only run while Terminal is open and also _dbux_run.js has not finished yet
Currently, there is no easy way to visit all traces in order by id, and "{Next,Previous}InContext" ignores ExpressionValue, which makes sense and keeps things nice and easy. However, sometimes we do want to go through all traces, and/or easily select values.
Add a new command Dbux: System Check to userCommands.js.
Place the following logic for the actual check in file checkSystem.js in dbux-projects (because it does not depend on VSCode).
Report results of all tools back in the form of a codeModal (one line per tool)
Basic Tools
Check if all dependent OS tools are available:
which or where.exe
bash (all shell execution must work)
node v12+
pm
git
System independence
On Cygwin, which returns a unix-style path, which needs further lookup via cygpath -w.
On Windows: Use where instead of which
For that, implement all functions in dbux-projects\src\util\which.js
Use which() from which.js instead of Process.execCapture...(which...)
in checkSystem.js
in terminalUtil.js
in checkSystem, first run hasWhich
if not available, report: "Cannot run the system check because you are on a legacy system. We need where to be available on Windows (available from Windows 7 and Windows Server 2003 onward), and which to be available on any other system. If you are on Windows, please refer to: https://superuser.com/questions/49104/how-do-i-find-the-location-of-an-executable-in-windows"
Run on activate, if not passed before
Use memento to remember if the command has passed all checks before. If not, call on start-up.
When clicking on a bug, open a TextEditor, showing a bug's introduction
Ideally, we want to use the builtin Markdown preview window, but that does not seem possible. Probably need to use Webview to render instead. Consider VSCode Webview sample for this.
patch string containing complete patch contents, obtained via Project.getPatchString
DataStructure: BugProgress
projectId + bugId (combined PrimaryKey)
createdAt + updatedAt
stopwatchEnabled + timePassed
status (BugStatus = new Enum({ None: 1, Solving: 2, Attempted: 3, Solved: 4 }))
NOTE: we will add id and uid columns to these data-structures when moving it online
Complete "PracticeSession" concept
When "activating" a bug for the first time, allow user to choose whether they want to time their results (see codeModals.js -> showInformationMessage)
If they choose to time it, start timer (implemented in PracticeStopwatch.js)
While timer is running, and user clicks the timer, selects another bug or cancels current bug run, confirm with user to stop timer. If confirmed, stop timer and store partial results (see below) before "cancelling" current bug.
when a bug testCommand has finished running:
Store a TestRun object in ProgressLog
the result statusCode obtained from the terminal is actually the number of failed tests. Show that result to the user. If testCommand executed and no test failed:
include a message of success and encouragement ๐๐๐
updateOrCreate corresponding BugStatus with status = BugStatus.Solved
"clear bug" from BugRunner so it is no longer active. Maybe we can just call cancel?
associate each session with a sessionId
store sessions, testRuns + bugStatus in db
In ProjectView, show bugs where status == BugStatus.Solved with a checkmark icon
When ProjectView is visible for the first time, load ProgressLog from Memento and render previous progress in UI
Allow user to switch between bugs without any problems
Always prompt user for confirmation before running gitResetHard when git diff result is not empty.
When activating a bug and user has an active bug (BugRunner.getActiveBug), store TestRun with nFailedTests = -1 (marking an attempt, but without test run)
If previous bug is unknown (e.g. because we restarted VScode):
get tag name (Project.getTagName)
get bugId from looking it up in the getOrLoadBugs array by tag name
store TestRun (with patch!)
then gitResetHard
When activating a bug that already has any TestRuns, choose the latest TestRun and apply that patch
Finish Project.applyPatchString in order to apply a patch from string (rather than file)
Allow user to review their changes using the VSCode built-in diff viewer (can we use code -d <file1> <file2>? -> might not be sufficient since there are multiple files and it cannot diff entire directories this way?)
Make sure, dbux-practice view is refreshed even if activateBug failed
It seems like we send out an update after disposal again :D
Two errors are reported in dbux-graph-client in Webview.
Stacktrace:
[DBUX [Highlighter]] Component update failed TypeError: Cannot read property 'classList' of null
at Highlighter.update (Highlighter.js?65ea:16)
at Highlighter._performUpdate (ClientComponentEndpoint.js?b2ab:83)
at Highlighter.updateClient (ClientComponentEndpoint.js?b2ab:121)
at Ipc._processRequest (Ipc.js?bb0f:162)
at Ipc.2 (Ipc.js?bb0f:254)
at Ipc._handleMessage (Ipc.js?bb0f:234)
at messageHandler (index.html:32)
[DBUX [ContextNode]] Component update failed TypeError: Cannot set property 'id' of null
at ContextNode.update (ContextNode.js?d8c9:67)
at ContextNode._performUpdate (ClientComponentEndpoint.js?b2ab:83)
at ContextNode.updateClient (ClientComponentEndpoint.js?b2ab:121)
at Ipc._processRequest (Ipc.js?bb0f:162)
at Ipc.2 (Ipc.js?bb0f:254)
at Ipc._handleMessage (Ipc.js?bb0f:234)
at messageHandler (index.html:32)
One great value of the traditional debugger is to see the value of variables in context in one table. Useful feature to have.
Dbux currently does not quite comprehend "variables", as we deal in traces and have not analyzed the role of individual variables yet - but at least we can show expression values of nearby expressions?
For now, we can:
Add new "Nearby Traces" node to traceDetailsView
One node per contexts on the current callstack just like the traditional debugger does it with scopes.
One node per staticTrace (that are expressions) of that context
Value of the actual trace that got executed in that context, closest to selectedTrace
Visualize with node's label = staticTrace label and node's description = traceValueLabel
The screenshot shows how VSCode is now grouping by block, but I think that just makes things more confusing.
We can just group by context, makes things more intuitive to my mind. (I think that is how they used to do it, too.)
Bugs from some projects (e.g. todomvc) are of our own design, and we have saved them as patch files. However, that defeats many of the assumptions we have when running certain git commands (such as git reset --hard).
fix eslint (webpack setup) to use webpack's env parameter to pass entry point information to webpack.config (Domi)
Convert todomvc Project to match eslint setup
During install, commit all patch files to "local tags"
When switching to bug, just checkout the corresponding local tag
In BugRunner.activateBug, we use await this._queue.enqueue but when any task fails, it skips right to the next. Make sure to clear the queue, if a task in it failed instead.
As mentioned in the README, observer effect can easily occur when Dbux tries to gather values from properties with side effects. We want to avoid that.
WARNING: This issue is only to collect well-thought-out and researched commentary. If you want to discuss or ask questions, please bring them to the Dbux Discord.
Just like ESLint configuration comments (// eslint-disable etc), Dbux should allow developers to turn off gathering data from specifically marked properties, or disable tracing on specific lines of code using directive comments.
Error thrown in try block does not get detected correctly.
If error was thrown, first trace of finally block will be flagged as error trace (incorrectly).
Examples
error4.js: error trace flagged incorrectly - should be todos (before todos.x) but is on = trace instead.
similar issue in error2.js -
using previousTraceInContext of PopImmediate does not work with try-catch cases
Solution
We are currently only resolving errors from PopImmediate in DataProvider.resolveErrorTraces. Currently, only traces that are PopImmediate and also !isTraceFunctionExit are error traces.
add and instrument new TraceTypes:
TraceType.TryBlockExit
TraceType.Catch
TraceType.Finally
fix up DataProvider.resolveErrorTraces
remove reliance on Trace.previousTrace (it seems to be inaccurate?)
use previousTrace = getPreviousInContext() instead?
remove getReturnTraceOfRealContext check (it's more complicated than that)
set error on previousTrace if...:
trace is Pop and !isTraceReturn(previousTrace) and !isTraceFunctionExit(previousTrace)
false-positive on try-finally, e.g. error-try-return-finally.js and error3.js
trace is TraceType.Catch
trace is TraceType.Finally and previousTrace is in try block and not TraceType.TryExit and !isTraceReturn(previousTrace) and !isTraceFunctionExit(previousTrace)
We currently call graphRoot.refresh on allApplications.selection.onApplicationsChanged in GraphDocument. However that causes race conditions when refreshing too fast because client-side nodes might get disposed while we are still adding them asynchronously. We want to prevent that.
Also, allApplications.addApplication might remove an old and add a new application in one go, thereby emiting applicationsChanged refreshing twice. It should tell the remove method not to emit the event instead (maybe add a silent parameter).
Test
Quickly select + deselect apps in ApplicationView without errors
The Actor Model centers around agents, which themselves are objects, and also "individual threads of execution". Agent state can only be modified by the agent itself by calling methods on each agent. Execution of methods is atomic in regard to the agent's own state, no two different methods can be running at the same time.
The ECS (commonly seen in games, and arguably made most popular through the Unity (and other) game engine(s)) shares the idea of agents (also called "Entities" or "GameObjects") managing their own state, but with less constraints and assurances (especially: no atomicity or isolated state!). However, viewing Entities (or GameObjects) in ECS game engines as "Actors" in the Actor Model might largely aid us in debugging their run-time.
We currently allow users to run dbux on arbitrary files, using the "dbux-run" launch config and its friends. That is not available to the deployed extension, so we need to migrate it to something more accessible.
uncaughtException does not really allow us to ensure data being sent out before the program halts.
Current workaround
Report that data did not get sent out on process.exit
Ideal workaround: monitoring parent process
NOTE: In general, process termination cannot be gracefully handled from within a process (at least not cross-platform). Thus, we need to monitor it externally, either from dbux-code, @dbux/cli, and/or some other meta/monitoring process.
Monitor the process to be debugged and notify the user about abnormalities going on, rather than failing silently and leaving the user waiting for some result.
NOTE: One odd side effect is that process.exit will not have the expected effect, which might (and probably will) introduce bugs in all kinds of programs, likely causing an abnormal exit anyway.