GithubHelp home page GithubHelp logo

domiii / dbux Goto Github PK

View Code? Open in Web Editor NEW
154.0 5.0 13.0 78.45 MB

Dbux is an Integrated Debugging Environment (IDbE) and Omniscient Debugger that makes JavaScript application's run-time behavior come alive, visible and interactive.

Home Page: https://domiii.github.io/dbux

License: Apache License 2.0

JavaScript 94.35% Shell 0.12% Python 0.14% Jupyter Notebook 1.76% HTML 0.12% CSS 0.60% MDX 2.90%
vscode vscode-extension monorepo traces debugging debugging-tool javascript js-runtime async-call-graph

dbux's People

Contributors

cherrydt avatar dependabot[bot] avatar domiii avatar edisonhello avatar miccwan avatar nehcled avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

dbux's Issues

dbux-projects: Add backend support

NOTE: You can test BackendAuth.login via the Dbux: Backend Login command.

  • add:

    async loginWithGoogleAccessToken(accessToken) {
      const cred = firebase.auth.GoogleAuthProvider.credential(googleAccessToken);
      return await firebase.auth().signInWithCredential(cred);
    }
  • implement BackendAuth.login()

    • check memento if googleAccessToken already present, and if so, try to loginWithGoogleAccessToken with that token:
      • await loginWithGoogleAccessToken(googleAccessToken)
      • if not existing, or if login failed:
        • this.loginWithLocalServer() (see below)
    • await loginWithGoogleAccessToken(googleAccessToken)
      • if we still fail to login, throw new Error
    • test if stuff works after login has succeeded
      • call await firestore.set('test', {x : 1});
  • implement BackendAuth.loginWithLocalServer (allow user to login via Google):

    • start local http server
      • yarn add serve
      • npx serve ${pathToLoginHtml} --port ${somePort}
    • host login.html, inside of login.html:
      • add <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/socket.io.js"></script> to <head>
        • NOTE: This way, io is globally available.
      • let user login via Google AuthProvider redirect and get the accessToken
      • send the accessToken back to the local http server
        • See dbux-runtime/src/client/Client.js -> _connect() for reference on how to connect to a server
    • open the website
    • wait until login.html sends back data
      • wrap the "wait" call in VSCode's window.withProgress and allow user to cancel while waiting
        • also need to expose withProgress and it's API as external
        • talk to Michael about this <3
    • if successful, store the received googleAccessToken in memento

Advanced Data Analysis

This issue is to allow for collecting initial feedback regarding more complex data analysis methods and tools to be used on Dbux data.

WARNING: This issue is only to collect well-thought-out and researched commentary. If you want to discuss or ask questions, please bring them to the Dbux Discord.

As a general hint, you probably want to get started with this task by looking at our two data-related folders:

  1. @dbux/data is the JavaScript module we also use internally to preprocess and manage all the data before visualizing it and making it interactive in the Dbux VSCode Plugin.
  2. analysis contains a few Python notebooks for rudimentary analysis on extracted data for testing and development purposes. We exported the data via the (also rather crude) dbux.exportApplicationData command

Button re-shuffle

  • Remove "trace" and "error" buttons from top right toolbar
    • If we can have conditional buttons (if we can hide them dynamically) in the toolbar, keep them, and only show them when relevant
  • Add "Run with Dbux" button to top right toolbar
  • Add "Error" button to traceDetails
  • Add "Toggle Call Graph" button (two buttons that switch with the call graph visible status) to traceDetails (merged from #194)
  • Add a "Help" button to "Applications". When clicked, open our Github repository (where they will see the README with more and more information available).

Complete (basic) async support

  • Fix ordering of staticTraces in dbux-babel-plugin/src/visitors/awaitVisitor.js
  • Probably also need to fix (if not entirely rethink) DataProvider.resolveCallIds
  • Fix rendering of TraceType.{Await,Resume} ContextNodes in Call Graph (broken loc and undefined displayName). We want to link them against their realContext.
  • TODO: probably more than just this.
    • traceDetailsView features might be bugged (not tested recently)

Overall it does not even seem entirely broken, but a lot more testing is needed:

image

NOTE: This issue only covers basic await support. The Async Call Graph feature is tracked in #210.

trace function parameters

E.g. in

function f(a, b, c) { ... }

We want to trace and step through a, b and c (probably after PushImmediate).

fix socket server for deployment

NOTE: The following goes for the two SocketServers that we are running:

  1. RuntimeServer (initRuntimeServer)
  2. TerminalSocketServer

Make sure, the servers don't start unless necessary:

  • don't start RuntimeServer unless necessary
    • when activating any bug in projects
    • when runCommands.js -> runFile function is called
  • If server fails to launch, logError properly, and also explain to the user why that might be the case (i.e. it might be because another instance of VSCode has already started such server)
  • register socket server with ExtensionContext (must have dispose method), so it will shut down when extension deactivates
  • TerminalServer should only run while Terminal is open and also _dbux_run.js has not finished yet

[trace navigation] Add "PreviousTrace" + "NextTrace" buttons

Currently, there is no easy way to visit all traces in order by id, and "{Next,Previous}InContext" ignores ExpressionValue, which makes sense and keeps things nice and easy. However, sometimes we do want to go through all traces, and/or easily select values.

Icons maybe just โ† and โ†’?

[dbux-project] version update resilience

When new versions are published, dbux-projects might bug out due to changes in assets, especially in _dbux_run.js.

Idea:

  • keep version of assets per project folder
  • just delete + re-install all previously installed projects on a wrong version by default

Command for system integrity check

  • Add a new command Dbux: System Check to userCommands.js.
  • Place the following logic for the actual check in file checkSystem.js in dbux-projects (because it does not depend on VSCode).
  • Report results of all tools back in the form of a codeModal (one line per tool)

Basic Tools

Check if all dependent OS tools are available:

  • which or where.exe
  • bash (all shell execution must work)
  • node v12+
  • pm
  • git

System independence

  • On Cygwin, which returns a unix-style path, which needs further lookup via cygpath -w.
    • On Windows: Use where instead of which
    • For that, implement all functions in dbux-projects\src\util\which.js
  • Use which() from which.js instead of Process.execCapture...(which...)
    • in checkSystem.js
    • in terminalUtil.js
  • in checkSystem, first run hasWhich

Run on activate, if not passed before

Use memento to remember if the command has passed all checks before. If not, call on start-up.

dbux-projects: Easily work through "sample bugs" + keep record of progress locally

  • When clicking on a bug, open a TextEditor, showing a bug's introduction

    • Ideally, we want to use the builtin Markdown preview window, but that does not seem possible. Probably need to use Webview to render instead. Consider VSCode Webview sample for this.
      • Need to prepare an html template for this.
  • Allow user to store their bug solutions locally:

    • Use VSCode's Memento API to store things locally
    • DataStructure: TestRun
      1. projectId + bugId
      2. createdAt
      3. nFailedTests number of tests failed (statusCode)
      4. timer time spent (or null, if no timer was used)
      5. patch string containing complete patch contents, obtained via Project.getPatchString
    • DataStructure: BugProgress
      1. projectId + bugId (combined PrimaryKey)
      2. createdAt + updatedAt
      3. stopwatchEnabled + timePassed
      4. status (BugStatus = new Enum({ None: 1, Solving: 2, Attempted: 3, Solved: 4 }))
    • NOTE: we will add id and uid columns to these data-structures when moving it online
  • Complete "PracticeSession" concept

    • When "activating" a bug for the first time, allow user to choose whether they want to time their results (see codeModals.js -> showInformationMessage)
      • If they choose to time it, start timer (implemented in PracticeStopwatch.js)
    • While timer is running, and user clicks the timer, selects another bug or cancels current bug run, confirm with user to stop timer. If confirmed, stop timer and store partial results (see below) before "cancelling" current bug.
    • when a bug testCommand has finished running:
      • Store a TestRun object in ProgressLog
      • the result statusCode obtained from the terminal is actually the number of failed tests. Show that result to the user. If testCommand executed and no test failed:
        • include a message of success and encouragement ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰
        • updateOrCreate corresponding BugStatus with status = BugStatus.Solved
        • "clear bug" from BugRunner so it is no longer active. Maybe we can just call cancel?
    • associate each session with a sessionId
    • store sessions, testRuns + bugStatus in db
  • In ProjectView, show bugs where status == BugStatus.Solved with a checkmark icon

  • When ProjectView is visible for the first time, load ProgressLog from Memento and render previous progress in UI

  • Allow user to switch between bugs without any problems

    • Always prompt user for confirmation before running gitResetHard when git diff result is not empty.
    • When activating a bug and user has an active bug (BugRunner.getActiveBug), store TestRun with nFailedTests = -1 (marking an attempt, but without test run)
      • If previous bug is unknown (e.g. because we restarted VScode):
        1. get tag name (Project.getTagName)
        2. get bugId from looking it up in the getOrLoadBugs array by tag name
        3. store TestRun (with patch!)
        4. then gitResetHard
    • When activating a bug that already has any TestRuns, choose the latest TestRun and apply that patch
      • Finish Project.applyPatchString in order to apply a patch from string (rather than file)
    • Allow user to review their changes using the VSCode built-in diff viewer (can we use code -d <file1> <file2>? -> might not be sufficient since there are multiple files and it cannot diff entire directories this way?)
  • Make sure, dbux-practice view is refreshed even if activateBug failed

[CallGraph -> Highlighter] Error when a trace was selected and application is deselected

Steps to reproduce

  1. Run something
  2. Select a trace
  3. Run it again

Error

It seems like we send out an update after disposal again :D

Two errors are reported in dbux-graph-client in Webview.

Stacktrace:

[DBUX [Highlighter]] Component update failed TypeError: Cannot read property 'classList' of null
at Highlighter.update (Highlighter.js?65ea:16)
at Highlighter._performUpdate (ClientComponentEndpoint.js?b2ab:83)
at Highlighter.updateClient (ClientComponentEndpoint.js?b2ab:121)
at Ipc._processRequest (Ipc.js?bb0f:162)
at Ipc.2 (Ipc.js?bb0f:254)
at Ipc._handleMessage (Ipc.js?bb0f:234)
at messageHandler (index.html:32)

[DBUX [ContextNode]] Component update failed TypeError: Cannot set property 'id' of null
at ContextNode.update (ContextNode.js?d8c9:67)
at ContextNode._performUpdate (ClientComponentEndpoint.js?b2ab:83)
at ContextNode.updateClient (ClientComponentEndpoint.js?b2ab:121)
at Ipc._processRequest (Ipc.js?bb0f:162)
at Ipc.2 (Ipc.js?bb0f:254)
at Ipc._handleMessage (Ipc.js?bb0f:234)
at messageHandler (index.html:32)

[dbux-projects] handle patch failures gracefully

In applyNewBugPatch, applying the patch might fail for a variety of reasons.

We want to make sure that even when patch cannot be applied, user can do their thing.

If previous user patch failed to apply:

  • warn user
    • allow user to look at patch
    • NOTE: in codeModals.js is our customized showWarningMessage which allows to show a warning message with action buttons
    • NOTE2: need to provide code-related functionality via externals
  • don't apply patch and keep going

show neighboring trace values in traceDetailsView

One great value of the traditional debugger is to see the value of variables in context in one table. Useful feature to have.

Dbux currently does not quite comprehend "variables", as we deal in traces and have not analyzed the role of individual variables yet - but at least we can show expression values of nearby expressions?

For now, we can:

  1. Add new "Nearby Traces" node to traceDetailsView
  2. One node per contexts on the current callstack just like the traditional debugger does it with scopes.
  3. One node per staticTrace (that are expressions) of that context
  4. Value of the actual trace that got executed in that context, closest to selectedTrace
  5. Visualize with node's label = staticTrace label and node's description = traceValueLabel

The screenshot shows how VSCode is now grouping by block, but I think that just makes things more confusing.

We can just group by context, makes things more intuitive to my mind. (I think that is how they used to do it, too.)

image

Thoughts?

allow configuring "run" + "debug" buttons

  • First step: Add config support using the default VSCode extension config API.
  • Name of the config option sub-trees: dbux.run and dbux.debug
    • Allow configuring run + debug configurations separately
  • enable-source-maps: toggle --enable-source-maps for run and debug (since it can be super slow)
  • dbuxArgs: custom dbux run command options (see buildCommonCommandOptions)
  • nodeArgs: Allow configuring node arguments
  • programArgs: Allow configuring program arguments, available via process.argv
  • env: Allow configuring program env, available via process.env

Talk to me first, if you have little to no experience in configuring CLI commands :)

[dbux-projects] fix "Bugs" that are stored in patch files

Bugs from some projects (e.g. todomvc) are of our own design, and we have saved them as patch files. However, that defeats many of the assumptions we have when running certain git commands (such as git reset --hard).

  • fix eslint (webpack setup) to use webpack's env parameter to pass entry point information to webpack.config (Domi)
  • Convert todomvc Project to match eslint setup
  • During install, commit all patch files to "local tags"
  • When switching to bug, just checkout the corresponding local tag

Deploy

Deploy packages to npm, and the extension to the VSCode Marketplace.

  • setup lerna
  • fix all eslint warnings + errors
  • fix up babel dependencies
  • fix up package inter-dependencies
  • copy all of dbux-code's required assets + dependencies into the resources/dist folder
  • Fix up + bundle all resource paths (specifically resources and assets folders).
  • Bundle VSCode extension [Publishing Tutorial]
    • Fix up .vscodeignore, .npmignore, package.json files
    • vsce publish
    • Allow testing deployed version before deployment (to make sure, paths are correct etc): add extension in dev mode to extensions folder
    • Allow to easily analyze and verify contents of final vsix file
    • Make sure, no resources spill into VSCode's own extensions folder (especially projects/)
  • Deploy to marketplace in a single script
  • Design a deployment protocol to make things no important steps are left out
  • Synchronize versions in all packages' package.json files. Easily allow version bumping.
  • Publish scripts: 1. Bump version (minor or major) of all packages. 2. Deploy all packages to npm. 3. Deploy to marketplace.

Potential Future Work

[trace navigation] `NextChildContext`does not work with getters/setters

Tested with calls1.js:

Cannot navigate into the getter call behind a['i'] or other getters.

NOTE: Getters and setters generate new contexts and function calls, but there are no CallExpression and no BCE associated with them.

NOTE2: We could insert a CallResultExpression after the fact, but we cannot guarantee for it to exist (e.g. when they throw Errors).

Try preventing Observer Effect bugs (e.g. in property accessors with side effects)

As mentioned in the README, observer effect can easily occur when Dbux tries to gather values from properties with side effects. We want to avoid that.

WARNING: This issue is only to collect well-thought-out and researched commentary. If you want to discuss or ask questions, please bring them to the Dbux Discord.

Just like ESLint configuration comments (// eslint-disable etc), Dbux should allow developers to turn off gathering data from specifically marked properties, or disable tracing on specific lines of code using directive comments.

traces inside of a try block are not flagged as error, even if error was thrown

Problem

  • Error thrown in try block does not get detected correctly.
  • If error was thrown, first trace of finally block will be flagged as error trace (incorrectly).

Examples

  • error4.js: error trace flagged incorrectly - should be todos (before todos.x) but is on = trace instead.
    • similar issue in error2.js -
    • using previousTraceInContext of PopImmediate does not work with try-catch cases

Solution

We are currently only resolving errors from PopImmediate in DataProvider.resolveErrorTraces. Currently, only traces that are PopImmediate and also !isTraceFunctionExit are error traces.

  • add and instrument new TraceTypes:
    • TraceType.TryBlockExit
    • TraceType.Catch
    • TraceType.Finally
  • fix up DataProvider.resolveErrorTraces
    • remove reliance on Trace.previousTrace (it seems to be inaccurate?)
      • use previousTrace = getPreviousInContext() instead?
    • remove getReturnTraceOfRealContext check (it's more complicated than that)
    • set error on previousTrace if...:
      • trace is Pop and !isTraceReturn(previousTrace) and !isTraceFunctionExit(previousTrace)

        • false-positive on try-finally, e.g. error-try-return-finally.js and error3.js
      • trace is TraceType.Catch

      • trace is TraceType.Finally and previousTrace is in try block and not TraceType.TryExit and !isTraceReturn(previousTrace) and !isTraceFunctionExit(previousTrace)

Fix GraphRoot.refresh race condition

We currently call graphRoot.refresh on allApplications.selection.onApplicationsChanged in GraphDocument. However that causes race conditions when refreshing too fast because client-side nodes might get disposed while we are still adding them asynchronously. We want to prevent that.

Also, allApplications.addApplication might remove an old and add a new application in one go, thereby emiting applicationsChanged refreshing twice. It should tell the remove method not to emit the event instead (maybe add a silent parameter).

Test

  • Quickly select + deselect apps in ApplicationView without errors

Add support for Actor Model

Explanation

Variants of the Actor Model (often in form of the Entity Component System (or "Paradigm") (ECS)) is commonly used in game development.

The Actor Model centers around agents, which themselves are objects, and also "individual threads of execution". Agent state can only be modified by the agent itself by calling methods on each agent. Execution of methods is atomic in regard to the agent's own state, no two different methods can be running at the same time.

The ECS (commonly seen in games, and arguably made most popular through the Unity (and other) game engine(s)) shares the idea of agents (also called "Entities" or "GameObjects") managing their own state, but with less constraints and assurances (especially: no atomicity or isolated state!). However, viewing Entities (or GameObjects) in ECS game engines as "Actors" in the Actor Model might largely aid us in debugging their run-time.

Checkpoints

  • Add a game project for testing to dbux-projects
  • Add FlowMode.AgentModel with only selected set of agents (i.e. those agents/entities/objects we want to investigate) as the parameter

[important] v0.1

When in that mode...

  • display all object executions of selected (entities) in horizontally parallel threads (similar to FlowMode.Async)`)
  • Only instrument/track calls/access of selected objects?
    • Q: Is this possible without significant reduction in FPS?

[nice-to-have] v0.2: Global System View

  • Add FlowMode.AgentModel without parameter -> Track all object executions?
  • Problem: At each update step, we might have hundreds, thousands or even more parallelly executing agents. Need to simplify/cull/group.
  • group execution of object methods by the objects that they belong to (i.e. `this
  • TODO: more

Remove asyncGenerator

Clean up our babel setup, make sure, we don't see asyncGenerator anymore. It just should not be necessary anymore and makes debugging a pain.

Change terminal into non-interactive

  • Process.js support getting err, out and code.
  • Listen on dispose of non-interactive terminal.
  • Promise.race to get result or canceled.
  • Get bash path on every OS.
    • Mac OS
    • Linux
    • Windows (will git bash acts differ from cygwin or not setting path in cmd? Ubuntu on Windows?)

[dbux-runtime] make sure data gets sent out, even upon `uncaughtException`

uncaughtException does not really allow us to ensure data being sent out before the program halts.

Current workaround

  • Report that data did not get sent out on process.exit

Ideal workaround: monitoring parent process

NOTE: In general, process termination cannot be gracefully handled from within a process (at least not cross-platform). Thus, we need to monitor it externally, either from dbux-code, @dbux/cli, and/or some other meta/monitoring process.

  • Monitor the process to be debugged and notify the user about abnormalities going on, rather than failing silently and leaving the user waiting for some result.

Old ideas

I briefly started experimenting in dbux-runtime/src/index.js , but did not get very far yet.

  • Maybe we can add a forcedShutdownDelay argument to the run command in dbux-cli/src/commands/run.js's builder object?
    • This might also allow us send out data before process.exit is called (e.g. by underlying framework), in which case we won't get any data.
  • on Node: override process.exit and add process.on('uncaughtException', ...) and unhandledRejection
  • check if node's process.exit can be overridden
    • it can! e.g.:
    const ex = process.exit.bind(process);
    process.exit = (...args) => {
      console.error(2);
      setTimeout(() => ex(...args), 1000);
    };
    
    process.exit(1);
    console.log(1);
    • NOTE: One odd side effect is that process.exit will not have the expected effect, which might (and probably will) introduce bugs in all kinds of programs, likely causing an abnormal exit anyway.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.