Misc collection of unsorted TODO items

[Domiii]

TODO

• fix: flame icons are not shown on ACG in details mode
• fix: Don't reveal Data Flow View nodes on trace select if Data Flow View is not currently open
• fix error logging: when error occures, debug console does not show the error information
• fix AsyncTDNode's "FROM" edges and "navigation up"
  • if the from node contains the scheduler, go there, else go to beginning
    • go to end, if navigating to "next parent"

Future Work

• Feature: Thread Selection #435
• can we add a new "highlight CallGraph roots" feature, that only highlights roots of given condition?
  • all roots where trace of selected staticTrace has the same value (ref or non-ref) as selected trace
    • test in todomvc -> dispatchEvent -> select selector and highlight all roots of same selector value
      • same with targetElement

Done

• fix: revealing Execution trace
  • When selecting a call graph node, the corresponding node in Executions should be reveal'ed (but is not).
  • When selecting the same call graph node twice and Executions is open, it throws an error: Failed to focus on TraceTDView...
  • Test w/ react-tic-tac-toe
• fix: ValueNode -> valueRender
  • test w/ objects0.js:
    • click into any value, it is incorrect in many cases
    • simple value without modification (-> null)
    • obj value without modification (seems to work, but need to re-test)
    • arr value without modification (-> renders internal representation with nodeId etc)
    • simple value after modification (seems to work, but need to re-test)
    • obj value after modification (seems to work, but need to re-test)
    • arr value after modification (-> renders internal representation with nodeId etc)
• bug: Value rendering of HTMLInputElement (any object) is broken
• When a CER or BCE is selected, and "step into" (in any direction) is clicked. Always step into (if the call has a context)!
  • test w/ nested function calls (e..g f(g(h()))
  • test w/ express#5 -> try to step into utils.isAbsolute as efficiently as possible
• When selecting applications of same group (same entry point?) -> de-select application of same group/entry point
  • 
• when pressing Error Button, also reveal the error node in GlobalView
• traces inside of a try block are not flagged as error, even if error was thrown #197
• Feature: Render AsyncGraph #452
• Global Analysis View: error + console output traces #618
• fix: stepping into and out of async function requires one extra button click (probably because of extra Resume/Await contexts)
• Select Trace button: given await x;, if cursor is on x, repeatedly clicking the button does not cycle to the await x trace, (probably because it is in a different context/run? However: note that they are still in the same real context)
• fix: PracticeSession loading
  • if session load fails, bring dbux-practice view to a sane state (currently, it remains empty)
  • test by throwing an error on purpose
• fix CallGraph: when toggling between sync and async too fast, it errors out
• fix AsyncGraph: of todomvc (or any other long running application)
  1. enable val and detail
  2. select a value trace -> some nodes now render a value
  3. interact with the GUI (which triggers data updates in dbux-code)
  4. -> after new data is received, all nodes are rendered empty, even if they have a value
• fix: todomvc practice session is not saved
  1. Run it
  2. Restart Dbux
  3. askForRecoverPracticeSession only finds a very small file. After loading it, applications are empty.
• remove TracesByCalleeTraceIndex in favor of ContextsByCalleeTraceIndex + TracesOfContextIndex (also see getCalledContext)
• better deal with enable-source-maps in ProjectsManager.runTest
  • maybe disable for all but Node@16+, since it was extremely slow in earlier versions
  • -> for now, we require node@16 in checkSystem() anyway; but that might be a too strong contraint...
• fix project installation: for todomvc
  • It calls WebpackBuilder.afterInstall (via await this.builder?.afterInstall?.() in install()) which installs webpack correctly
  • but at some point later in time, webpack is removed from package.json again
  • then in installBug -> await project.npmInstall(); it now uninstalls webpack again because it is not in package.json anymore.
  • -> Why is package.json reset? Is it one of the git operations?
• fix Executions group -> Expand button does not do anything?
  • 
  • NOTE: Not sure, if we are already using TreeView.reveal API here?
• feature: when selecting a new trace, don't scroll lists back to the top in TDView (or other views?)
  • When selecting a trace (or doing certain other things), all nodes get re-created, and because of that, scroll status is moved back to the top
  • -> Scroll status is reset because we manually focus on NavigationNode when traceSelected to show navigation buttons, not because the view is refreshed. VSCode tends to stay at the original scroll height.
  • Once re-built, use TreeView.reveal to go to the last clicked node (according to this issue)
    • NOTE: You should be able to use node.id to uniquely identify a node, even if it was re-built. Probably want to start by making sure that that works as anticipated (see _decorateNewNode and makeNodeId)
    • We might have to remember, how the click relates to the selected trace. If the last click is unrelated, maybe don't reveal?
    • -> Try debugging javascript-algorithms -> BubbleSort.sort and see if it feels right.
  • IF that does NOT work: instead, we might want to only update changed nodes if possible? (hopefully we don't need to!)
    • e.g. re-render only the previously and newly selected traces (2 traces total)
    • Maybe add some sort of rerenderCondition function to nodes and traverse all nodes top-to-bottom throughout the view?
    • NOTE: you can re-render only the sub-tree node by changing BaseTreeViewNodeProvider.repaint -> this._onDidChangeTreeData.fire(node); (mentioned in VSCode API -> onDidChangeTreeData)
• require node@16 (reduce monkey-patch-induced infinite loops) #593
• change CallGraph toolbar buttons so that the situation where nothing is shown is prevented
  • remove off from the GraphMode button
  • instead, add a third toggle option to stack button which makes it full screen, but does not bother the graph if there is no stack
  • when stack is full screen, change label to Stack
• Currently when using "Delete Project" while in project's own workspace -> then Run a bug in the same project again (which should re-install the project) -> files seem to be missing, installation does not seem to be complete.
  • test with hexo#4
• CallGraph: if someone tries to expand a node with more than 10000 children -> show confirm and warn them that it might take a while
• [dbux-projects] BugRunner.testBug returns invalid code
  • disable the feature that determines "success" for now, since it is not reliable. Need to fix it in the future. For now, only determine whether the user "found the bug".
    • meaning: PathwaysDataProvider.addTestRun should not interpret nFailedTests (which is just the code of the run application)
  • NOTE: running test command can return all kinds of meaningless codes.
    • e.g. {code:0} while there is actually a failed test?
      • e.g. express#1?
• fix CallGraph rendering for HoF's (map et al.) is confusing: it looks like every call leads to the caller's return value to be returned again
  • Problem: we currently render call and value for every callback invocation
  • Sln (hackfix): only render call and value for first iteration
    • -> we are currently using getCallerOrSchedulerTraceOfContext for this
    • -> maybe add some sort of getCallerOrSchedulerTraceOfFirstContext which returns null, if it is not the first context of the caller trace?
  • test w/ arrayMap.js sample
• fix ValueTDNode rendering for strings -> render double quotation marks on strings
  • test w/ optionalCalls4.js -> compare rendering of obj vs. s
  • consider using JSON.stringify (that would also escape everything, which might be desirable)
    • -> we already do this when clicking the "Inspect this value" 🔎 magnifying glass button
    • -> also consider new value rendering of DebugTDNode and renderValueSimple
• fix: "step into" does not work anymore?
  • 
  • test w/ javascript-algorithms -> BubbleSort.sort
• fix CallGraph: call does not show up anymore
  • 
• fix: ValueTDRefNode - some nested objects repeat infinitely
  • test with any function f() { } -> investigate prototype property
  • e.g.:
• fix: error button is not clickable when re-selecting application
  • steps to reproduce:
    • run 2 applications (one without error A, one with error B)
    • -> error button still works
    • under Applications, toggle off B
    • under Applications, toggle on B
    • -> error button not clickable anymore
    • -> this bug does not appear if you only have one application
• add an indication when file has diverged from recording
  • add a visual indicator (e.g. something like ⚠️) in Trace Details
    • Consider existing document changed warning
  • add description to Trace Details tooltip
    • -> VSCode API does not support custom tooltip
    • -> use additional button instead
  • store file status per Application, so that when switching between applications, status is updated correctly
    • test: run file -> change file (indicator should show up) -> run again (indicator should be gone)
• fix value descriptions. They are buggy for nested values, e.g. array-of-object:
• review comments: many fixes to Issue#561 #594 (comment)
• Feature: Proper async stacktrace visualization + interaction #566
• When a trace is selected, the red border decoration around the selected piece of code does not get re-rendered, if the file moves out of (and back into) focus.
  • to re-produce: 1. select any trace. 2. select something that is not the file (all decorations are gone). 3. Select the file again. 4. -> the trace is not bordered anymore
• Replace "Delete Project" button with "Cleanup". Ask user whether they want to "Flush Cache", "Clear Log", "Delete Project" or "Cancel" (ordered by severity)
• implement DataProviderUtil.getArrayDataNodes (for arguments<->parameter linking of apply)
  • -> test Data Flow Graph with bind_apply_call.js
• review PR: Add producer_consumer_callback.js, promiseTicks.js, AsyncValueLabel and implement dp.util.getNestedAncestors #580
• AsyncGraph rendering:
  • sometimes, thread#1 nodes are moved to the right. -> leave thread#1 nodes in first lane (for now)
  • e.g. sequelize: findOrCreate-serial
• small AsyncCallGraph bugs:
  • it always says "(no applications selected)"
  • when toggling detail mode, it does not re-select the active node if follow is enabled
• when switching between bugs: files are not changed correctly
  • make sure that async-js#2's patch is not applied anymore when switching back to async-js#1 (it currently stays applied)
  • test by switching between async-js#1 and async-js#2 several times, also store research app data and sometimes choose to load and sometimes not
  • same for sequelize#4 and sequelize#5 (the difference being a node_modules file. Maybe that causes some issues?)
• in case of sequelize#5, make sure the node_moduels file is always undone when switching to any other sequelize bug, not just sequelize#4
• fix "open in VSCode workspace" (3)
  • [bug] when selecting "Create and open...", after the workspace is opened, the bug is not automatically selected
    • NOTE: it appears to have tried to load a different bug from the PracticeSession, but I deleted that bug from the project's list
      • -> nevertheless, it should always open the newly selected bug, and ignore the previously stored PracticeSession bug
  • if the "default workspace" exists, show "open..." instead of "create and open..."
  • [new feature] when selecting a workspace (during bug init) or when "running" a bug -> store lastWorkspacePath of that bug in localStorage
  • if the project is not in workspace, offer new option: lastWorkspacePath (if the file exists)
    • if lastWorkspacePath is the currently open workspace, change button label to reflect that. When button is pressed, add to workspace.
• Editor.md#1 has incorrect code - bug patch is suddenly always applied?
  • steps to reproduce: 1. Run Editor.md#1, 2. git status -> see uncommitted changes in src/editormd.js
  • Find out where those changes come from and make sure they don't happen. Note that there is onload.patch but it should not be applied to bug#1, only to bug#2.
• fix "open in VSCode workspace" (2)
  • only ask the user when selecting a bug
  • check if project is in current workspace
    • if project is in current workspace, proceed without asking
    • if project is not in current workspace, present user with modal
  • modal: Project "X" is currently not in your workspace (which makes it harder to work with it).
    • choices: add to current workspace (if there is an open workspace); create + open new workspace for project, "Continue", cancel
  • test all modal options with all different workspace scenarios - open bug...
    • from a dedicated workspace of the project
    • from a workspace that has the project, but also other projects
    • from a workspace that does not have the project
    • with no workspace open
• fix: if AsyncGraph is large (e.g. for webpack), follow starts jerking around and does not land exactly on the right node anymore after traceSelection
• cannot import Editor.md application data (applicationId of null)
• identify data dependencies between threads
  • add the following as a node to the nodes/research folder (see EdgeAnalysisNode for comparison)
  • print out all variables (by accessId) that are written in thread a and later read by another thread b
    • -> when printing out, only print out variable names (static trace displayName); not individual array elements; i .e. print a, not a[3]
    • -> ignore program context roots (check the root of all accessIds, ignore if that root's StaticContext is of type Program)
  • run on producer_consumer
    • -> needs re-writing producer_consumer to only use one file (since we don't yet track data dependencies between files)
    • -> should identify all "global queue" variables
  • test w/ all producer_consumer samples
• nesting
  • -> create a new copy of UP getNestedAncestors(rootId) that gathers all updates and does not stop recursion until it hits a "top-level CGR".
  • getNestingParent(rootId) -> next of rootId in s
  • getNestedDepth(rootId) -> s.length
  • CallStack build algo: when traversing an AsyncEdge -> skip roots that are more deeply nested than self
    • i.e. CS should only contain all previous siblings and ancestors by NESTING property
  • test in complex scenarios (basic1, all sync* samples + sequelize)
• render SYNC roots:
  • when a root is selected: render all getAllSyncRoots(fromEdgeOfCurrentRoot.syncPromiseIds) with a bright-red, DASHED border
  • algo:

    getAllSyncRoots(rootId) {
      const { syncPromiseIds } = asyncNode(rootId);
      return syncPromiseIds?.flatMap(p => {
        var postUpdateData = { links: [], syncPromiseIds: [] }; // ignore for now
        const roots = DOWN(p, postUpdateData);
        
        // handle special Promisify synchronization -> test w/ new producer_consumer_async
        for (const p of postUpdateData.syncPromiseIds) {
          const link = dp.indexes.promiseLinks.to.getUnique(p);
          if (link && link.type === PromiseLinkType.Promisify && !roots.includes(link.rootId)) {
            roots.push(link.rootId);
          }
        }
    
        /* handle: r is maybe null, or maybe array (in case of Promise.all) */
        return roots.flatMap(r => 
          all i starting at i=r, going up all AsyncEdges from i while i >= root(getFirstTraceByRefId(p))
        );
      });
    }

• producer_consumer
  • re-write remaining producer_consumer samples using wait/notify (to replace current "poll/idle" approach) -> see async version
  • make sure, RNG works correctly and produces the same order of high-level events
  • produce the 3 graphs
  • Write down all differences + the rules (+ reasons) behind the differences (possibly using our previous analysis results)
• Callstack bugs
  • in sequelize - findOrCreate-parallel, the first FORKed node (and many others) are missing synchronous call stack nodes of previous roots (sync and main calls in this case)
  • 
  • fix call stack algorithm: while going up from root at depth d2, when encountering the first root at level d1 < d2, make sure to add all roots of depth d2-1, d2-2, d2-3, ..., d1, instead of only d1 (since we likely have popped multiple nested awaits in this case).
    • -> maybe an even easier way is to just go to d1 and add its synchronous stack, since it would include all those nested awaits already.
  • make sure to always add intermediate synchronous roots between any two roots d1 and d0 (not only the ones at the top (highest depth))
• ACG rendering
  • sometimes, ACG node width is too small, putting the error symbol onto a second line, and then gets cut off
  • fix multiple incoming chains: some lines are not showing (test w/ icse-2022-seip/basic1)
  • render CHAIN + FORK lines (https://jsfiddle.net/n2x5z9m8/)
  • remove brightness on lines (test w/ socket.io#1)
  • remove terminal node border (will fix terminal node designation later)
  • make sure, nodes have a bit of a border (test w/ sync2)
• AsyncGraph: remove call site and reduce box height if toolbar's loc is disabled
• bug: panzoom zoom buttons don't work anymore
• fix auto-focus: in CallStack (and probably also in CallGraph) -
  • when a node is "focused", it automatically pans to the right of the node, but it should actually always try to stay as far to the "left" as possible (without losing sight of the selected node itself)
  • also, do not pan, if node is already (mostly) in view
• AsyncGraph.resolvePositionData bugs out with __samplesInput__/async/sync-asyncReturn1.js
• analyze difference between producer_consumer_async and producer_consumer_promise
• Error button is gone - can we bring it back?
  • new error trace collection algorithm dp.util.getAllErrorTraces (for reference we are also using ValueRef.isError in Feature: Proper async stacktrace visualization + interaction #566)
    • Gather all "error traces"
      • Also add all traces of TraceType.ThrowArgument
      • Also, of all traces with !!error (that is one Pop trace per context that was interrupted by an error), select the bottom-most trace on the call graph (ignore all error traces whose context parents have an error trace).
      • NOTE: we need these for runtime errors that did not manifest in form of an actual error (e.g. syntax error etc.). We would not be able to record such an error object if there is no catch.
    • Sort by traceId
    • (Probably need one or two indexes for this, to make sure, we don't iterate over all traces every time we re-render or click the error button)
  • test with error{1,2}.js + add more samples as needed
    • also test with sequelize#2
  • when clicking the Error button, cycle through all such traces (possibly need to remember selectedErrorRefId in traceSelection?)
  • also render fire in any AsyncGraph node whose root has an error
• bug: cannot select traces inside of asynchronously executed inline callback functions
  • NOTE: the callback executes in a later root than the call trace
  • try almost any async sample with inline asynchronously called callbacks (incl. promises and inline then callbacks)
  • e.g. producer_consumer_async.js -> select node inside of tryConsume
• in AsyncTDNode's "root edges" node: when clicking a to edge, go to the to roots first trace (not to schedulerTraceId)
• assign getStaticContextColor to AsyncGraph nodes
• change Async Edges TD node to Async node with two children:
  • Async Edges in Root (${n})
    • add inline buttons: "go to root" and "go to scheduler trace" (basically always the same buttons as the one on the nodes in the AsyncCallGraph)
  • Scheduled by trace
    • Children: all Pre and Post updates of trace
      • for Post update: render contextLabel of rootId in description (if it has any; e.g. PostAwait would not have a meaningful label; but PostThen represent a function that has a name)
• ValueTDNode bugs:
  • renders array contents as undefined
  • renders array label as [object Object]
  • renders [undefined] instead of empty array in first DataNode of non-empty ArrayExpression (e.g. [1, 2, 3])
  • test on: arrays1 + arrays2 samples and also on deeply nested objects with arrays with objects with arrays etc.
• in askForRecoverPracticeSession, offer 3 options: Yes, Not now (ask again next time), No (log should be discarded and the user should not be asked again)
• PathwaysDataProvider.resolveVisitedStaticTracesIndex: If application not found, don't error out, but only raise a warning instead (if it does not mess with data integrity further down the line)
  • test by throwing (artificial?) error during load; make sure, its handled properly
• fix shallow-nested promises
  • Promise.resolve/reject returns a new Promise if argument is a promise of different type (e.g. PatchedPromise vs. native)
  • return promise from async function -> nested with async function's return promise
  • return promise from non-async function (e.g. hexo#4)
• new, simplified CHAIN model
• fix existing producer_consumer_* case studies
• make sure that every root is recorded, even if it does not have a POST event
  • i.e. register any root, even if no async events happend (or were not recorded)
• DataNode -> varAccess incorrect for skipped array values, e.g. [, , 1, 2, , 4] (test w/ arrays2)
• lodash bug: lodash suddenly observes Object.defineProperty being undefined (in sequelize#1)
• fix msgpack: cannot encode symbol (node_modules/notepack.io/lib/encode.js) (-> moved to fix notepack's encoding problems #570)
• fix: async error handling
  • properly clean-up Await context and deal with catch + finally blocks in async function after error
  • test
    • todomvc
    • stream1.js
    • sequelize (Database callback edges)
• fix resume

  Trace: resume received while already executing - not handled properly yet. Discarding executing stack.

• fix getPreviousPostOrResolveAsyncEventOfPromise: prevent infinite loop
  • NOTE: promise can nest itself: var p = Promise.resolve().then(() => p)
    • var p = (async function() { await 0; return p; })()
    • -> Node + Chrome both throw error: Uncaught (in promise) TypeError: Chaining cycle detected
• basic optimization: be able to run webpack with pw=.* (the default socket.io parser, it uses JSON.stringify which caps out at Node's max string length of 512MB)
  • [wont-fix] even with msgpack, it just straight up runs out of memory during encoding. The odd thing is that before encoding, it only uses 1/3 of the ram. might need further debugging of the notepack algorithm implementation.
  • use msgpack parser: https://socket.io/docs/v3/custom-parser/#The-msgpack-parser
  • partition queue buffer by run
• fix: schedulerTraceId button does not show up (webpack#1)
• visualize unassigned AsyncNodes
• fix all bugs, s.t. at least webpack (among one of the biggest and most complex runtime behaviors) can run
• new X() traces show Value as undefined
  • test with new1.js
• fix wrapValue <-> unwrapValue to properly deal with identity of patchedFunctions
• in TraceDetails -> Executions -> change "group by run" to "group by root"
• also make parent nodes clickable (go to trace (or begining of context/root) on click)
• Wrap RuntimeDataProvider._handleData into a progress bar
  • (because this can take a while with large applications)
• implement DataProviderUtil.getRealCalleeTrace (use bceTrace.data.specialCallType)
• DataProviderUtil.getCallArgTraces
• fix monkey patched calls argument linkage
• fix lodash and other libraries not being instrumented
• fix dynamic callback patching: do not patch new calls (for now)
• fix caching sourceRoot + flushCache button
• improve callback async node CHAINing of event listeners, recursive callback calls etc.
• fix instrumentation for re-declaration of variables and their bindings
• fix es6 class instrumentations: computed methods, super calls, getters, setters etc.
• make cache configurable in @dbux/cli
• add a "flush cache" inline button on top of dbux-projects -> "Run" button
  • check if projectFolder/node_modules/.cache/@babel/register exists, and delete if it does. Report result back to user (using modal).
• CallGraph: when switching between sync and async mode, make sure, correct node is highlighted, followed (if follow enabled)(in PR Refactor AsyncGraph & add ThreadSelection(support multi apps) #560 )
• CallGraph: follow does not seem to highlight the active context anymore
• CallGraph: make sure, there is only one set of scrollbars

Future Work

• Initializing dbux-project progress bar needs to be cancellable
  • Sometimes loading a big project can lead to long load times
  • Once cancelled, it should:
    • stop loading
    • if necessary, undo previous loading progress
    • Won't try to load it again when re-starting VSCode

[Domiii]

@MiccWan 2 more small issues with value rendering