How python debugging works
This page talks about how the python extension debugs a local python script from the point of view of an extension developer. It's being discussed here because it's really the basis for the rest of debugging in the Jupyter extension.
If you want to know how to use the python debugger, go here
(Borrowed from https://code.visualstudio.com/api/extension-guides/debugger-extension)
In order to hook into that workflow, the Python extension calls registerDebugAdapterDescriptorFactory with a type of python. (See here for a description of the registerDebugAdapterDescriptorFactory API)
This sets up VS code to call the Python extension whenever a debug launch of type python occurs.
This would look like so in a launch.json:
{
"name": "Python: Current File",
"type": "python", // Notice 'python' here. It corresponds to the registered factory
"request": "launch",
"program": "${file}",
"console": "integratedTerminal"
},On launch, VS code will use the launch.json entry descriptors and call into the DebugAdapterDescriptorFactory::createDebugAdapterDescriptor for the registered type.
The python extension would then return a structure describing an executable to run. That executable is a 'debug' server that is expected to communicate with VS code over stdio. That server handles 'debugging' the python file.
The structure describing the server would look something like so:
{
executable: "c:\\users\\rich\\miniconda\\envs\\myenv\\scripts\\python.exe"
args: [
"'c:\\Users\\rich\\.vscode\\extensions\\ms-python.python-2022.10.1\\pythonFiles\\lib\\python\\debugpy\\launcher'",
"57624",
"--",
"foo.py"
]
}The 'debugpy' launcher creates a server that will sit and listen on stdin for messages.
What sort of messages does it listen to?
It uses the Debug Adapter Protocol (or DAP for short).
The DAP has messages for stuff like:
- Launching a new process
- Setting breakpoints
- Querying for variables, stack frames, and threads
- Send step into/step out/step over commands
- Responding with break states from breakpoints or stepping or exceptions
VS code sends DAP messages to the debugger to prepare it for debugging. They generally follow this order:
- Initialize - First message always. Indicates capabilities supported by the client and the server responds with which capabilities it allows. This might be something like whether or not it supports fetching variable values.
- Launch - Second message when launching. Contains the data in the launch.json. In debugpy's case, this would be python interpreter to use, the script to debug, arguments, environment, and a bunch of other flags.
- Set Breakpoints - Breakpoints are sent prior to a 'configurationDone' message so that the process has all of its breakpoints before the debugging actually starts
- Configuration Done - VS code sends this if the debugger supports it, but basically it means no more messages, go ahead and start debugging.
Debugging a process isn't really useful unless you can actually 'stop' at some point in the process and inspect state. This is handled by the server sending a stopped event. The stopped event indicates why it stopped (you stepped, you hit a breakpoint).
VS code then needs to fill out its variable windows, so it sends a bunch of requests:
- Threads to get the list of threads
- Stack Trace for the active thread to get the list of frames
- Scopes to get scopes for variables (this would be like current function, global, etc). Scope values contain a reference to retrieve variables for a scope.
- Variables to get the list of variables for the currently showing scope.
You can see in VS code what the result of those different requests look like:
In this image there were multiple scopes for the current stack frame.
Debugpy supports logging all of the DAP messages by setting a few environment variables:
- PYDEVD_DEBUG=1
- DEBUGPY_LOG_DIR=
- PYDEVD_DEBUG_FILE=
There's a batch script to set these before starting VS code here.
This generates logs with data like so:
D+00000.094: Client[1] --> {
"seq": 1,
"type": "request",
"command": "initialize",
"arguments": {
"clientID": "vscode",
"clientName": "Visual Studio Code",
"adapterID": "python",
"pathFormat": "path",
"linesStartAt1": true,
"columnsStartAt1": true,
"supportsVariableType": true,
"supportsVariablePaging": true,
"supportsRunInTerminalRequest": true,
"locale": "en-us",
"supportsProgressReporting": true,
"supportsInvalidatedEvent": true,
"supportsMemoryReferences": true
}
}
D+00000.094: /handling #1 request "initialize" from Client[1]/
Capabilities: {
"supportsVariableType": true,
"supportsVariablePaging": true,
"supportsRunInTerminalRequest": true,
"supportsMemoryReferences": true
}
D+00000.094: /handling #1 request "initialize" from Client[1]/
Expectations: {
"locale": "en-us",
"linesStartAt1": true,
"columnsStartAt1": true,
"pathFormat": "path"
}
D+00000.094: /handling #1 request "initialize" from Client[1]/
Client[1] <-- {
"seq": 3,
"type": "response",
"request_seq": 1,
"success": true,
"command": "initialize",
"body": {
"supportsCompletionsRequest": true,
"supportsConditionalBreakpoints": true,
"supportsConfigurationDoneRequest": true,
"supportsDebuggerProperties": true,
"supportsDelayedStackTraceLoading": true,
"supportsEvaluateForHovers": true,
"supportsExceptionInfoRequest": true,
"supportsExceptionOptions": true,
"supportsFunctionBreakpoints": true,
"supportsHitConditionalBreakpoints": true,
"supportsLogPoints": true,
"supportsModulesRequest": true,
"supportsSetExpression": true,
"supportsSetVariable": true,
"supportsValueFormattingOptions": true,
"supportsTerminateDebuggee": true,
"supportsGotoTargetsRequest": true,
"supportsClipboardContext": true,
"exceptionBreakpointFilters": [
{
"filter": "raised",
"label": "Raised Exceptions",
"default": false,
"description": "Break whenever any exception is raised."
},
{
"filter": "uncaught",
"label": "Uncaught Exceptions",
"default": true,
"description": "Break when the process is exiting due to unhandled exception."
},
{
"filter": "userUnhandled",
"label": "User Uncaught Exceptions",
"default": false,
"description": "Break when exception escapes into library code."
}
],
"supportsStepInTargetsRequest": true
}
}This is the debugpy.adapter log and shows the DAP messages and their responses.