fortls crashes during initialization due to RecursionError

[tomoto]

Describe the bug

fortls crashes during initialization due to RecursionError when the workspace contains source files with deep nesting levels (about 200 nestings of IF~ENDIF). While it is acceptable to skip parsing such files, the bigger issue is fortls goes down entirely without any information to help me troubleshoot.

The output message:

["INFO" - 10:24:53 PM] Fortran Language Server
["INFO" - 10:24:53 PM] Initialising Language Server for workspace: /home/tomoto/dev/fortran/test1.f with command-line options: --enable_code_actions, --hover_signature, --use_signature_help, --nthreads=4, --notify_init, --incremental_sync
[WARN - 22:24:54] error handling request {'jsonrpc': '2.0', 'id': 0, 'method': 'initialize', 'params': {'processId': 1682974, 'clientInfo': {'name': 'Visual Studio Code', 'version': '1.95.3'}, 'locale': 'en', 'rootPath': '/home/tomoto/dev/fortran', 'rootUri': 'file:///home/tomoto/dev/fortran', 'capabilities': {'workspace': {'applyEdit': True, 'workspaceEdit': {'documentChanges': True, 'resourceOperations': ['create', 'rename', 'delete'], 'failureHandling': 'textOnlyTransactional', 'normalizesLineEndings': True, 'changeAnnotationSupport': {'groupsOnLabel': True}}, 'configuration': True, 'didChangeWatchedFiles': {'dynamicRegistration': True, 'relativePatternSupport': True}, 'symbol': {'dynamicRegistration': True, 'symbolKind': {'valueSet': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]}, 'tagSupport': {'valueSet': [1]}, 'resolveSupport': {'properties': ['location.range']}}, 'codeLens': {'refreshSupport': True}, 'executeCommand': {'dynamicRegistration': True}, 'didChangeConfiguration': {'dynamicRegistration': True}, 'semanticTokens': {'refreshSupport': True}, 'fileOperations': {'dynamicRegistration': True, 'didCreate': True, 'didRename': True, 'didDelete': True, 'willCreate': True, 'willRename': True, 'willDelete': True}, 'inlineValue': {'refreshSupport': True}, 'inlayHint': {'refreshSupport': True}, 'diagnostics': {'refreshSupport': True}}, 'textDocument': {'publishDiagnostics': {'relatedInformation': True, 'versionSupport': False, 'tagSupport': {'valueSet': [1, 2]}, 'codeDescriptionSupport': True, 'dataSupport': True}, 'synchronization': {'dynamicRegistration': True, 'willSave': True, 'willSaveWaitUntil': True, 'didSave': True}, 'completion': {'dynamicRegistration': True, 'contextSupport': True, 'completionItem': {'snippetSupport': True, 'commitCharactersSupport': True, 'documentationFormat': ['markdown', 'plaintext'], 'deprecatedSupport': True, 'preselectSupport': True, 'tagSupport': {'valueSet': [1]}, 'insertReplaceSupport': True, 'resolveSupport': {'properties': ['documentation', 'detail', 'additionalTextEdits']}, 'insertTextModeSupport': {'valueSet': [1, 2]}, 'labelDetailsSupport': True}, 'insertTextMode': 2, 'completionItemKind': {'valueSet': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]}, 'completionList': {'itemDefaults': ['commitCharacters', 'editRange', 'insertTextFormat', 'insertTextMode']}}, 'hover': {'dynamicRegistration': True, 'contentFormat': ['markdown', 'plaintext']}, 'signatureHelp': {'dynamicRegistration': True, 'signatureInformation': {'documentationFormat': ['markdown', 'plaintext'], 'parameterInformation': {'labelOffsetSupport': True}, 'activeParameterSupport': True}, 'contextSupport': True}, 'definition': {'dynamicRegistration': True, 'linkSupport': True}, 'references': {'dynamicRegistration': True}, 'documentHighlight': {'dynamicRegistration': True}, 'documentSymbol': {'dynamicRegistration': True, 'symbolKind': {'valueSet': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]}, 'hierarchicalDocumentSymbolSupport': True, 'tagSupport': {'valueSet': [1]}, 'labelSupport': True}, 'codeAction': {'dynamicRegistration': True, 'isPreferredSupport': True, 'disabledSupport': True, 'dataSupport': True, 'resolveSupport': {'properties': ['edit']}, 'codeActionLiteralSupport': {'codeActionKind': {'valueSet': ['', 'quickfix', 'refactor', 'refactor.extract', 'refactor.inline', 'refactor.rewrite', 'source', 'source.organizeImports']}}, 'honorsChangeAnnotations': False}, 'codeLens': {'dynamicRegistration': True}, 'formatting': {'dynamicRegistration': True}, 'rangeFormatting': {'dynamicRegistration': True}, 'onTypeFormatting': {'dynamicRegistration': True}, 'rename': {'dynamicRegistration': True, 'prepareSupport': True, 'prepareSupportDefaultBehavior': 1, 'honorsChangeAnnotations': True}, 'documentLink': {'dynamicRegistration': True, 'tooltipSupport': True}, 'typeDefinition': {'dynamicRegistration': True, 'linkSupport': True}, 'implementation': {'dynamicRegistration': True, 'linkSupport': True}, 'colorProvider': {'dynamicRegistration': True}, 'foldingRange': {'dynamicRegistration': True, 'rangeLimit': 5000, 'lineFoldingOnly': True, 'foldingRangeKind': {'valueSet': ['comment', 'imports', 'region']}, 'foldingRange': {'collapsedText': False}}, 'declaration': {'dynamicRegistration': True, 'linkSupport': True}, 'selectionRange': {'dynamicRegistration': True}, 'callHierarchy': {'dynamicRegistration': True}, 'semanticTokens': {'dynamicRegistration': True, 'tokenTypes': ['namespace', 'type', 'class', 'enum', 'interface', 'struct', 'typeParameter', 'parameter', 'variable', 'property', 'enumMember', 'event', 'function', 'method', 'macro', 'keyword', 'modifier', 'comment', 'string', 'number', 'regexp', 'operator', 'decorator'], 'tokenModifiers': ['declaration', 'definition', 'readonly', 'static', 'deprecated', 'abstract', 'async', 'modification', 'documentation', 'defaultLibrary'], 'formats': ['relative'], 'requests': {'range': True, 'full': {'delta': True}}, 'multilineTokenSupport': False, 'overlappingTokenSupport': False, 'serverCancelSupport': True, 'augmentsSyntaxTokens': True}, 'linkedEditingRange': {'dynamicRegistration': True}, 'typeHierarchy': {'dynamicRegistration': True}, 'inlineValue': {'dynamicRegistration': True}, 'inlayHint': {'dynamicRegistration': True, 'resolveSupport': {'properties': ['tooltip', 'textEdits', 'label.tooltip', 'label.location', 'label.command']}}, 'diagnostic': {'dynamicRegistration': True, 'relatedDocumentSupport': False}}, 'window': {'showMessage': {'messageActionItem': {'additionalPropertiesSupport': True}}, 'showDocument': {'support': True}, 'workDoneProgress': True}, 'general': {'staleRequestSupport': {'cancel': True, 'retryOnContentModified': ['textDocument/semanticTokens/full', 'textDocument/semanticTokens/range', 'textDocument/semanticTokens/full/delta']}, 'regularExpressions': {'engine': 'ECMAScript', 'version': 'ES2020'}, 'markdown': {'parser': 'marked', 'version': '1.1.0'}, 'positionEncodings': ['utf-16']}, 'notebookDocument': {'synchronization': {'dynamicRegistration': True, 'executionSummarySupport': True}}}, 'trace': 'off', 'workspaceFolders': [{'uri': 'file:///home/tomoto/dev/fortran', 'name': 'fortran'}]}}
Traceback (most recent call last):
  File "/home/tomoto/dev/fortls/fortls/langserver.py", line 173, in handle
    resp = handler(request)
  File "/home/tomoto/dev/fortls/fortls/langserver.py", line 213, in serve_initialize
    self.workspace_init()
  File "/home/tomoto/dev/fortls/fortls/langserver.py", line 1492, in workspace_init
    result_obj = result.get()
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 774, in get
    raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<fortls.parsers.internal.parser.FortranFile object at 0x7fc3f96db5e0>'. Reason: 'RecursionError('maximum recursion depth exceeded while pickling an object')'
[Error - 10:24:54 PM] Server initialization failed.
  Message: Error sending result: '<fortls.parsers.internal.parser.FortranFile object at 0x7fc3f96db5e0>'. Reason: 'RecursionError('maximum recursion depth exceeded while pickling an object')'
  Code: -32603 
[object Object]
[Error - 10:24:54 PM] Fortran Language Server client: couldn't create connection to server.
  Message: Error sending result: '<fortls.parsers.internal.parser.FortranFile object at 0x7fc3f96db5e0>'. Reason: 'RecursionError('maximum recursion depth exceeded while pickling an object')'
  Code: -32603 
[object Object]

How it looks like in the VSCode GUI:

As far as I can tell, the RecursionError occurs while pickling (serialization/deserialization) the AST to pass the parse result from the worker process to the main process. It can happen when the AST has a deep nesting of scopes, and possibly for many other reasons.

To Reproduce

1. Place a source file with deep nesting levels in the workspace. Use the Python script below to create such a file:

   levels = 200
   print("      PROGRAM NESTED_IF")
   print("      IF (.TRUE.) THEN\n" * levels, end="")
   print("      END IF\n" * levels, end="")
   print("      END PROGRAM NESTED_IF")

2. Open the workspace and start fortls.
3. Observe "Server initialization failed" error and you cannot use any function of fortls. You see no information about what caused it.

Expected behavior

Fortls should gracefully ignore such files instead of crashing entirely. Besides, user should be informed about which files caused the issue, so they can choose to remove those files from the workspace if desired.

I understand that such deeply nested source files are very rare, possibly machine-generated. I don't necessarily expect fortls to process them, as they may require additional resources and time that may not be worthwhile. I'm fine as long as fortls continues to function without being impacted by such exotic files.

Screenshots & Animations

Nothing to add.

Setup information (please complete the following information):

• OS: Linux
• Python Version: 3.10.12
• fortls Version: 3.1.3
• Code editor used: VS Code
• the Fortran extension for the code editor and its version: Modern Fortran v3.2.0

Configuration information (please complete the following information):

Nothing special.

Additional context

I will post a PR of the suggested fix. Thank you very much!

[tomoto]

@gnikit I posted the PR at #446. Appreciate your time!