Support comments in JSON configuration (aka JSONC)

[Avasam]

Problem

If my JSON in contains_json contains comments, the parser will fail.
Example:

[".vscode/extensions.json".contains_json]
"unwantedRecommendations" = """
  [
    // VSCode has implemented an optimized version
    "coenraads.bracket-pair-colorizer",
    "coenraads.bracket-pair-colorizer-2",
    // Lots of conflicts
    "esbenp.prettier-vscode",
    // Replaced by ESLint
    "eg2.tslint",
    "ms-vscode.vscode-typescript-tslint-plugin",
    // Obsoleted by Pylance
    "ms-pyright.pyright",
    // Not configurable per workspace, tends to conflict with other linters
    // Use eslint-plugin-sonarjs for JS/TS projects
    "sonarsource.sonarlint-vscode"
  ]
"""
"[python]" = """
  {
    "editor.tabSize": 4,
    "editor.rulers": [
      72, // PEP8-17 docstrings
      // 79, // PEP8-17 default max
      // 88, // Black default
      99, // PEP8-17 acceptable max
      120 // Our hard rule
    ]
  }
"""

Results in

".vscode/extensions.json".contains_json.unwantedRecommendations.value: Invalid JSON (json.decoder.JSONDecodeError: Expecting value: line 2 column 5 (char 8))
".vscode/settings.json".contains_json.[python].value: Invalid JSON (json.decoder.JSONDecodeError: Expecting value: line 4 column 11 (char 62))

Possible solution

Support parsing JSON with Comments (aka JSONC)
Note that this is different than #406 . I am not asking for the comments to be kept when using nitpick fix, I just want the parser to at least ignore comments and not fail.

For more information, see the CONTRIBUTING guide.

[Avasam]

Add to this that it also crashes when trying to check JSON files with comments:

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python39\Scripts\nitpick.exe\__main__.py", line 7, in <module>
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1130, in __call__        
    return self.main(*args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Program Files\Python39\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\click\decorators.py", line 26, in new_func    
    return f(get_current_context(), *args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\nitpick\cli.py", line 116, in check
    common_fix_or_check(context, verbose, files, True)
  File "C:\Program Files\Python39\lib\site-packages\nitpick\cli.py", line 82, in common_fix_or_check
    for fuss in nit.run(*files, autofix=not check_only):
  File "C:\Program Files\Python39\lib\site-packages\nitpick\core.py", line 63, in run
    yield from chain(
  File "C:\Program Files\Python39\lib\site-packages\nitpick\core.py", line 118, in enforce_style  
    yield from plugin_class(info, config_dict, autofix).entry_point()
  File "C:\Program Files\Python39\lib\site-packages\nitpick\plugins\base.py", line 106, in entry_point
    yield from self._enforce_file_configuration()
  File "C:\Program Files\Python39\lib\site-packages\nitpick\plugins\base.py", line 112, in _enforce_file_configuration
    yield from self.enforce_rules()
  File "C:\Program Files\Python39\lib\site-packages\nitpick\plugins\json.py", line 48, in enforce_rules
    comparison = Comparison(json_doc, self.expected_dict_from_contains_keys(), self.special_config)()
  File "C:\Program Files\Python39\lib\site-packages\nitpick\blender.py", line 294, in __init__    
    self.flat_actual = flatten_quotes(actual.as_object)
  File "C:\Program Files\Python39\lib\site-packages\nitpick\blender.py", line 456, in as_object   
    self.load()
  File "C:\Program Files\Python39\lib\site-packages\nitpick\blender.py", line 672, in load        
    self._object = flatten_quotes(json.loads(self._string))
  File "C:\Program Files\Python39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Python39\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\Python39\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Files I'm trying to check:
https://github.com/Avasam/Auto-Split/tree/nitpick/.vscode
https://github.com/Avasam/speedrun.com_global_scoreboard_webapp/tree/nitpick/.vscode

[andreoliwa]

Thanks for the report. 🙏🏻

I am not asking for the comments to be kept when using nitpick fix, I just want the parser to at least ignore comments and not fail.

Got it. 👌🏻
Way easier than #406. 😅

---

The current JSON parser is just not ready to handle JSONC.

Maybe this could work: NickolaiBeloguzov/jsonc-parser: Parsing JSON with comments like it's our job. Oh wait, it is!

I quickly skimmed through these and they don't seem to handle JSONC either:

• ijl/orjson: Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy
• ultrajson/ultrajson: Ultra fast JSON decoder and encoder written in C with Python bindings

[Avasam]

Add to this that it also crashes when trying to check JSON files with comments: [...]

Sorry to bump on this. I still haven't found any open-source all-in-one solution like nitpick, yet this issue is preventing us from using it. I haven't seen any activity in a while so I'm wondering if nitpick is still actively being developed, if I should try forking it, or if I should keep looking for an alternative solution (currently considering custom NX generators and CI tasks that check for file changes)

[andreoliwa]

I haven't seen any activity in a while so I'm wondering if nitpick is still actively being developed, if I should try forking it

Hi! It's not abandoned, I will review and interact on pull requests when possible.
It's just that I didn't have time in the last 12 months or so due to personal reasons.

If you create a PR I will help push it through.
Thanks!