feat: add support for pkl files #331

akhil-rasheed · 2025-10-07T07:11:09Z

Closes #141

Addresses following comments left on the original (abandoned) PR here #164

Moves PKL binary validation to the pkl validator
Ensures we don't run into race conditions during testing by using a mutex lock
Adds a more complete example for good.pkl
Alphabetises list of supported config filetypes

The existing test for ignoring PKL files when the binary is missing was not safe for parallel execution, as it modified a global function variable, creating a data race condition. This commit resolves the issue by refactoring the validation logic: 1. The responsibility for checking for the pkl binary is moved from the CLI into the `PklValidator`. 2. The validator now returns a specific `ErrSkipped` error if the binary is not found. 3. A thread-safe `SetPklBinaryChecker` function is introduced, allowing tests to safely mock the check. 4. The test is updated to use this new thread-safe mechanism.

…fig filetypes

- Removes a redeclared isPklBinaryPresent function that was causing a build failure. - Updates the test logic to check for the new ErrSkipped sentinel error, aligning it with the recent refactoring. - Removes an unused import.

- Fixes a build failure in the filetype package by using the correct `tools` import. - Updates the `good.pkl` test fixture with valid syntax so that tests pass when the pkl binary is installed.

kehoecj · 2025-10-07T14:17:53Z

@akhil-rasheed Thanks for the PR! Please address the issues identified in the unit and mega linter jobs.

Refactors the pkl validator to improve testability and adds tests for uncovered code paths. - Modifies the `PklValidator` to allow injecting an evaluator factory, making it easier to test error conditions. - Adds a test to verify that `ErrSkipped` is returned when the `pkl` binary is not found. - Adds a test to verify that an error is returned when the pkl evaluator fails to be created.

akhil-rasheed · 2025-10-08T11:32:19Z

I've addressed the coverage issues and some whitespace in the workflow file which was causing the yaml linter to fail @kehoecj

kehoecj · 2025-10-08T13:21:11Z

I've addressed the coverage issues and some whitespace in the workflow file which was causing the yaml linter to fail @kehoecj

Still seeing at least one test failure: #331 (comment)

Removes `t.Parallel()` from tests that modify the global `isPklBinaryPresent` variable. This prevents potential race conditions between tests that rely on the state of this variable.

akhil-rasheed · 2025-10-08T13:29:30Z

I've addressed the coverage issues and some whitespace in the workflow file which was causing the yaml linter to fail @kehoecj

Still seeing at least one test failure: #331 (comment)

Resolved

kehoecj · 2025-10-08T13:35:41Z

I've addressed the coverage issues and some whitespace in the workflow file which was causing the yaml linter to fail @kehoecj

Still seeing at least one test failure: #331 (comment)

Resolved

Can you take another look? I'm still seeing a failure in the pipeline

akhil-rasheed · 2025-10-08T13:55:18Z

Total Summary: 35 succeeded, 0 failed
--- FAIL: Test_TripleGroupOutput (0.11s)
=== RUN   Test_IncorrectSingleGroupOutput
--- PASS: Test_IncorrectSingleGroupOutput (0.10s)
=== RUN   Test_IncorrectDoubleGroupOutput
--- PASS: Test_IncorrectDoubleGroupOutput (0.06s)
=== RUN   Test_IncorrectTripleGroupOutput
--- PASS: Test_IncorrectTripleGroupOutput (0.07s)
FAIL
coverage: 93.2% of statements
FAIL	github.com/Boeing/config-file-validator/pkg/cli	0.781s
?   	github.com/Boeing/config-file-validator/pkg/filetype	[no test files]

This is on the CI, I assume this is where it's failing, but the same test passes for me locally

Total Summary: 35 succeeded, 0 failed
--- PASS: Test_TripleGroupOutput (0.07s)
=== RUN   Test_IncorrectSingleGroupOutput
--- PASS: Test_IncorrectSingleGroupOutput (0.06s)
=== RUN   Test_IncorrectDoubleGroupOutput
--- PASS: Test_IncorrectDoubleGroupOutput (0.07s)
=== RUN   Test_IncorrectTripleGroupOutput
--- PASS: Test_IncorrectTripleGroupOutput (0.06s)
PASS
ok  	github.com/Boeing/config-file-validator/pkg/cli	1.423s

Am I just not identifying the failure correctly?

kehoecj · 2025-10-08T14:01:33Z

Total Summary: 35 succeeded, 0 failed
--- FAIL: Test_TripleGroupOutput (0.11s)
=== RUN   Test_IncorrectSingleGroupOutput
--- PASS: Test_IncorrectSingleGroupOutput (0.10s)
=== RUN   Test_IncorrectDoubleGroupOutput
--- PASS: Test_IncorrectDoubleGroupOutput (0.06s)
=== RUN   Test_IncorrectTripleGroupOutput
--- PASS: Test_IncorrectTripleGroupOutput (0.07s)
FAIL
coverage: 93.2% of statements
FAIL	github.com/Boeing/config-file-validator/pkg/cli	0.781s
?   	github.com/Boeing/config-file-validator/pkg/filetype	[no test files]

This is on the CI, I assume this is where it's failing, but the same test passes for me locally

Total Summary: 35 succeeded, 0 failed
--- PASS: Test_TripleGroupOutput (0.07s)
=== RUN   Test_IncorrectSingleGroupOutput
--- PASS: Test_IncorrectSingleGroupOutput (0.06s)
=== RUN   Test_IncorrectDoubleGroupOutput
--- PASS: Test_IncorrectDoubleGroupOutput (0.07s)
=== RUN   Test_IncorrectTripleGroupOutput
--- PASS: Test_IncorrectTripleGroupOutput (0.06s)
PASS
ok  	github.com/Boeing/config-file-validator/pkg/cli	1.423s

Am I just not identifying the failure correctly?

That appears to be the correct failure - let me see if it passes locally for me as well. There may be a timing issue that is present in the pipeline causing it to fail

pkg/validator/pkl.go

akhil-rasheed · 2025-10-09T09:02:24Z

I have made the changes that @ccoVeille requested. @kehoecj were you able to replicate the pipeline issue locally?

pkg/cli/cli_test.go

pkg/validator/pkl.go

pkg/validator/validator_test.go

Co-authored-by: ccoVeille <[email protected]>

kehoecj

When the PKL executable is not installed, a runtime error is generated:

> .\validator.exe .\test\                         
Warning: 'pkl' binary not found, file C:\Users\se456c\src\github.com\prs\pkl\config-file-validator\test\fixtures\good.pkl will be ignored.
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7ff78ccb90f0]

goroutine 22 [running]:
github.com/apple/pkl-go/pkl.(*execEvaluator).deinit(0xc000206140)
        C:/Users/se456c/go/pkg/mod/github.com/apple/[email protected]/pkl/evaluator_manager_exec.go:214 +0x70        
github.com/apple/pkl-go/pkl.(*evaluatorManager).closeErr(0xc000193bc0, {0x7ff78cee41c0?, 0xc000116e80?})
        C:/Users/se456c/go/pkg/mod/github.com/apple/[email protected]/pkl/evaluator_manager.go:308 +0x82
github.com/apple/pkl-go/pkl.(*evaluatorManager).listenForImplClose(0xc000193bc0)
        C:/Users/se456c/go/pkg/mod/github.com/apple/[email protected]/pkl/evaluator_manager.go:245 +0x46
created by github.com/apple/pkl-go/pkl.NewEvaluatorManagerWithCommand in goroutine 1
        C:/Users/se456c/go/pkg/mod/github.com/apple/[email protected]/pkl/evaluator_manager_exec.go:61 +0x256

kehoecj · 2025-10-13T13:55:09Z

.github/workflows/go.yml

        with:
          go-version: "1.25"

+      - name: Set up pkl


The unit tests shouldn't rely on PKL being present - that should be mocked so the tests are deterministic

Ok we can do this. But I'll have to extract out the pkl tests to their own function, deviating from the existing pattern of putting tuples in the testData array

That's fine - it's the first time we have to rely on the presence of a local binary

@akhil-rasheed Looking at the PKL Go documentation, I do not think the binary is required anymore. The PKL binary is only required when using code generation and it looks like you can load PKL into a struct without using Code Generation: https://pkl-lang.org/go/current/evaluation.html#without-code-generation. This may only apply to a pre-defined struct though which wouldn't work.

Please take a look and see if this is a viable path. This would greatly simplify the implementation.

kehoecj · 2025-10-13T14:51:45Z

I have made the changes that @ccoVeille requested. @kehoecj were you able to replicate the pipeline issue locally?

Tests are failing with the latest changes due to the runtime error

olunusib and others added 10 commits August 20, 2024 21:58

feat: add support for pkl files

395392c

feat: skip pkl files when binary isn't in PATH

b706fe9

Merge branch 'main' into feat/pkl-validation

80c436b

chore: use the new action tag in ref comment

1fb119b

fix: move PKL to alphabetically correct spot in list of supported con…

df3e3de

…fig filetypes

test: add fleshed out example for good.pkl

e33c522

Merge branch 'main' into feat/pkl-validation

4f54b22

fix(test): repair test suite after pkl validator refactor

a3f1ca6

- Removes a redeclared isPklBinaryPresent function that was causing a build failure. - Updates the test logic to check for the new ErrSkipped sentinel error, aligning it with the recent refactoring. - Removes an unused import.

fix: correct build and pkl test fixture

1b7e7cd

- Fixes a build failure in the filetype package by using the correct `tools` import. - Updates the `good.pkl` test fixture with valid syntax so that tests pass when the pkl binary is installed.

akhil-rasheed marked this pull request as ready for review October 7, 2025 07:38

kehoecj added the waiting-on-maintainer-review PR is waiting to be reviewed and functionally tested by the maintainers label Oct 7, 2025

kehoecj self-requested a review October 7, 2025 13:49

kehoecj added OSS Community Contribution Contributions from the OSS Community pr-action-requested PR is awaiting feedback from the submitting developer and removed waiting-on-maintainer-review PR is waiting to be reviewed and functionally tested by the maintainers labels Oct 7, 2025

Akhil Rasheed added 2 commits October 8, 2025 16:24

fix: remove whitespace in workflow file

e59a61c

akhil-rasheed force-pushed the feat/pkl-validation branch 2 times, most recently from ade80f0 to 309847e Compare October 8, 2025 11:57

fix(test): Prevent race conditions in pkl validator tests

8ffa41d

Removes `t.Parallel()` from tests that modify the global `isPklBinaryPresent` variable. This prevents potential race conditions between tests that rely on the state of this variable.

ccoVeille reviewed Oct 8, 2025

View reviewed changes

pkg/validator/pkl.go Outdated Show resolved Hide resolved

pkg/validator/pkl.go Outdated Show resolved Hide resolved

refactor: rename ErrSkipped to ErrPklSkipped

158c3c6

akhil-rasheed and others added 3 commits October 9, 2025 14:15

refactor: improve pkl binary detection

7fe75c1

revert: add back validator to .gitignore

5a7e6c0

Merge branch 'main' into feat/pkl-validation

b2e4e29

ccoVeille approved these changes Oct 9, 2025

View reviewed changes

pkg/cli/cli_test.go Show resolved Hide resolved

pkg/validator/pkl.go Outdated Show resolved Hide resolved

akhil-rasheed added 2 commits October 9, 2025 21:15

fix: remove whitespace

29461bc

fix: remove whitespace

8398985

ccoVeille reviewed Oct 10, 2025

View reviewed changes

pkg/validator/validator_test.go Outdated Show resolved Hide resolved

pkg/validator/validator_test.go Outdated Show resolved Hide resolved

refactor(suggestion): mark unused parameters

306e190

Co-authored-by: ccoVeille <[email protected]>

ccoVeille approved these changes Oct 11, 2025

View reviewed changes

kehoecj requested changes Oct 13, 2025

View reviewed changes

feat: add support for pkl files #331

Are you sure you want to change the base?

feat: add support for pkl files #331

Uh oh!

Conversation

akhil-rasheed commented Oct 7, 2025

Uh oh!

kehoecj commented Oct 7, 2025

Uh oh!

akhil-rasheed commented Oct 8, 2025

Uh oh!

kehoecj commented Oct 8, 2025

Uh oh!

akhil-rasheed commented Oct 8, 2025

Uh oh!

kehoecj commented Oct 8, 2025

Uh oh!

akhil-rasheed commented Oct 8, 2025

Uh oh!

kehoecj commented Oct 8, 2025

Uh oh!

Uh oh!

Uh oh!

akhil-rasheed commented Oct 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kehoecj left a comment

Choose a reason for hiding this comment

Uh oh!

kehoecj Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

akhil-rasheed Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

kehoecj Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

kehoecj Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kehoecj commented Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kehoecj Oct 14, 2025 •

edited

Loading