Is it possible to load data from filepaths within KCL?

[polarathene]

This may be a documentation issue. I'd like to source data from a variety files and merge that data into a dict.

I've seen:

• Embedded YAML/JSON decoding in .k files.
• Loading KCL files (filesystem hierarchy gets in the way, requiring workarounds)
• Providing dynamic data upfront via YAML to the CLI (viable, but not as useful, requires pre-processing).

Below I share:

• A use-case scenario
• An existing approach with yq all in the CLI + YAML data.
• An attempt adapting that to KCL
• Feedback on the experience, mostly about potential documentation improvements.

From what I understand KCL is focused on config generation (JSON / YAML output), but it seems a bit limited with support to import data sources. Perhaps this is not a suitable use for KCL and the concern with loading dynamic data from files should be handled prior to using KCL?

I also noted some examples that implied ENV variables, but this was just naming an attribute with env / _env and using option() to read a key from -D or -Y inputs. It'd seem like a useful feature to have access to ENV, but this can be worked around like using yq to generate YAML that can supply it to -Y option.

---

Recently I was looking into generating some config for Rspamd dkim_signing.conf which has a domains field/key, that can represent data as:

# Domain specific settings
domain {
  # Domain name is used as key
  example.com {
    # Private key path
    path = "/var/lib/rspamd/dkim/example.key";
    # Selector
    selector = "ds";
  }
}

or (A domain has selectors array property instead, with elements of the path + selector map/object):

domain {
  alpha.example.org {
    selectors [
     {
       path: ".../configs/dkim.key";
       selector: "dkim";
     },
     {
       path: ".../configs/dkim-eddsa.key";
       selector: "eddsa";
     }
   ]
 }
}

UCL can be represented as JSON

UCL can alternatively be represented in JSON, which is useful since there doesn't appear to be support for generating UCL configs from code/schemas.

Thus I could generate JSON like this:

  "domain": {
    "example.org": {
      "path": "/config/dkim/keys/example.org/mail-rsa/private.pem",
      "selector": "mail-rsa"
    },
    "example.test": {
      "selectors": [
        {
          "path": "/config/dkim/keys/example.test/mail-ec/private.pem",
          "selector": "mail-ec"
        },
        {
          "path": "/config/dkim/keys/example.test/mail-rsa/private.pem",
          "selector": "mail-rsa"
        }
      ]
    }
  }

YAML snippets to unified JSON config via yq

To accomplish that, when generating the DKIM keypair, I could add a rspamd.yaml snippet at the same location:

# Find all config snippets per domain + selector; a `rspamd.yaml` file beside associated PEM files:
# /config/dkim/keys/${DOMAIN}/${SELECTOR}/{private,public}.pem
find /config/dkim/keys -type f -name "*.yaml" -exec cat {} \;

Results in:

example.com:
  selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
example.com:
  selectors:
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa
example.org:
  selectors:
    - path: /config/dkim/keys/example.org/mail-rsa/private.pem
      selector: mail-rsa

With these YAML files, I can then collect and process them into a single YAML document with yq:

# Find the config files, pipe into xargs to spread as args into `yq`.
# yq will merge the individual documents, group them under `domain` map key, and output as JSON.
# An optional operation prior to final JSON output will find any single length `selectors` arrays and convert
# to the config representation without `selectors` array, instead a single `path` + `selector` object.
find /config/dkim/keys -type f -name "*.yaml" \
  | xargs yq -o json eval-all '. as $item ireduce ({}; . *+ $item ) | { "domain": . }
  | with(.domain[]; . |=  select(.selectors | length == 1) |= .selectors[0])'

That produces this YAML (with the equivalent JSON output as shown earlier):

domain:
  example.com:
    selectors:
      - path: /config/dkim/keys/example.com/mail-ec/private.pem
        selector: mail-ec
      - path: /config/dkim/keys/example.com/mail-rsa/private.pem
        selector: mail-rsa
  example.org:
    path: /config/dkim/keys/example.org/mail-rsa/private.pem
    selector: mail-rsa

KCL

I was curious to try KCL, so I tried to adapt this.

• The config wants the domain as the field name, which to no surprise a . is not valid in KCL identifiers. That can be post-processed later, so I just replaced . with _.
• I also ran into issues with attempting to merge the separate configs due to the filesystem hierarchy and immutability features of KCL. I explain this and my workaround with the kcl command below.

_domain |= {
  example_com: {
    selectors += [
      {
        path = "/config/dkim/keys/example.com/mail-ec/private.pem",
        selector = "mail-ec"
      }
    ]
  }
}

_domain |= {
  example_com: {
    selectors += [
      {
        path = "/config/dkim/keys/example.com/mail-rsa/private.pem",
        selector = "mail-rsa"
      }
    ]
  }
}

_domain |= {
  example_org: {
    selectors += [
      {
        path = "/config/dkim/keys/example.org/mail-rsa/private.pem",
        selector = "mail-rsa"
      }
    ]
  }
}

I thought I could pass a list of .k files and they'd merge, but it seems KCL is trying to be helpful with the module system due to different locations?

So my workaround was to use cat with file redirection (this combines all the files into a single .k input file passed to kcl):
 

$ kcl <(cat start.k example.com/mail-ec/rspamd.k example.com/mail-rsa/rspamd.k example.org/mail-rsa/rspamd.k end.k)

# Outputs this YAML
domain:
  example_com:
    selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa
  example_org:
    selectors:
    - path: /config/dkim/keys/example.org/mail-rsa/private.pem
      selector: mail-rsa

The start.k and end.k were part of another workaround.

• I wanted to merge/union the original snippets, however even with the cat trickery, the immutability prevented my original attempt of merging the common example_com key as they were seen as conflicting IIRC.
• This seemed to require defining a parent as _domain = {} (start.k) and using _domain |= with my actual inputs to append to it mutably, with domain = _domain (end.k) to finalize an output.

A better approach?

The DKIM domain snippets aren't too useful in KCL it'd seem. Perhaps a better approach would be to have KCL define the domains object (dict), and some way to bring in that data?

Looks like I'd need to generate the YAML document anyway to provide as input options to kcl? (like described here)

I didn't see any way for the .k files to import dynamic data.

• As noted, the multiple .k files CLI example doesn't work as the file locations on disk seems to prevent the ability to collect/merge multiple inputs into a single .k file (unless all files are siblings on disk).
• There are the examples for embedded YAML + JSON decoding, but I don't think there's any documentation for providing an array of filepaths to load and decode into .k schema / dict?
• Using top-level arg with YAML data to use with the option() function seems to be the only viable way?

Using option() approach, I could have a .k file process the input, but without being able to load that data in, it kind of needs to already be in a prepared format 🤷‍♂️

UPDATE: I managed to get this alternative approach working:

schema Selector:
   path: str
   selector: str

schema Selectors:
   selectors: [Selector]

type SelectorKind = Selector | Selectors
type Domains = { str:SelectorKind }

# This will ensure the input matches the above schema + types.
domain: Domains = option("domain")

# NOTE: `: Domains` could be omitted, along with the schema + types,
# but then KCL isn't really doing anything helpful 🤷‍♂️ 

Additional KCL to handle both Selector kinds (assuming all input is of schema Selectors):

# Below is optional processing to rewrite the structure,
# I'm not sure how to be more verbose with multi-line construction
# for iteration over the input data. The comprehension syntax is
# awkward the longer it gets, so I've used a separate function
# for better readability.

# Collapse single item Selectors into Selector kind:
# **v destructures props into schema
# https://kcl-lang.io/docs/reference/lang/tour/#dict
_into_selectorkind = lambda v: Selectors -> SelectorKind {
  v if len(v?.selectors) > 1 else Selector { **v.selectors[0] }
}

# Dict comprehension on `domain` dict input of {str:Selectors} into {str:SelectorKind}
# https://kcl-lang.io/docs/reference/lang/tour/#comprehension
domain: Domains = { str(k): _into_selectorkind(v) for k,v in option("domain") }

I found that rather frustrating to get working, jumping through the reference docs a fair bit. I'm aware that KCL was originally Python based, which would explain why this type of data manipulation reminds me of Python 😭 (I do not find it a good UX personally, I'm biased more towards Rust)

Only missing functionality now would be to load the YAML data from a list of filepaths, and handling the merging. Interestingly, the dict keys could use . this way, probably because I'm not ever interacting with those directly in KCL? (or a feature as a string key, although I'm not sure how I'm meant to use that within KCL EDIT: via bracket notation)

Input data via options.yaml:

kcl_options:
  - key: domain
    value:
      example.com:
        selectors:
          - path: /config/dkim/keys/example.com/mail-ec/private.pem
            selector: mail-ec
          - path: /config/dkim/keys/example.com/mail-rsa/private.pem
            selector: mail-rsa
      example.org:
        selectors:
          - path: /config/dkim/keys/example.org/mail-rsa/private.pem
            selector: mail-rsa

Applying it:

$ kcl main.k -Y options.yaml

domain:
  example.org:
    path: /config/dkim/keys/example.org/mail-rsa/private.pem
    selector: mail-rsa
  example.com:
    selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa

Feedback

You'll notice I used |=, I don't think the docs actually have any mention of this operator. Only some examples like | operator for merging/union.

This was a little confusing at first, I needed the : operator (also considered for union?), which looked useful due to the labels example. That handled merging the domain fields like example_com.

Then finally the += for concatenating the selectors array. Interestingly this works even with selectors being defined for the first time, while |= did not and required _domain = {} to be declared first.

Might be worth documenting |=? And better distinction between | and : operators 🤷‍♂️ I think I used them correctly, but looking over the docs I'm not quite sure how to explain their differences, they seem to serve a similar purpose for merging a value into an existing one?

[Peefy]

Make sense. I guess what you need is to write the configuration in blocks and merge the same fields? This is my personal solution:

schema Selector:
    path: str
    selector: str

schema Selectors:
    selectors: [Selector]

type SelectorKind = Selector | Selectors
schema Domains:
    [...str]: SelectorKind

domain: Domains {
    "example.com": {selectors += [{
        path = "/config/dkim/keys/example.com/mail-ec/private.pem"
        selector = "mail-ec"
    }]}
}
domain: Domains {
    "example.com": {selectors += [{
        path = "/config/dkim/keys/example.com/mail-rsa/private.pem"
        selector = "mail-rsa"
    }]}
}
domain: Domains {
    "example.org": {selectors += [{
        path = "/config/dkim/keys/example.org/mail-rsa/private.pem"
        selector = "mail-rsa"
    }]}
}

Of course, you can split the three domains into three files.

Besides, we are considering supporting reading data from KCL files, but the initial reason for not supporting IO capability was due to security concerns. Issue is here: #313

[polarathene]

Of course, you can split the three domains into three files.

That runs into the issue I mentioned earlier?

$ find /config/dkim/keys -type f -name "rspamd.k" | xargs kcl main.k

error[E3M38]: conflict kcl.mod file paths: {"/config/dkim/keys/example.org/mail-rsa": "/config/dkim/keys/example.org/mail-rsa", "/config/dkim/keys/example.com/mail-ec": "/config/dkim/keys/example.com/mail-ec", "/config": "/config", "/config/dkim/keys/example.com/mail-rsa": "/config/dkim/keys/example.com/mail-rsa"}

/config/main.k:

schema Selector:
    path: str
    selector: str

schema Selectors:
    selectors: [Selector]

type SelectorKind = Selector | Selectors
schema Domains:
    [...str]: SelectorKind

I would need to make a copy to a single directory for all files, no subdirectories?

Or the workaround I showed with <(cat $file_paths), to create a temporary file that concatenates the base with each domain snippet config?

$ kcl <(cat main.k $(find /config/dkim/keys -type f -name "rspamd.k"))

domain:
  example.com:
    selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa
  example.org:
    selectors:
    - path: /config/dkim/keys/example.org/mail-rsa/private.pem
      selector: mail-rsa

---

the initial reason for not supporting IO capability was due to security concerns.
Issue is here: #313

I understand that security concern 👍

You could use config to:

• Whitelist which .k file(s) have permission to access the filesystem
• Alternatively what locations in the filesystem are whitelisted? (read / write)
• Or passing in a list of files in the config (perhaps most appropriate?)

Generating a list of filepaths and passing that to the config could work, instead of having to preprocess by parsing the files into a single YAML to append to kcl_options config? (not as nice for updates either)

Globs would be nice, but at the same time restricting file access. As you might tell from the example discussed here, the content snippets live beside their keypair, malicious access to the private.pem key data would be bad indeed 😅

[Peefy]

Ping @zong-zhe, we can complete the document on the differences between the : and | operators.

[Peefy]

@polarathene May I ask what is your KCL version? I did not reproduce this issue in kcl v0.7.0-alpha.1

[polarathene]

$ docker run --rm kcllang/kcl kcl -v

Version: 0.6.0-20ab3eb4b9179219d6837a57f5d35286
Platform: x86_64-unknown-linux-gnu

[Peefy]

Thanks for @zong-zhe, he has fixed this issue, #819

[polarathene]

Verified, thanks! ❤️

Using the earlier attempt with base main.k file with separate rspamd.k files (to merge into main.k which provides the base schema + types).

# `--pull=always` is only relevant to ensure any local image of the same tag is updated if necessary:
# `bash -c` used due to pipe operator
$ docker run --rm --pull=always -v "/tmp/config:/config:ro" kcllang/kcl:main bash -c \
  'find /config/dkim/keys -type f -name "rspamd.k" | xargs kcl /config/main.k'

main: Pulling from kcllang/kcl
Digest: sha256:094150067c1c1079e320dc1f6a1c2ed68dcf92d09d1f428e8dd760c96f619ab6
Status: Image is up to date for kcllang/kcl:main

domain:
  example.com:
    selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa
  example.org:
    selectors:
    - path: /config/dkim/keys/example.org/mail-rsa/private.pem
      selector: mail-rsa

Only other improvement to the UX would be for a way to run kcl main.k with a config that permits glob merging (importing without module scope?) the .k files from /config/dkim/keys without any risk of reading / modifying the .pem files (which should already be viable technically with posix permissions + ownership).

---

UX issue

If I understand correctly, passing files in via CLI or via -Y / --setting should have worked the same?

kcl_cli_configs:
  files:
    - /config/dkim/keys/example.com/mail-ec/rspamd.k
    - /config/dkim/keys/example.com/mail-rsa/rspamd.k
    - /config/dkim/keys/example.org/mail-rsa/rspamd.k

# Blank output:
$ kcl /config/main.k --setting /config/kcl.yaml


# For some reason a value after `--setting` is included? (bug?)
# Docs show example of `-Y` followed by yaml then k files, then `-Y` with two yaml files:
# https://kcl-lang.io/docs/reference/lang/tour#arguments-with-setting-files
$ kcl --setting /config/kcl.yaml /config/main.k
EvaluationError
Failed to load '/config/main.k', invalid setting file format


# --verbose outputs, but otherwise blank:
$ kcl --verbose /config/main.k --setting /config/kcl.yaml
Version: 0.7.0-alpha.1-20ab3eb4b9179219d6837a57f5d35286
Platform: x86_64-unknown-linux-gnu
GitCommit: 09b8ffe5927d26df32c17d832d6cfa219f806ed0


# Blank, no verbose output (position of --verbose arg matters?):
$ kcl --setting /config/kcl.yaml --verbose /config/main.k
$ kcl /config/main.k --setting /config/kcl.yaml --verbose

Perhaps that's not the expected behaviour of --verbose, it seems to match kcl --version when it did output anything extra 🤷‍♂️

---

If I ensure /config/main.k is also added into the settings yaml, this works:

kcl_cli_configs:
  files:
    - /config/main.k
    - /config/dkim/keys/example.com/mail-ec/rspamd.k
    - /config/dkim/keys/example.com/mail-rsa/rspamd.k
    - /config/dkim/keys/example.org/mail-rsa/rspamd.k

# Now works without requiring `find` + `xargs`
$ kcl --verbose --setting /config/kcl.yaml

domain:
  example.com:
    selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa
  example.org:
    selectors:
    - path: /config/dkim/keys/example.org/mail-rsa/private.pem
      selector: mail-rsa

The kcl.yaml can be generated easily enough with:

# Find the `rspamd.k` files and output a YAML list (filepath prefixed with `- `) as input into `yq`:
$ find /config/dkim/keys -type f -name "rspamd.k" -exec printf '- %s\n' {} + \
  | yq '{ "kcl_cli_configs": { "files": . } }'

kcl_cli_configs:
  files:
    - /config/dkim/keys/example.com/mail-ec/rspamd.k
    - /config/dkim/keys/example.com/mail-rsa/rspamd.k
    - /config/dkim/keys/example.org/mail-rsa/rspamd.k

---

Cache is not configurable?

Another potential bug I caught while verifying is I mounted the config volume as read-only. My first attempt was to run kcl from that location but it failed:

$ kcl --verbose --setting /config/kcl.yaml
error[E3M38]: Read-only file system (os error 30)

I switched to /tmp and tried again (with absolute paths as seen above), and that was successful, so I checked what it was trying to write to succeed. It's the hidden folder .kclvm which contains a cache subdirectory.

I did not see a settings config or option to opt-out of cache or change the cache storage location. You may want to consider supporting that, and possibly handle that error message to better communicate why write access was required (at least for the attempt to write .kclvm).

Locked Github Action

This is a bit off-topic, but I noticed you're using a Github Action to lock a PR after merging? (eg: #819 ) What motivated that?

It would not only prevent any future engagement but any other issue or PR that references it will not get a back reference which affects traceability (a feature I often use when navigating Github issues and PRs to troubleshoot other projects and decisions).

[Peefy]

The latest kcl docker image is kcllang/kcl:main. Reference: https://github.com/kcl-lang/kcl/actions/workflows/daily-release.yaml cc @zong-zhe

[zong-zhe]

Hi @polarathene 😊

Appreciate the questions you've raised, as they are instrumental in our improvement. Here are some more detailed information, hope it can help you.

# Blank output:
$ kcl /config/main.k --setting /config/kcl.yaml


# For some reason a value after `--setting` is included? (bug?)
# Docs show example of `-Y` followed by yaml then k files, then `-Y` with two yaml files:
# https://kcl-lang.io/docs/reference/lang/tour#arguments-with-setting-files
$ kcl --setting /config/kcl.yaml /config/main.k
EvaluationError
Failed to load '/config/main.k', invalid setting file format

Firstly, the issue you pointed out is a mistake in our documentation https://kcl-lang.io/docs/reference/lang/tour#arguments-with-setting-files, which we will correct. The correct usage is :

$ kcl /config/main.k --setting /config/kcl.yaml

And the blank output is the expected output. Let me provide more detailed information.

The main reason you received a blank output from this command is because the behavior of this command is to filling in non-empty arguments, not only merge them.

kcl can accept many arguments through the command line, with *.k file paths being just one of them. These arguments can be written in the kcl.yaml file.

You can use many kcl.yaml and manually entered arguments at the same time.

If merging them, it could cause these arguments to interact with each other, leading to many unpredictable results.

Therefore, we have adopted the approach of filling in non-empty arguments.

That is, if you have manually entered a argument, and then later input the same argument in kcl.yaml, the manually entered argument will take effect, because this argument has been manually entered and it is not empty.

If you haven't entered a argument, then this argument is empty. If you then write this argument in kcl.yaml, the argument in kcl.yaml will take effect.

Taking the command you provided above as an example.

$ kcl /config/main.k --setting /config/kcl.yaml

the *k file path is also one of kcl's input arguments. You have manually entered the path /config/main.k.

And you have added the following path in kcl.yaml.

kcl_cli_configs:
  files:
    - /config/dkim/keys/example.com/mail-ec/rspamd.k
    - /config/dkim/keys/example.com/mail-rsa/rspamd.k
    - /config/dkim/keys/example.org/mail-rsa/rspamd.k

Because you have manually entered the path, the *.k file path argument is not empty, so the content in kcl.yaml cannot take effect. In other words, the following two commands are equivalent in your case.

# The file path is '/config/main.k', it is not empty, so the *.k file path in kcl.yaml cannot take effect
$ kcl /config/main.k --setting /config/kcl.yaml
$ kcl /config/main.k

I believe you have found the correct way to use them through your intelligence and thoughtful thinking. 👍

If I ensure /config/main.k is also added into the settings yaml, this works:

kcl_cli_configs:
  files:
    - /config/main.k
    - /config/dkim/keys/example.com/mail-ec/rspamd.k
    - /config/dkim/keys/example.com/mail-rsa/rspamd.k
    - /config/dkim/keys/example.org/mail-rsa/rspamd.k

# Now works without requiring `find` + `xargs`
$ kcl --verbose --setting /config/kcl.yaml

domain:
  example.com:
    selectors:
    - path: /config/dkim/keys/example.com/mail-ec/private.pem
      selector: mail-ec
    - path: /config/dkim/keys/example.com/mail-rsa/private.pem
      selector: mail-rsa
  example.org:
    selectors:
    - path: /config/dkim/keys/example.org/mail-rsa/private.pem
      selector: mail-rsa

The kcl.yaml can be generated easily enough with:

# Find the `rspamd.k` files and output a YAML list (filepath prefixed with `- `) as input into `yq`:
$ find /config/dkim/keys -type f -name "rspamd.k" -exec printf '- %s\n' {} + \
  | yq '{ "kcl_cli_configs": { "files": . } }'

kcl_cli_configs:
  files:
    - /config/dkim/keys/example.com/mail-ec/rspamd.k
    - /config/dkim/keys/example.com/mail-rsa/rspamd.k
    - /config/dkim/keys/example.org/mail-rsa/rspamd.k

[polarathene]

Firstly, the issue you pointed out is a mistake in our documentation https://kcl-lang.io/docs/reference/lang/tour#arguments-with-setting-files, which we will correct. The correct usage is

I think given the error, it was more unexpected that --setting takes a list of files, as opposed to using --setting / -Y each time per file, or a --settings that would imply more than one input.

I know some CLI cannot support repeating an option like that to collect multiple inputs, but it can be a UX issue when the CLI supports multiple inputs for different options/default.

Some CLI set an options first (if any), followed by a list of file inputs, and handle this issue with a -- delimiter. This way you could have --setting file_one file_two --another-option file_three file_four -- file_five file_six and because of the options and --, the inputs are not mistakenly assigned incorrectly.

---

And the blank output is the expected output. Let me provide more detailed information.

The main reason you received a blank output from this command is because the behavior of this command is to filling in non-empty arguments, not only merge them.

kcl can accept many arguments through the command line, with *.k file paths being just one of them. These arguments can be written in the kcl.yaml file.

You can use many kcl.yaml and manually entered arguments at the same time.

If merging them, it could cause these arguments to interact with each other, leading to many unpredictable results.

Therefore, we have adopted the approach of filling in non-empty arguments.

That is, if you have manually entered a argument, and then later input the same argument in kcl.yaml, the manually entered argument will take effect, because this argument has been manually entered and it is not empty.

This makes sense, but due to the blank output I just assumed something was broken, not that main.k was being processed (which results in blank output).

At least with the --verbose option, I am expecting the UX to provide some sort of warning/hint such as the YAML setting equivalent is ignored because of CLI input (which I understand is how many CLI tools commonly handle config precedence with (left-most is highest priority) "CLI option" > "ENV vars" > "Settings file").

The main issue was how to troubleshoot the blank output:

• I don't recall a --files option supported by the kcl CLI, so the conflict did not occur to me.
• Trying to get more information about what was going on was attempted with --verbose but as shown, not helpful (--debug did nothing either).
• If I had my main.k file output something by default, perhaps it would have been more obvious what went wrong.

Perhaps if there was some way to verify what kcl settled on processing with (after resolving the different settings from CLI and YAML) it would be more apparent.

Docker Compose has command docker compose config that outputs the final compose.yaml after it resolves any imports/merges, which helps verify the configuration used before other docker compose commands continue. I find that useful for troubleshooting.

[zong-zhe]

Cache is not configurable?

Another potential bug I caught while verifying is I mounted the config volume as read-only. My first attempt was to run kcl from that location but it failed:

$ kcl --verbose --setting /config/kcl.yaml
error[E3M38]: Read-only file system (os error 30)

I switched to /tmp and tried again (with absolute paths as seen above), and that was successful, so I checked what it was trying to write to succeed. It's the hidden folder .kclvm which contains a cache subdirectory.

I did not see a settings config or option to opt-out of cache or change the cache storage location. You may want to consider supporting that, and possibly handle that error message to better communicate why write access was required (at least for the attempt to write .kclvm).

@polarathene Additionally, this is a very constructive and thoughtful suggestion. 👍 We have decided to accept this requirement, and it will be supported. 😊

I've created an issue to track this feature #835. Additionally, the implementation for this feature is not difficult. If you're interested in KCL, we warmly welcome you to participate in the development of KCL. 😃

[polarathene]

We have decided to accept this requirement, and it will be supported. 😊

That's great to know, thanks! ❤️

If you're interested in KCL, we warmly welcome you to participate in the development of KCL. 😃

Thanks, but unfortunately I do not have enough time to spare with existing open-source commitments 😭

---

I am impressed with KCL. My main complaints are only:

• More advanced features only have their Pythonic way supported (eg: list comprehension vs Rust iterators/collect or regular for loop).
• Lack of proper ENV access (requires -D args or pre-process to generate YAML), similarly file access (security sandbox by default is fine) but that matters less as KCL is more config generation oriented.
• There is a goal to release more modules, and the docs link to a Github packages page for this, but there is a lack of context / docs to understand them at a glance. At some point, it would be helpful for those to more accessible to browse/filter and communicate briefly the value each offers.

[zong-zhe]

@polarathene And sorry ❤️ , we did not notice the PR was locked. It gets locked automatically upon close.

Locked Github Action

This is a bit off-topic, but I noticed you're using a Github Action to lock a PR after merging? (eg: #819 ) What motivated that?

It would not only prevent any future engagement but any other issue or PR that references it will not get a back reference which affects traceability (a feature I often use when navigating Github issues and PRs to troubleshoot other projects and decisions).

[polarathene]

It gets locked automatically upon close.

Yes I know that it does. I just don't understand how that helps your project 😅

Maybe misconfiguration and not intended.