Reinstalling the new custom pack is not getting rendered on the Web UI

[Bindu-yadav8]

We have uninstalled one of the custom pack, and reinstalled it by using the command st2 pack install (File path of all the actions, rules, workflows).

By running this command, the files were installed on the K8 cluster. However, On the Stackstorm GUI, the workflows, actions are not rendering properly. The pack is still showing the older actions even after uninstalling the pack.

Uninstalled the custom pack:

Reinstalled the same custom pack:

We are not able to see the new actions on the K8 instance Web UI, and the edit parameters, executions options not visible on the Right Hand side of the Actions tab:

Is this a know issue? How to enable the options on the RHS of the Action Tab?

[Bindu-yadav8]

@armab Could you please look into this issue?

[cognifloyd]

You have not answered the question here:
#313 (comment)

This issue and #313 probably have the same cause. To help you we need to know how you have configured packs. In particular what are your values for st2.packs.volumes and st2.packs.images?

[Bindu-yadav8]

@cognifloyd @armab We have used Shared Volumes to install the custom packs as per the documentation.
https://docs.stackstorm.com/install/k8s_ha.html#custom-st2-packs

One of the API Calls is showing Response 404. We have not used any Custom APIs apart from default StackStorm APIs.

[Bindu-yadav8]

kapildev.misra@accenture.com - Please look into this

[Bindu-yadav8]

While trying to run the action from the Custom pack, it is giving error. However, the action from in-built pack "core" is getting executed.

st2 run cnas.trigger_notification is the action from custom pack cnas.

st2 run core.local cmd="echo 'OK'" - is working without any error

[cognifloyd]

Again: please show me your values. Feel free to replace any secrets with ****.

[amanda11]

Looking at the command you are running then you need to be careful about your use of '.

You look like you are trying to send through a parmetr of message with a json string of {'title':'..', ... }. But as your outer quote is the same type as the inner ones then it's going to get all a bit messed up.

You get away with it on the core.local as sending cmd='echo 'OK'' - is actually treated as the concatenaton of strings:

• 'echo '
• OK
• ''

Hence why in the parameters to the first call, it says the cmd passed was echo OK, rather than echo 'OK'.

So I'd suggest using "" as the outer quotes, so that the single quotes are kept.

[Bindu-yadav8]

Again: please show me your values. Feel free to replace any secrets with ****.

@cognifloyd Here is the vaule.yaml file for your reference below. We have used Shared Volumes for the custom pack. Initially, when we tried to install the custom pack CNAS, the pack and its underlying actions/workflows got registered. However, when we are trying to update the pack, (by uninstalling the previous pack and reinstalling the cnas pack). The files are not getting rendered properly on the UI, it still showing the older files. You can see the snapshots above

[Bindu-yadav8]

Looking at the command you are running then you need to be careful about your use of '.

You look like you are trying to send through a parmetr of message with a json string of {'title':'..', ... }. But as your outer quote is the same type as the inner ones then it's going to get all a bit messed up.

You get away with it on the core.local as sending cmd='echo 'OK'' - is actually treated as the concatenaton of strings:

• 'echo '
• OK
• ''

Hence why in the parameters to the first call, it says the cmd passed was echo OK, rather than echo 'OK'.

So I'd suggest using "" as the outer quotes, so that the single quotes are kept.

@amanda11 Thanks for the suggestion. Yes, I gave the parameter with double quotes and tried executing the action. But it failed.

[cognifloyd]

Ok, so you are using persistentVolumeClaims for the packs and virtualenvs volumes. Could you show the values for the virtualenvs volume and the configs volume as well?

Can also share your persistentVolume resource definition? (You would have created this outside the chart)

[Bindu-yadav8]

@cognifloyd Here is the values for the virtualenvs volume and the configs volume:

Also, the persistentVolume resource definition outside the chart:

[cognifloyd]

Access Mode: RWO is probably the source of your issues.

RWO, or ReadWriteOnce: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes

the volume can be mounted as read-write by a single node. ReadWriteOnce access mode still can allow multiple pods to access the volume when the pods are running on the same node.

Are all of you pods on the same node? If not, then only the pods that are on the node with the read/write mount will be able to successfully modify the files. Plus, the read/write mount for virtualenvs and packs could be on different nodes, leading to partially successful pack installs.

At a minimum, you need a PVC+PV with ReadWriteMany access mode.

[arm4b]

Good find @cognifloyd 💯

While https://github.com/StackStorm/stackstorm-k8s#method-2-shared-volumes mentions the use of NFS shares as an example (that provide RWX capabilities under the hood), looks like it would make sense to mention explicitly the ReadWriteMany requirements in the Readme for this type of sharing the pack content.

@Bindu-yadav8 could you please create a PR against the https://github.com/StackStorm/stackstorm-k8s#method-2-shared-volumes and mention that somewhere? It would definitely help others.

[Bindu-yadav8]

@cognifloyd @armab We tried to delete the existing PVC in order to reapply the changes, but the status of PVC is in "terminating" state as shown below:

We also tried to check for volume attachments and tried to patch the PVC considering it a fix for the issue but this didn't work.