Cannot write to /opt/jfrog/router/app/router/router.pid: Permission denied

[oniabifo]

Artifactory installed from official helm chart is not running due to a permission denied issue

Is this a request for help?:
Yes

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT

Version of Helm and Kubernetes:
107.77.3

Which chart:
artifactory

Which product license (Enterprise/Pro/oss):

JFrog support reference (if already raised with support team):

What happened:
Artifactory is not running on kubernetes

What you expected to happen:
Artifactory UP and running.

How to reproduce it (as minimally and precisely as possible):
install helm chart with mentioned version and default values.

Anything else we need to know:
Our kubernetes clusters prevent apps from writing on filesystem

[chukka]

@oniabifo Thanks for reaching out ! we need more details to reproduce . Can you share below information ?

1. kubernestes flavour and version ( is it openshift ,if yes , what is the version used)
2. helm command used
3. complete console logs

[oniabifo]

Hello @chukka Thanks for reaching out

kubernestes flavour and version
v1.21.5 and it is on AWS

helm command used
This is the helm command used

helm upgrade --install artifactory jfrog/artifactory \
      --set nginx.service.ssloffload=$SSL_OFFLOAD \
      --set nginx.tlsSecretName=tls-ingress \
      --set artifactory.masterKey=$MASTER_KEY \
      --set artifactory.joinKey=$JOIN_KEY \
      --set artifactory.license.secret=artifactory-license \
      --set artifactory.persistence.enabled=false \
      --set artifactory.nginx.enabled=false \
      --set artifactory.uid=1000 \
      --set artifactory.gid=1000 \
      --set artifactory.license.dataKey=$ARTIFACTORY_DATA_KEY $NAMESPACE_COMMAND

complete console logs

This is the output of kubectl logs podname
Testing directory /var/opt/jfrog/router has read/write permissions for user id 1000
Permissions for /var/opt/jfrog/router are good
Setting JF_SHARED_NODE_ID to artifactory-0
Setting JF_SHARED_NODE_IP to 10.98.136.72
Setting JF_SHARED_NODE_NAME to artifactory-0
Using default router's certificate and private key
======= Router PID 265
/opt/jfrog/router/app/bin/entrypoint-router.sh: line 112: /opt/jfrog/router/app/router/router.pid: Permission denied
runtime/cgo: pthread_create failed: Operation not permitted
SIGABRT: abort
PC=0x7f00665a054c m=0 sigcode=18446744073709551610

goroutine 0 [idle]:
runtime: g 0: unknown pc 0x7f00665a054c
stack: frame={sp:0x7ffccb238fa0, fp:0x0} stack=[0x7ffccaa3a520,0x7ffccb239530)
0x00007ffccb238ea0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238eb0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238ec0: 0x6e696c2d72657475 0x3436646d612d7875
0x00007ffccb238ed0: 0x454e524542554b00 0x565245535f534554
0x00007ffccb238ee0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238ef0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f00: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f10: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f20: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f30: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f40: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f50: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f60: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f70: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f80: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f90: 0x0000000000000000 0x00007f00665a053e
0x00007ffccb238fa0: <0x0000000000000000 0x0000000000000000
0x00007ffccb238fb0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238fc0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238fd0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238fe0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238ff0: 0x0000000000000000 0x0000000000000000
0x00007ffccb239000: 0x0000000000000000 0x0000000000000000
0x00007ffccb239010: 0x0000000000000000 0x0000000000000000
0x00007ffccb239020: 0x0000000000000000 0xf8b108eb38f38600
0x00007ffccb239030: 0x00007f00664fc740 0x0000000000000006
0x00007ffccb239040: 0x000000000856b2e0 0x0000000000000000
0x00007ffccb239050: 0x000000000729da00 0x00007f0066553d06
0x00007ffccb239060: 0x00007f00666fae90 0x00007f00665277f3
0x00007ffccb239070: 0x0000000000000020 0x0000000000000000
0x00007ffccb239080: 0x0000000000000000 0x0000000000000000
0x00007ffccb239090: 0x0000000000000000 0x0000000000000000
runtime: g 0: unknown pc 0x7f00665a054c
stack: frame={sp:0x7ffccb238fa0, fp:0x0} stack=[0x7ffccaa3a520,0x7ffccb239530)
0x00007ffccb238ea0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238eb0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238ec0: 0x6e696c2d72657475 0x3436646d612d7875
0x00007ffccb238ed0: 0x454e524542554b00 0x565245535f534554
0x00007ffccb238ee0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238ef0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f00: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f10: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f20: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f30: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f40: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f50: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f60: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f70: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f80: 0x0000000000000000 0x0000000000000000
0x00007ffccb238f90: 0x0000000000000000 0x00007f00665a053e
0x00007ffccb238fa0: <0x0000000000000000 0x0000000000000000
0x00007ffccb238fb0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238fc0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238fd0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238fe0: 0x0000000000000000 0x0000000000000000
0x00007ffccb238ff0: 0x0000000000000000 0x0000000000000000
0x00007ffccb239000: 0x0000000000000000 0x0000000000000000
0x00007ffccb239010: 0x0000000000000000 0x0000000000000000
0x00007ffccb239020: 0x0000000000000000 0xf8b108eb38f38600
0x00007ffccb239030: 0x00007f00664fc740 0x0000000000000006
0x00007ffccb239040: 0x000000000856b2e0 0x0000000000000000
0x00007ffccb239050: 0x000000000729da00 0x00007f0066553d06
0x00007ffccb239060: 0x00007f00666fae90 0x00007f00665277f3
0x00007ffccb239070: 0x0000000000000020 0x0000000000000000
0x00007ffccb239080: 0x0000000000000000 0x0000000000000000
0x00007ffccb239090: 0x0000000000000000 0x0000000000000000

goroutine 1 [running]:
runtime.systemstack_switch()
runtime/asm_amd64.s:474 +0x8 fp=0xc0000a6740 sp=0xc0000a6730 pc=0x5f0448
runtime.main()
runtime/proc.go:169 +0x6d fp=0xc0000a67e0 sp=0xc0000a6740 pc=0x5bd4cd
runtime.goexit()
runtime/asm_amd64.s:1650 +0x1 fp=0xc0000a67e8 sp=0xc0000a67e0 pc=0x5f2421

rax 0x0
rbx 0x7f00664fc740
rcx 0x7f00665a054c
rdx 0x6
rdi 0x109
rsi 0x109
rbp 0x109
rsp 0x7ffccb238fa0
r8 0x7ffccb239070
r9 0x7f00666b04e0
r10 0x8
r11 0x246
r12 0x6
r13 0x0
r14 0x729da00
r15 0x6
rip 0x7f00665a054c
rflags 0x246
cs 0x33
fs 0x0
gs 0x0

[chukka]

@oniabifo can you let us know the license that you are using Enterprise/Pro/oss ?

[oniabifo]

@chukka I am using the 30 days free trial from the jfrog website for this

[chukka]

@oniabifo I tried with below commands , it works with out any issues , can you try again with latest version 107.77.3

helm repo update jfrog
helm upgrade --install artifactory jfrog/artifactory --set artifactory.uid=1000,artifactory.gid=1000 --namespace test --create-namespace --version 107.77.3

[oniabifo]

@chukka still failing. Also noticed nginx isn't starting up

Using deprecated password for user _internal.
2024-02-01 14:58:17  [169 entrypoint-nginx.sh] Preparing to run Nginx in Docker
2024-02-01 14:58:17   [14 entrypoint-nginx.sh] Dockerfile for this image can found inside the container.
2024-02-01 14:58:17   [15 entrypoint-nginx.sh] To view the Dockerfile: 'cat /docker/nginx-artifactory-pro/Dockerfile.nginx'.
2024-02-01 14:58:17   [19 entrypoint-nginx.sh] Setting up directories if missing
2024-02-01 14:58:17  [160 entrypoint-nginx.sh] Generating ssl example.key and example.crt
2024-02-01 14:58:17   [83 entrypoint-nginx.sh] Artifactory configuration already in /var/opt/jfrog/nginx/conf.d/artifactory.conf
2024-02-01 14:58:17  [177 entrypoint-nginx.sh] Adding logrotate configuration
2024-02-01 14:58:17   [92 entrypoint-nginx.sh] Adding logrotate to crontab
2024-02-01 14:58:17  [187 entrypoint-nginx.sh] Starting updateConf.sh in the background
Using deprecated password for user _internal.
2024-02-01 14:58:17  [192 entrypoint-nginx.sh] Starting nginx daemon...

[chukka]

@oniabifo can you confirm if you are using latest version of chart 107.77.3 ?

Can you provide me output of this command
helm list -n $NAMESPACE_COMMAND

[oniabifo]

@chukka it returned nothing.

I also have another question, i pulled this image releases-docker.jfrog.io/jfrog/nginx-artifactory-pro:7.77.3 and i tried running it locally on docker and i noticed it never got past this stage

Using deprecated password for user _internal.
2024-02-06 22:39:52  [169 entrypoint-nginx.sh] Preparing to run Nginx in Docker
2024-02-06 22:39:52   [14 entrypoint-nginx.sh] Dockerfile for this image can found inside the container.
2024-02-06 22:39:52   [15 entrypoint-nginx.sh] To view the Dockerfile: 'cat /docker/nginx-artifactory-pro/Dockerfile.nginx'.
2024-02-06 22:39:52   [19 entrypoint-nginx.sh] Setting up directories if missing
2024-02-06 22:39:52  [160 entrypoint-nginx.sh] Generating ssl example.key and example.crt
2024-02-06 22:39:52   [77 entrypoint-nginx.sh] Unable to find Artifactory configuration (/var/opt/jfrog/nginx/conf.d/artifactory.conf). Copying a default one
2024-02-06 22:39:52  [177 entrypoint-nginx.sh] Adding logrotate configuration
2024-02-06 22:39:52   [92 entrypoint-nginx.sh] Adding logrotate to crontab
2024-02-06 22:39:52  [187 entrypoint-nginx.sh] Starting updateConf.sh in the background
Using deprecated password for user _internal.
2024-02-06 22:39:52  [192 entrypoint-nginx.sh] Starting nginx daemon...

Could there be a reason why? it seems it is running updateConf.sh in the background and there might be something going on in that file

[gitta-jfrog]

Closing, If additional support is required please let us know.

[nola-it]

Has this been closed as a resolution has been provided? Could you clarify as I am getting a similar error while trying to install xRay?

[gitta-jfrog]

The issue closed without resolution as our team wasn't able to reproduce the issue with the information provided.
Please open a new ticket with all the details for additional investigation. In addition, We are recommending to reach out JFrog Support team if possible.
Thanks!

[awolde]

Hitting this error as well, I have a cluster with kubeadm, k8s version v1.29.3. Closing it without a resolution doesn't make sense @gitta-jfrog . This is the error I'm getting from the router container:

Testing directory /var/opt/jfrog/router has read/write permissions for user id 1000
Permissions for /var/opt/jfrog/router are good
Setting JF_SHARED_NODE_ID to artifactory-0
Setting JF_SHARED_NODE_IP to 192.168.145.218
Setting JF_SHARED_NODE_NAME to artifactory-0
Using default router's certificate and private key
======= Router PID 256
/opt/jfrog/router/app/bin/entrypoint-router.sh: line 112: /opt/jfrog/router/app/router/router.pid: Permission denied
2024-04-11T14:20:58.192Z [jfrou] [FATAL] [64938e467fb39198] [home.go:56                    ] [main                ] [] - Could not create service home: stat /opt/jfrog/router/var/data/router: permission denied

Im getting similar issues with the other containers 'permission denied'. I think there is some setup we need to do for the PV. This helm chart expects a pv to exists before it can claim it with pvc. My pv setup is with nfs. Here is my setup:

awolde@k8s-lab-01:~$ k -n jfrog get pvc
NAME                               STATUS   VOLUME         CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
artifactory-volume-artifactory-0   Bound    db-pv-volume   300Gi      RWO,RWX        standard       <unset>                 9h
data-artifactory-postgresql-0      Bound    cc-pv-volume   300Gi      RWO,RWX        standard       <unset>                 9h
awolde@k8s-lab-01:~$ k -n jfrog get pv 
NAME           CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                    STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
cc-pv-volume   300Gi      RWO,RWX        Delete           Bound    jfrog/data-artifactory-postgresql-0      standard       <unset>                          9h
db-pv-volume   300Gi      RWO,RWX        Delete           Bound    jfrog/artifactory-volume-artifactory-0   standard       <unset>                          9h

[oumkale]

Hi @awolde and @oniabifo,

Thank you for raising these concerns. However, we were unable to reproduce them, and it seems the issue stems from Docker on the host. Could you please take a look into below ref and let us know if you need any help?

Reference issue: docker-library/golang#467

[Momotoculteur]

Hello,
Same problem here, running on HA version chart, on a EKS (BottlerocketOS)

I was on 107.59.X, chart based on 107.63.X is broken by a malformated yaml chart, so i'm trying to update to 107.68.X

I already had that error on router component :

But that was jut a warning, and now it don't want to boot up

┌──────────────────────────────────────────────────────────────────── Logs(jfrog/artifactory-ha-107-77-14-artifactory-ha-primary-0:router)[tail] ────────────────────────────────────────────────────────────────────┐
│                                                                         Autoscroll:On      FullScreen:Off     Timestamps:Off     Wrap:On                                                                           │
│ Testing directory /var/opt/jfrog/router has read/write permissions for user id 1030                                                                                                                                │
│ Permissions for /var/opt/jfrog/router are good                                                                                                                                                                     │
│ Setting JF_SHARED_NODE_ID to artifactory-ha-107-77-14-artifactory-ha-primary-0                                                                                                                                     │
│ Setting JF_SHARED_NODE_IP to 10.207.57.179                                                                                                                                                                         │
│ Setting JF_SHARED_NODE_NAME to artifactory-ha-107-77-14-artifactory-ha-primary-0                                                                                                                                   │
│ Using default router's certificate and private key                                                                                                                                                                 │
│ ======= Router PID 267                                                                                                                                                                                             │
│ /opt/jfrog/router/app/bin/entrypoint-router.sh: line 112: /opt/jfrog/router/app/router/router.pid: Permission denied                                                                                               │
│ 2024-08-07T17:31:26.235Z [jfrou] [WARN ] [                ] [config_holder.go:325          ] [main                ] [] - Could not resolve config value for key 'router.serviceRegistry.accessUrl', although it wa │
│ s just returned as part of all keys... Ignoring.                                                                                                                                                                   │
│ 2024-08-07T17:31:26.235Z [jfrou] [WARN ] [                ] [config_holder.go:325          ] [main                ] [] - Could not resolve config value for key 'router.serviceRegistry.topologyUrl', although it  │
│ was just returned as part of all keys... Ignoring.                                                                                                                                                                 │
│ 2024-08-07T17:31:26.236Z [jfrou] [FATAL] [648ea037a14a2f64] [config_holder.go:50           ] [main                ] [] - Could not initialize configuration: unable to resolve service registry URL (please config │
│ ure the JFrog Platform Deployment URL explicitly, e.g. using the environment variable: JF_SHARED_JFROGURL)                                                                                                         │
│ Stream closed EOF for jfrog/artifactory-ha-107-77-14-artifactory-ha-primary-0 (router)

[gitta-jfrog]

Hi @Momotoculteur

If you are having support as part of your subscription - please open support ticket and our support engineers will work with you up to full resolution.

if you are not having support - I'll ask you to open a new Github issue with all the details (version before, version after upgrade, database type and values yaml) and I'll try to assist as much as I can. Upgrade to newer version (Artifactory 7.68 is already released one year ago is highly recommended, and will make easier for us to provide support)

Thanks
Yarden