Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NetworkTimeoutError with tus for file uploads #10857

Open
Pherkel opened this issue Jan 12, 2025 · 1 comment
Open

NetworkTimeoutError with tus for file uploads #10857

Pherkel opened this issue Jan 12, 2025 · 1 comment
Labels
Type:Bug

Comments

@Pherkel
Copy link

Pherkel commented Jan 12, 2025

Describe the bug

I am getting slow uploads, especially for many smaller files.
Looking at the logs, it seems that the upload via tus does not work correctly.

Steps to reproduce

  1. Follow the setup described
  2. Upload larger file (>50Mb in my case)

Behavior

The upload is initially normal for a few seconds but then stalls. Log is spammed with

{
  "level": "warn",
  "service": "storage-users",
  "pkg": "rhttp",
  "datatx": "tus",
  "method": {},
  "path": {},
  "requestId": {},
  "id": {},
  "time": "2025-01-12T13:02:44Z",
  "line": "github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251",
  "message": "NetworkTimeoutError"
}

After a while, the upload is then resumed and continues as expected. NetworkTimeoutError is still logged.
Upload finishes normally.

For many smaller files (~5Mb), this causes the upload to take a really long time.

Setup

OCIS set up with docker compose, following the ocis-full example but having stripped out anything but the core ocis functionality.
Using the Posix filesystem driver.

Replaced Traefik reverse proxy with Caddy, followed the guide from tus on how to configure proxies

OCIS:

Image: owncloud/ocis-rolling:7.0.0

OCIS_INSECURE: true
OCIS_ADD_RUN_SERVICES: "notifications"

PROXY_TLS: false
PROXY_ENABLE_BASIC_AUTH: false

MICRO_REGISTRY_ADDRESS: 127.0.0.1:9233
NATS_NATS_HOST: 0.0.0.0
NATS_NATS_PORT: 9233

STORAGE_USERS_DRIVER: "posix"
STORAGE_USERS_POSIX_ROOT: "/var/lib/ocis/posix-storage"
STORAGE_USERS_POSIX_WATCH_TYPE: "inotifywait"
STORAGE_USERS_ID_CACHE_STORE: "nats-js-kv"
STORAGE_USERS_ID_CACHE_STORE_NODES: "localhost:9233"

docker-compose.yaml

---
services:
  ocis:
    image: ${OCIS_DOCKER_IMAGE:-owncloud/ocis}:${OCIS_DOCKER_TAG:-latest}
    entrypoint:
      - /bin/sh
    command: [ "-c", "ocis init || true; ocis server" ]
    environment:
      OCIS_ADD_RUN_SERVICES: ${START_ADDITIONAL_SERVICES}
      OCIS_URL: https://${OCIS_DOMAIN:-ocis.owncloud.test}
      OCIS_LOG_LEVEL: ${LOG_LEVEL:-info}
      OCIS_LOG_COLOR: "${LOG_PRETTY:-false}"
      OCIS_LOG_PRETTY: "${LOG_PRETTY:-false}"

      PROXY_TLS: "false"

      GATEWAY_GRPC_ADDR: 0.0.0.0:9142
      OCIS_INSECURE: "${INSECURE:-false}"

      PROXY_ENABLE_BASIC_AUTH: "${PROXY_ENABLE_BASIC_AUTH:-false}"

      IDM_ADMIN_PASSWORD: "${ADMIN_PASSWORD:-admin}" # this overrides the admin password from the configuration file

      IDM_CREATE_DEMO_USERS: "${DEMO_USERS:-false}"

      NOTIFICATIONS_SMTP_HOST: "${SMTP_HOST}"
      NOTIFICATIONS_SMTP_PORT: "${SMTP_PORT}"
      NOTIFICATIONS_SMTP_SENDER: "${SMTP_SENDER:-oCIS notifications <notifications@${OCIS_DOMAIN:-ocis.owncloud.test}>}"
      NOTIFICATIONS_SMTP_USERNAME: "${SMTP_USERNAME}"
      NOTIFICATIONS_SMTP_INSECURE: "${SMTP_INSECURE}"

      MICRO_REGISTRY_ADDRESS: 127.0.0.1:9233
      NATS_NATS_HOST: 0.0.0.0
      NATS_NATS_PORT: 9233
      PROXY_CSP_CONFIG_FILE_LOCATION: /etc/ocis/csp.yaml

      OCIS_PASSWORD_POLICY_BANNED_PASSWORDS_LIST: banned-password-list.txt

      STORAGE_USERS_DRIVER: "posix"
      STORAGE_USERS_POSIX_ROOT: "/var/lib/ocis/posix-storage"
      STORAGE_USERS_POSIX_WATCH_TYPE: "inotifywait"
      STORAGE_USERS_ID_CACHE_STORE: "nats-js-kv"
      STORAGE_USERS_ID_CACHE_STORE_NODES: "localhost:9233"

    volumes:
      - ./config/ocis/app-registry.yaml:/etc/ocis/app-registry.yaml
      - ./config/ocis/csp.yaml:/etc/ocis/csp.yaml
      - ./config/ocis/banned-password-list.txt:/etc/ocis/banned-password-list.txt

      - ${OCIS_CONFIG_DIR:-ocis-config}:/etc/ocis
      - ${OCIS_DATA_DIR:-ocis-data}:/var/lib/ocis
    restart: always

volumes:
  ocis-config:
  ocis-data:

networks:
  default:
    name: caddy_net
    external: true

.env

LOG_DRIVER=
INSECURE=true
OCIS_DOCKER_IMAGE=owncloud/ocis-rolling
OCIS_DOCKER_TAG=7.0.0
OCIS_DOMAIN=<my_domain>
ADMIN_PASSWORD=
DEMO_USERS=
LOG_LEVEL=
# LOG_PRETTY=true
# OCIS_CONFIG_DIR=/your/local/ocis/config
# OCIS_DATA_DIR=/your/local/ocis/data

SMTP_HOST=
SMTP_PORT=
SMTP_SENDER=
SMTP_USERNAME=
SMTP_PASSWORD=
SMTP_AUTHENTICATION=
SMTP_INSECURE=

START_ADDITIONAL_SERVICES="notifications"

TIKA=:tika.yml
TIKA_IMAGE=

COMPOSE_FILE=docker-compose.yml${TIKA:-}

Caddy:

Caddyfile

# have also tried setting the `Host` header with this directive
# then the origin does not match for auth
{
	# servers :443 {
	# 	trusted_proxies static 10.0.0.1/24
	# }
	# servers :80 {
	# 	trusted_proxies static 10.0.0.1/24
	# }
}

cloud.{$MY_DOMAIN} {
	reverse_proxy ocis:9200 {
		# header_up Host {upstream_hostport}
		flush_interval -1
	}
}

Additional context

Another point is that Caddy itself is behind a wireguard tunnel used to traverse a CGNAT, I am not sure if that might be part of the problem.

Relevant OCIS logs:

ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"a1a9e7f811cbb10aa2021fa8cc01001c","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"a1a9e7f811cbb10aa2021fa8cc01001c","data-server":"http://localhost:9216/data/simple/dc49d326-021b-4918-afa5-e8fc1168a48c","fn":"./settings/bundles/71881883-1768-46bd-a24d-a356a2afdf7f","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"a1a9e7f811cbb10aa2021fa8cc01001c","data-server":"http://localhost:9216/data/tus/dc49d326-021b-4918-afa5-e8fc1168a48c","fn":"./settings/bundles/71881883-1768-46bd-a24d-a356a2afdf7f","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"ea6dc3fab1cc324728806154d444a3f5","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"ea6dc3fab1cc324728806154d444a3f5","data-server":"http://localhost:9216/data/simple/66f62bff-dcb8-41ed-ab0f-b01e9792a10e","fn":"./settings/bundles/d7beeea8-8ff4-406b-8fb6-ab2dd81e6b11","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"ea6dc3fab1cc324728806154d444a3f5","data-server":"http://localhost:9216/data/tus/66f62bff-dcb8-41ed-ab0f-b01e9792a10e","fn":"./settings/bundles/d7beeea8-8ff4-406b-8fb6-ab2dd81e6b11","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"b0d75dccb734fdde857b838e1cd8673d","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"b0d75dccb734fdde857b838e1cd8673d","data-server":"http://localhost:9216/data/simple/5bbb2e65-7caa-465f-9696-810c25c388bf","fn":"./settings/bundles/38071a68-456a-4553-846a-fa67bf5596cc","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"b0d75dccb734fdde857b838e1cd8673d","data-server":"http://localhost:9216/data/tus/5bbb2e65-7caa-465f-9696-810c25c388bf","fn":"./settings/bundles/38071a68-456a-4553-846a-fa67bf5596cc","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"e60728f6cf4d284b15af4292dcecb807","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"e60728f6cf4d284b15af4292dcecb807","data-server":"http://localhost:9216/data/simple/455b9515-4f88-4cc3-b30d-e56a42813991","fn":"./settings/bundles/2a506de7-99bd-4f0d-994e-c38e72c28fd9","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"e60728f6cf4d284b15af4292dcecb807","data-server":"http://localhost:9216/data/tus/455b9515-4f88-4cc3-b30d-e56a42813991","fn":"./settings/bundles/2a506de7-99bd-4f0d-994e-c38e72c28fd9","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"e2d590112184ed764a0d64b481b5bee7","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
tika-1  | INFO  [main] 13:38:46,548 org.eclipse.jetty.server.AbstractConnector Started ServerConnector@323e8306{HTTP/1.1, (http/1.1)}{0.0.0.0:9998}
tika-1  | INFO  [main] 13:38:46,555 org.eclipse.jetty.server.Server Started Server@308a6984{STARTING}[11.0.24,sto=0] @1072ms
tika-1  | INFO  [main] 13:38:46,575 org.eclipse.jetty.server.handler.ContextHandler Started o.a.c.t.h.JettyContextHandler@606fc505{/,null,AVAILABLE}
tika-1  | INFO  [main] 13:38:46,581 org.apache.tika.server.core.TikaServerProcess Started Apache Tika server 13d8db4e-1cc3-499e-8eb3-d62e271f4f5c at http://0.0.0.0:9998/
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"e2d590112184ed764a0d64b481b5bee7","data-server":"http://localhost:9216/data/simple/41d10f56-c367-45e2-98e4-83f7bcc7ffec","fn":"./settings/bundles/2aadd357-682c-406b-8874-293091995fdd","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"e2d590112184ed764a0d64b481b5bee7","data-server":"http://localhost:9216/data/tus/41d10f56-c367-45e2-98e4-83f7bcc7ffec","fn":"./settings/bundles/2aadd357-682c-406b-8874-293091995fdd","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"3da8b8cc6d75f87be18209e173fbbc1e","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"3806b3c4be49ca640c60d604b674595d","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"a425ea0a66bde49f4ed344e55398575b","time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"a425ea0a66bde49f4ed344e55398575b","data-server":"http://localhost:9216/data","ref":{"resource_id":{"opaque_id":"f1bdd61a-da7c-49fc-8203-0558109d1b4f","space_id":"f1bdd61a-da7c-49fc-8203-0558109d1b4f"},"path":"./settings/accounts/87b3fadd-7628-4b7a-866f-4f548bc51801/e64ec9c6-23bf-4a73-9222-db57aaac17b0"},"time":"2025-01-12T13:39:19Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:297","message":"file download"}
ocis-1  | {"level":"info","service":"auth-machine","pkg":"rgrpc","traceid":"3f1e1409686c9b6b2b83a73b1f022553","time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"https://cloud.domain.com\" opaque_id:\"87b3fadd-7628-4b7a-866f-4f548bc51801\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,CREATE,4591228f-e91b-41fd-b2a4-d4a2987584b4
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",4591228f-e91b-41fd-b2a4-d4a2987584b4
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,CREATE,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info1509438866
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",.4591228f-e91b-41fd-b2a4-d4a2987584b4.info1509438866
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,DELETE,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info1509438866
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,CREATE,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info8277508481237985037
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",.4591228f-e91b-41fd-b2a4-d4a2987584b4.info8277508481237985037
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,MOVED_FROM,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info8277508481237985037
ocis-1  | 2025/01/12 13:39:39 /var/lib/ocis/posix-storage/uploads/,MOVED_TO,4591228f-e91b-41fd-b2a4-d4a2987584b4.info
ocis-1  | {"level":"info","service":"storage-users","pkg":"rgrpc","traceid":"3f1e1409686c9b6b2b83a73b1f022553","data-server":"http://localhost:9158/data/simple/4591228f-e91b-41fd-b2a4-d4a2987584b4","fn":"./example.zip","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rgrpc","traceid":"3f1e1409686c9b6b2b83a73b1f022553","data-server":"http://localhost:9158/data/tus/4591228f-e91b-41fd-b2a4-d4a2987584b4","fn":"./example.zip","xs":"map[md5:100 unset:1000]","time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/storageprovider/storageprovider.go:471","message":"file upload"}
ocis-1  | {"level":"info","service":"frontend","pkg":"rhttp","traceid":"6e38d937ca0fa6a5f5e69a8dca452e0f","time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/internal/http/interceptors/auth/auth.go:195","message":"skipping auth check for: /data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsicmV2YSJdLCJleHAiOjE3MzY3NzU1NzksImlhdCI6MTczNjY4OTE3OSwidGFyZ2V0IjoiaHR0cDovL2xvY2FsaG9zdDo5MTU4L2RhdGEvdHVzLzQ1OTEyMjhmLWU5MWItNDFmZC1iMmE0LWQ0YTI5ODc1ODRiNCJ9._UYR-0g82S-O7ZVOsh_wiFKWmvJ9h8RRZydEUmvKKpI"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","traceid":"c89435eb3504c20e525a975b8c11a499","time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/internal/http/interceptors/auth/auth.go:195","message":"skipping auth check for: /data/tus/4591228f-e91b-41fd-b2a4-d4a2987584b4"}
ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkControlError"}
ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkControlError"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"RequestIncoming"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"ChunkWriteStart"}
ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:39Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkTimeoutError"}

ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:45Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkTimeoutError"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:45Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"ChunkWriteComplete"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:45Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"ResponseOutgoing"}
ocis-1  | 2025/01/12 13:39:45 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",4591228f-e91b-41fd-b2a4-d4a2987584b4
ocis-1  | {"level":"info","service":"proxy","proto":"HTTP/1.1","request-id":"1280ab9abba4/lAeV3j4wFN-000026","traceid":"3f1e1409686c9b6b2b83a73b1f022553","remote-addr":"10.0.0.1","method":"PATCH","status":204,"path":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsicmV2YSJdLCJleHAiOjE3MzY3NzU1NzksImlhdCI6MTczNjY4OTE3OSwidGFyZ2V0IjoiaHR0cDovL2xvY2FsaG9zdDo5MTU4L2RhdGEvdHVzLzQ1OTEyMjhmLWU5MWItNDFmZC1iMmE0LWQ0YTI5ODc1ODRiNCJ9._UYR-0g82S-O7ZVOsh_wiFKWmvJ9h8RRZydEUmvKKpI","duration":6716.474046,"bytes":0,"time":"2025-01-12T13:39:45Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/accesslog.go:34","message":"access-log"}
ocis-1  | {"level":"info","service":"proxy","proto":"HTTP/1.1","request-id":"2f6d4f6b-4192-4219-ba75-bac5c798edfc","traceid":"3f1e1409686c9b6b2b83a73b1f022553","remote-addr":"10.0.0.1","method":"POST","status":201,"path":"/remote.php/dav/spaces/f31ab3b9-b377-40e8-a212-44f84d8e2555$e58614db-323d-43d9-9742-210861d04875","duration":6917.030684,"bytes":0,"time":"2025-01-12T13:39:45Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/accesslog.go:34","message":"access-log"}
ocis-1  | {"level":"info","service":"frontend","pkg":"rhttp","traceid":"ab447dd420ffa5b9a664efab218684b6","time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/internal/http/interceptors/auth/auth.go:195","message":"skipping auth check for: /data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsicmV2YSJdLCJleHAiOjE3MzY3NzU1NzksImlhdCI6MTczNjY4OTE3OSwidGFyZ2V0IjoiaHR0cDovL2xvY2FsaG9zdDo5MTU4L2RhdGEvdHVzLzQ1OTEyMjhmLWU5MWItNDFmZC1iMmE0LWQ0YTI5ODc1ODRiNCJ9._UYR-0g82S-O7ZVOsh_wiFKWmvJ9h8RRZydEUmvKKpI"}
ocis-1  | {"level":"warn","service":"frontend","pkg":"rhttp","traceid":"ab447dd420ffa5b9a664efab218684b6","time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/internal/http/interceptors/auth/auth.go:248","message":"core access token not set"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","traceid":"2d168e170928c4c7ba6feed3a4f49f19","time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/internal/http/interceptors/auth/auth.go:195","message":"skipping auth check for: /data/tus/4591228f-e91b-41fd-b2a4-d4a2987584b4"}
ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","traceid":"2d168e170928c4c7ba6feed3a4f49f19","time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/internal/http/interceptors/auth/auth.go:248","message":"core access token not set"}
ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkControlError"}
ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkControlError"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"RequestIncoming"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:46Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"ChunkWriteStart"}

ocis-1  | {"level":"warn","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:251","message":"NetworkTimeoutError"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"ChunkWriteComplete"}
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",4591228f-e91b-41fd-b2a4-d4a2987584b4
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/admin/,CREATE,example.zip.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/admin/,CREATE,example.zip
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/,CREATE,admin.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/,DELETE,admin.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,CREATE,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info2552782423
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",.4591228f-e91b-41fd-b2a4-d4a2987584b4.info2552782423
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,DELETE,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info2552782423
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,CREATE,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info909687815827874621
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,"CLOSE_WRITE,CLOSE",.4591228f-e91b-41fd-b2a4-d4a2987584b4.info909687815827874621
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,MOVED_FROM,.4591228f-e91b-41fd-b2a4-d4a2987584b4.info909687815827874621
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/uploads/,MOVED_TO,4591228f-e91b-41fd-b2a4-d4a2987584b4.info
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/admin/,"CLOSE_WRITE,CLOSE",example.zip.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/admin/,DELETE,example.zip.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/,CREATE,admin.mlock
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"UploadFinished"}
ocis-1  | {"level":"info","service":"storage-users","pkg":"rhttp","datatx":"tus","method":{},"path":{},"requestId":{},"id":{},"time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/pkg/rhttp/datatx/manager/tus/tus.go:249","message":"ResponseOutgoing"}
ocis-1  | {"level":"info","service":"proxy","proto":"HTTP/1.1","request-id":"e695100c-a383-4f33-99d6-299f3ea8f949","traceid":"250f733949eaae6d2444904ac4490d6b","remote-addr":"10.0.0.1","method":"PATCH","status":204,"path":"/data/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsicmV2YSJdLCJleHAiOjE3MzY3NzU1NzksImlhdCI6MTczNjY4OTE3OSwidGFyZ2V0IjoiaHR0cDovL2xvY2FsaG9zdDo5MTU4L2RhdGEvdHVzLzQ1OTEyMjhmLWU5MWItNDFmZC1iMmE0LWQ0YTI5ODc1ODRiNCJ9._UYR-0g82S-O7ZVOsh_wiFKWmvJ9h8RRZydEUmvKKpI","duration":823.559716,"bytes":0,"time":"2025-01-12T13:39:56Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/accesslog.go:34","message":"access-log"}
ocis-1  | 2025/01/12 13:39:56 /var/lib/ocis/posix-storage/users/admin/,"CLOSE_WRITE,CLOSE",example.zip
ocis-1  | {"level":"info","service":"auth-machine","pkg":"rgrpc","traceid":"be1bc61e320913459dad928e048f9657","time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"https://cloud.domain.com\" opaque_id:\"87b3fadd-7628-4b7a-866f-4f548bc51801\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"graph","request-id":"538733bb-11e5-4e3f-8074-a01e4236a56f","query":{},"time":"2025-01-12T13:39:56Z","line":"github.com/owncloud/ocis/v2/services/graph/pkg/service/v0/drives.go:249","message":"calling get drive"}
ocis-1  | {"level":"info","service":"proxy","proto":"HTTP/1.1","request-id":"538733bb-11e5-4e3f-8074-a01e4236a56f","traceid":"be1bc61e320913459dad928e048f9657","remote-addr":"10.0.0.1","method":"GET","status":200,"path":"/graph/v1.0/drives/f31ab3b9-b377-40e8-a212-44f84d8e2555$e58614db-323d-43d9-9742-210861d04875","duration":9.468646,"bytes":743,"time":"2025-01-12T13:39:56Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/accesslog.go:34","message":"access-log"}
ocis-1  | {"level":"info","service":"auth-machine","pkg":"rgrpc","traceid":"a86c94313b367dacdc24ff8d0da1a316","time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"https://cloud.domain.com\" opaque_id:\"87b3fadd-7628-4b7a-866f-4f548bc51801\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"gateway","pkg":"rgrpc","traceid":"a86c94313b367dacdc24ff8d0da1a316","time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/gateway/publicshareprovider.go:92","message":"listing public shares"}
ocis-1  | {"level":"info","service":"sharing","pkg":"rgrpc","traceid":"a86c94313b367dacdc24ff8d0da1a316","publicshareprovider":"list","time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/publicshareprovider/publicshareprovider.go:450","message":"list public share"}
ocis-1  | {"level":"info","service":"storage-system","pkg":"rgrpc","traceid":"c8d93b921a1ded319b20eadf55894abc","time":"2025-01-12T13:39:56Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"internal\" opaque_id:\"99367729-b364-4db0-a254-b79ad972e1e7\" type:USER_TYPE_PRIMARY authenticated"}
ocis-1  | {"level":"info","service":"proxy","proto":"HTTP/1.1","request-id":"645f23f4-39f3-4d30-bbe8-e6c40fd908e5","traceid":"a86c94313b367dacdc24ff8d0da1a316","remote-addr":"10.0.0.1","method":"PROPFIND","status":207,"path":"/remote.php/dav/spaces/f31ab3b9-b377-40e8-a212-44f84d8e2555$e58614db-323d-43d9-9742-210861d04875","duration":13.063341,"bytes":2781,"time":"2025-01-12T13:39:56Z","line":"github.com/owncloud/ocis/v2/services/proxy/pkg/middleware/accesslog.go:34","message":"access-log"}
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,DELETE,admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/uploads/,DELETE,4591228f-e91b-41fd-b2a4-d4a2987584b4
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/uploads/,DELETE,4591228f-e91b-41fd-b2a4-d4a2987584b4.info
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/admin/,CREATE,example.zip.flock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/admin/,"CLOSE_WRITE,CLOSE",example.zip.flock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/admin/,DELETE,example.zip.flock
ocis-1  | {"level":"info","service":"auth-service","pkg":"rgrpc","traceid":"453922c2f8d43c394445a3a7ed9500f4","time":"2025-01-12T13:39:57Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"none\" opaque_id:\"90b1f729-6c5a-4f39-9ac8-e109e652ddf6\" type:USER_TYPE_SERVICE authenticated"}
ocis-1  | {"level":"info","service":"auth-service","pkg":"rgrpc","traceid":"f740c7c55dc0d27ac1cf799b2da3e5cf","time":"2025-01-12T13:39:57Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"none\" opaque_id:\"90b1f729-6c5a-4f39-9ac8-e109e652ddf6\" type:USER_TYPE_SERVICE authenticated"}
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,CREATE,admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/admin/,CREATE,example.zip.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,DELETE,admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,CREATE,admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | 2025/01/12 13:39:57 /var/lib/ocis/posix-storage/users/,DELETE,admin.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/,CREATE,admin.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/admin/,"CLOSE_WRITE,CLOSE",example.zip.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/admin/,DELETE,example.zip.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/admin/,"CLOSE_WRITE,CLOSE",example.zip.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/,"CLOSE_WRITE,CLOSE",admin.mlock
ocis-1  | 2025/01/12 13:39:58 /var/lib/ocis/posix-storage/users/,DELETE,admin.mlock
ocis-1  | {"level":"info","service":"auth-service","pkg":"rgrpc","traceid":"756230f471587d9e5c1ff57d078f3ce6","time":"2025-01-12T13:39:58Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"none\" opaque_id:\"90b1f729-6c5a-4f39-9ac8-e109e652ddf6\" type:USER_TYPE_SERVICE authenticated"}
ocis-1  | {"level":"info","service":"auth-service","pkg":"rgrpc","traceid":"6af6cb3eeb091828f50f1151ab5d330e","time":"2025-01-12T13:39:58Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"none\" opaque_id:\"90b1f729-6c5a-4f39-9ac8-e109e652ddf6\" type:USER_TYPE_SERVICE authenticated"}
ocis-1  | {"level":"info","service":"search","ResourceID":{"storage_id":"f31ab3b9-b377-40e8-a212-44f84d8e2555","opaque_id":"e58614db-323d-43d9-9742-210861d04875","space_id":"e58614db-323d-43d9-9742-210861d04875"},"Name":"admin","time":"2025-01-12T13:39:58Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/content/tika.go:66","message":"file exceeds content extraction size limit. skipping."}
ocis-1  | {"level":"info","service":"auth-service","pkg":"rgrpc","traceid":"cf9b5a8b6b8c4fdf137c40b2729a33b5","time":"2025-01-12T13:39:58Z","line":"github.com/cs3org/reva/[email protected]/internal/grpc/services/authprovider/authprovider.go:146","message":"user idp:\"none\" opaque_id:\"90b1f729-6c5a-4f39-9ac8-e109e652ddf6\" type:USER_TYPE_SERVICE authenticated"}
ocis-1  | {"level":"info","service":"search","ResourceID":{"storage_id":"f31ab3b9-b377-40e8-a212-44f84d8e2555","opaque_id":"b5290ffc-0f8f-41ed-85c5-8703b7d04eab","space_id":"e58614db-323d-43d9-9742-210861d04875"},"Name":"example.zip","time":"2025-01-12T13:39:58Z","line":"github.com/owncloud/ocis/v2/services/search/pkg/content/tika.go:66","message":"file exceeds content extraction size limit. skipping."}

Full OCIS log
Caddy Access Log
@bitbytenybble110
Copy link

I've seen this exact error spam my container logs while I'm uploading files but everything seems to work so I've come to ignore them.

I'd be interested to see what the team has to say.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type:Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bitbytenybble110 @Pherkel