You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am attempting to run Selenium inside security hardened Kubernetes cluster. Current policy does now allow read-write access to the root filesystem of the container.
After deploying Selenium with read-only filesystem, the container will run, but it is not possible to connect to it using external client.
The only breaking flag in below configuration is
readOnlyRootFilesystem: true
If I deploy exact same configuration as attached below, just changing the flag to false, everything will work fine.
It seems that Selenium container under the hood writes (unrestricted) to all kind of various filesystem paths. So this issue could be solved by allowing specify separate list of emptyDir volumes mounted in all locations that selenium container writes / caches data to. Searching through the documentation and open issues on github, I did not found a list of folders that Selenium container is writing to, and the purpose of each writable volume. Please assist :)
Reproduction - deploy selenium using attached manifest, then run following script:
Traceback (most recent call last):
File "/home/use/demo/repro_sel_minimal.py", line 15, in <module>
browser = webdriver.Remote(
^^^^^^^^^^^^^^^^^
File "/home/use/demo/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in __init__
self.start_session(capabilities)
File "/home/use/demo/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 342, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/use/demo/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
self.error_handler.check_response(response)
File "/home/use/demo/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Could not start a new session. No nodes support the capabilities in the request
Host info: host: 'chromedriver-6f448dcfbb-7vbhk', ip: '172.20.8.192'
Build info: version: '4.28.1', revision: '73f5ad48a2'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-112-generic', java.version: '17.0.13'
Driver info: driver.version: unknown
Stacktrace:
at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.checkMatchingSlot (LocalDistributor.java:841)
at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.run (LocalDistributor.java:823)
at org.openqa.selenium.concurrent.GuardedRunnable.lambda$guard$0 (GuardedRunnable.java:35)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:539)
at java.util.concurrent.FutureTask.runAndReset (FutureTask.java:305)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:305)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
at java.lang.Thread.run (Thread.java:840)
Command used to start Selenium Grid with Docker (or Kubernetes)
2025-02-07 12:18:23,274 INFO Included extra file "/etc/supervisor/conf.d/chrome-cleanup.conf" during parsing
2025-02-07 12:18:23,275 INFO Included extra file "/etc/supervisor/conf.d/recorder.conf" during parsing
2025-02-07 12:18:23,275 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2025-02-07 12:18:23,275 INFO Included extra file "/etc/supervisor/conf.d/uploader.conf" during parsing
2025-02-07 12:18:23,278 INFO RPC interface 'supervisor' initialized
2025-02-07 12:18:23,278 INFO supervisord started with pid 8
2025-02-07 12:18:24,281 INFO spawnerr: unknown error making dispatchers for'xvfb': EROFS
2025-02-07 12:18:24,282 INFO spawnerr: unknown error making dispatchers for'vnc': EROFS
2025-02-07 12:18:24,282 INFO spawnerr: unknown error making dispatchers for'novnc': EROFS
2025-02-07 12:18:24,283 INFO spawned: 'selenium-standalone' with pid 9
E: [pulseaudio] main.c: Daemon startup failed.
2025-02-07 12:18:24,299 INFO gave up: xvfb entered FATAL state, too many start retries too quickly
2025-02-07 12:18:24,299 INFO gave up: vnc entered FATAL state, too many start retries too quickly
2025-02-07 12:18:24,299 INFO gave up: novnc entered FATAL state, too many start retries too quickly
2025-02-07 12:18:24,299 INFO success: selenium-standalone entered RUNNING state, process has stayed up for> than 0 seconds (startsecs)
Failed to create secure directory (/home/seluser/.config/pulse): Read-only file system
No PulseAudio daemon running, or not running as session daemon.
Failed to create secure directory (/home/seluser/.config/pulse): Read-only file system
No PulseAudio daemon running, or not running as session daemon.
Failed to create secure directory (/home/seluser/.config/pulse): Read-only file system
No PulseAudio daemon running, or not running as session daemon.
Appending Selenium option: --heartbeat-period 30
Appending Selenium option: --log-level INFO
Appending Selenium option: --http-logs false
Appending Selenium option: --structured-logs false
Appending Selenium option: --reject-unsupported-caps true
Appending Selenium option: --newsession-threadpool-size 32
/opt/bin/generate_config: line 19: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 21: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 26: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 28: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 29: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 30: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 31: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 33: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 61: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 62: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 63: /opt/selenium/config.toml: Read-only file system
/opt/bin/generate_config: line 65: /opt/selenium/config.toml: Read-only file system
Selenium Grid Standalone configuration:
Starting Selenium Grid Standalone...
Appending Selenium option: --tracing false
Tracing is disabled
Using JAVA_OPTS: -XX:ActiveProcessorCount=4
Feb 07, 2025 12:18:24 PM org.openqa.selenium.grid.log.LoggingOptions getTracer
INFO: Using null tracer
Feb 07, 2025 12:18:25 PM org.openqa.selenium.grid.log.LoggingOptions getTracer
INFO: Using null tracer
Feb 07, 2025 12:18:25 PM org.openqa.selenium.grid.node.config.NodeOptions getSessionFactories
INFO: Detected 4 available processors
Feb 07, 2025 12:18:25 PM org.openqa.selenium.grid.node.Node <init>
INFO: Binding additional locator mechanisms: relative
Feb 07, 2025 12:18:25 PM org.openqa.selenium.grid.commands.Standalone execute
INFO: Started Selenium Standalone 4.28.1 (revision 73f5ad48a2): http://172.20.8.192:4444
Feb 07, 2025 12:18:28 PM org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable checkMatchingSlot
INFO: No nodes support the capabilities in the request: [Capabilities {browserName: chrome, goog:chromeOptions: {args: [--disable-infobars, --disable-extensions, --disable-popup-blocking, --headless, --user-data-dir=/tmp/0decd9...], extensions: []}, pageLoadStrategy: normal}]
Feb 07, 2025 12:18:31 PM org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable checkMatchingSlot
INFO: No nodes support the capabilities in the request: [Capabilities {browserName: chrome, goog:chromeOptions: {args: [--disable-infobars, --disable-extensions, --disable-popup-blocking, --headless, --user-data-dir=/tmp/947d2d...], extensions: []}, pageLoadStrategy: normal}]
Feb 07, 2025 12:18:55 PM org.openqa.selenium.concurrent.GuardedRunnable lambda$guard$0
WARNING: Unable to execute task
java.lang.IllegalArgumentException: Make sure that a driver is available on $PATH
at org.openqa.selenium.internal.Require.positive(Require.java:136)
at org.openqa.selenium.grid.data.NodeStatus.<init>(NodeStatus.java:62)
at org.openqa.selenium.grid.node.local.LocalNode.getStatus(LocalNode.java:1015)
at org.openqa.selenium.grid.node.local.LocalNode.lambda$new$12(LocalNode.java:305)
at org.openqa.selenium.concurrent.GuardedRunnable.lambda$guard$0(GuardedRunnable.java:35)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Operating System
Kubernetes Server Version: v1.29.4+vmware.3-fips.1
Docker Selenium version (image tag)
4.28.1-20250202
Selenium Grid chart version (chart version)
Standalone (no chart)
The text was updated successfully, but these errors were encountered:
@majusmisiak, thank you for creating this issue. We will troubleshoot it as soon as we can.
Info for maintainers
Triage this issue by using labels.
If information is missing, add a helpful comment and then I-issue-template label.
If the issue is a question, add the I-question label.
If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.
If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C),
add the applicable G-* label, and it will provide the correct link and auto-close the
issue.
After troubleshooting the issue, please add the R-awaiting answer label.
What happened?
I am attempting to run Selenium inside security hardened Kubernetes cluster. Current policy does now allow read-write access to the root filesystem of the container.
After deploying Selenium with read-only filesystem, the container will run, but it is not possible to connect to it using external client.
The only breaking flag in below configuration is
If I deploy exact same configuration as attached below, just changing the flag to
false
, everything will work fine.It seems that Selenium container under the hood writes (unrestricted) to all kind of various filesystem paths. So this issue could be solved by allowing specify separate list of
emptyDir
volumes mounted in all locations that selenium container writes / caches data to. Searching through the documentation and open issues on github, I did not found a list of folders that Selenium container is writing to, and the purpose of each writable volume. Please assist :)Reproduction - deploy selenium using attached manifest, then run following script:
Client:
Result:
Command used to start Selenium Grid with Docker (or Kubernetes)
Relevant log output
Operating System
Kubernetes Server Version: v1.29.4+vmware.3-fips.1
Docker Selenium version (image tag)
4.28.1-20250202
Selenium Grid chart version (chart version)
Standalone (no chart)
The text was updated successfully, but these errors were encountered: