[ntpl] Allow XRootD redirection with EOS files #20272

vepadulano · 2025-11-03T09:54:08Z

Fixes #20213

With these changes I can see the redirection happening when reading an RNTuple on EOS from lxplus (going from ntpl001_staff.root to root://eoshome-v.cern.ch//eos/user/v/vpadulan/ntpl001_staff.root

Breakpoint 1, ROOT::Internal::RRawFile::Create (url=..., options=...) at /tmp/vpadulan/root/io/io/src/RRawFile.cxx:88       
88         std::string transport = GetTransport(url);                                                                       
(gdb) n  
89         if (transport == "file") {
(gdb) p url                                                                                                                 
$1 = {_M_len = 18, _M_str = 0x4030a3 "ntpl001_staff.root"}  
[...]
103                    if (auto baseName = fileNameFromUrl.substr(fileNameFromUrl.find_last_of("/") + 1);
(gdb) 
105                       return Create(xurl, options);
(gdb) s
[...]
Breakpoint 1, ROOT::Internal::RRawFile::Create (url=..., options=...) at /tmp/vpadulan/root/io/io/src/RRawFile.cxx:88       
88         std::string transport = GetTransport(url);                                                                       
(gdb) n                                                                                                                     
89         if (transport == "file") {                                                                                       
(gdb) 
113        if (transport == "http" || transport == "https" ||
(gdb) p url
$8 = {_M_len = 64, _M_str = 0x136a790 "root://eoshome-v.cern.ch//eos/user/v/vpadulan/ntpl001_staff.root"}
(gdb) p transport
$9 = {static npos = 18446744073709551615, 
  _M_dataplus = {<std::allocator<char>> = {<std::__new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    _M_p = 0x7fffffffa230 "root"}, _M_string_length = 4, {
    _M_local_buf = "root\000\177\000\000\220\242\377\377\377\177\000", _M_allocated_capacity = 139639930187634}}
(gdb)

There might be more places where the redirection needs to happen though.

jblomer

Thanks for addressing this!

io/io/src/RRawFile.cxx

github-actions · 2025-11-03T13:10:29Z

Test Results

0 tests 0 ✅ 0s ⏱️
0 suites 0 💤
0 files 0 ❌

Results for commit 45b2a23.

♻️ This comment has been updated with latest results.

io/io/src/IOUtils.cxx

The logic to extract the extended attribute 'eos.url.xroot' from a path to an EOS file on a FUSE mount is centralised in a separate header in RIO, with an internal function called GetEOSRedirectedXRootUrl. This function can be called anywhere needed in the rest of ROOT. With this commit, the following places use this functionality: - TFile::Open (done before this commit) - RLoopManager::ChangeSpec (done before this commit) - RRawFile::Create (introduced in this commit) In particular the last means introducing the redirection also for RNTupleReader, which follows a different logic to open the TFile.

vepadulano · 2025-11-05T10:26:03Z

To the best of my understanding, this PR is simply moving the code that used to live separately in TFile::Open and RLoopManager::ChangeSpec extracting it into a common place. I have been testing these changes on EOS. Both TFile::Open and RNTupleReader::Open work flawlessly. For some reason, with the redirection happening in RDataFrame to store also the redirected URL as a sample ID, I see this error

terminate called after throwing an instance of 'std::runtime_error'
  what():  Full sample identifier 'root://eoshome-v.cern.ch//eos/user/v/vpadulan/rdf_sample.root/events' cannot be found in the available samples.

which is the same that triggered #17544 . What's even more worrying is that I don't see the error on every test execution, but sometimes the test also runs without failing. Before merging this PR I need to understand why that happens. It's possible the problem is still there somehow also without the code changes of this PR.

jblomer

In principle looks good to me! We should think about how to test it. Extended attributes are file system dependent, so we may need to add some code just for testing, e.g. something that lets us control what GetEOSRedirectedXRootURL returns under unit tests.

jblomer · 2025-11-05T13:20:52Z

io/io/inc/ROOT/InternalIOUtils.hxx

+#include <string>
+#include <optional>


Reorder alphabetically

jblomer · 2025-11-05T13:26:54Z

io/io/src/InternalIOUtils.cxx

+   if (gEnv->GetValue("TFile.CrossProtocolRedirects", 1) == 1) {
+      if (auto xurl = ROOT::Internal::GetXAttrVal(inputURL, "eos.url.xroot")) {
+         // Sometimes the `getxattr` call may return an invalid URL due
+         // to the POSIX attribute not being yet completely filled by EOS.
+         if (inputSV.back() != '/') {
+            if (auto baseName = inputSV.substr(inputSV.find_last_of("/") + 1);
+                std::equal(baseName.crbegin(), baseName.crend(), xurl->crbegin())) {
+               // Ensure the redirected URL actually starts with the XRootD protocol string
+               if (ROOT::StartsWith(*xurl, "root://")) {
+                  return xurl;
+               }
+            }
+         }
+      }
+   }


To increase readability, I'd reverse the if statements, like

if (gEnv->GetValue("TFile.CrossProtocolRedirects", 1) != 1) return std::nullopt; auto xurl = ROOT::Internal::GetXAttrVal(inputURL, "eos.url.xroot"); if (!xurl) return std::nullopt; ...

jblomer · 2025-11-05T13:29:51Z

io/io/src/InternalIOUtils.cxx

+            if (auto baseName = inputSV.substr(inputSV.find_last_of("/") + 1);
+                std::equal(baseName.crbegin(), baseName.crend(), xurl->crbegin())) {
+               // Ensure the redirected URL actually starts with the XRootD protocol string
+               if (ROOT::StartsWith(*xurl, "root://")) {


The schema can also be xroot://

jblomer · 2025-11-05T13:30:36Z

io/io/src/InternalIOUtils.cxx

+         // Sometimes the `getxattr` call may return an invalid URL due
+         // to the POSIX attribute not being yet completely filled by EOS.


Hm, that doesn't sound great. Is there a corresponding bug report for EOS?

vepadulano requested a review from jblomer November 3, 2025 09:54

vepadulano self-assigned this Nov 3, 2025

vepadulano requested a review from pcanal as a code owner November 3, 2025 09:54

vepadulano added the in:RNTuple label Nov 3, 2025

jblomer reviewed Nov 3, 2025

View reviewed changes

io/io/src/RRawFile.cxx Outdated Show resolved Hide resolved

io/io/src/RRawFile.cxx Outdated Show resolved Hide resolved

vepadulano force-pushed the rntuple-eos-xrootd-redirection branch from 3e7059b to ab97475 Compare November 3, 2025 22:36

vepadulano requested review from bellenot and martamaja10 as code owners November 3, 2025 22:36

vepadulano force-pushed the rntuple-eos-xrootd-redirection branch 7 times, most recently from 0ec1f2f to 4422f5e Compare November 4, 2025 07:46

pcanal reviewed Nov 4, 2025

View reviewed changes

io/io/src/IOUtils.cxx Outdated Show resolved Hide resolved

pcanal reviewed Nov 4, 2025

View reviewed changes

io/io/src/IOUtils.cxx Outdated Show resolved Hide resolved

vepadulano force-pushed the rntuple-eos-xrootd-redirection branch from 4422f5e to c6f6051 Compare November 5, 2025 09:28

vepadulano force-pushed the rntuple-eos-xrootd-redirection branch from c6f6051 to 45b2a23 Compare November 5, 2025 10:22

jblomer reviewed Nov 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ntpl] Allow XRootD redirection with EOS files #20272

[ntpl] Allow XRootD redirection with EOS files #20272

vepadulano commented Nov 3, 2025

Uh oh!

jblomer left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 3, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

vepadulano commented Nov 5, 2025

Uh oh!

jblomer left a comment

Uh oh!

jblomer Nov 5, 2025

Uh oh!

jblomer Nov 5, 2025

Uh oh!

jblomer Nov 5, 2025

Uh oh!

jblomer Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// Sometimes the `getxattr` call may return an invalid URL due
		// to the POSIX attribute not being yet completely filled by EOS.

		#include <string>
		#include <optional>

[ntpl] Allow XRootD redirection with EOS files #20272

Are you sure you want to change the base?

[ntpl] Allow XRootD redirection with EOS files #20272

Conversation

vepadulano commented Nov 3, 2025

Uh oh!

jblomer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

Uh oh!

Uh oh!

vepadulano commented Nov 5, 2025

Uh oh!

jblomer left a comment

Choose a reason for hiding this comment

Uh oh!

jblomer Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

jblomer Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

jblomer Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

jblomer Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Nov 3, 2025 •

edited

Loading