LabRecorder crashed while closing the file; saved XDF cannot be read by load_xdf

[mcvain]

Hello, I am using load_xdf for Matlab to import some XDF files. It of course works well, but I noticed that my LabRecorder was crashing when writing files sometimes (conditions to replicate unknown; Windows 11). On the console window, I believe it crashed before the usual "Closing the file" message; rather, it would get stuck at one of the "Wrote footer for... " lines. But I don't see/have any crash logs. When this crash happens, the XDF file still gets saved, and it has data (filesize appears normal), but it cannot be read by load_xdf. It throws the following error:

xdf_viewer
Error using xmlread
Java exception occurred:
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1838; XML document structures must start and end within the same entity.

	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)

	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)

	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)

	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)

	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)

	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.endEntity(Unknown Source)

	at org.apache.xerces.impl.XMLDocumentScannerImpl.endEntity(Unknown Source)

	at org.apache.xerces.impl.XMLEntityManager.endEntity(Unknown Source)

	at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source)

	at org.apache.xerces.impl.XMLEntityScanner.skipString(Unknown Source)

	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)

	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)

	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)

	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)

	at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)

	at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)


Error in load_xdf>parse_xml_struct (line 766)
result = parseChildNodes(xmlread(tmp));

Error in load_xdf (line 411)
            footer = parse_xml_struct(fread(f,len-6,'*char')');

Error in xdf_viewer (line 8)
[streams,fileheader] = load_xdf(strcat(filepath, filename));

LabRecorder version info:

I was wondering if this is a known issue, and if there is a way to recover any data from these XDF files. It appears that it contains data but was not formatted properly for load_xdf to read. I have attached an example corrupted XDF file.
004.zip

[cboulay]

Here is where LabRecorder writes the footer:

App-LabRecorder/src/recording.cpp

Lines 258 to 270 in becc09e

	footer << "<?xml version=\"1.0\"?><info><first_timestamp>" << first_timestamp
	<< "</first_timestamp><last_timestamp>" << last_timestamp
	<< "</last_timestamp><sample_count>" << sample_count << "</sample_count>";
	footer << "<clock_offsets>";
	{
	// including the clock_offset list
	std::lock_guard<std::mutex> lock(offset_mut_);
	for (const auto pair : offset_lists_[streamid]) {
	footer << "<offset><time>" << pair.first << "</time><value>" << pair.second
	<< "</value></offset>";
	}
	footer << "</clock_offsets></info>";
	}

You can see that it's writing offset_lists_, which are important for synchronizing streams from different originating PCs.

However, LabRecorder also has a thread that writes offsets continuously throughout the recording, here:

App-LabRecorder/src/recording.cpp

Line 314 in becc09e

file_.write_stream_offset(streamid, now, offset);

As long as your XDF importer is capable of grabbing the clock offsets distributed throughout the file, and it does not rely on the footer, then the footer loading is optional and that can be fixed in the XDF loader.

[pietrocima]

I had the same error after getting an .xdf from LabRecorder 1.12 who crashed during the "stop recording" process. The recording was related to one of many experimental trials that had all the same recording structure. If you are lucky enough to have an openable file with the same recording structure (let's call it the "good" file), you can solve the issue and save your data by changing the extension of both the corrupted and the good file to .txt. Then you go to the last line of both file, you will see that they look almost identical, but the one of the corrupted file is suddenly interrupted. If you copypaste the missing piece from the good file into the corrupted one, then convert it back do xdf, you will then be able to open it.