An example with double channel string marker #4

cbrnr · 2025-02-10T07:13:01Z

How did you get this timestamp? When loading the file, I get the following output for that stream:

{'clock_times': [6.1], 'clock_values': [-0.1], 'footer': {'info': defaultdict(<class 'list'>, {'first_timestamp': ['10'], 'last_timestamp': ['10.001'], 'sample_count': ['1']})}, 'info': defaultdict(<class 'list'>, {'channel_count': ['2'], 'channel_format': ['string'], 'created_at': ['10'], 'effective_srate': np.float64(999.9999999987779), 'name': ['SendMarker'], 'nominal_srate': ['1000'], 'segments': [(np.int64(0), np.int64(0))], 'stream_id': 3735928559, 'type': ['Marker']}), 'time_series': [['Marker 0A', 'Marker 0B']], 'time_stamps': array([16.9])}

The time stamp is 16.9, or no? This also means that the clock offset measurements precede the timestamp and not the other way round.

Hi Clemens,
This PR, was suggested by you in cbrnr/XDF.jl#14, I loaded it with https://github.com/cbrnr/XDF.jl

Here's the output for twochannel_string_marker.xdf for every stream using XDF.jl:

Pair{Int64, Any}(3735928559, Dict{String, Any}( "time" => [16.725987961266686], "data" => ["Marker 0A" "Marker 0A"], "nchannels" => 2, "name" => "SendMarker", "dtype" => String, "srate" => 1000.0f0, "clock" => [6.1], "offset" => [-0.1], "footer" => "<?xml version=\"1.0\"?><info><first_timestamp>10</first_timestamp><last_timestamp>10.001</last_timestamp><sample_count>1</sample_count></info>", "header" => "<?xml version=\"1.0\"?><info><name>SendMarker</name><type>Marker</type><channel_count>2</channel_count><nominal_srate>1000</nominal_srate><channel_format>string</channel_format><created_at>10</created_at></info>", "type" => "Marker") ) Pair{Int64, Any}(46202862, Dict{String, Any}( "time" => [16.725987961266686], "data" => [0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0], "nchannels" => 64, "name" => "SendDataC", "dtype" => Float64, "srate" => 1000.0f0, "clock" => [6.1], "offset" => [-0.1], "footer" => "<?xml version=\"1.0\"?><info><first_timestamp>10</first_timestamp><last_timestamp>10.001</last_timestamp><sample_count>1</sample_count></info>", "header" => "<?xml version=\"1.0\"?><info><name>SendDataC</name><type>EEG</type><channel_count>64</channel_count><nominal_srate>1000</nominal_srate><channel_format>double64</channel_format><created_at>10</created_at></info>", "type" => "EEG") )

I get the same output as you from pyxdf.

Testing the outputs between the two libraries on minimal.xdf they seem to match:

( [ { 'info': defaultdict(<class 'list'>, { 'name': ['SendDataC'], 'type': ['EEG'], 'channel_count': ['3'], 'nominal_srate': ['10'], 'channel_format': ['int16'], 'created_at': ['50942.723319709003'], 'desc': [None], 'uid': ['xdfwriter_11_int'], 'stream_id': 0, 'effective_srate': np.float64(10.000000000000025), 'segments': [(np.int64(0), np.int64(8))] }), 'footer': { 'info': defaultdict(<class 'list'>, { 'writer': ['LabRecorder xdfwriter'], 'first_timestamp': ['5.1'], 'last_timestamp': ['5.9'], 'sample_count': ['9'], 'clock_offsets': [ defaultdict(<class 'list'>, { 'offset': [ defaultdict(<class 'list'>, { 'time': ['50979.76'], 'value': ['-.01'] }), defaultdict(<class 'list'>, { 'time': ['50979.86'], 'value': ['-.02'] }) ] }) ] }) }, 'time_series': array([[192, 255, 238], [ 12, 22, 32], [ 13, 23, 33], [ 14, 24, 34], [ 15, 25, 35], [ 12, 22, 32], [ 13, 23, 33], [ 14, 24, 34], [ 15, 25, 35]], dtype=int16), 'time_stamps': array([5. , 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8]), 'clock_times': [6.1, 7.1], 'clock_values': [-0.1, -0.1] }, { 'info': defaultdict(<class 'list'>, { 'name': ['SendDataString'], 'type': ['StringMarker'], 'channel_count': ['1'], 'nominal_srate': ['10'], 'channel_format': ['string'], 'created_at': ['50942.723319709003'], 'desc': [None], 'uid': ['xdfwriter_11_str'], 'stream_id': 46202862, 'effective_srate': np.float64(10.000000000000016), 'segments': [(np.int64(0), np.int64(8))] }), 'footer': { 'info': defaultdict(<class 'list'>, { 'writer': ['LabRecorder xdfwriter'], 'first_timestamp': ['5.1'], 'last_timestamp': ['5.9'], 'sample_count': ['9'], 'clock_offsets': [ defaultdict(<class 'list'>, { 'offset': [ defaultdict(<class 'list'>, { 'time': ['50979.76'], 'value': ['-.01'] }), defaultdict(<class 'list'>, { 'time': ['50979.86'], 'value': ['-.02'] }) ] }) ] }) }, 'time_series': [ [ '<?xml version="1.0"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp> <sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value ></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>' ], ['Hello'], ['World'], ['from'], ['LSL'], ['Hello'], ['World'], ['from'], ['LSL'] ], 'time_stamps': array([5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9]), 'clock_times': [], 'clock_values': [] } ], {'info': defaultdict(<class 'list'>, {'version': ['1.0']})} )

and Julia

Pair{Int64, Any}(0, Dict{String, Any}( "time" => [5.0, 5.1000000000000005, 5.200000001490117, 5.300000002980233, 5.4, 5.5, 5.600000001490116, 5.700000002980232, 5.800000004470348], "data" => Int16[192 255 238; 12 22 32; 13 23 33; 14 24 34; 15 25 35; 12 22 32; 13 23 33; 14 24 34; 15 25 35], "nchannels" => 3, "name" => "SendDataC", "dtype" => Int16, "srate" => 10.0f0, "clock" => [6.1, 7.1], "offset" => [-0.1, -0.1], "footer" => "<?xml version=\"1.0\"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp><sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>", "header" => "<?xml version=\"1.0\"?><info><name>SendDataC</name><type>EEG</type><channel_count>3</channel_count><nominal_srate>10</nominal_srate><channel_format>int16</channel_format><created_at>50942.723319709003</created_at><desc/><uid>xdfwriter_11_int</uid></info>", "type" => "EEG") ) Pair{Int64, Any}(46202862, Dict{String, Any}( "time" => [5.1, 5.2, 5.300000001490116, 5.400000002980232, 5.5, 5.6, 5.700000001490116, 5.800000002980232, 5.900000004470348], "data" => ["<?xml version=\"1.0\"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp><sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>"; "Hello"; "World"; "from"; "LSL"; "Hello"; "World"; "from"; "LSL";;], "nchannels" => 1, "name" => "SendDataString", "dtype" => String, "srate" => 10.0f0, "clock" => Float64[], "offset" => Float64[], "footer" => "<?xml version=\"1.0\"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp><sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>", "header" => "<?xml version=\"1.0\"?><info><name>SendDataString</name><type>StringMarker</type><channel_count>1</channel_count><nominal_srate>10</nominal_srate><channel_format>string</channel_format><created_at>50942.723319709003</created_at><desc/><uid>xdfwriter_11_str</uid></info>", "type" => "StringMarker") )

This doesn't look like a rounding error. Do you see where the difference is? I can't seem to find the source of it.

This is strange. I don't have the time to dive into this right now, but maybe XDF.jl does clock correction by default?

Hi Clemens, I can work on it on short periods, but continuously. If you check it out every once in a while and point me in the right direction is very helpful.

Both do clock correction by default (clock syncing):

pyxdf https://github.com/xdf-modules/pyxdf/blob/main/src/pyxdf/pyxdf.py#L77

xdf.jl https://github.com/cbrnr/XDF.jl/blob/d3bb58bd11e1699471e747974125e40e1143a5fc/src/XDF.jl#L34C45-L34C60

However, pyxdf also does de-jittering by default. https://github.com/xdf-modules/pyxdf/blob/main/src/pyxdf/pyxdf.py#L79. XDF.jl still has it in pr: cbrnr/XDF.jl#13, so we should consider turn it it on as default on that branch. I think I left it off by default.

Still, when loading pyxdf.load_xdf("twochannel_string_marker.xdf", dejitter_timestamps=False) I get the same array:

... 'name': ['SendMarker'], 'type': ['Marker'], 'channel_count': ['2'], 'nominal_srate': ['1000'], 'channel_format': ['string'], 'created_at': ['10'], 'stream_id': 3735928559, 'effective_srate': 0.0, 'segments': [(0,0)] ... 'footer': { 'info': defaultdict(<class 'list'>, { 'first_timestamp': ['10'], 'last_timestamp': ['10.001'], 'sample_count': ['1'] }) }, 'time_series': [['Marker 0A', 'Marker 0B']], 'time_stamps': array([16.9]), 'clock_times': [6.1], 'clock_values': [-0.1] ...

I think it's unlikely to be the de-jittering. It doesn't really do anything to the time stamp, (at least for this file at the floating point precision we use to compare the two libraries). I guess next I'll go debugging pyxdf, to see what it actually does. Unless you have another idea of what it could be.

Thanks!

-Original file line number
+Diff line change
@@ Expand Up / @@ -480,3 +480,101 @@ As above. @@
     #### Clock offsets
     As above.
+    ## twochannel_string_marker.xdf
+    An example file with two streams, one with string markers.
+    ### Stream 3735928559
+`string` channels, 1 sample
+    #### Header
+    ``` xml
+    <info>
+        <name>SendMarker</name>
+        <type>Marker</type>
+        <channel_count>2</channel_count>
+        <nominal_srate>1000</nominal_srate>
+        <channel_format>string</channel_format>
+        <created_at>10</created_at>
+    </info>
+    ```
+    #### Footer
+    ``` xml
+    <info>
+        <first_timestamp>10</first_timestamp>
+        <last_timestamp>10.001</last_timestamp>
+        <sample_count>1</sample_count>
+    </info>
+    ```
+    #### Time-series data
+    Data:
+    ```
+    ["Marker 0A", "Marker 0B"]
+    ```
+    Timestamp: 16.725987961266686
+    These time-stamps precede the first clock offset measurement, but for
+    synchronization they will be handled with respect to the first (and
+    only in this case) detected clock segment.
+    #### Clock offsets
+    |   time |   value |
+    |--------|---------|
+    |    6.1 |    -0.1 |
+    ### Stream 46202862
+`Float64` channels, 1 sample
+    #### Header
+    ``` xml
+    <info>
+        <name>SendDataC</name>
+        <type>EEG</type>
+        <channel_count>64</channel_count>
+        <nominal_srate>1000</nominal_srate>
+        <channel_format>double64</channel_format>
+        <created_at>10</created_at>
+    </info>
+    ```
+    #### Footer
+    ``` xml
+    <info>
+        <first_timestamp>10</first_timestamp>
+    	<last_timestamp>10.001</last_timestamp>
+    	<sample_count>1</sample_count>
+    </info>
+    ```
+    #### Time-series data
+    Data:
+    ```
+    [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,]
+    ```
+    Timestamp: 16.725987961266686
+    These time-stamps precede the first clock offset measurement, but for
+    synchronization they will be handled with respect to the first (and
+    only in this case) detected clock segment.
+    #### Clock offsets
+    |   time |   value |
+    |--------|---------|
+    |    6.1 |    -0.1 |

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

An example with double channel string marker #4

Uh oh!

Diff view

Diff view

There are no files selected for viewing

cbrnr Feb 10, 2025

Uh oh!

abcsds Feb 18, 2025

Uh oh!

cbrnr Feb 18, 2025

Uh oh!

abcsds Feb 21, 2025

Uh oh!

An example with double channel string marker #4

Are you sure you want to change the base?

Uh oh!

An example with double channel string marker #4

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

cbrnr Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

abcsds Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

cbrnr Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

abcsds Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!