BUG: Resampled column has one too many rows #358
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
effigies
commented
Jan 27, 2019
| layout = BIDSLayout('.', validate=False) | ||
| coll = load_variables(layout).get_collections('run')[0] | ||
| dense_var = coll.variables['val'].to_dense(coll.sampling_rate) | ||
| regressor = dense_var.resample(1.0 / TR).values |
There was a problem hiding this comment.
Should probably note that you do get the right shape if you go directly to 1/TR:
regressor = coll.variables['val'].to_dense(1.0 / TR).values
|
It's possible that the "bug" is simply sampling to freq A and then to freq B does not necessarily produce the same shape as sampling directly to freq B, which is fine, but what we really need to do is in methods that take |
Codecov Report
@@ Coverage Diff @@
## master #358 +/- ##
==========================================
+ Coverage 73.27% 73.34% +0.07%
==========================================
Files 24 24
Lines 2604 2604
Branches 640 640
==========================================
+ Hits 1908 1910 +2
+ Misses 513 511 -2
Partials 183 183
Continue to review full report at Codecov.
|
1 similar comment
Codecov Report
@@ Coverage Diff @@
## master #358 +/- ##
==========================================
+ Coverage 73.27% 73.34% +0.07%
==========================================
Files 24 24
Lines 2604 2604
Branches 640 640
==========================================
+ Hits 1908 1910 +2
+ Misses 513 511 -2
Partials 183 183
Continue to review full report at Codecov.
|
|
Well, if that was a fix, it wasn't the only fix needed. |
|
Not sure if its relevant, but let me share some of the obscure observations I made (without the change in f936aed) with this issue:
(i.e. this model):{
"name": "FEF_localizer",
"Input": {
"session": "movie"
},
"Steps": [
{
"Level": "run",
"Model": {
"X": [
"amplitude_.RIGHT"
]
},
"Contrasts": [],
"AutoContrasts": true,
"Transformations": [{
"Name": "Split",
"Input": ["amplitude_"],
"By": ["trial_type"]
},
{
"Name": "Convolve",
"Input": ["amplitude_.RIGHT"],
"Model": "spm"
}]
},
{
"Level": "subject",
"AutoContrasts": true
},
{
"Level": "dataset",
"AutoContrasts": true
}]
}
(i.e. this model:){
"name": "FEF_localizer",
"Input": {
"session": "movie"
},
"Steps": [
{
"Level": "run",
"Model": {
"X": [
"amplitude_.RIGHT",
"amplitude_.LEFT"
]
},
"Contrasts": [],
"AutoContrasts": true,
"Transformations": [{
"Name": "Split",
"Input": ["amplitude_"],
"By": ["trial_type"]
},
{
"Name": "Convolve",
"Input": ["amplitude_.RIGHT", "amplitude_.LEFT"],
"Model": "spm"
}]
},
{
"Level": "subject",
"AutoContrasts": true
},
{
"Level": "dataset",
"AutoContrasts": true
}]
}Traceback for the failure:190129-10:33:21,320 nipype.workflow WARNING:
[Node] Error on "fitlins_wf.loader" (/home/adina/Documents/MasterMagdeburg/USA_Dartmouth/content/localizer_project/BIDS_sacc/Jan_29_2ev_wd/fitlins_wf/loader)
190129-10:33:23,119 nipype.workflow ERROR:
Node loader failed to run on host odin.
190129-10:33:23,119 nipype.workflow ERROR:
Saving crash info to /home/adina/Documents/MasterMagdeburg/USA_Dartmouth/content/localizer_project/BIDS_sacc/crash-20190129-103323-adina-loader-e8a1d682-f64c-46f9-ba51-680f9f68a132.pklz
Traceback (most recent call last):
File "/home/adina/Repos/nipype/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
result['result'] = node.run(updatehash=updatehash)
File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 473, in run
result = self._run_interface(execute=True)
File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 557, in _run_interface
return self._run_command(execute)
File "/home/adina/Repos/nipype/nipype/pipeline/engine/nodes.py", line 637, in _run_command
result = self._interface.run(cwd=outdir)
File "/home/adina/Repos/nipype/nipype/interfaces/base/core.py", line 369, in run
runtime = self._run_interface(runtime)
File "/home/adina/Repos/fitlins/fitlins/interfaces/bids.py", line 181, in _run_interface
self._load_level1(runtime, analysis)
File "/home/adina/Repos/fitlins/fitlins/interfaces/bids.py", line 195, in _load_level1
for sparse, dense, ents in step.get_design_matrix():
File "/home/adina/Repos/pybids/bids/analysis/analysis.py", line 263, in get_design_matrix
for n in nodes]
File "/home/adina/Repos/pybids/bids/analysis/analysis.py", line 263, in <listcomp>
for n in nodes]
File "/home/adina/Repos/pybids/bids/analysis/analysis.py", line 402, in get_design_matrix
sampling_rate=sampling_rate, **kwargs)
File "/home/adina/Repos/pybids/bids/variables/kollekshuns.py", line 348, in to_df
in_place=False).values())
File "/home/adina/Repos/pybids/bids/variables/kollekshuns.py", line 279, in resample
kind=kind)
File "/home/adina/Repos/pybids/bids/variables/variables.py", line 439, in resample
var.resample(sampling_rate, True, kind)
File "/home/adina/Repos/pybids/bids/variables/variables.py", line 454, in resample
f = interp1d(x, self.values.values.ravel(), kind=kind)
File "/usr/local/lib/python3.5/dist-packages/scipy/interpolate/interpolate.py", line 433, in __init__
_Interpolator1D.__init__(self, x, y, axis=axis)
File "/usr/local/lib/python3.5/dist-packages/scipy/interpolate/polyint.py", line 60, in __init__
self._set_yi(yi, xi=xi, axis=axis)
File "/usr/local/lib/python3.5/dist-packages/scipy/interpolate/polyint.py", line 125, in _set_yi
raise ValueError("x and y arrays must be equal in length along "
ValueError: x and y arrays must be equal in length along interpolation axis.Maybe relevant facts:
nib-ls sub-01/ses-movie/func/sub-01_ses-movie_task-avmovie_run-*_space-MNI152NLin6Sym_desc-highpass_bold.nii.gz
sub-01/ses-movie/func/sub-01_ses-movie_task-avmovie_run-1_space-MNI152NLin6Sym_desc-highpass_bold.nii.gz float32 [ 68, 84, 58, 451] 2.50x2.50x2.50x2.00
sub-01/ses-movie/func/sub-01_ses-movie_task-avmovie_run-2_space-MNI152NLin6Sym_desc-highpass_bold.nii.gz float32 [ 68, 84, 58, 441] 2.50x2.50x2.50x2.00
sub-01/ses-movie/func/sub-01_ses-movie_task-avmovie_run-3_space-MNI152NLin6Sym_desc-highpass_bold.nii.gz float32 [ 68, 84, 58, 438] 2.50x2.50x2.50x2.00I'm pushing the slightly larger dataset to branch |
|
I'd be kind of surprised if this has anything to do with the event structure per se... I think what's probably going on is that only some models have timing information that's causing rounding irregularities. I do plan to look at this and come up with a fix, but am deferring all non-essential work until I have a draft of 0.8 pushed. |
This was referenced Jan 29, 2019
|
Closed by #365 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here's a test to demonstrate what I described in #355. There's probably a way to do this without a simulated layout, but hopefully this demonstrates the issue in a plausible context.
I don't understand the cause yet. @tyarkoni Please let me know if it's obvious to you.