step type in a staircase #18
Comments
|
Maybe on Friday we can look at some data files from the RAs runs and make sure the staircase is adjusting accuracy the way we want? |
|
Once the code is set and I'm able to have RAs run through the entire experiment without errors, I'll need to make sure the accuracy is adjusting appropriately. If not, it's possible we may need to either 1) change the steptype, or 2) change the way the staircase is working and use pilot data to calculate specific parameters... |
|
So, I just did a quick check on the longest run that we've gotten so far, and I looked at the accuracy in the Environmental condition. For the "easy" staircase, the RA only got 18/60 trials correct. For the "hard" staircase, she got 31/60 correct. So obviously something isn't working correctly. Will see what I can figure out... |
|
It could be that they are just equally hard.
…On Thu, Feb 2, 2017 at 3:27 PM, kcm1231 ***@***.***> wrote:
So, I just did a quick check on the longest run that we've gotten so far,
and I looked at the accuracy in the Environmental condition. For the "easy"
staircase, the RA only got 18/60 trials correct. For the "hard" staircase,
she got 31/60 correct. So obviously something isn't working correctly. Will
see what I can figure out...
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#18 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF0T7N-_QBLH73CIKnPSp679nVANRAEjks5rYjw0gaJpZM4LiJjB>
.
|
|
I think that I figured out what was happening here. Traditional staircase procedures are used to discover a threshold (ie a JND), and they also usually change the level of the noise rather than the signal. With our staircase, we want to keep them at a predetermined level, and we are changing the value of the signal rather than the noise. So, this means that instead of using the 3up 1down and 1up 1down, we need to reverse the step directions, and probably also adjust the values. I'm going to run some tests using a 1up 5down for easy and a 1up 2down for hard. We'll see how that works out. After talking with Peter, we may try to implement a different check on accuracy for the main staircase. The baseline staircase will hopefully get them closer to the desired easy and hard thresholds, and then in the main staircase, we want to implement a piece of code that checks the accuracy using a sliding window (i.e. rolling average) of the previous 10 trials, then adjusts either up or down depending on if accuracy is below or above the desired threshold. Our new postdoc, Andrew, has a bunch of coding experience but hasn't really used Python, but he said that if you're busy working on the other edits and your own stuff, he can try to implement this accuracy check for us. However, let me know if this is clear, and if it sounds like something that would be really quick for you to add in. If it's an easy change, I don't want him to mess with your code since you have it very well organized and formatted... |
|
It is easier for me to edit later if people haven't added things... but if
you need it asap that is fine.
Checking the accuracy isn't going to be straightforward because there are
several interleaved staircases... it isn't hard to do, you just have to
make sure that you are checking the previous trials in the correct
staircase and level.
…On Fri, Feb 3, 2017 at 10:24 AM, kcm1231 ***@***.***> wrote:
I think that I figured out what was happening here. Traditional staircase
procedures are used to discover a threshold (ie a JND), and they also
usually change the level of the noise rather than the signal. With our
staircase, we want to keep them at a predetermined level, and we are
changing the value of the signal rather than the noise. So, this means that
instead of using the 3up 1down and 1up 1down, we need to reverse the step
directions, and probably also adjust the values. I'm going to run some
tests using a 1up 5down for easy and a 1up 2down for hard. We'll see how
that works out.
After talking with Peter, we may try to implement a different check on
accuracy for the main staircase. The baseline staircase will hopefully get
them closer to the desired easy and hard thresholds, and then in the main
staircase, we want to implement a piece of code that checks the accuracy
using a sliding window (i.e. rolling average) of the previous 10 trials,
then adjusts either up or down depending on if accuracy is below or above
the desired threshold. Our new postdoc, Andrew, has a bunch of coding
experience but hasn't really used Python, but he said that if you're busy
working on the other edits and your own stuff, he can try to implement this
accuracy check for us. However, let me know if this is clear, and if it
sounds like something that would be really quick for you to add in. If it's
an easy change, I don't want him to mess with your code since you have it
very well organized and formatted...
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#18 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF0T7CnqS0Xp4FkLvjH91JCwo5AQlTUkks5rY0aTgaJpZM4LiJjB>
.
|
|
Ok! Yeah, I figured it would be better to have you do it yourself, but wanted to make sure. I'll tell him not to worry about it. We don't need that any sooner than any of the other edits we've already talked about, just wanted to make sure that wasn't going to add a huge chunk of time onto what you already had to do! |
|
Hey there, Experimenting with some different nUP and nDOWN numbers, and on the last couple of runs it's been doing something weird. I'll send the parameter and exp file used, but I had it set at an 1up 10down for easy and 2up 3down for hard, with a step size of 6 for the baseline and 2 for the main staircase. I'm looking at the intensity levels of the stimuli, and on the easy baseline staircase instead of changing by a step size of 6, on one trial the intensity jumps from 7 to 15, and then on another trial from 15 to 31, and then from 31 to 63 (meaning the sounds are super loud and distorted). The step type is db, not log. I have no clue why this could be happening. |
|
If you just want it to step up by the step size (6), then you need it to be
'lin' not 'db'.
…On Tue, Feb 7, 2017 at 1:22 PM, kcm1231 ***@***.***> wrote:
Hey there,
Experimenting with some different nUP and nDOWN numbers, and on the last
couple of runs it's been doing something weird. I'll send the parameter and
exp file used, but I had it set at an 1up 10down for easy and 2up 3down for
hard, with a step size of 6 for the baseline and 2 for the main staircase.
I'm looking at the intensity levels of the stimuli, and on the easy
baseline staircase instead of changing by a step size of 6, on one trial
the intensity jumps from 7 to 15, and then on another trial from 15 to 31,
and then from 31 to 63 (meaning the sounds are super loud and distorted).
The step type is db, not log. I have no clue why this could be happening.
Rami_1_A_AV-2017-Feb-07-115207.xlsx
<https://github.com/kschuler/words-in-noise-EEG/files/758599/Rami_1_A_AV-2017-Feb-07-115207.xlsx>
experiment_Rae_edit.txt
<https://github.com/kschuler/words-in-noise-EEG/files/758601/experiment_Rae_edit.txt>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#18 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AF0T7FfHc3SWz5HRflpBqIhDSINFU1M8ks5raLZhgaJpZM4LiJjB>
.
|
The text was updated successfully, but these errors were encountered: