Desktop PIP runner

[baranan]

We have a desktop installation for running PIP locally using a local installation of node.js. However, because the code still runs from a browser, I guess that the time precision is significantly inferior to most other desktop programs that run experiments.
If there was a solution for running PIP scripts locally with good time precision, it could encourage labs start using PIP code for all their studies, in the lab and online. That will increase PIP's popularity and encourage others to contribute to PIP's development. It will also help the core PI labs.

My ideas so far:

1. Use an existing open-source software (e.g., PsychoPy) to write an interface between PIP's code and PsychoPy implementation.
2. Perhaps there are programs that run javascript/html code locally with good time precision.
3. Perhaps time precision is much better when running PIP code locally using a local installation of node.js on one of the modern browsers, or in a certain system configuration.

Perhaps others have more ideas how to achieve this goal.

[eladzlot]

1. Possible. Seems like an over extension of our resources to me (see below).
2. https://github.com/rogerwang/node-webkit offers one of many such interfaces. It is worth a try...
3. The limitation on precision is in the browser. There is no difference where the server resides. (The only difference is what browser you use, and a local installation could force users to use modern browsers - read: no IE).

I think that we should concentrate our efforts on creating one high quality service and not trying to overwhelm the whole market at once.
Regarding precision, there are some tricks http://www.html5rocks.com/en/tutorials/audio/scheduling/ that may allow us to improve precision and timing, but they are available only on modern browsers. It seems chromium may have somethings to add to this discussion but I haven't really put the time to look into all the really cutting edge technologies out there...

On Mon, Jun 30, 2014 at 5:28 PM, baranan [email protected] wrote:

We have a desktop installation for running PIP locally using a local
installation of node.js. However, because the code still runs from a
browser, I guess that the time precision is significantly inferior to most
other desktop programs that run experiments.
If there was a solution for running PIP scripts locally with good time
precision, it could encourage labs start using PIP code for all their
studies, in the lab and online. That will increase PIP's popularity and
encourage others to contribute to PIP's development. It will also help the
core PI labs.

My ideas so far:

1. Use an existing open-source software (e.g., PsychoPy) to write an
   interface between PIP's code and PsychoPy implementation.
2. Perhaps there are programs that run javascript/html code locally with
   good time precision.
3. Perhaps time precision is much better when running PIP code locally
   using a local installation of node.js on one of the modern browsers, or in
   a certain system configuration.

Perhaps others have more ideas how to achieve this goal.

—
Reply to this email directly or view it on GitHub
https://github.com/ProjectImplicit/PIPlayer/issues/19.

[baranan]

1. The limitation is in the browser, but perhaps if we disable any competing program and run the experiment in a plugins-clean FF or Chrome, time precision becomes acceptable.
2. node-webkit seems interesting. They mention having timer that is superior to WebKit's (https://github.com/rogerwang/node-webkit/wiki/Console-and-timers). And, if it is open-source, we might be able to improve the timing and display, perhaps using code from one of the open-source experiment programs available.
3. Regarding improving timing in PIP: @eladzlot, do you know window.requestAnimationFrame()? It reminds me a function I used when programming in E-Prime, to improve the precision of stimuli presentation duration.
4. Regarding on what to concentrate, it is too early to decide. It will be good to first discuss our options. If there is an easy option worth trying, I might decide to devote a few days for this, or perhaps I'll stumble on a student who can do that for cheap.

[eladzlot]

1. The browser limitation does not come from competition. Some of browser optimization is polling the system time at a relatively large intervals, that cause timing problems.
2. This depends on a node.js environment. Worth a try.
3. Yes I do, the problem is browser support. If we decide to go into high precision timing we should take into consideration that primitive browsers (read IE9 and older at the very least), won't be able to support the high quality timing (although I don't see a reason that they won't be able to run any more...). Also, it gives us a minimum time resolution of 10 micro seconds (what is the meaningful resolution for cognitive science?).
4. I agree, I think the first thing that we need in order to assess these things correctly is a quantitative measure for accuracy. As well as clear definitions of what timings are critical for us (stimulus display, response latency etc.).

[JeffSpies]

Have you guys looked at (parentheses are in the url):

https://developer.mozilla.org/en-US/docs/Web/API/Performance.now()

?

More info:

http://updates.html5rocks.com/2012/08/When-milliseconds-are-not-enough-performance-now

And spec:

http://www.w3.org/TR/hr-time/

On Tue, Jul 1, 2014 at 3:13 AM, eladzlot [email protected] wrote:

The browser limitation does not come from competition. Some of browser
optimization is polling the system time at a relatively large intervals,
that cause timing problems.
2.

This depends on a node.js environment. Worth a try.
3.

Yes I do, the problem is browser support. If we decide to go into high
precision timing we should take into consideration that primitive browsers
(read IE9 and older at the very least), won't be able to support the high
quality timing (although I don't see a reason that they won't be able to
run any more...).
4.

I agree, I think the first thing that we need in order to assess these
things correctly is a quantitative measure for accuracy. As well as clear
definitions of what timings are critical for us (stimulus display, response
latency etc.).

—
Reply to this email directly or view it on GitHub
https://github.com/ProjectImplicit/PIPlayer/issues/19#issuecomment-47624061
.

[eladzlot]

@JeffSpies We're already using performance.now() when possible.
I'm thinking our bottleneck is with timeouts that get called at the wrong time (because js is one threaded), and possibly with measuring latency from the exact time that stimuli are displayed.

These are already old, but worth the read:
http://ejohn.org/blog/how-javascript-timers-work/
http://ejohn.org/blog/accuracy-of-javascript-time/

[baranan]

Here is an article reporting accuracy tests of an open-source experiments software written in Flash:
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0067769
The first three studies can provide us some ideas about the kind of hardware that we could use for similar tests for PIP.

[eladzlot]

There are three ways that we can assess the speed/efficiency of the player:

• Analysing user data (this is expensive in time and resources)
• Using automated testing; relying on selenium/dev tools etc. to analyse response times.
• Using spectroscopes (or other external equipment) to test real time performance.

This isn't currently high performance, but in order for us to start improving performance we need clear criteria for testing it. The criteria should take the form of propositions such as "it should display stimuli within 16ms of assigned time-frame" so that we can create specific benchmarks.

• Write test specifications ( @baranan )
• Create simple test instances ( probably simple PIP trials @eladzlot )
• Choose appropriate testing technology ( @eladzlot )
• optimize! ( @eladzlot )

[eladzlot]

Good (but old) article about requestAnimationFrame http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/