CIID 2019 Insights week 1

Please write your name and your 3 insights on this page:

Suniti Thapa

Insights about your project:

1. Spatial immersion: What could have been done to better achieve spatial immersion for the project? After noticing a couple of other projects, the issue of spatial immersion for our project felt pertinent
2. Movement of the fish flock – Disconnect in how the fish flock organized themselves while moving towards a direction vs. while moving in their destination grid.
3. Water as an input medium – After seeing hesitation among the guests to dip fingers in the water containers, I questioned whether water was the right medium? What else could have been done/used for input interactions?

Insights about any project:

1. Circular interaction loop: Is it possible to complete the virtual to physical and physical to virtual loop? What kind of conflict in the immersion would this lead to?
2. Think about the journey: What happens to the game objects at the end of their journey? How do they end? People are curious to see the ending if looking for complete immersion.
3. Accessibility use cases: After reflecting on the projects exhibited, I realized that the discussion around accessibility was completely missing. Is it because that the mixed reality is an emerging field and we are busy just focusing on the tech aspect of it?

Silvia Sgualdini

Insights about your project:

1. It’s difficult to create a coherent loop! It’s not as intuitive as it seems to create a coherent narrative that connects the real and the digital in a meaningful way, so that actions in the real justify those in the digital realm and viceversa. It took a while to understand the consequences and reactions caused by each action.
2. The butterfly became a character. The presence of a ‘living’ being creates empathy in the user. The introduction of the butterfly as agent pressing the button in the digital real to switch the light off in the real caused a moment of delight in the user. I wonder if it had been a lever or another object pressing the button whether it would have had a different reaction?
3. Sound is important as a ‘cause and effect’ signal. I became aware of the sound of the relay playing a part in the project only at the demo phase. The digital butterfly ‘caused’ the sound of the relay in the real world by gently landing on the digital button. The sound made this transition clear and justified the light switch turning off.

Insights about any project:

1. The fiducial can be any size and anywhere! (sort of) The possibility of wearing the fiducial in the ‘rocket’ project created a whole new dimension - the possibility of triggering something through one’s own body and position in space. The awareness of this possibility opened reflections both on self-image and its manipulation and potential developments in advertising and sponsorship.
2. When you see it, it hurts. In the finger chopping project, what made the reaction so visceral was both the image onscreen and the anticipation of risk (when/will it really happen?) as well as the solenoid valve noise, which created a rhythm in synch with the action onscreen. Having one’s own body as a vulnerable part in the ‘game’ is extremely powerful.
3. You see what you get. The water/salt pouring project played on our expectations from the real world. Gravity makes things flow downwards so the confirmation of this behaviour on the digital display caused by the accelerometer is reassuring. A moment of delight happens when we see this particles sprinkling into a real container. Nothing out of the ordinary except that the particles are not really there but they are imaginary! The colour blue contributed to this feeling of ‘magic’. Water is actually transparent but we can imagine it as blue in cartoons and drawings.

Yang Li

Insights about your project:

1. The imperfect perfect of the virtual building. In the creation of the virtual part of the narrative, the fussiness is hiding behind the rendering. Or, in other words, the fussiness is being replaced by the banality or the presetting. But how to bring this side to the audience?
2. The blend of overview and the small moments in the whole make the Rube Goldberg machine exciting. People understand what will happen but the excitement is about whether it will make it or not. The delicacy of the physical setting adds up the excitement.
3. Back and forth loop. There is the singularity from the virtual to the real in the project, but what could be more interesting to help build the MR feeling would be the back and forth of different components triggering each other.

Insights about any project:

1. The scale and the placement of fiducials have the power to hide and transform. The rocket project raises an interesting point when the human body becomes an activation point for the MR, it carries the part of hiding part of the body and replaces with something else. It would be interesting to play with this quality of the MR into different contexts.
2. There is no single presence in the MR world, thus it is important to tightly attach different components from the two sides to be able to hold the narrative. The moth project, the moment the moth stops the switch, there is the sound from the relay and the light puts out. It would be totally different if there is a delay of the time in between.
3. MR may affect other perceptions too. The key is about the story hiding behind the invisibility hiding behind the diversion between the virtual and real. From the crystal shaker project, the actions are well executed that it challenges people’s belief system about what is being added in the water. What happens after? What is being added in the water? Will it taste different?

Laurin Kraan:

Insights about your project:

1. Details matter: small deviations between expectation and reality (in our case the interactive behavior of the fish) can lead to an unfulfilled experience.
2. Placement of screen: there might be a better, more engaging way than just placing the digital output (computer display) next to the physical input (pond).
3. Moment of joy: the actual moment of joy is often unplanned and subjective. In our case it was not the interaction with the water, but fooling around with the fish by never letting them arrive at the destination they’re heading to.

Insights about any project:

1. Closing the loop: most projects were unidirectional: a physical input triggers a virtual output (e.g. the falling ball switches on the fan). Moving to a bidirectional interaction might make the experience more engaging (the fan could trigger something in Unity).
2. Randomness: adding randomness in the right place (e.g. in the knife game) could make the experience much more intense and exciting.
3. Tricking the mind is an interesting field to explore with MR. This can be done in different ways, e.g. by establishing a connection between physical haptics and virtual output (knife game). Or by the uncertainty/hope if a certain event can be avoided (e.g. butterfly switch off the light bulb).

Aditi Vijay

Insights about your project:

1. Introducing cloth merchandise(t-shirt) adds to the interaction. It’s interesting how it can be controlled by manipulating the scale, distance, and placement between the fiducial and the camera.
2. Rocket and Astronaut visualization explained the idea of digital to physical through LEDs.
3. Rocket being on wall Adds as an ambient element with the visuals on the screen.

Insights about any project:

1. Details and focus matter: In the knife and finger game, what made it feel so real was the onscreen video in sync with the solenoid valve noise and that you actually felt it in on your finger in the end.
2. Blue Imaginary crystals: It was magical to see how the blue lights on the sense hat looked so real on the screen being poured into a real container.
3. Feed the fish: Putting the finger in actual water to see the fish move towards your finger on the screen was playful. Details like the school of fish and blue water to make it resemble the actual abode was interesting.

Marc Raum:

Insights about your project:

1. Extending the Do-feel-know loop to the virtual part of the AR is essential to make the AR experience plausible - and don't forget to close it by linking it to the conventional reality

• Think about whether virtual controls conventional reality or vice versa (and if so: which part does what)
• Think about what the AR characters know and how they get that information
• Think about how it feels for the AR characters
• Think about the mental model of the user in the conventional reality - can you exploit an existing metaphor?

(In a sense that looks like cycling through the infinity symbol with the left half being the AR and the right part being the conventional reality)

2. Microinteractions matter a lot
3. Setting up the scene in a reliable way is an art in itself - angles and distances can change the placement of the objects a little and so does switching from preview- to fullscreen-mode. That sucks a lot if you want to control your collisions in a precise way

Insights about any project:

1. Storytelling and framing the AR scene can go a long way - your imagination completes what you've not really implemented
2. Spatial overlays that are timed well make for plausible AR experiences
3. Fiducial objects are salient objects. This feature can be exploited in the storytelling and interaction.

Fei Kwok:


Insights about your project:


1. Clothing as a medium for advertising (e.g., fiducial printed on garments). How would people feel/engage with this (both as a bystander and ”user”)? How would this change our social norms or the way we engage with our environment? Could this lead to deceptive advertising such that our real world is augmented/distorted in misleading ways?

2. Would mixed reality make it harder for humans to differentiate between what is real and what is virtual? How would we define reality? 

3. Ethical quandaries - environment is designed and controlled by developers. Are users (without programming ability) merely a recipient with little autonomy for creation? Does this leave room for manipulation of our environment? 


Insights about any project:


1. Changes in social norms and privacy - broader spectrum of information collected to render content for more immersive experiences (e.g., biometric data, behavioural monitoring).

2. Blending reality with digitally created assets will change our relationship with computers - one that is more collaborative, natural and intuitive?
 The wow factor of the digital experience extending into the physical world (e.g., pouring of the blue crystals).
3. Speed of technological advancement vs. time it takes for humans’ mental models to adapt to and be fully comfortable to this technology.

Mitsu Shah:

Insights about your project (Harvesting LED crystals):

1. When merging the real and virtual, storytelling plays an important role. It helps feed in to the audience’s (people experiencing it) imagination and builds a good start point for them to experience the merged reality.
2. Framing matters. What part of the experience do you want the audience to see? How do you want them to form the relationship between the real and the virtual- makes a big difference to the audience experiencing the installation/ project.
3. Working on peoples’ known psyche. Paying attention to micro interactions (and adding a twist to them- probably changing the expected outcome?) can bring about a very fun, unexpected and ‘magical’ experience.

Insights about other projects:

1. An interaction needs to complete the do-feel-know loop to bind the entire MR experience together.
2. Sound, visual cues and story telling are all important elements to make the MR experience more credible.
3. To exploit the possibilities of the fiducial and not just think of it as a static point of reference in AR.

Lea Lemaire:

Insights about your project (Pinball game):

1. It is very hard to make a coherent story jumping back and forth from real to digital. We need to make them interact more.
2. In order to make the Pinball game more AR-relevant, we would need to only have the ball to be digital and have it bounce on physical objects, perhaps the user places them himself. *Think of the screen as a passthrough from the real to digital.
3. Sounds could be a good addition as feedback.

Insights about any project:

1. Play and explore with the fiducial.
2. The element of surprise, aligning with the real world the expectations can really push a "wow" effect! It's all about timing (Water on crystals, Finger-knife game, Lamp-butterfly...).
3. Microinteractions + a good story is the key.

Minatsu Homma

Insights about your project:

1. Know user's conventional interactive expectation towards an object and design to exceed it by adding some surprising moment. Most users already have set of expectation for pinball game, and just replacing the component with physical object is not enough to create an immersive experience.
2. It is important to design do-feel-know loop and user journey when designing the immersive experience. We should discuss what feeling the object will bring to the users for what and how.
3. When you choose a physical component, make sure you can prepare supported all software environment. (e.g. Python 2 vs 3)

Insights about any project:

1. MR can create an emotional attachment to the experience which would be annoying without MR (butterfly switch)
2. Physical interaction can create 'wilder' feedback compared to the digital one. (Tick-tock sound of knife game)
3. When physical and digital feedback are aligned well, it can create perceptual illusion ('painful' moment from knife game)

Aakash Aggarwal

Insights about my project:

1. ‘Flocking algorithms’ can be used to make so many cool installations with birds, butterflies, fishes etc. if done correctly.
2. While creating AR simulations inspired by nature, a lot of focus needs to be given on minute details (response time, the direction of movement) to make the entire experience feel more organic rather than robotic
3. Water switches were not the best input; in our case, if we had used some other sensors for detecting the touch of the user it would have prevented us from putting water into compartments

Insights about other projects:

1. AR can bring out extremely real emotions and make people feel something is happening while actually nothing is happening
2. AR as 3D Storytelling: Putting fiducial on a person opens up a lot of possibilities of storytelling in the 3D space
3. Mixed reality can be a very powerful way to explain concepts that are hard to visualize. Eg: Structure of Atom, Bonding in Crystalline Solids