Interaktionstaxonomie AR

This page is a sort of lookup table of different modes of interaction using Augmented reality. Eventually, this should be converted into a card game.

Input - technical

exposed in unity:

• Touch
• Multitouch
• Parsing Textfield
• Timer
• Time

• GPS (location)
• Gyroskop
• Accelerometer
• Light Input
• ARCore (position)
• Image Marker 2D (pixelcluster - ARCore, Vuforia)
• Object Marker 3D - non-moving (Object Targets - Vuforia)

networking:

• WiFi Network strength
• Cellphone Network Coverage
• Network SSIDs
• Network commands (OSC)
• Bluetooth Beacon

non-trivial:

• Gesture recognition
• Voice
• Object Classification (ML - Tensorflow)
• Face Tracking (OpenCV - ML Tensorflow)
• Body Segmentation (ARKit only - for now)

Input - human

bodily interaction

• Shake phone
• Turn around
• tap on screen (on object)
• move device to frame Image
• Walk to Location
• Turn on/off Flashlight
• Talk to device
• Talk to other Person

cognitive framing

• cast magic
• start tricorder analysis
• throw virtual pokeball
• go to the virtual representation of the Planet Pluto
• stand inside the particle stream
• do a sensor sweep
• engage warp drive
• run away from spiders
• throw rock at spider
• magnify spider
• send message in a bottle
• remove your own shadow (skill level five thousand)
• level up (from skill level fourthousandninehundredandninentynine)
• answer a question
• ask a question
• answer a riddle
• solve a puzzle

...

Output - technical

direct

• Text appears/disappears on screen
• Mesh appears/disappears on screen
• Skinned Mesh Animation starts/stops
• Play/Stop Animation
• Move Camera
• UI appears/disappears
• Sound gets played/stopped
• Video starts/stops
• Phone vibrates/stops vibrating
• Send Network message
• Camera turns on/off
• App closes/starts

indirect

• Simulation gets updated
• Method of Interaction changes
• World State changes (Portal opens, etc…)

Output - human

on-device

• receive a question
• receive information
• receive new mode of interaction
• receive new task
• receive message from other human
• hear/read someone’s thoughts
• listen to music
• feel vibration
• see a specific place
• watch a movie
• look at a different time
• look at a different place

in context

• be in the center of the universe
• stand on Mars
• Travel through Wormhole
• Move Towards Black hole
• feel Neutrinos