I had the pleasure of meeting Josh Blake last week on the LEF Study Tour, those that know him, will know that my mind is now filled with the wonders of Natural User Interfaces or NUIs
Natural... Hmmm .... of Nature
So they must at least appear Natural... which would mean that they would appear to be unsupported by technology.
I reach to Clarke's Third Law:
Any sufficiently advanced technology is indistinguishable from magic.
So actually we are really after building MUIs!
Over the week we started exploring the various dimensions of NUIs
On the plane home I extended our list and mapped some of the relationships. I so needed Josh's patient thinking support
(Josh: I really did appreciate your gentle patience as you lowered me into this new space.... but I still have problems with your definition of NUI :-) )
So here in no particular order ( he says lying through his teeth) are my latest dimensions:
Agency (OK so you knew I'd put that one first! For it's surely the "raison d'être" of any interface!)
Affordability (ie Affordance - look it up it does't relate to monetary cost, it's more like learning cost)
Efficacy = Affordability / Effort ?
Efficiency = Utility / Effort ?
Manageability, describes the ease of adding new things that can be controlled
Utility, Outcome Value
Magic was added after the experience of sitting at the back of the room at Microsoft watching Josh drive his presentation by apparently effortlessly interacting with, who he later revealed as, "Preso", it certainly appeared to be Magic.
I have all the bits in my lounge (apart from the magic linking software) that would allow me to gain the attention of my imagined Home Agent "Auto" , point at a light switch and indicate what I want to do with it... Switch On, Dim, Swich Off, Wait why would I point at the switch? I would of course point at the light, that would be more natural right... ? But last night as I was practicing interacting with the imagined "Auto"... I pointed at the switch. I wonder why.... I suppose I am not yet conditioned to interact with MUIs
For those interested in developing the software, I have
an X Box and Kinect
my light switch is X10 enabled
A Mac Mini running Home automation software Indigo,
Indigo Touch on my iphone and iPad
So the real magic will be to make it a mundane exercise to connect Kinect & Indigo
I tried explaining this to the folks who develop such standards and APIs and failed badly!
Such things should be able to, when I ask them to engage in the exercise, discover all the things that they can interact with and associate with them.
Until all the Things are that smart, I need an API Agent, who I have called Api , that helps me connect all my Things together... now that would be Magic!
(Aside I hope the FIDO Alliance keeps focussed on Things!)
Josh I have developed My hand commands for Auto
Double Silent Finger Click = Attention Auto
Point = Select Thing I want to interact with
Palm up and still = STOP!
( Attention followed by Stop = STOP ALL a sort of emergency Stop Command)
Palm Down drop hand = Lower
Palm Up raise hand = Raise
Imaginary Knob Grip Twist Left = Turn Down
Imaginary Knob Grip Twist Right = Turn Up
Finger to my Lips = Go into Silent Mode Auto
Palm forwards and forward movement = Close
Palm face inwards an backwards movement = Open
On and Off are a lot harder to imagine as natural commands the closest I have come is
On start with finger and thumb open then touching the two
Off start with finger and thumb touching then opening the two
Though I can also imagine touching finger and thumb as the command for both
A set of Hidden Commands acts as Duress Commands which mean
don't do this or call for help, or emergency erase sensitive info
Actually all the commands should be able to be personalized
I want Auto and Api now!
I do know that Smart Things will take a little longer
magic and Interfaces have been explored before.... darn again I wasn't original
Some other reading on the subject
Side note: The exclusionists in Wikipedia have gained ascendancy and they eradicate terms like Magical User Interfaces as they consider them proto-neologisms and refuse to use Wikipedia to be used to track the birth of knowledge. I find it hateful, but then I would as I am an inclusionist! So don't bother to look for the term in Wikipedia yet, as you will only find the Amiga software!