On Magical User Interfaces

I had the pleasure of meeting Josh Blake last week on the LEF Study Tour, those that know him, will know that my mind is now filled with the wonders of Natural User Interfaces or NUIs

Natural... Hmmm .... of Nature

So they must at least appear Natural... which would mean that they would appear to be unsupported by technology.

I reach to Clarke's Third Law:

    Any sufficiently advanced technology is indistinguishable from magic.

So actually we are really after building MUIs!

Over the week we started exploring the various dimensions of NUIs

On the plane home I extended our list and mapped some of the relationships. I so needed Josh's patient thinking support

(Josh: I really did appreciate your gentle patience as you lowered me into this new space.... but I still have problems with your definition of NUI :-) )

So here in no particular order ( he says lying through his teeth)  are my latest dimensions:

Agency (OK so you knew I'd put that one first! For it's surely the "raison d'être" of any interface!)

Affordability (ie Affordance - look it up it does't relate to monetary cost, it's more like learning cost)

Efficacy = Affordability / Effort ?

Efficiency = Utility / Effort ?

Manageability, describes the ease of adding new things that can be controlled

Utility, Outcome Value

Spookiness

Mundaneness

Effort

Magic

Magic was added after the experience of sitting at the back of the room at Microsoft watching Josh drive his presentation by apparently effortlessly interacting with, who he later revealed as, "Preso", it certainly appeared to be Magic.

I have all the bits in my lounge (apart from the magic linking software) that would allow me to gain the attention of my imagined Home Agent "Auto" , point at a light switch and indicate what I want to do with it... Switch On, Dim, Swich Off, Wait why would I point at the switch? I would of course point at the light, that would be more natural right... ? But last night as I was practicing interacting with the imagined "Auto"... I pointed at the switch. I wonder why....  I suppose I am not yet conditioned to interact with MUIs

For those interested in developing the software, I have

an X Box and Kinect

my light switch is X10 enabled

A Mac Mini running Home automation software Indigo, 

Indigo Touch on my iphone and iPad

So the real magic will be to make it a mundane exercise to connect Kinect & Indigo

I tried explaining this to the folks who develop such standards and APIs and failed badly!

Such things should be able to, when I ask them to engage in the exercise, discover all the things that they can interact with and associate with them.

Until all the Things are that smart, I need an API Agent, who I have called Api , that helps me connect all my Things together...   now that would be Magic!

(Aside I hope the FIDO Alliance keeps focussed on Things!)

Josh I have developed My hand commands for Auto

Double Silent Finger Click = Attention Auto

Point  = Select Thing I want to interact with

Palm up and still = STOP!

( Attention followed by Stop = STOP ALL a sort of emergency Stop Command)

Palm Down drop hand  = Lower

Palm Up raise hand = Raise

Imaginary Knob Grip Twist Left = Turn Down

Imaginary Knob Grip Twist Right = Turn Up

Finger to my Lips = Go into Silent Mode Auto

Palm forwards and forward movement = Close

Palm face inwards an backwards movement = Open

On and Off are a lot harder to imagine as natural commands the closest I have come is

On start with finger and thumb open then touching the two

Off start with finger and thumb touching then opening the two

Though I can also imagine touching finger and thumb as the command for both

A set of Hidden Commands acts as Duress Commands which mean

don't do this or call for help, or emergency erase sensitive info

Actually all the commands should be able to be personalized 

I want Auto and Api now!

I do know that Smart Things will take a little longer

magic and Interfaces have been explored before.... darn again I wasn't original

Some other reading on the subject

http://www.wired.com/design/2013/08/design-and-the-digital-world/

http://research.microsoft.com/pubs/191705/uiphysics09.pdf

Side note: The exclusionists in Wikipedia have gained ascendancy and they eradicate terms like Magical User Interfaces as they consider them proto-neologisms and refuse to use Wikipedia to be used to track the birth of knowledge. I find it hateful, but then I would as I am an inclusionist! So don't bother to look for the term in Wikipedia yet, as you will only find the Amiga software!