Microsoft HoloLens: Controller
Imagining what a controller for the Microsoft HoloLens could look like, and how it could improve the product experience.
Project Overview
Microsoft + WWU
Western Washington University (WWU) and Microsoft teamed up for a ten week project imagining future interactions with Microsoft’s HoloLens. The goal was to research a specific area the HoloLens could benefit and design an enchanted object for that space. An enchanted object is a physical object that is able to interact in both physical and mixed reality spaces.
10 weeks
Project Team
Edward Hanko | Jill Kuyt | Peter Jacobson | Trevan Strean
Individual Role
Human Interaction Research / Ideation / User Experience / Conceptual Development / Illustration / Graphic Design / Presentation Design / Project Management
Don't know what HoloLens or Mixed Reality is? Watch this brief video from Microsoft to familiarize yourself!
Form Exploration
| What forms and tactile features communicate control, intuition, and delight?
The team's initial explorations focused around the dimensional nature of mixed reality. We wanted to create interactions that would mimic how humans traditionally interact with their environment. However, we discovered that we had been saturated by the 2D screens in our world and had to turn elsewhere for inspiration.
Interaction Research
| What is an intuitive 3D gesture? What interactions do certain forms communicate? We turned to kids for help.
Children became a prominent influence on the creative process because of their natural creativity and imagination, tactile interaction with objects, and their lack of preexisting understanding of two dimensional input methods.
Insights
| The Mixed Reality experience is magical, so controlling it should be equally as delightful.
The concept of delightful control was a synthesis of our research in human physicality, childlike imagination, and physical and digital interaction. Maintaining the magic and wonder of the holographic world through a fluid digital experience can make using the HoloLens more human and delightful.
TECHNICAL
Levels of Control
Precision
Fatigue Management
INTERACTION
Gesture
Shareability
Physical Feedback
FORM
Comfortable to Hold
Dignified as an object
Communicates function
MAGIC
Physical Magnetism
Mixed Reality Effects
Discoverability
Product Development | Hardware
| Form and conceptual refinement of the controller with the addition of a stand to give the object a home and stance.
We explored a multitude of concepts, almost all of which could be categorized into three categories: Simple, directional, one handed controllers; Two distinct controllers for advanced two-handed control; Two-handed control combined into one object. However, the device itself didn't provide enough allure. The device needed stance in order to achieve the dynamic attraction the vision required. A thorough exploration of companion objects provided a solution while also adding appeal to the object when at rest.
Product Development | UI / UX
| Developing the relationship between the physical world and mixed reality.
It was important to create interactions that were both digital and tactile, so different control modes were explored to create different types of control and functionality. In order to achieve the greatest amount of control in the simplest object, a storyboard was created that outlined the different inputs and how they could be combined or used to improve productivity while working in HoloLens. The team also developed a sharing platform to reduce the isolation experienced by users of the HoloLens, the team explored different ways to facilitate physical and digital interactions.
Solution
| A dual mode controller with a simple, versatile form, finished to fit within the HoloLens suite of products.
To enhance the control experience within mixed reality, the device supports a collection of holographic tools that mimic real world applications. A haptic engine creates interactions with the digital world, giving the user feedback taps whenever the device encounters the surface of a hologram.
Hammer Mode
| Hammer is intended for gross control.
Hammer grip allows for fluid three dimensional manipulation of objects through a touch sensitive hemisphere. Utilizing the gaze control of the HoloLens, Hammer orientation lets the user perform more actions by combining gesture and directional controls with a traditional clicker function. The controller’s Hammer inputs can be used outside of field of view to reduce fatigue.
Scalpel Mode
| Scalpel is intended for precise interaction with holographic content.
Scalpel grip allows the user to interact precisely with the holographic world. The HoloLens recognizes the tip of the device as the cursor, allowing intimate control within the field of view. In specific applications, a variety of tip options will appear, facilitating different types of control that mimic real world activities.
Storytelling | Presentation
To capture the magical feeling we sought throughout the project, the team presented the final solution through an illustrated children's story.
Special Thanks
The project was made possible by many individuals at Microsoft. We would specifically like to thank them for providing us with their time and expertise. Thank you for inspiring us!