Microsoft • May 2015 - November 2018
Microsoft. Mixed Reality Viewer
Within the 3D4E program and while working on Paint3D, the team acknowledged the crucial role of 3D model viewing, essential from desktop to mobile, to augmented reality. This led our team to develop 3D abd mixed reality viewer technology, a universal gateway for 3D applications from mobile to Hololens. This technology enhances visualization, improves communication, and aids in understanding complex spatial relationships. It facilitates better decision-making, encourages creativity, and reduces reliance on physical models. Additionally, it enables immersive exploration across multiple platforms, becoming vital element in numerous applications.
User Research
User testing
Experience Strategy
Rapid Prototyping
Design Direction
User Experience
Task
To build a versatile and user-friendly platform that could seamlessly integrate with a range of applications, from traditional desktop environments to cutting-edge augmented reality experiences.
MIXED REALITY VIEWER
My Role in the Project
My role:
Product Design Lead
Team:
UX Designer
UI designers
Creative Tecnologist
VF OMNICHANNEL
VF OMNICHANNEL
As the Design Lead in the project, my role was pivotal in steering the development of the 3D viewer application. I oversaw the design team, ensuring that the software not only met technical specifications but also provided an intuitive and user-friendly experience. My expertise in design principles guided the aesthetic and functional aspects of the application, making it accessible to a diverse range of users. I worked closely with developers, balancing technical feasibility with innovative design solutions. I was instrumental in fostering a collaborative environment, where creativity and practicality merged to create a product that was both technically robust and visually appealing. My attention to detail ensured that the final product aligned with the project's broader goals, making it a crucial tool for users across various platforms.
In the development of the mixed reality viewer application, the main challenge was twofold: ensuring technical coherence of diverse 3D models and providing an effortless user experience. The application must effectively handle 3D models from various sources, each potentially differing in format, complexity, and design. Achieving a natural and cohesive presentation of these models is a significant technical hurdle.
Simultaneously, the user interface must be intuitive, allowing users to navigate and interact with these models easily, regardless of their technical expertise. Balancing the intricate technical requirements for model compatibility with a user-friendly interface poses a complex challenge, requiring innovative solutions to bridge the gap between advanced functionality and accessibility for all users.
MICROSOFT 3D4E PROGRAM
The Challenge
How can we allow our users to view and engage with 3D content on any Windows device in a way that feels natural and captivating?
DESIGN FUNDAMENTAL QUESTION
The Tech Stack
The mixed reality viewer project presented the team with a series of extremely intricate technical challenges, encompassing four critical phases.
The 'model inbound', where models from various sources, each distinct in format and complexity, were imported. The second phase, 'Transcoding', involved a detailed process of decoupling and interpreting the data from each model, which was then translated into a universal language for platform-wide compatibility. In the 'rendering' stage, this new updated rendering data was visualized in a way that felt natural and intuitive, a key factor in guaranteeing user engagement. The final phase, 'model output', was about fine-tuning these rendered models to suit a range of devices such as mobiles, tablets, desktops, and Hololens. This step demanded a nuanced approach to each device's specific output capabilities, ensuring that the models were not just efficiently displayed, but also tailored to offer a coherent and immersive experience across the diverse array of hardware, being "correct and compelling visualisation" the key goal of the application.
Interaction and Prototyping
For the viewing experience, we wanted interactions and cinematics to be as natural and fun as possible. We explored how we could ground the 3D model by leveraging interaction patterns typical of the real world. We also introduced model 'shaking' on long-press to give a 3D feel to the flat representation and leveraged accelerometers to allow the user to 'peep over' the item as the camera titled following the tablet's motion or device. However, what was crucial for an optimal experience was to provide simple cues for primary functions like 3D model rotation and scale by limiting the scope of movement to specific axis points and preventing disorientation, as we discovered that for novice users, this represents a common problem. Here a snapshot on the work done on the tablet version, whic entailed exploring how natural gesture could lead to a more seamless user experience and educate user to also hololens interaction patterns.
VP design Windows Next
Mixed reality viewer is all about enabling those next-gen XR experiences to really put Microsoft at the forefront in this space.
Kudo Tsonoda
Assets rendering
Individuals make a judgment in the first 50 milliseconds, therefore providing a pleasant model experience since the first moment was imperative. The visual design direction for the 3DViewer was holistic and comprehensive. My objective was always to consider the multifaceted nature of each model and ensure that each visualization could represent the author's intentions coherently. Ultimately, I was laser-focused in pursuing optimal digital representation of the model and showcase the 3D model in the best possible way within any environment. Here the deep studies I conducted in order to maximise the visualisation and rendering ot 3d models on any device and any condition, in particular when streching the engagement on mixed reality user scenarios.
End Result
The 3D viewer technology is a relatively simple application, indeed achieved thanks to a highly complex backend architecture. Working with an incredible team of engineers, interaction and visual designers, we released software that is opening the doors for spatial computing today. Thanks to our efforts, the result looked finely integrated into any environment and tangibly rendered within desktop, mobile and the rest of Windows applications and products including Hololens.
What I Learned
The learnings from developing a 3D viewer application are multifaceted. Firstly, working with 3D assets is processor-intensive, necessitating consideration of varying device capabilities to ensure smooth performance across platforms. Secondly, since users often lack familiarity with depth perception in digital environments, effectively communicating dimensionality presents its own unique challenge.
Additionally, creating a seamless user experience that functions consistently across different platforms is essential for scalability; this needs to be a priority from the project's onset.
Lastly, planning for diverse interaction inputs across devices is crucial. Tailoring the interface to suit each device's unique characteristics is key to maximizing user adoption and enhancing accessibility.
-
In technology development, user testing is crucial to align products with customer goals and desires. This approach grounds the development in real-world feedback, ensuring that the technology not only addresses unmet needs but also resonates with users. It helps refine features, identify issues, and keep the product in tune with evolving user expectations. By integrating user feedback, the development remains focused on solving actual problems.
This user-centric method is key to ensuring technology serves as a practical tool rather than just a technical showcase
-
