Vodafone DesignOps • May 2015 - November 2018
Microsoft. 3D for Everyone
Microsoft was the first tech giant showcasing to the world the potential of spatial computing. With Hololens v1, a new vision for the future was spearheaded and the audience marvelled with the promise of new ways of interacting with computers thanks to holograms and spatial interactions. However, the usability and performances gap to take full advantage of this new technology ware profound. I was lucky to be part of the Microsoft 3D for Everyone project, a monumental initiative aiming to democratize the potential of 3D visualization and share ideas like never before.
Experience strategy
Service design
User experience
User interface
Rapid prototyping
Design Direction
Motion design
Branding
Task
Understand the impact of 3D on new generations and define a new design language across devices, including AR\VR headsets, desktops, tablets, and mobile. To elevate the adoption of 3D as a vehicle to better express ideas and enhance productivity.
How can we extend the Paint3D Creation to non-traditional windows devices in a way that feels seamless and intuitive?
DESIGN FUNDAMENTAL QUESTION
MICROSOFT 3D FOR EVERYONE
My Role in the Project
My role:
Product Design UI Lead
Creative team:
UI designers
Graphic designers
3D Artists
Tecnical Artist
VF OMNICHANNEL
VF OMNICHANNEL
After just a few months as Senior UX\UI Motion Designer contractor, I was asked to join the team as a permanent UI Lead on Paint3D and Mixed Reality Viewer. Partly as a hands-on designer, I acted as the second arm for the Principal Design Director based in Redmond. In this position, I advocated design thinking and UDC methodologies working in close partnership with the UX team and provided steers on all the creative output for the mentioned products. I was also fortunate to collaborate side by side with the Remix 3D and 3D Scan teams and influence product development. My role entailed mentoring and inspiring the team and help individuals to think outside the box, come up with new innovative immersive concepts, and create experiences never imagined before. I also liaised with program managers, researchers and engineers to identify customer journeys, define roadmaps and contribute to the broader 3D4E ecosystem development.
Windows Next
The scale of the challenge was massive, and Microsoft set up a monumental program to tackle the matter from different angles. Few departments were created to build a 3D ecosystem and enable global communities to consume 3D content intuitively. 3D for Everyone was made up of four main areas, including Mixed Reality Viewer, Paint 3D, Remix 3D Community and 3D Scan, with additional support for integration on the Office and Minecraft platforms. Each product was powered by entire studios located in London, Vancouver and Redmond, collaborating remotely daily. The 3D4E program was part of Windows NEXT, the most advanced technology wing within Windows, responsible for bringing innovation and original solutions across the Microsoft Windows OS and to more than 900M daily users.
How can we democratize 3D creation and enable our customers to quickly and intuitively consume three-dimensional content and achieve more?
PROGRAM FUNDAMENTAL QUESTION
Service Design
The Hololens technology was at its infancy and far to be perfect and broadly accessible. The 3D for Everyone project focused on the new generations as the primary target audience, looking to establish a distinctive personality for this new Windows ecosystem. 3D for Everyone tools and products were very compelling. We discovered that these solutions could capture casual people and the most proficient creators outside of the GenZ category. This knowledge pushed us to be inclusive and create products and services that feel part of the Microsoft ecosystem, still giving hints of the future thanks to distinctive experiences and design languages generating differentiation and demand even beyond the already vast Microsoft offering.
As the creative lead for the project and to illustrate the art direction for the experience, its environments and props, I created reference maps and mood boards for all the assets needed for the demo. These documents were instrumental in inspiring and aligning the team and ensuring that all the VR experience elements would work suitably and aligned with the overarching goals.
Visual Explorations
Since early phases, we explored how the Paint3D user interface could also enable audiences to understand the third dimension. The goal was to convey the notion of depth and space and empower customers with an experience that would start from the desktop and end in the physical space leveraging the Hololens device. My approach was holistic and analytical. I went through deep phases of researches and realized that the simplest spatial interface affordance can determine a clear understanding of the interaction and guide the user through the layout hierarchy layers, still providing great accessibility and usability. I learned how the use of materials, lights, shadows, motion and environment could influence the user in determining the meaning of each UI component, still making sure that the visual treatments would resemble the Windows visual identity.
Vision Video
As a starting point of this adventure with the team, we worked on a video to create a solid vision for product development. We were at the beginning of our journey, and the name wasn't even definitive. We wanted Paint 3D to be comprehensive and bundled with all the platforms and capabilities, later to be split into separate integrated services like Remix 3D and 3D scan. The goal was to describe a digital product that could enable users to create three-dimensional content in few taps and remix existing objects from the community. True to our belief of being compelling to younger generations, we envisioned a young student's journey looking to submit an assignment to a school context. For this video, I provided the design direction and worked side by side with the video team and provided interaction concepts and visual designs for the storytelling. The outcome was presented several times to C-level stakeholders to get buy-in and ultimately influence the engineering teams. Many of the features we envisioned during those days are today a reality in the Paint3D end product, such as "Paint on 3D objects", 3D Doodle, "3D stickers", and "Magic selection".
Rapid Prototyping
Fluent design is designed to scale across everything Microsoft. From desktop, tablet, mobile and ultimately VR and Hololens.
As user experience practitioners, our work is to make easy things for people to use. Order and relation is something we try to bring into whatever area we are tackling. When we started working on the Paint3D interface, we wanted people to immerse themselves in the digital experience. However, at that time, there was a lack of knowledge in spatial UI. This gap pushed us to ask ourselves how people can interact in the physical, multi-sensory environment as we unconsciously learnt as children. These thoughts were at the core of our studies. The outcomes profoundly influenced the development of the Windows Fluent Design, as the outcome of regular meetings with the Principal Creative Director, Mike Harnish and Principal Art Director, Alberto Cerriteno, based in Redmond.
Interaction Design
The challenge of representing 3D on a 2D screen is enormous. In addition, letting complete beginners create in 3D the problem can suddenly evolve into a riddle-like fitting an elephant through a keyhole. I worked on several critical features through Paint3D, working as solo or as part of a larger team of incredibly talented individuals within the Windows division. At the beginning of the Paint3D development journey, I worked on the application information architecture and onboarding to accelerate the platform's understanding and make sure any audience would start using the software straight away and without any formal training and\or experience in the field.
I tackled fundamental functions such as object selection, position, rotation, and scale as part of my contribution. We came up with an original idea called "unipolator", essentially a graphical device that takes cues from the 2D selection tool in Office. We expanded its capabilities by allowing the user to interact with this tool to manipulate the object dynamically in a 3D environment. I also participated in intense sprints to determine assets library operations, painting with 3D brushes, smart cut-outs, and material properties. I worked side-by-side with UX and 3d engineers teams to determine the best practices and supported the implementation in a very collaborative environment.
I had the great fortune to work with a talented team made of graphic, UI and 3D artists and creative technologists that shaped the evolution of the application. Thanks to the knowledge gathered in more complex 3d packages, such as Cinema4D and Maya, we advised UX designers and engineers with new features and patterns to help expand the Paint3D capabilities and enable users to create with more confidence. We took advantage of rapid prototyping in several circumstances and for different features to communicate effectively with the developers and get the best output from our conversations.
Paint 3D
To address the challenge of democratizing 3D creation, the team took advantage of the beloved old Paint to compel the user with new dimensional capabilities. While adopting the already familiar common 2D creation interaction patterns, we ported the extra dimension in an intuitive way, escalating the 3D experience to a whole new level. The simplicity and ease of use that characterizes Paint 3D, supported by the launch of the Remix 3D community - a place where anyone could share and remix 3D creations - delivered to more than 27M monthly active users the immense potential of creating, sharing and consuming 3D content as never before. Paint 3D is today installed on any Windows 10 machine with a current user base of more than 1Billions individuals globally.
How can we allow Windows users to easily create 3D content with the familiar tools they already know and share their creations with the communities they love?
DESIGN FUNDAMENTAL QUESTION
The road to moving people and letting them start adopting emerging technology is long and curvy. 3D and computer graphics have been established concepts for almost 60 years. However, creating a tridimensional model and communicating via its representation is complex. A challenge mainly induced by two factors: the flat screen and the lack of 3D expertise to execute the design.
When Microsoft leaders recognized the potential of Holograms and spatial computing, it was immediately apparent that a platform was needed to educate users about 3D content, democratize its use and then bridge consumers to the next generation of personal computing.
MICROSOFT 3D4E PROGRAM
The Challenge
Paint 3D in VR
Paint3D is a revolutionary application, and within the broader "3D for Everyone" Windows program, the ultimate goal was to break down the physical barriers and allow users to draw or create whatever they'd want in the space around them. With Paint3D, the ambition was to turn the area around the user into an infinite canvas, offering unlimited creative freedom. The business, therefore, sought the opportunity to explore how the desktop Paint3D experience would unfold on other devices as a natural extension. As proof of concept, we developed a demo to extend the experience in virtual reality and fill the device gap. The goal was to enable the user to create 3D content in a fully immersive environment and take advantage of the spatial cues typical of physical interactions.
Design Process
During the conception phase, we analyzed and described different scenarios as use cases to inform the storytelling and effectively target the hypothesis that creating within an immersive environment could unlock further opportunities for the user. This exercise was also an opportunity to establish and develop a few other creation tools leveraging unique VR interactions, expanding the Paint3D VR capabilities, and enable the user to become even more performing and creative.
Once immersed in the 3D environment, the challenge to create with confidence becomes enormous. The depth of the space can be daunting, leading the user not to feel inspired. We experimented with creative technology artists and created immersive VR prototypes in Unity to iterate on features and validate our hypothesis. In the below video, we wanted to demonstrate that hard modelling can be achieved with very few steps. We came up with an easy and quick solution that, thanks to point-to-point interpolation, enables the user to extrude simple shapes. Via sculpting, the user can carve parts out to remove the unnecessary and achieve the result. 3D creation within a three-dimensional environment doesn't necessarily require highly complex functionalities, as long as the users can find workarounds and achieve their intent following the creative flow.
Paint 3D in VR - E2e Journey
The result was a working prototype along with a vision video that tells the story of a young entrepreneur looking to design his new coffee store. We created an end to end customer journey that leverages different devices to maximize the technological potential and takes advantage of the most common creation ux patterns on mobile and desktop. For any business, achieving a cross-device experience is a critical element for success. This opportunity lets the user transcend the specific device and think laterally by unlocking possibilities otherwise impossible on a single touchpoint. We considered the VR experiment a huge success. The individuals who tried the experience confirmed that creating within an immersive environment was more creative, accessible, and fun than working on a traditional flat screen. Unfortunately, Microsoft's focus and investment weren't much on VR at that time, given the low penetration of Microsoft owned devices. The level of effort to develop the project would have been too large to gain momentum and profit; therefore, the project was put on hold. But the learnings we gathered are still today vivid and clear, allowing me to tackle any new interaction challenge within an immersive environment with great confidence.
Mixed Reality Viewer
Unlocking 3D and Windows is not only unlocking creativity; it is unlocking whole new productivity for creativity. It will enable people to share their ideas in new ways, to express and comprehend stories like never before. It will allow people to capture what is happening in their real world and what's happening in their digital world, and the blurring of those two dimensions will unlock an entirely new wave of innovation. The gateway to enable this unique opportunity is the ability for the user to access and interact with 3D content without friction and in an intuitive way. The 3Dviewer technology allows any Windows user to visualize a 3D asset in a beautiful and meaningful way. Today any Windows 10 device embeds this technology natively, allowing to project three-dimensional digital content into the real world.
How can we allow our users to view and engage with 3D content on any Windows device in a way that feels natural and captivating?
DESIGN FUNDAMENTAL QUESTION
We researched extensively about the UI and adopted visual patterns that would feel with familiar the Windows design language. For the main menus, we amplified affordances thanks to motion and light design. Buttons would vary their state according to the pointer distance and activate on controller click. Menus and iconography were tangible and reachable within arms distance. Once selected, object items would transition in the digital real world, allowing the user to seamlessly control their scale, rotation and position following natural interactions. We aimed to create an interface perfectly integrated with the digital environment and fulfilling users' creative needs by allowing them to switch on/off this element depending on necessity during the creation flow.
Interaction and Prototyping
For the viewing experience, we wanted interactions and cinematics to be as natural and fun as possible. We explored how we could ground the 3D model by leveraging interaction patterns typical of the real world. We also introduced model 'shaking' on long-press to give a 3D feel to the flat representation and leveraged accelerometers to allow the user to 'peep over' the item as the camera titled following the tablet's motion or device. However, what was crucial for an optimal experience was to provide simple cues for primary functions like 3D model rotation and scale by limiting the scope of movement to specific axis points and preventing disorientation, as we discovered that for novice users, this represents a common problem.
VP design Windows Next
Mixed reality viewer is all about enabling those next-gen XR experiences to really put Microsoft at the forefront in this space.
Kudo Tsonoda
Look and Feel
Individuals make a judgment in the first 50 milliseconds, therefore providing a pleasant model experience since the first moment was imperative. The visual design direction for the 3DViewer was holistic and comprehensive. My objective was always to consider the multifaceted nature of each model and ensure that each visualization could represent the author's intentions coherently. Ultimately, I was laser-focused in pursuing optimal digital representation of the model and showcase the 3D model in the best possible way within any environment.
End Result
The 3D viewer technology is a relatively simple application, indeed achieved thanks to a highly complex backend architecture. Working with an incredible team of engineers, interaction and visual designers, we released software that is opening the doors for spatial computing today. Thanks to our efforts, the result looked finely integrated into any environment and tangibly rendered within desktop and mobile on windows applications. Below you can see some examples without taking any advantage of post-processing. I took these images on a casual day around the office. I wanted to have some fun and simulate a Galactic Empire invasion to celebrate The Last Jedi - Star War movie launch. I was delighted by the result, and even if we were ahead of time, I knew these experiences would have changed forever the way we interact with computers and software.
What I Learned
Working on the "3D for everyone" program was a superb experience. I had the opportunity to work with many incredibly talented people, which made me learn and grow professionally and from a human perspective. The level and scale of innovation we were looking to achieve is something hard to describe. We created the condition for the masses to start utilizing 3D, which sounds like a utopia. We soon realized that we were too ahead and pushing the envelope, lacking a specific customer need. We found out that even if our products were extremely compelling to use and highly accessible, unfortunately, they didn't serve a precise scope for most of our users.
The first impression and usability results were great, as proved by the high scores on social sentiment. But people were not coming back to share their creations iteratively. Unless they were avant-garde artists, there wasn't such a strong need for them to continue the journey on these platforms and feed the loop. Later, the company decided to extend 3D integration on Office products to solve more productivity tasks with this technology. But still, that felt like an afterthought patch.
Ultimately, I learned that technology is worthwhile only when there's a clear unmet need to fulfil and it is mature enough to guarantee a smooth experience. When these conditions are true, digital solutions become successful and widely adopted. Even if the vision of future technology can promise exceptional outcomes, it is always imperative to determine a clear north light to follow and ground dreams of a better future to the current user's expectations and desires.
-
Technology is worthwhile only when there's a clear unmet need to fulfil and it is mature enough to guarantee a smooth experience. When these conditions are met, those digital solutions become successful and widely adopted. Even if the vision of future technology can promise exceptional outcomes, it is always imperative to determine a clear north light to follow and ground dreams of a better future to the current user's expectations and desires.
-
Visual Design
My primary responsibility was to unlock the potential of 3D by providing intuitive and meaningful visual cues and leverage depth as a new variable for the user interface. In addition, my objective was to achieve a distinctive and recognizable identity that would allow the product to stand out within the crowded digital creation products landscape. The 3D4E and Paint 3D visual styles take inspiration from the spectrum. The name recalls crucial visual elements like colours, light and depth and conveys the range of destinations the product will lead the user to. The intent is to feel harmonious but not a subset of the Windows style. By channelling the customer on a playful journey that brings productive and practical outcomes, such as 3D printing, creating a hologram or sharing a 3d model with the community.