oceanx

Microsoft Mesh entertainment partnership with OceanX, James Cameron, National Geographic and Disney

problem

Ocean science produces some of the most extraordinary data on earth, and almost none of it is visible to the human eye. Scientists aboard research expeditions work with vast spatial datasets, tracking animal behavior, mapping migration patterns, and reconstructing underwater events that no camera can fully capture. The gap wasn't a lack of discovery. It was a fundamental inability to translate that discovery into something a television audience could see, feel, and understand in real time. The tools of science and the tools of storytelling had never been designed to speak the same language, and for a six-part documentary series that depended entirely on that translation, that gap was the whole problem.

solution

I led my team in designing and building a Hololab command center, installed aboard the world's most advanced oceanographic research vessel, the OceanXplorer, housing a fully interactable Holotable that allowed scientists to view, manipulate, and simulate 3D spatial data live on camera. The goal was to give scientists the power to facilitate mission planning, execution, and analysis, with each of the six episodes requiring a completely different version of the tool to serve its underlying narrative. We built animal viewers, bathymetry terrain maps, biomass trackers, ocean current visualizers, and a first-person immersion mode for experiencing glacial melt from the inside. We had four weeks between shoots to design, build, and validate each iteration, and we delivered all six on time, at high quality, without exception. The room itself was fabricated and installed by our design team, repurposing the ship's gym, and it remains an active holographic science lab aboard the vessel today.

I served as Design Director and Creative Director across the full scope of this project, from the Hololab episodic content to the traveling museum experience. My responsibility was to work directly with my team, the showrunner, Avatar productions, National Geographic, and OceanX to determine how best to represent each episode's story on the Holotable, then lead my team through design and production on a relentless four-week cycle per episode.

I directed the central art team in the creation of all 3D models, animations, and rigging, oversaw rigorous end-to-end testing in our Redmond-based Hololab studio, and validated interaction models, accessibility, and NCAM integration before every shoot.

Beyond the show, I co-created a traveling Mesh-enabled museum experience that guided virtual avatars and physical guests through mission-based adventures inside a hybrid digital and physical twin of the ship, part of a larger entertainment vision I developed called The Oceanverse.

We then went a step further and explored what the future of streaming media could look like when immersive smart glasses were applied. I led a 6 week discovery and prototyping cycle, honing the vision into a compelling platform strategy that earned high praise from Disney and James Cameron's Avatar Alliance team.

The build

Our first task was to design and build out a physical Hololab that would house the experience on the ship. We re-purposed the ship’s gym facility to achieve this.

There were several considerations that needed to be accounted for:

The room needed to accommodate 4-6 scientists plus a 3 person film crew
It needed to have an integrated NCAM setup so folks not wearing HL can see holograms
Hololens charging stations needed to be built in to the room
The table itself needed to be future-proofed for eventual holoportation functionality
The ship would only port for two weeks, which made installation a challenge

Our design team fabricated and installed the entire setup and the room is in full use today on the ship as an active holographic science lab.

Constructing an episode

Let’s take a look at one of the 6 episodes we created, the ORCA PREDATION EVENT.

For this episode, the showrunner wanted to tell the entire predation story in the Hololab, interspersed with live shots from the field. My task was to figure out how to do this.

CHALLENGE

While on mission, the OceanXplorers witnessed the first ever orca predation event of a blue whale mother and calf. The show captured the event on camera, but all the action was underwater and impossible to see

CONSIDERATIONS

Double-blind science study
Needed to give scientists complete control over the beats
Audio visualizations were required for animal communications

To frame the problem space, I led interviews with featured scientists, the film crew and alignment meetings with key stakeholders.

Built a Creative Brief with reference images, sample footage and inspiration.

Conducted brainstorms with the team and directed the design of a prototype 2D click-through.

Directed central art team in the creation of 3D models, animations and rigging. Here you see the progression from literal physical models of the animal ecotype to end result.

Testing & validation

Next, we ran rigorous testing of the e2e experience in our clone of the Hololab on campus in Redmond.

In this step, we validate interaction models, accessibility and check for any NCAM discrepancies.

OceanXperience

Our partners were so impressed with what we delivered we were asked to co-create a traveling Mesh-enabled oceanographic museum exhibit to compliment the show. Our design guides groups of virtual avatars and physical guests through mission-based adventures on a hybrid digital/physical twin of the ship.