Mixed reality dreams: one look into Magic Leap

Magic Leap is a pretty secretive* yet awe- (or hype-)inspiring startup that promises a world-changing new technology.  That tech is a combination of virtual reality with augmented reality, often dubbed “mixed reality” or “artificial reality”.  It’s hard to get information on Magic Leap – did I mention they’re secretive? – but this Kevin Kelly piece for Wired is probably the best account so far.  It has fascinating implications for the future of technology and education, if the main idea enters the real world.

The article waxes lyrical from start to finish.  Enjoy that or grit your teeth and push on, as Kelly’s conclusions are really interesting. Let me pull out the points which struck me as most provocative.

Computing in space If a user wears… whatever headgear ML uses (goggles? visor? glasses? monocle? contacts?), they could see digital documents like web pages, spreadsheets, videos, 3d models, and so on.  Using some other tech (glove? puck? nothing? as yet unclear) the user can manipulate those documents.  That could lead to a computing experience without screen or tangible interface: computing in space.

At Magic Leap, the development team will soon abandon desktop screens altogether in favor of virtual displays. Meron Gribetz, founder of Meta, says that its new Meta 2 mixed-reality glasses will replace monitors in his company of 100 employees within a year.

It’s no great leap to imagine such glasses also replacing the small screens we all keep in our pockets. In other words, this is a technology that can simultaneously upend desktop PCs, laptops, and phones.

We know that the first step in exploring a new tech involves copying practices and forms from old tech, so if ML pulls this off, we shouldn’t be surprised to see people clicking on spreadsheets floating over their desks.

While I was wearing the photonic spectacles of Magic Leap, I watched an HD movie on a virtual movie screen. It looked as bright and crisp as my 55-inch TV at home.

buy filitra online buy filitra no prescription generic

With Microsoft’s HoloLens on, I watched a live football game on a virtual screen hovering next to a web browser window, alongside a few other virtual screens.

The second step, though, is when imagination takes over.  Those practices and forms then mutate into new shapes. That would be interesting to see.  Kelly starts to think this way: “I could fill my office with as many screens as I wanted, as big (or small) as I desired. I could click for a screen overlaid anywhere in the real world.”

And this, when venerable MS Paint becomes something new:

I had an aha moment inside a VR app called Tilt Brush that was purchased by Google. I was using a brush to paint with light in three dimensions. My traces in the air could be thin, thick, flickering, pulsating, solid sheets, of any color. I was inside my creation, moving around with my whole body, working up a sweat. I was sketching a sculpture or sculpting a sketch or architecting a drawing or dancing up a building of light—I don’t know what to call it, but it was the most fun I’ve ever had in VR.

Peter Jackson appears in the article to offer his take on MR as succeeding film.  “Once you can create the illusion of solid objects anywhere you want, you create new entertainment opportunities.”  (Which reminds me to start blogging about VR/MR and storytelling)

Walled garden or Wikipedia? Kelly offers this tantalizing idea: “With a VR platform we will create a Wikipedia of experiences, potentially available to anyone, anywhere, anytime.”  The long-time copyright observer in my head replies, “Maybe not.”  Imagine, instead, VR/MR walled gardens, locked-down and licensed content available through some version of the iTunes store or the way we currently use console games.

Social or asocial? Kelly insists that VR/AR will be, even is, inherently social. He describes powerful emotional responses to seeing someone else’s avatar, then goes on to proclaim: “The time is coming when, if someone says ‘let’s meet,’ everyone will know that means let’s meet in VR.

The default mode of VR is ‘together.'”

Yet what we’ve seen of VR has been deeply isolating, even antisocial.  We’ve already seen embarrassing or creepy photos of people using headsets.  The touted immersive quality of VR is perhaps akin to what some of us find in reading: pleasurable, consuming, and very antisocial.  Perhaps we should watch VR/MR unfold in a social-asocial dialectic.

A diversity engine In thinking of different ML uses, Kelly suggests sharing unusual perspectives:

[E]xperiences to be shared [will be accessible to anyone with a VR rig]: : marching with protesters in Iran; dancing with revelers in Malawi; how about switching genders? Experiences that no humans have had: exploring Mars; living as a lobster; experiencing a close-up of your own beating heart, live.

Much of that is classic VR pedagogy, and very sound.  But now we should also note the bit about changing perspective.  In an American education system more attuned to diversity than it has been for a while, we can imagine campuses, museums, and libraries deploying this to encourage tolerance and discourage prejudice.

A surveillance utopia Although he often rhapsodizes in the story, Kelly concludes on a grim note.  Recall that the computing needs to make VR/MR work are enormous.

“The scale of the servers, bandwidth, processing, storage, and cleverness required to run networked virtual places at the scale of the planet for billions of people is beyond Big Data. It is Ginormous Data.”  And so:

Which raises another issue. One of the underappreciated aspects of synthetic reality [a classic 1990s term! ah, nostalgia – Bryan] is that every virtual world is potentially a total surveillance state. By definition, everything inside a VR or MR world is tracked. After all, the more precisely and comprehensively your body and your behavior are tracked, the better your experience will be.

During a virtual journey, whether it lasts two minutes or two hours, the things your gaze lingers on, the places you choose to visit, how you interact with others and in what mood could all be captured in great detail to customize the experiences to your preferences and tendencies. But many other uses for this data are also obvious…

If a smartphone is a surveillance device we voluntarily carry in our pocket, then VR will be a total surveillance state we voluntarily enter.

Who’s talking about this?  Are any policymakers paying attention?  Are educators on the case?

The art of the prank About one-third of the way into this article Kelly describes a mixed reality hack that has all kinds of possibilities.  A VR/MR creator and stage magician invented  redirected walking”, a way of using digital feedback (visual, haptic) to convince the brain of a different reality than that brain perceived through its own optics.

As an example, whenever you turn 90 degrees in the room your VR will show you the room turning only 80 degrees. You don’t notice the difference, but the VR accumulates those small 10-degree cheats on each turn until it redirects your route away from a wall or even gets you to walk in a circle while making you think you’ve walked a mile in a straight line.

There’s also “[r]edirected touching”.  As with early cinema, we can imagine all kinds of creative storytelling and artistic possibilities.

Stairs can be made to feel endless if they drop down as you walk upward. In fact, at one point in the Void a decaying floor collapses while you’re walking across it, and you see, hear, and feel—in all your body—a plunge down to the floor below. But in fact the real floor only sinks 6 inches. You can easily imagine a room 60 by 60 feet packed with a minimal set of elemental shapes, ramps, and seats, all recycled and redirected for a variety of multihour adventures.

We can also envision some abuses.  Will the latter encourage people to prefer MR walled gardens instead of an experiential Wikipedia?

Stacked media generations It’s important to remember that we like to layer different strata of media when we make new ones.  Audio, once the dominant media life form through radio and gramophone, lived again through cinema and television.  Text plays a role in all visual media.  Kelly’s article mentions one way this could play out in MR/VR:

Researchers found that the you-person view that VR creates is so intense that it’s emotionally taxing. People need a break after an hour. Curiously, if someone stays inside VR but pulls up a virtual flat-screen version of Minecraft and continues playing in the traditional 2-D first-person view on a virtual monitor (still wearing the VR gear), they will feel more at ease. Once rested enough by playing in first-person mode, they often switch back to the fully immersive VR.

I can imagine listening to podcasts while watching video feeds of the day’s news pass before my photonic lens-equipped eyes, for example.

The ongoing irrelevance of Second Life In an article that thinks about the past, with references to the 1990s, 1980s science fiction, car phones, and early cinema, I’m struck by the way nobody mentions Second Life.  That most famous virtual reality project is either just not on anyone’s radar, or its spectacular collapse from mega-hype to irrelevance is radioactive.  Either way, interesting.

I’ll be blogging more about VR – we really should think of MR, or a better term – as the year progresses.

*secretive?  Look how short the Wikipedia entry is.

Liked it? Take a second to support Bryan Alexander on Patreon!