Is there room for FOSS in the AI, VR, AR, MR, ML and XR revolutions—or vice versa?

Will the free and open-source revolution end when our most personal computing happens inside the walled gardens of proprietary AI VR, AR, MR, ML and XR companies? I ask, because that's the plan.

I could see that plan when I met the Magic Leap One at IIW in October (only a few days ago as I write this). The ML1 (my abbreviation) gave me an MR (mixed reality) experience when I wore all of this:

• Lightwear (a headset).
• Control (a handset).
• Lightpack (electronics in a smooth disc about the size of a saucer).

So far, all Magic Leap offers is a Creator Edition. That was the one I met. Its price is $2,295, revealed only at the end of a registration gauntlet that requires name, email address, birth date and agreement with two click-wrap contracts totaling more than 7,000 words apiece. Here's what the page with the price says you get:

Magic Leap One Creator Edition is a lightweight, wearable computer that seamlessly blends the digital and physical worlds, allowing digital content to coexist with real world objects and the people around you. It sees what you see and uses its understanding of surroundings and context to create unbelievably believable experiences.

Also recommended on the same page are a shoulder strap ($30), a USB (or USB-like) dongle ($60) and a "fit kit" ($40), bringing the full price to $2,425.

Buying all this is the cost of entry for chefs working in the kitchen, serving apps and experiences to customers paying to play inside Magic Leap's walled garden: a market Magic Leaps hopes will be massive, given an investment sum that now totals close to $2 billion.

The experience it created for me, thanks to the work of one early developer, was with a school of digital fish swimming virtually in my physical world. Think of a hologram without a screen. I could walk through them, reach out and make them scatter, and otherwise interact with them. It was a nice demo, but far from anything I might crave.

But I wondered, given Magic Leap's secretive and far-advanced tech, if it could eventually make me crave things. I ask because immersive doesn't cover what this tech does. A better adjective might be invasive.

See, the Lightwear headset has cameras facing both outward at the physical world and inward at your eyes (each of which, as the saying goes, is a "window to your soul"). The outward ones take in your physical world, while the inward ones profile your eyeballs and project those 3D images directly onto the retinas of your eyes. They do that using "light wave" (aka "waveguide") technology. The control has onboard electronics (such as GPS) and connections to Magic Leap's cloud, which, I am told, map both the users and their physical environments to maximal depths, surely for purposes far beyond what any of us can guess at.

I'll spare you other details, most of which you can read about elsewhere. (One place to start: Karl Guttag's Magic Leap One – FOV and Tunnel Vision.) I have learned enough, so far, to make two points:

1. Magic Leap's MR experience is of a proprietary and closed digital world mixed with the free and open physical one.
2. No matter how appealing they may be, proprietary and closed digital worlds are all quarantined futures. And maybe this is a good thing, because it limits the degrees to which it can infect the open world where most useful development takes place.

Those quarantined futures are modeled currently by the video game industry. Nearly all electronic gaming in the world today happens with closed and proprietary applications running on closed and proprietary hardware. The main exception to that is a conditional one: Windows games running on the same kind of open hardware most Linux developers and users run. But hey, it's still on Windows, which remains a closed and proprietary platform, even though it can run on open hardware.

And yes, there are native games that run on Linux, and Windows ones you can run in emulation. We've been covering both of those here in Linux Journal for decades. Still, those are beside my point here, which is that the electronic gaming industry is a vast mosaic of walled gardens. On the MR front, in addition to Magic Leap, there's Windows Mixed Reality (formerly Microsoft HoloLens), HTC Vive, Lenovo Daydream Mirage, Oculus Rift, Sony Playstation VR and Samsung Gear VR.

Each points toward a future that presumably requires massive investments in science and manufacture: investments that can be recouped only by trapping customers inside corporate gardens, each walled in by patents, proprietary commercial licenses and restrictive one-sided agreements between owners of those gardens and their paying visitors.

So the smart guess is that the primary use for all of them will be gaming. I won't wish them good luck with that, because they'll have it. What they won't have is any more leverage into the open world than we already see with gaming on headset-less hardware—in other words, limited and likely to stay that way. In its quarantined state, the gaming world has thrived in blissful near-oblivity to the FOSS world that grew outside its gardens—for going on 50 years. (The first video game console was the Magnavox Odyssey, released in 1972, and built to run on the open-source hardware of its time: televisions.)

Our job in this space is to write and build the tech required for a free and open digital world we can mix with the free and open physical one we entered at birth. In other words, we need to do for all those two-letter acronyms what Linus did for UNIX with Linux.

Can we do that at a time when nearly all the big bucks are flowing to companies making closed worlds, each on the old proprietary mainframe model that personal computing, the internet and the web were all designed to obsolesce?

My first source of optimism in that midst is Liam Broza, (@LiamBroza), co-founder of Bitscoop Labs and main developer of LifeScope. He's the guy who brought the Magic Leap One to IIW, treated a bunch of us to experiences with it, and salted those experiences with dark warnings about what will likely become of our virtual worlds if only tech's scary giants and their billionaire friends provide them.

I love that Bitscoop and LifeScope aren't just about FOSS, but start, as Linus did, with the individual. Says the home page, "Control of your personal data is a human right. LifeScope is an open platform for personal data whereby ownership is returned to the user." LifeScope's Manifesto goes farther. Here's the whole thing:

Built by millions of individuals for everyone to use

The internet is the single most inspiring achievement of engineering and collaboration in human history. Contained within the internet’s data is both the promise of enriching the human condition as well as the danger of spreading misinformation, seeding divisiveness, and propagating mass manipulation. Our photos, emails, social media, biometrics, geolocation, and more tell the story of us. It is our personal contribution to the global scale dataset of human interaction. But each of us can only see and control a small fraction of the overwhelming data cloud we give off. We have left a record of reality in a digital memory we can’t trust.

Silicon Valley can't be trusted with our history

We create everything on the internet, but we have power over none of it. Large organizations gather our individual data to understand and control our psychographic and psychometric profiles. Incumbent powers use machine learning to gain insights and influence over our behavior to advance their own agenda. A handful of giant companies are centralizing command of the internet, and our courts and government are going along with it. We, as a civilization, are at a crossroads between Black Mirror and Star Trek.

There is a better way

By incorporating the power of big data, blockchain, and machine learning, LifeScope gives everyone a perfect digital memory allowing a truly objective reflection of themselves. As our data become more organized we become more directionalized. We are seeing a million fold efficiency in human understanding. Who am I? How do we fit together? Data can give us better answers. Trustworthy, complete, and organized data can tell our true story. Freedom, privacy, decentralization, and openness are the values that drive us. We aim to work together with people and organizations everywhere who share these goals to restore trust and restore control over our personal data.

In the FOSS tradition, @LifescopeLabs on Twitter throws credit toward developers of a similar mind, for example, @Mozilla, @MozillaReality (the Mozilla mixed reality team), @TensorFlow ("a fast, flexible, and scalable open-source machine learning library for research and production") and other allies, including @Sketchfab ("the largest platform to publish and find 3D models online"), @MongoDB and @GraphQl.

Liam presented his Bitscoop and LifeScope work both at IIW (see Sessions 4 and 5 for Thursday, 25 October) and VRM Day (which ProjectVRM held in advance of IIW). A talk he gave in July is on YouTube here. His slides from that talk are here. I advise checking out all of it. (That's him at IIW in the photo at the top, by the whiteboard where he detailed the open foundations of his work.)

The same goes for everything Evo Heyning (@EvoHeyning, @amoration) touches: Light Lodges(@LightLodges), XRStudio (@XRStudioSF), @PlayFiles, Toyshoppe Systems (@TSSystems), ExO Works (@Exo_Works), Hyperledger (@hyperledger), ExO Lever (@theExOLever), Unity (@Unity3D) and much more. She's my second source on all this stuff (to which I am still very new). When I asked her by email to review a draft of this piece, she encouraged me to stress the importance of WebXR, calling it "the open web solution to walled gardens," adding, "Mozilla's Hubs and Spoke are just one example—we will see many of the main tech players publishing to WebXR directly in the future." When I asked for more context on WebXR, she replied:

WebXR today probably feels like UNIX early on....the forks are experimental as hackers try to build tools and platforms on top of the open source code. It's a bit early web wild Westworld in terms of consistent immersive experiences across interfaces. Fully 4D interactive experiences on URLs will flip content and media models as much as YouTube & Netflix changed TV but that will take us through the next decade to realize. 5G<ubiquity. Smartglasses with full HUDs change our data integration equation and you are right—it's invasive and deeply intimate.

Others working in personal AI and adjacent spaces are being added to and updated frequently on ProjectVRM's VRM Development Work page as well. Please check those out too, and let us know what you think. Better yet, tell us what you're working on. If it's free and open, we need it.