Facebook’s Chief Executive Mark Zuckerberg has long rued the day that Apple and Google beat him to building smartphones, which now underpin many people’s digital lives. Ever since, he has searched for modern computing’s next frontier and how to be a part of it from the start.

Now, Zuckerberg is betting he has found it: the real world. On Tuesday, Zuckerberg introduced what he positioned as the first mainstream augmented reality platform, a way for people to view and digitally manipulate the physical world around them through the lens of their smartphone cameras.

What that means today is fairly limited. Augmented reality is nascent — people can add simple flourishes on top of their photos or videos, like sticking a pixelated blue beard on a selfie, or adding puppy dog stickers in the front yard of a photo of their house.

But in Zuckerberg’s telling, there are few boundaries for how this technology would evolve. He said he envisioned a world in which people can eventually point smartphone cameras at a bowl of cereal and have an app create tiny sharks swimming in the bowl of milk. Friends can leave virtual notes for one another on the walls outside their favourite restaurants, noting which menu item is the most delicious.

Apps like Pokemon Go, the breakout augmented reality hit that went viral globally last year, are just the beginning for Zuckerberg. One day, he mused, household objects could perhaps be replaced entirely by software.

“Think about how many of the things around us don’t actually need to be physical,” Zuckerberg said in an interview last week. “Instead of a $500 TV sitting in front of us, what’s to keep us from one day having it be a $1 app?”

Facebook does not expect to build all of these software experiences itself. At its annual developer conference on Tuesday, the social network called for computer programmers to assist the company by building augmented reality-based apps to work with what Facebook calls its Camera Effects Platform. Facebook announced a new set of tools to help developers and will begin the initiative with a small handful of partners in a closed test.

Zuckerberg’s goal is ambitious — perhaps overly so. Many augmented reality efforts have flopped in the past, including Google’s much-hyped attempt around spectacles with the technology, known as Google Glass. Facebook has previously gambled on other futuristic technologies — including virtual reality with a $2 billion purchase of Oculus, the virtual reality goggles maker, in 2014 — but Zuckerberg has acknowledged that it has had difficulty finding traction.

The chief executive is also grappling with wide range of issues that have the potential to distract Facebook. The company is under scrutiny for its position as an arbiter of mass media, and questions as to what role Facebook should play in policing content across its platform of nearly two billion regular users. That issue was thrust to the forefront this week after a man posted on his Facebook page a video of a murder he committed.

Still, Zuckerberg said he intended to create the next major app ecosystem that would work with Facebook’s in-app camera. If successful, Facebook could be in a position similar to that of Apple, which relies on the hundreds of millions of apps in its store to keep users buying the company’s smartphones and tablets every year. Facebook, in turn, wants developers to build experiences that entice people to visit its website and apps on a daily — if not hourly — basis.

“Just like Apple built the iPod and iTunes ecosystem before the iPhone, you want to make sure there’s a set of content there, even if there’s not everything,” Zuckerberg said.

Facebook has been building towards this goal for some time. Zuckerberg has spent the past 18 months reorganising his company and its suite of consumer apps — Facebook, Instagram, WhatsApp and Messenger — around a new interface, focused almost entirely on the camera. Slowly, the company has downplayed the role of text inside its apps, instead encouraging people to take and send photos and videos to one another by accessing the in-app camera features.

In time, Facebook hopes that companies like Electronic Arts, Nike and Warner Bros — which are part of the initial set of partners — will be the ones to bring immersive augmented reality experiences to Facebook’s platform.

One early partner app is Giphy Thoughts, made by Giphy, a short-form video startup that acts as a search engine for animated GIFs. With Giphy Thoughts, for instance, people can attach cartoon thought bubbles above the heads of people they view through their Facebook camera lens.

Creative expression

“It goes back to creative expression,” said David Rosenberg, director of business development at Giphy. “Facebook Camera is just going to be this massive audience of people ready to make deeply personal content they can share with their friends.”
Facebook’s past attempts to be at the centre of an apps ecosystem have not been particularly successful. In 2012, the company released App Centre, a central hub within Facebook to discover third-party apps — like Farmville, Goodreads and Spotify — and use them on the Facebook desktop site. But that initiative fizzled as consumers slowly shifted away from desktops to smartphones.

One year later, Facebook tried to emulate Apple and Google’s platform strategy more directly with its own Facebook-branded smartphone, called Facebook Home. The phone, a product of a partnership with AT&T and HTC, sold poorly and was eventually abandoned.

Then came Facebook’s most aggressive move, with the 2014 acquisition of Oculus. Facebook is spending hundreds of millions of dollars more investing in virtual reality content and apps in the hopes that it will mature into a full-fledged ecosystem similar to Apple’s App Store, but sales of the Oculus Rift goggles have been slow.

Facebook is not alone in its quest to establish a foothold in augmented reality. Hours before Facebook’s developer conference, Snapchat — a professed “camera company” and Facebook’s competitor for attention — announced its own take on augmented reality, offering new 3D lenses for use inside of its app. The effect is similar to some of the ideas Zuckerberg has described. Snapchat also offers Spectacles that allow users to record video from their face.

Another dark horse is Magic Leap, the secretive augmented reality startup that has an enormous investment from Google and Alibaba and is working on hardware to offer a similar experience. The company has yet to unveil an official product.

For the near term, however, Zuckerberg sees the smartphone camera as the first step forward. “We want to get to this world in the future where you eventually have glasses or contact lenses where you can mix digital or physical objects in the digital world,” Zuckerberg said.