Facebook Developing ‘Egocentric’ Artificial Intelligence

A Facebook-led artificial-intelligence research project hopes to make machines think more like the people who use them.

The Ego4D programme is training AIs to interact with the world from an “egocentric” or first-person perspective.

The aim is to allow wearable devices such as augmented reality (AR) glasses to assist their users in everyday tasks such as cooking or remembering where they put the keys.

The project is training systems using a massive trove of video recorded from a first-person point of view.

‘Centre of the action’

“Next-generation AI will need to learn from videos that show the world from the centre of the action,” Facebook said in a blog post.

It said it wants “immersive devices” such as AR glasses and VR headsets to become as useful as smartphones.

Facebook owns VR headset maker Oculus and is also developing AR spectacles.

At the centre of Ego4D is a “massive-scale egocentric video dataset” developed with 13 universities in nine countries.

It includes 3,025 hours of video of daily life activities from 855 individuals wearing cameras, researchers said.

Massive data set

Users filmed while doing construction work, baking, playing with pets, socialising and carrying out other activities in countries including Saudi Arabia, Tokyo, Los Angeles and Colombia.

The company said the dataset includes 20 times more footage than any other, and will be available from November to researchers who sign a data use agreement.

Facebook also developed benchmarks for what it hopes to achieve, including “episodic memory” (“Where did I leave my keys?”), forecasting what the user is likely to do next (“Wait, you’ve already added salt to this recipe”), hand and object manipulation, keeping a record of who said what when (for instance, during a class), and social interaction.

But the project is likely to raise privacy concerns, based as it is on the idea of a computer system that’s constantly recording data from the user’s surroundings.

Privacy

The company has been heavily criticised and fined for its previous record on privacy.

And its Ray-Ban Stories smart glasses, which allow the user to record short video clips, prompted a response from the data protection regulators of Ireland and Italy earlier this year.

Technology news site The Verge noted that Facebook’s benchmarks for the project don’t include privacy parameters, such as refraining from recording people who don’t wish to be recorded when creating diaries or recording social interactions.

Facebook told the site that safeguards would be brought in as applications were developed.