Meta’s super-sensing AI glasses are still in the works, and I don’t know if I should be excited or terrified  | amznusa.com

• Meta reportedly has a new AI glasses prototype
• It is helping the company finalize a super sensing AI model
• Meta execs apparently aren’t yet certain what rules the all seeing AI should follow

Meta is reportedly trialling a new super-sensing AI glasses prototype that aim to offer an ultimate level of assistance, for the small price of capturing your every moment.

The basic idea is that, because personal AI assistance gets better the more the AI knows you, by having you smart specs watch every detail of your life, the assistant can then offer more insightful help.

It could, for example, know if your fridge has milk in because it has seen inside it, or remind you not to forget your keys as you go to leave home. It could remember that gift your friend was desperately hoping to get for their birthday, or remind you the name of someone you’ve met before, if you’re forgotten and are too embarrassed to ask.

This kind of tool would undeniably be pretty handy, but the extreme cost would be that your glasses would have to be always on — otherwise the AI might not see or hear the crucial information you’ll need later.

However, this incredible level of insight could be very easily abused — so you’d either have to really trust the company capturing it, or just not care if it knows everything about you.

This latest super-sensing leak comes via the Financial Times (behind a paywall) which reports that Meta is currently testing a pair of specs that achieve super-sensing by capturing continuous audio, and snapping photos every few seconds.

The more staccato image feed is likely an effort to conserve battery — according to previous leaks, existing Meta glasses have all the hardware they need to pull off super sensing, but their batteries wouldn’t be able to last long enough for it to be widely useful.

According to the report, Meta is also still trying to work on some other aspects, like whether the recording light LED should be displayed or not. When you’re recording it normally would be — super sense would be capturing people around you who might wish to know if they’re on camera — though sources have told the FT that there are plans for it to not to be displayed.

Considering that Meta just made a big post about privacy, and even updating the specs to disable tricks modders have been using to allow the specs to record without the light coming on, I’d be surprised if it took the no-light approach, but we’ll have to wait and see.

Man vs machine

You might also remember a story we shared in March about Meta contractors who claimed to be able to see images and videos taken with the glasses.

Essentially, in order to use Meta’s AI you agree to allow the company to see information about your AI communications, including videos and photos. This seemingly includes photos and videos you take using hands-free voice controls — which does technically involve Meta AI, as it has to action your request.

If an always-on AI is seeing every detail of your life there’s potentially a lot more scope for you to capture sensitive data that you’d rather not let Meta’s contractors (or anyone else frankly) see. Encouragingly, according to people familiar with Meta’s super-sensing technology, there are plans to never store the raw footage and audio — meaning Meta nor the user could access it.

Instead, the system would extract the metadata from the capture, and only that would be uploaded — metadata is data about data, so imagine if instead of showing someone a photo you just described what’s in it and where it is.

There’s still some room for personal data to leak through this system, but with the correct safeguards it would have far fewer privacy implications. The big obvious gap would be the privacy of those around you — people who, unlike you the glasses wearer, might not have consented for Meta to store any kind of data about them.

The FT report adds there are also debates over whether Meta’s AI should be allowed to use this metadata for training purposes, in order for it to keep up with the capabilities of OpenAI, Google, and Anthropic’s models.

While leaks should be taken with a pinch of salt, Meta CEO Mark Zuckerberg has made it clear that he wants to offer these kinds of features on his company’s glasses. Most recently he said in an investor call that he wants smart glasses “to be a personal agent that’s with you all day long, helping you remember things and achieve your goals.”

So the advent of technology like super-sense is most likely a matter of when not if, though with battery hardware constraints persisting (especially if you want slim and light glasses), and with privacy being such a hot-button issue, I expect it might be a while before we see super sense in action.

Whenever it arrives, we’ll just have to hope that it’s implemented in the right way, and with appropriate safeguards. Such a tool could potentially offers some incredible accessibility benefits, but if the privacy cost is too great the I don’t see it taking off in the way I’m sure Meta would want it to.