Meta Files Patent for AI That Can Listen All Day and Track How You're Feeling

Meta has filed a patent application for an AI system designed to continuously monitor a user’s voice, eye movements, and device usage to analyze their emotional state and provide personalized feedback. The system would track speech patterns, tone, pace, and even biometric data like pupil size and blink rate, creating a detailed emotional profile over time. While similar technology has been explored before, notably by Amazon with its Halo wearable in 2020, Meta’s system distinguishes itself by potentially storing data on-device in some versions and logging it to servers in others.

Each read would be pinned to the moment it happened: the time, your location, what you were doing, even how you were using your phone. Some versions in the filing would listen all day; others would check in only at set times. The system would then generate an ‘example readout’ like, "You sigh most frequently before bed, and you’re happiest when with friends. You’ve expressed more gratitude this month."

The patent’s other half is a workout coach. Smart glasses would watch your form in a mirror and talk you through the set, telling you to sink deeper into a squat, then cheering you on for a few more reps. The coach would read your mood, too. If it senses you are tired or discouraged, it eases off. If it decides you have energy to spare and are slacking, the patent says it may "admonish" you.

Amazon put voice mood-reading into its Halo wearable in 2020. Its Tone feature listened to your pitch and pace and told you how you came across through the day, calm, frustrated, and the like, and it processed those samples on your phone and deleted them, never touching the cloud. It drew scrutiny anyway: in December 2020, Senator Amy Klobuchar pressed federal health regulators over Halo’s collection of voice-tone and body-scan data, calling it unusually intrusive. Amazon shut the whole line down in 2023, though it never tied that to privacy.

The EU’s AI Act has already implemented restrictions on emotion-reading AI in certain workplaces and schools, except for medical or safety reasons, with fines up to 35 million euros or 7% of a company’s global turnover, whichever is more. Its drafters flagged the thin science directly: emotional expression varies from person to person, culture to culture, and moment to moment.

One figure logs a single person across a day: passive language on a morning video call from home, a laugh with a friend at dinner, a sigh at 9:15 PM caught by a smart home speaker. The system would then hand the user an example readout like this: "You sigh most frequently before bed, and you’re happiest when with friends. You’ve expressed more gratitude this month."

The system would also watch how you use your devices, down to the posts you view or like, your screen time, and how fast you switch between apps. All of it feeds one emotional profile.

The workout coach is one use. Underneath it is a running log of everything the system decided you felt, keyed to where you were and what you were doing. The system would then hand the user an example readout like, "You sigh most frequently before bed, and you’re happiest when with friends. You’ve expressed more gratitude this month."

The Hacker News has reached out to Meta for comment on whether the application reflects any product plans and how such a system would handle user data, and will update this story with any response.