Amazon’s new fitness tracker listens to your voice to figure out your mood

Modern fitness trackers often try to set themselves apart by adding more advanced hardware and sensors inside. Now, many high-end trackers and smart watches include ECG devices for tracking heart rhythms—one company, Withings, even promises sleep apnea detection in its upcoming wearable.

Amazon, however, has decided to set its new fitness device apart from the rest by relying heavily on AI algorithms and listening to your voice. The $99 device is called Halo and it has a heart rate monitor inside, as well as an accelerometer, temperature sensor, and no screen all. It tracks the typical fitness stuff you’d expect like steps and pulse trends. But, It also has a pair of microphones, which are important for the device’s most interesting—and so far, controversial—feature, called Tone.

When you first set up Halo, it asks you to read a few quotes from classic literature. It uses that information to build a voice profile of how you typically speak. The device will then listen to you as you talk during the day and use an algorithm to give you a positivity score regarding your speech. If it determines that you don’t have enough positivity when you talk, it will give you suggestions about how to cheer up your speech pattern.

Many people already fear that smart assistants like Amazon’s own Alexa are already listening to everything they say, which makes the prospect of strapping a dedicated listening device to your wrist all day seem dubious. Amazon says that the watch will intermittently listen throughout the day, or you can specifically trigger it to listen for up to 30 minutes by pressing the button. There’s a hardware switch to turn off the microphones completely, but that defeats one of the key functions of the device.

Amazon has tried to get out ahead of the obvious security concerns involved with Tone. The company claims that all of the speech processing and analysis happens in the app or on the device itself—it never makes its way to the cloud. Once the processing is done, the recording gets deleted and isn’t saved anywhere. That’s a departure from how Alexa works—you can go and listen to recordings of your Alexa requests right now by logging into your Amazon account.

Even with secure recordings, there’s still the somewhat murky issue of just how the algorithm perceives your speech patterns. Obviously, a person’s speech patterns can change for a variety of reasons, whether it’s their mood or a simple stuffed nose.

Amazon isn’t alone in this department: Grammarly, the online copy editing tool that checks your spelling and grammar as you type, has a similar tone detection service that analyzes the phrases you type and tells you how it perceives your words with a simple emoji. It can rate your typing with qualifiers such as “accusatory,” “formal,” “encouraging,” or “regretful.” Since it’s happening with typing, it’s obviously working on a different level than Amazon’s technology, but it could have a similar effect on guiding how users communicate. The “neutral” rating in Grammarly comes with a straight-faced emoji that looks unimpressed or negative. I have it on right now and it suggests the tone of this post is “concerned,” which isn’t totally inaccurate.

It’s clear that Amazon plans to expand on its Halo brand more in the future. The band costs just $99, but it will likely enjoy the regular cycle of deep price cuts that Amazon typically employs in order to bring people into its infrastructure of services. Right now, you can pre-order a Halo for $65, which includes six months of the Halo subscription (typically $3.99 per month).

Once you’re in the system, Amazon plans to keep users by implementing a health points system, a familiar technique for fitness trackers. You get a base goal of 150 somewhat nebulous points per week. You can earn them by walking, but you can earn more by running. You lose a point for every sedentary hour you spend per day beyond the eight it expects you to spend at your desk. It’s reminiscent of Nike’s Fuel Points from when it made its own fitness tracker.

Voice tone doesn’t factor into your points total, but the features together put a strong emphasis on encouraging users to shape their behavior around the AI-driven suggestions. It’s not uncommon to feel like you’re letting your fitness tracker down when you don’t hit your goals. Now, your tracker can hear the sadness in your voice when you talk about your failure—and it can give you some AI-based tips on how to sound more chipper.