Microsoft patents new software that can detect when employees are lazy


As if workers weren’t self-conscious enough during a video conference.

United States Patent and Trademark Office records indicate that Microsoft is staking claim over novel software that allows employers to monitor staffers’ body language and facial expressions during virtual and in-person meetings, and delivers a numeric “productivity score,” topping out at 800, for companies overall.

The technology garnered little buzz during Microsoft’s Ignite conference in October, Forbes reported last week. Since then, security experts have had a chance to take a deeper look at Productivity Score.

“It’s horrendous,” said right-to-privacy researcher J.S. Nelson, associate professor of law at Villanova University. “Why are they monitoring people this way and what is that telling people about the relationship they should have with their employers in the workplace? What message are you sending?” he told Forbes.

The software essentially taps office computers to give managers access to an overview of companywide activity: what devices employees are using; how many meetings they’re attending on Microsoft Teams, and how many hours they spend on the Microsoft Office suite or Skype. To some extent, emails are also subject to scrutiny, and live or virtual meetings are subject to monitoring for body language, facial expressions and meeting engagement (as opposed to looking down at your smartphone).

“The quality parameters each quantify meeting conditions during one or more previously elapsed meetings and are usable to determine an overall quality score for each of the one or more previously elapsed meetings,” the wordy patent filing stated, adding that a “quality monitoring device” may record “the body language and/or facial expressions of meeting participants.”

“This type of employee surveillance software obstructs diversity in workplaces by operating on the false premise that there is a uniform, normative way that people work optimally,” Silkie Carlo, director of UK civil liberties group Big Brother Watch, told the BBC. “A lot of surveillance tech is marketed as ‘innovative’ but in reality is astoundingly retrograde.”

In an October blog post, the computer giant denied that Productivity Score is invasive.

“Let me be clear: Productivity Score is not a work monitoring tool,” wrote Microsoft 365 Corporate Vice President Jared Spataro. “Productivity Score is about discovering new ways of working, providing your people with great collaboration and technology experiences.”

In an interview with Forbes, Melissa Grant, Microsoft 365 Product Marketing Director, reiterated that the tool couldn’t be used to track an individual employee’s every move. Rather, it delivers an aggregate of data, which also disappears monthly.

“Having the numbers on something doesn’t mean you have any idea what’s going on with your employees,” she said. Grant also acknowledged a potential loophole, as workers could simply launch numerous Microsoft programs on one’s desktop — and then take a long lunch.