Facebook does have one exception to its claim that it doesn’t look at message contents: Messenger’s abuse-reporting mechanism has long included a feature that decrypts encrypted messages when a user flags them. That gives Facebook another potential clue as to what other bad behavior that sender might be up to, but isn’t generally considered a breach of end-to-end encryption since the recipient of an encrypted message can always choose to share the decrypted version with a third party.
For now, the safety notices feature will only explicitly suggest blocking or ignoring a potential abuse. (Users can still report abusive behavior with the usual method of tapping on the sender’s name and then “Something’s Wrong,” then specifying what happened.) “We wanted to give people an immediate action, and blocking is the most immediate action someone can take to avoid harm,” a Facebook spokesperson says. “Reporting is something we’re looking at bringing to the feature as well.”
Facebook Messenger’s addition of abuse alerts based on metadata alone is a “good start,” says Alex Stamos, the former chief security officer of Facebook. But he argues that the company could—and should—do more. Stamos, who now leads the Stanford Internet Observatory, has argued that Facebook, Google, Microsoft, Snap, and others should all monitor for signs of bad behavior on user devices.
“I think they should have client-side looking-at-content. And once they do that, they should prompt people to be able to report it,” Stamos says. That on-device content analysis and reporting allow Facebook to find bad actors faster than mere metadata scanning, Stamos says, while still maintaining end-to-end encryption.
Stamos also notes that if Facebook emphasized reporting rather than mere blocking in its alerts to, those reports could create evidence that law enforcement could use against serious criminals. “You’re not going to arrest somebody because your data shows that they tried to ‘friend’ a bunch of teenage girls,” Stamos adds. “Whereas if somebody actually sends a request for nudes to a kid, that is probably illegal in most cases. That could be used to get a search warrant and to possibly prosecute the person. You really want the content of the communications, and the best way to do that in end-to-end encryption is just to encourage the recipient to report the conversation.”
A Facebook spokesperson, asked about that possibility of client-side content analysis, says the measure “wasn’t considered and isn’t necessary for this safety feature.”