Flock Safety has launched new search tools that enable police to identify individuals captured on camera, extending beyond just vehicle license plates. These tools are part of a software suite currently involved in various police-related cases. WIRED analyzed the software and created a mockup of the interface used by officers.
One feature is an AI-powered watchlist, allowing officers to designate a specific area and conduct automated searches for individuals matching a written description.
Over the past year, Flock has encountered criticism regarding its technology, leading to the cancellation of contracts by police departments and city councils reversing previous approvals of the camera systems. Concerns were raised after a Texas deputy searched over 83,000 cameras for a woman who had an abortion, prompting two members of Congress to contact CEO Garrett Langley. Additionally, an investigation in Illinois revealed that Flock allowed federal immigration agents access to state camera data, violating state law.
In response to the backlash, Flock announced in August a series of changes, including reducing the default data retention period, requiring case codes for searches, and implementing automated audits to monitor misuse. These changes are expected to be mandatory by the end of the year. However, WIRED's analysis indicated that while Flock's measures may deter some misuse, they do not prevent it entirely.
Experts in automated screening and police technology have expressed concerns about the reliability of Flock's AI system, which determines the descriptions officers can search for. The accuracy of such a system is questioned, as no external party can assess its error rates. Flock warns that results may be inaccurate or incomplete and emphasizes that officers bear the responsibility for any inaccuracies.
Flock's system evaluates officer descriptions against various sensitive categories, including race and religion, but allows searches based on political, social, and cultural expression. Tom Bowman from the Center for Democracy and Technology highlighted that this category is heavily protected under the First Amendment, raising concerns about its treatment in Flock's system.
Flock provided a statement asserting that its search tools are designed to help police while maintaining safeguards. However, the company did not disclose details about the AI model's construction or the criteria for its decision-making.
The software records data flow, including the language used by officers during searches and the model's assigned confidence scores. Flock did not clarify how long these records are retained or who has access to them.
Flock's interface includes various tools, each functioning differently. For instance, a text-based search generates a ranking of footage that matches the description provided. The system also features a Smart Sort function, allowing officers to refine search results based on their feedback.
In terms of vehicle searches, officers can select from predefined attributes, while person searches rely solely on written descriptions. Flock has marketed its FreeForm search for years, which allows officers to search using natural language descriptions.
WIRED's analysis of Flock's code revealed that departments can configure searches without needing a reason or case number, and the company indicated that controls will be enforced by year-end. Search activities are logged, but it remains unclear if searches without proper justification can be blocked.
Reports of police officers misusing Flock's software have emerged, with at least 50 officers facing charges or accusations of improper use of license plate readers, 46 of which involve Flock's technology. Instances of officers tracking personal acquaintances or romantic interests have been documented.
Flock's software includes two notification systems: one based on police search queries and another called People Detection Alert, which notifies officers when a person enters a designated area. The software requires a minimum confidence level for alerts.
Flock's moderation system aims to uphold First Amendment protections, with the AI model scoring officer descriptions against sensitive content categories before processing footage. The company claims that attempts to search using prohibited attributes will be blocked, but did not clarify the conditions under which warnings might be issued instead.
Flock's system has provisions for logging searches that trigger warnings, requiring officers to acknowledge the notification before proceeding. The company argues that this allows for legitimate investigative work while maintaining oversight.
Experts have raised concerns about the accuracy of automated moderation systems, particularly in analyzing video content. Kate Ruane from the Center for Democracy and Technology noted the challenges of accurately categorizing searches involving political and social expressions, which could lead to potential misuse of the technology.