Clearview AI lets law enforcement agencies search its database of faces.
But the Office of the Australian Information Commissioner (OAIC) ordered it to stop collecting photos taken in Australia and remove ones already in its collection.
A lawyer representing the firm said it would seek a review of the decision.
Clearview AI's system allows a user - for example, a police officer seeking to identify a suspect - to upload a photo of a face and find matches in a database of billions of images it has collected from the internet and social media.
The system then provides links to where matching images appeared online.
Clearview AI has promoted its service to police as resembling a "Google search for faces".
The system is primarily marketed as a tool for law enforcement agencies although its clients are not limited to that. The investigation said a recent patent filing revealed interest in other uses for the technology, including: "Dating, retail, granting or denying access to a facility, venue, or device, accurately dispensing social benefits and reducing fraud."
On its website, the firm says it has "the largest known database of 10+ billion facial images", although the report into the investigation uses a lower figure of more than three billion.
The company maintains that it only collects publicly available images from the open web, but that did not satisfy investigators.
"The covert collection of this kind of sensitive information is unreasonably intrusive and unfair," Australia's Information Commissioner, Angelene Falk, wrote.
"It carries significant risk of harm to individuals, including vulnerable groups such as children and victims of crime, whose images can be searched on Clearview AI's database".
The OAIC found the firm's breaches of the Australian Privacy Act included:
- collecting sensitive information without consent
- collecting personal information by unfair means
- not notifying individuals of the collection of personal information
- not ensuring that personal information it disclosed was accurate
It said Clearview must "cease collecting facial images and biometric templates from individuals in Australia, and to destroy existing images and templates collected from Australia".
The investigation was carried out jointly with the UK Information Commissioner's Office (ICO).
The ICO says it is still considering its next steps and any formal regulatory action that may follow.
But Elizabeth Denham, the UK Information Commissioner, said it was "an investigation that will protect consumers in both the UK and Australia".
Mark Love, a lawyer representing Clearview, told the BBC in a statement that the firm had "gone to considerable lengths to co-operate" with the Australian investigation.
He said the commissioner "has not correctly understood how Clearview AI conducts its business" and plans to appeal.
"Not only has the commissioner's decision missed the mark on the manner of Clearview AI's manner of operation, the commissioner lacks jurisdiction," he said.
"Clearview AI," Mr Love wrote, "has not violated any law nor has it interfered with the privacy of Australians. Clearview AI does not do business in Australia, does not have any Australian users."
The OAIC accepted the company's assertion that it had instituted "a policy of refusing all requests for user accounts from Australia, and that there is no evidence of Australian users since March 2020".
The company's founder and chief executive, Hoan Ton-That, wrote in a statement that as a dual citizen of Australia and the United States: "My company and I have acted in the best interests of these two nations and their people by assisting law enforcement in solving heinous crimes against children, seniors and other victims of unscrupulous acts."
Clearview's website says its images are "sourced from public-only web sources, including news media, mugshot websites, public social media, and other open sources".
Automated bulk data collection from public websites is often referred to as "scraping".
The OAIC investigation report notes Clearview AI "collects images from social media websites, including Facebook and YouTube".
"When Australians use social media or professional networking sites, they don't expect their facial images to be collected without their consent by a commercial entity to create biometric templates for completely unrelated identification purposes," Ms Falk wrote.
The report notes that the terms and conditions of the sites "each prohibit this kind of scraping and a number of social media companies have sent the respondent cease and desist letters in relation to alleged scraping from their sites".
However, Ms Falk was critical of the social media firms, saying the case raised the question whether "online platforms are doing enough to prevent and detect scraping of personal information".
There are some signs that big tech companies are growing wary of face recognition.
On Tuesday, Facebook announced that it would no longer use facial recognition software to identify faces in photographs and videos.
But online tools, and search engines, using facial recognition technology continue to operate online, privacy campaigners warn.