Uploaded Thousands of Photos How Long for Facial Recognition

Credit... Adam Ferriss

The Secretive Company That Might End Privacy as We Know It

A lilliputian-known start-up helps law enforcement match photos of unknown people to their online images — and "might lead to a dystopian future or something," a backer says.

Credit... Adam Ferriss

• Published Jan. xviii, 2020 Updated Nov. 2, 2021

Until recently, Hoan Ton-That'due south greatest hits included an obscure iPhone game and an app that allow people put Donald Trump's distinctive yellow hair on their own photos.

So Mr. Ton-That — an Australian techie and former model — did something momentous: He invented a tool that could end your ability to walk down the street anonymously, and provided it to hundreds of law enforcement agencies, ranging from local cops in Florida to the F.B.I. and the Section of Homeland Security.

His tiny company, Clearview AI, devised a groundbreaking facial recognition app. Yous accept a moving-picture show of a person, upload it and go to see public photos of that person, along with links to where those photos appeared. The system — whose courage is a database of more than than iii billion images that Clearview claims to have scraped from Facebook, YouTube, Venmo and millions of other websites — goes far beyond anything ever constructed past the U.s. government or Silicon Valley giants.

Federal and country law enforcement officers said that while they had only express knowledge of how Clearview works and who is behind it, they had used its app to help solve shoplifting, identity theft, credit menu fraud, murder and child sexual exploitation cases.

Until now, technology that readily identifies everyone based on his or her face has been taboo because of its radical erosion of privacy. Tech companies capable of releasing such a tool have refrained from doing and then; in 2011, Google's chairman at the time said it was the one technology the company had held back because information technology could be used "in a very bad way." Some large cities, including San Francisco, have barred police from using facial recognition technology.

But without public scrutiny, more than 600 police enforcement agencies have started using Clearview in the past year, according to the company, which declined to provide a list. The computer lawmaking underlying its app, analyzed by The New York Times, includes programming language to pair information technology with augmented-reality spectacles; users would potentially be able to place every person they saw. The tool could identify activists at a protestation or an bonny stranger on the subway, revealing not just their names but where they lived, what they did and whom they knew.

And it'south not only law enforcement: Clearview has too licensed the app to at least a handful of companies for security purposes.

"The weaponization possibilities of this are countless," said Eric Goldman, co-director of the High Tech Police force Plant at Santa Clara Academy. "Imagine a rogue law enforcement officer who wants to stalk potential romantic partners, or a foreign authorities using this to dig upward secrets near people to blackmail them or throw them in jail."

Clearview has shrouded itself in secrecy, avoiding argue about its boundary-pushing engineering science. When I began looking into the visitor in November, its website was a bare folio showing a nonexistent Manhattan address every bit its identify of business. The company's one employee listed on LinkedIn, a sales manager named "John Adept," turned out to be Mr. Ton-That, using a fake name. For a month, people affiliated with the visitor would not return my emails or phone calls.

While the company was dodging me, it was also monitoring me. At my asking, a number of police officers had run my photo through the Clearview app. They soon received telephone calls from company representatives asking if they were talking to the media — a sign that Clearview has the ability and, in this instance, the ambition to monitor whom police force enforcement is searching for.

Facial recognition technology has e'er been controversial. It makes people nervous about Big Blood brother. It has a tendency to deliver faux matches for certain groups, similar people of colour. And some facial recognition products used by the police — including Clearview'due south — haven't been vetted by independent experts.

Clearview's app carries extra risks because law enforcement agencies are uploading sensitive photos to the servers of a visitor whose ability to protect its information is untested.

The company somewhen started answering my questions, saying that its earlier silence was typical of an early-stage start-up in stealth mode. Mr. Ton-That acknowledged designing a prototype for employ with augmented-reality glasses but said the visitor had no plans to release it. And he said my photograph had rung alarm bells because the app "flags possible anomalous search beliefs" in gild to foreclose users from conducting what it accounted "inappropriate searches."

In add-on to Mr. Ton-That, Clearview was founded past Richard Schwartz — who was an aide to Rudolph West. Giuliani when he was mayor of New York — and backed financially by Peter Thiel, a venture capitalist behind Facebook and Palantir.

Another early investor is a modest business firm called Kirenaga Partners. Its founder, David Scalzo, dismissed concerns about Clearview making the internet searchable past face up, saying information technology'south a valuable criminal offense-solving tool.

"I've come to the conclusion that because information constantly increases, in that location's never going to be privacy," Mr. Scalzo said. "Laws have to determine what's legal, but you can't ban technology. Sure, that might lead to a dystopian future or something, just you lot tin can't ban it."

Image

Credit... Amr Alfiky for The New York Times

Addicted to A.I.

Mr. Ton-That, 31, grew up a long way from Silicon Valley. In his native Australia, he was raised on tales of his regal ancestors in Vietnam. In 2007, he dropped out of college and moved to San Francisco. The iPhone had just arrived, and his goal was to get in early on what he expected would be a vibrant market for social media apps. But his early ventures never gained real traction.

In 2009, Mr. Ton-That created a site that let people share links to videos with all the contacts in their instant messengers. Mr. Ton-That shut it down afterward it was branded a "phishing scam." In 2015, he spun up Trump Hair, which added Mr. Trump's distinctive coif to people in a photo, and a photo-sharing program. Both fizzled.

Dispirited, Mr. Ton-That moved to New York in 2016. Tall and slender, with long black hair, he considered a modeling career, he said, but later on one shoot he returned to trying to figure out the side by side big thing in tech. He started reading academic papers on artificial intelligence, prototype recognition and auto learning.

Mr. Schwartz and Mr. Ton-That met in 2016 at a book issue at the Manhattan Institute, a conservative think tank. Mr. Schwartz, now 61, had amassed an impressive Rolodex working for Mr. Giuliani in the 1990s and serving equally the editorial page editor of The New York Daily News in the early on 2000s. The two soon decided to go into the facial recognition business organisation together: Mr. Ton-That would build the app, and Mr. Schwartz would use his contacts to drum upwardly commercial involvement.

Law departments have had access to facial recognition tools for nigh 20 years, merely they have historically been limited to searching government-provided images, such equally mug shots and driver's license photos. In recent years, facial recognition algorithms take improved in accuracy, and companies similar Amazon offer products that can create a facial recognition programme for any database of images.

Mr. Ton-That wanted to go way beyond that. He began in 2016 past recruiting a couple of engineers. I helped design a program that can automatically collect images of people'southward faces from across the internet, such as employment sites, news sites, educational sites, and social networks including Facebook, YouTube, Twitter, Instagram and fifty-fifty Venmo. Representatives of those companies said their policies prohibit such scraping, and Twitter said information technology explicitly banned employ of its data for facial recognition.

Another engineer was hired to perfect a facial recognition algorithm that was derived from academic papers. The outcome: a organisation that uses what Mr. Ton-That described as a "state-of-the-art neural net" to convert all the images into mathematical formulas, or vectors, based on facial geometry — like how far apart a person's optics are. Clearview created a vast directory that clustered all the photos with similar vectors into "neighborhoods." When a user uploads a photo of a confront into Clearview's organisation, information technology converts the face into a vector and then shows all the scraped photos stored in that vector'due south neighborhood — along with the links to the sites from which those images came.

Mr. Schwartz paid for server costs and bones expenses, but the performance was bare bones; everyone worked from home. "I was living on credit card debt," Mr. Ton-That said. "Plus, I was a Bitcoin believer, then I had some of those."

Image

Credit... Amr Alfiky for The New York Times

Going Viral With Law Enforcement

By the end of 2017, the visitor had a formidable facial recognition tool, which it called Smartcheckr. Simply Mr. Schwartz and Mr. Ton-That weren't sure whom they were going to sell it to.

Possibly it could be used to vet babysitters or as an addition feature for surveillance cameras. What about a tool for security guards in the lobbies of buildings or to help hotels greet guests by name? "We thought of every idea," Mr. Ton-That said.

One of the odder pitches, in late 2017, was to Paul Nehlen — an anti-Semite and self-described "pro-white" Republican running for Congress in Wisconsin — to use "unconventional databases" for "farthermost opposition inquiry," according to a document provided to Mr. Nehlen and later posted online. Mr. Ton-That said the visitor never really offered such services.

The visitor shortly inverse its proper noun to Clearview AI and began marketing to police force enforcement. That was when the company got its first round of funding from outside investors: Mr. Thiel and Kirenaga Partners. Among other things, Mr. Thiel was famous for secretly financing Blob Hogan'southward lawsuit that bankrupted the popular website Gawker. Both Mr. Thiel and Mr. Ton-That had been the subject of negative articles past Gawker.

"In 2017, Peter gave a talented young founder $200,000, which ii years later converted to disinterestedness in Clearview AI," said Jeremiah Hall, Mr. Thiel's spokesman. "That was Peter's only contribution; he is not involved in the visitor."

Even after a 2nd funding circular in 2019, Clearview remains tiny, having raised $7 million from investors, according to Pitchbook, a website that tracks investments in commencement-ups. The company declined to ostend the corporeality.

In February, the Indiana State Law started experimenting with Clearview. They solved a case inside 20 minutes of using the app. Two men had gotten into a fight in a park, and it concluded when one shot the other in the tummy. A bystander recorded the crime on a phone, and then the police had a still of the gunman's face to run through Clearview's app.

They immediately got a lucifer: The man appeared in a video that someone had posted on social media, and his name was included in a caption on the video. "He did not have a driver's license and hadn't been arrested as an developed, so he wasn't in government databases," said Chuck Cohen, an Indiana State Police force captain at the time.

The homo was arrested and charged; Mr. Cohen said he probably wouldn't have been identified without the power to search social media for his confront. The Indiana State Police became Clearview's starting time paying customer, co-ordinate to the company. (The constabulary declined to comment beyond saying that they tested Clearview'due south app.)

Clearview deployed current and former Republican officials to approach police force forces, offering gratuitous trials and almanac licenses for as little as $ii,000. Mr. Schwartz tapped his political connections to help make regime officials aware of the tool, according to Mr. Ton-That. ("I'm thrilled to have the opportunity to assistance Hoan build Clearview into a mission-driven organization that's helping constabulary enforcement protect children and enhance the safe of communities across the land," Mr. Schwartz said through a spokeswoman.)

The visitor'southward main contact for customers was Jessica Medeiros Garrison, who managed Luther Strange's Republican entrada for Alabama chaser general. Brandon Fricke, an North.F.L. amanuensis engaged to the Pull a fast one on Nation host Tomi Lahren, said in a financial disclosure report during a congressional campaign in California that he was a "growth consultant" for the company. (Clearview said that it was a cursory, unpaid role, and that the company had enlisted Democrats to help market its production also.)

The company's most effective sales technique was offering thirty-day complimentary trials to officers, who and so encouraged their acquisition departments to sign upward and praised the tool to officers from other police departments at conferences and online, co-ordinate to the company and documents provided by police force departments in response to public-record requests. Mr. Ton-That finally had his viral striking.

In July, a detective in Clifton, N.J., urged his captain in an electronic mail to buy the software because it was "able to place a suspect in a matter of seconds." During the department's costless trial, Clearview had identified shoplifters, an Apple tree Store thief and a practiced Samaritan who had punched out a man threatening people with a knife.

Photos "could be covertly taken with telephoto lens and input into the software, without 'called-for' the surveillance operation," the detective wrote in the email, provided to The Times by ii researchers, Beryl Lipton of MuckRock and Freddy Martinez of Open up the Government. They discovered Clearview belatedly last twelvemonth while looking into how local police departments are using facial recognition.

Co-ordinate to a Clearview sales presentation reviewed past The Times, the app helped identify a range of individuals: a person who was defendant of sexually abusing a child whose confront appeared in the mirror of someone's else gym photograph; the person behind a cord of mailbox thefts in Atlanta; a John Doe found dead on an Alabama sidewalk; and suspects in multiple identity-fraud cases at banks.

Listen to 'The Daily': The End of Privacy equally We Know It?

An unregulated facial recognition app can probably tell the law your name, and help them find out where yous live and who your friends are.

transcript

transcript

Heed to 'The Daily': The End of Privacy as We Know It?

Hosted by Michael Barbaro; produced by Annie Brownish and Daniel Guillemette; with assist from Michael Simon Johnson; and edited by Paige Cowett and Larissa Anderson

An unregulated facial recognition app can probably tell the police your name, and help them notice out where you lot live and who your friends are.

[music]

michael barbaro

From The New York Times, I'm Michael Barbaro. This is "The Daily."

Today: A secretive company promising the next generation of facial recognition software has compiled a database of images far bigger than anything always constructed past the U.Southward. government. The Daily's Annie Chocolate-brown speaks to reporter Kashmir Hill about whether the technology is a breakthrough for law enforcement or the end of privacy every bit we know it.

Information technology's Mon, February 10.

annie brown

Kashmir, how did this story come to you?

kashmir hill

So I got an email. It was a Midweek morning. I was checking my phone. And it was from a tipster who had gotten a bunch of documents from police departments. And one of the constabulary departments had sent along this memo virtually a private visitor that was offer a radical new tool to solve crimes using facial recognition.

annie dark-brown

And what would make a facial recognition tool radical?

kashmir hill

So law enforcement has for years had admission to facial recognition tools. But what this visitor was offering was different any other facial recognition tools that police force accept been using, because they had scraped the open up spider web of public photos — from Facebook, from Venmo, from Twitter, from education sites, employment sites — and had a massive database of billions of photos. So the pitch is that you can accept a picture of a criminal suspect, put their confront into this app and place them in seconds.

annie brown

And when you read this memo, what do you make of what this company is offering?

kashmir hill

So I've been covering privacy for ten years, and I know that a technology like this in public hands is the nightmare scenario.

[music]

This has been a tool that was too taboo for Silicon Valley giants who were capable of building it. Google in 2011 said that they could release a tool like this, simply it was the one engineering they were holding back because it could be used in a very bad way.

annie brown

And why exactly is this kind of technology this line in the sand that no one will cross? What makes it so dangerous?

kashmir colina

So imagine this technology in public hands. It would hateful that if you were at a bar and someone saw y'all and was interested in you lot, they could take your photo, run your face through the app, and and so it pulls up all these photos of you lot from the internet. Information technology probably takes them back to your Facebook page. So at present they know your proper name, they know who you're friends with, they can Google your name, they can see where you live, where y'all work, perhaps how much coin yous brand. Let'due south say y'all're a parent and you're walking downwardly the street with your three-year-one-time. Somebody can take a photo of you and know where the two of you live. Imagine you lot're a protester in the U.S. or in a more than authoritarian government. All of a sudden they know everything about you lot, and you lot tin can face repercussions for just trying to exercise your political opinions. If this app were made publicly available, it would be the finish of being bearding in public. You would accept to assume anyone can know who you are any fourth dimension they're able to accept a photograph of your face.

annie brown

Then that technology is what this company is pitching these police departments?

kashmir hill

Exactly.

annie brown

And what exercise you know about this company at this signal?

kashmir hill

So at this bespeak, all I really know is that the visitor is called Clearview AI. And so the first thing I do is Google it. And I find their website, which is clearview.ai. And the website is pretty bare, but there'south as well an role accost listed there, 145 West 41st Street, which happens to be but a couple of blocks from The New York Times function.

annie brown

Right.

kashmir hill

Then I decided to walk over at that place, and there merely is no 145 West 41st Street. So that was weird. So now I have this company that's offer this radical new tool —

annie dark-brown

Information technology's got a fake accost.

kashmir hill

It's got a fake address, which is a huge reddish flag.

annie brown

And then what yous do side by side?

kashmir hill

I found the company on LinkedIn. It only had ane employee listed, a sales director named John Good, which —

annie brownish

John Skillful.

kashmir hill

John Good. It seemed like information technology could also be fake. And I sent that person a LinkedIn message and never heard back. So one of the things I find online is a website called PitchBook that lists investments in start-ups. And then information technology says that this Clearview AI has received $seven million from a venture capital letter house and from Peter Thiel — you know, a big name in Silicon Valley, invested in Facebook and Palantir. Then I reach out to his spokesperson, and he says I'll get back to you. I never hear from him again. And then one day, I open up Facebook, and I have a message from a friend whose name I don't recognize. And he says, hey, I hear you're looking into Clearview AI. I know them. They're a bully company. How can I aid?

annie chocolate-brown

And you don't know who this guy is?

kashmir hill

I don't. I mean, it's a guy I met once x years agone. And somehow he knows that I'1000 looking into this company. But I'll take information technology. You know, finally —

annie brown

Right!

kashmir hill

— somebody wants to talk to me nearly Clearview AI. So I say, hey, can I requite you a call? And then he doesn't respond, which I'm getting used to.

annie dark-brown

You just can't catch a interruption.

kashmir colina

I know. I'm like, I cannot believe this is another dead end.

So phone and e-mail are non working for me. So I just need to effigy out another door to knock on to try to talk to a real man existence. And one of the investors in the company is this venture capital business firm that has an office in Bronxville, New York. So on a common cold, rainy Tuesday, I got on the train and headed to Bronxville. I become to the company'due south accost. It's just like in a retail space. And go inside. In that location's this long, tranquility hallway of role suites, and this venture capital house is at the very end. And I knock on the door, and there'southward no one there. So I start trying to talk to their neighbors, and a adult female who works adjacent door says, oh yeah, they're never here. And then I'g walking downward the stairs to get back out of the building, and two guys walk through the door. They're both in dark suits with lavander and pinkish shirts underneath, and they merely kind of expect like V.C.s to me. And so I say, hey, are you with this venture majuscule business firm? And they say, we are. Who are you? And I was like, I'grand the New York Times reporter who'south been trying to make it impact with you. And they said, the company has told us not to talk to you. And I said, well I've come all the fashion out to Bronxville. Can nosotros just chat for a little scrap? And they say, O.K. If probably helps that I'm very pregnant, and they offered me h2o. And they just start telling me everything.

[music]

annie dark-brown

And what exercise they tell you?

kashmir hill

They ostend that they've invested in Clearview AI and that Peter Thiel has also invested. They identified the genius coder behind the company, this guy named Hoan Ton-That. And they say he's Vietnamese royalty but he's from Australia. And they besides tell me that Hoan is the ane that was using the faux name John Skillful on LinkedIn.

annie brown

He'due south John Proficient.

kashmir hill

He's John Good.

And they confirm that law enforcement is already using the app. And that police enforcement loves it and that it'southward spreading like wildfire.

annie dark-brown

Wow.

kashmir loma

And so I've learned some stuff from these 2 investors, just no one from the company is talking to me still. So in the meantime, I am also reaching out to police force enforcement, because I want to know if this app really works as well as the company claims. By this point, I had learned that over 600 police force enforcement agencies had tried the app, including the Department of Homeland Security and the F.B.I.

annie brown

Wow. Information technology'south non just local police departments. This is being used by the federal regime already.

kashmir colina

Yep, I mean, I was just shocked to discover how easily government agencies can merely endeavour a new technology without apparently knowing much about the visitor that provides information technology. And then I talked to a retired police force primary from Indiana, who was actually i of the first departments to use the app. And they solved a case inside twenty seconds, he said.

annie brown

A case they hadn't been able to solve?

kashmir colina

That they hadn't been able to solve. One of the officers told me that he went back through like thirty dead-cease cases that hadn't had any hits on the authorities database, and he got a bunch of hits using the app. So they were really excited about information technology.

annie brownish

This is fashion more than effective than what they were using before.

kashmir hill

Exactly. With the regime databases they were previously using, they had to accept a photo that was just a direct full-face photograph of a suspect — like mug shots and driver'due south license photos. But with Clearview, it could exist a person wearing glasses, or a hat, or part of their face was covered, or they were in profile, and officers were still getting results on these photos.

annie chocolate-brown

Wow.

kashmir loma

Just the nearly astounding story I was told was that investigators had this child exploitation video, and there was an developed who was visible in the video just for a few seconds in the background. So they had this person's face. They had run information technology through their usual databases and not gotten anything back. But then they ran his face through Clearview'southward app, and he turned up in the background of someone else's gym selfie. You could see his face in the mirror. And so they figured out what gym this photo was taken out. They went to the gym. They asked the employees, do you know who this is? And the employee said, we can't tell you. We have to protect our members' privacy. Only then later, the detectives got a text from somebody who worked there identifying the person. And that — I mean, that's just something that would not have been possible without Clearview's app.

So because officers were telling me the tool works so well, I wanted to meet it for myself, on myself. And I asked them if they would run my photo through the app. Merely every time I did this, things would get weird. The officers would tell me that they ran my photo and there were no results.

annie brown

No pictures of you?

kashmir loma

There were no pictures of me, which was really weird, because I accept a lot of photos of myself online. Then officers would just stop responding to me or talking to me. And I had no idea what was going on until one officeholder was kind enough to explain to me.

[phone ringing]

officer

Hello, how are yous.

kashmir loma

Hey. It's Kashmir.

officeholder

Yep, hi. Mm-hmm.

kashmir colina

I'm keeping this officer anonymous because he could make it serious trouble for talking to me and so openly about Clearview.

kashmir hill

If y'all could just depict yourself, to the extent that you can describe yourself.

officer

I'thousand a law officeholder at a large metropolitan police department.

kashmir loma

Then he's a cop who was doing a thirty-twenty-four hour period costless trial of the app. And he was actually impressed with information technology. So I asked him if he wouldn't heed running my photograph.

annie brown

And what did he tell y'all happened when he sent your picture through?

officer

Yes, zero. I didn't get a response at all.

kashmir colina

No results?

officer

No results. And inside a couple of minutes of me putting your photo up in that location — maybe five, less than 10 — I got a phone call from the Clearview company. They wanted to know why I was uploading a New York Times reporter'south photo.

kashmir loma

That is and so wild. I don't know. [LAUGHS] It creeps me out as a reporter. I hateful yeah, it just —

officer

It kind of creeped me out equally a user.

kashmir hill

And then this implied that Clearview flagged my face in their system such that they got an warning when a police officer ran my face. Which I constitute —

annie brown

Wow.

kashmir hill

— very alarming, because this is telling me for the first time that this visitor is able to monitor who law enforcement is looking for, and not just know who they're looking for, just manipulate the results. And so then that fabricated me become back to the before officers who had run my photograph. And they all confirmed, yes, I got a telephone call from the visitor, and they said, nosotros're not supposed to be talking to the media.

[music]

kashmir hill

So were you lot able to continue using the app later that?

officer

My business relationship was deactivated.

kashmir hill

Did you ever get access dorsum?

officer

I never did. Just I take colleagues that have access. So if I were to demand a picture searched, I could simply email it to them and they can email me the results.

kashmir colina

And you call back the merchandise-offs are worth information technology, in terms of what the company has admission to?

officer

Practise I think it'due south worth it? And then from a police enforcement perspective, it's worth it. Nosotros get a lot of cases, and we don't normally accept a lot of leads. And so anything that can — honestly, anything that tin help us solve a law-breaking is a win for united states. From a privacy perspective, it's rather frightening the amount of information that they were able to get and provide. Every bit long equally they're doing information technology for the right reasons, then everything will piece of work out. Permit's put it that style.

[music]

kashmir loma

Only the trouble is we don't know anything near the company at this indicate. We don't know if there's whatever kind of oversight. We don't know who the people are that are operating this and what their intentions are with their product. The person in charge of the visitor won't talk to me. Just then, it'south the end of December when I get a call from the company'southward spokeswoman. And she says that the founder, Hoan Ton-That, is ready to talk.

michael barbaro

We'll be right back.

kashmir hill

Do you take a hard stop?

hoan ton-that

No I don't really. 12:thirty.

lisa linden

12:00 noon.

hoan ton-that

Oh, 12:00 apex.

kashmir hill

I accept no hard cease.

lisa linden

Oh.

kashmir hill

And I have lots of questions, and so I'll take equally much time equally you can requite me.

annie brown

Then Kashmir, you finally got an interview with the founder of Clearview, this man named Hoan Ton-That. Where practice you meet him?

kashmir hill

Then we met in a WeWork in Chelsea. He came downward to the lobby.

kashmir hill

You similar New York, y'all're going to stay here?

hoan ton-that

Oh, yeah.

kashmir colina

And his appearance surprised me, because I had Googled him online and there are a lot of photos of him. And he'south usually pretty eccentric — similar a lot of paisley shirts, he'southward at Burning Man.

hoan ton-that

Let'southward go to the back room.

kashmir loma

But in person he was very bourgeois. He was in this dark blue navy adapt with a white push-up and leather shoes. And then he looked very much like the security outset-upwardly entrepreneur.

annie brown

He was looking the part.

kashmir hill

He was looking the part.

kashmir hill

When were you lot born? How quondam are you lot?

hoan ton-that

'88, so I'm 31.

kashmir hill

O.K.

annie chocolate-brown

And what do you learn virtually him?

kashmir hill

And so he is 31. He grew up in Australia, but you tin can't hear that in his voice.

hoan ton-that

I dear computers, apparently.

kashmir hill

Yes, and so how did you get interested in engineering?

hoan ton-that

We had a computer, of course, when I was four or five years old.

kashmir loma

And then his family got a figurer when he was 3 or four, and he was e'er tinkering with computers growing up.

hoan ton-that

We got the net when I was 10, I think. And then you could find all these things online. But Linux, I was like I take to get this thing. It's the nerdiest thing ever. I convinced my dad. Nosotros installed it, and I would spend the whole summer reinstalling and learning Linux stuff, staying domicile from high schoolhouse and learning programming for fun. So that's — I merely actually liked it.

kashmir hill

He enrolled in college, decided to drop out like many technologists practice, and moved to San Francisco when he was 19.

hoan ton-that

— 2007, earlier it was a big affair, correct? It was kind of getting there, but it wasn't huge.

kashmir loma

This is 2007, and this is kind of a boom time. The iPhone has just come out.

hoan ton-that

That's the Facebook app era. Remember that?

kashmir hill

Yes.

kashmir hill

People are condign millionaires by making Facebook games. And he wants to be the side by side big app guy.

hoan ton-that

Being there is a lot unlike from reading about it online. Yous absorb a lot more of how people get things done. And you learn a lot more secrets.

annie dark-brown

What did he built?

kashmir hill

And then the Facebook apps were similar "would you rather" apps and kind of like romantic GIFs.

hoan ton-that

Did Some of the start iPhone games as well.

kashmir hill

I of his most recent apps was called Trump Pilus, and it was an app for adding Trump'southward pilus to your photos.

annie brown

That's it?

kashmir hill

That'south it. The tagline was, "It's gonna be yuge!"

annie brown

O.K. [LAUGHS] So how do you move from a Donald Trump hair app to something that seems like it could revolutionize police work?

kashmir loma

Well, he moved to New York. And that seemed to be a big modify for him. And he started meeting very different people. And one of the nigh of import people he met was Richard Schwartz.

hoan ton-that

I concluded up meeting Richard at a party.

kashmir hill

This 61-twelvemonth-old guy who worked for Mayor Rudy Giuliani in the 1990s. He was just very politically connected.

hoan ton-that

I actually loved that. He had a lot of stories. And then we talked for an hr about different ideas. Because I was similar, this is what I do — technology. I can brand anything. And it went from there.

kashmir hill

And the two of them decided, with Hoan Ton-That'due south tech know-how and Richard's Rolodex, that they want to try to start a facial recognition company together.

annie brown

And why facial recognition? Why did the 2 of them choose that?

kashmir colina

I retrieve it was considering Hoan had started reading a lot of papers near facial recognition and machine learning.

hoan ton-that

I had never really studied AI stuff earlier, but I could pick upwardly a lot of it.

kashmir colina

And I think they realized they could make money doing it.

kashmir hill

What would you say, in terms of the range of ideas at first, what were yous thinking?

hoan ton-that

A lot. I could keep, actually crazy, just —

kashmir hill

At that place's a lot of face up recognition algorithms out there, and a lot that work pretty well. What was different about what Hoan Ton-That and Richard Schwartz were doing is they had been willing to scrape all of these photos from the internet. So they just had a huge database of photos.

annie brown

Correct, the billions of photos.

kashmir hill

Exactly.

hoan ton-that

And so we had this betoken where we got to 99 percent accuracy. I remember that, it was just in the office. And he was similar, wow, it works. Endeavour that one again. Endeavour that one again. And just every time, information technology would option the right person out. And that'southward when we knew, this is crazy. This actually works.

annie brown

Is that legal? Tin can you just have photographs from anywhere on the internet and apply them for this kind of thing?

kashmir hill

There was a ruling in a federal courtroom this fall that said, yeah, this kind of public scraping seems to be legal.

annie brown

And what are they hoping to practise with this software at this point?

kashmir loma

I mean, they're but trying to effigy out how they tin brand coin off of the app. And then they somewhen end up settling on police force enforcement.

hoan ton-that

And they start solving cases from grainy A.T.M. photos, cases they would've never solved. And so this spread to different departments, and then from one bureau to other agencies.

annie brown

And do y'all ask him about that thing that happened with the officer who couldn't find your photos?

kashmir hill

Yeah, and so that was ane of my questions, and I wasn't entirely satisfied by his answer.

hoan ton-that

Then —

kashmir hill

Ane matter that surprised me — some of the officers I talked to tried to run my photo through it, and they got no hits. And I tons of photos online.

hoan ton-that

[LAUGHS] It must have been a bug.

kashmir hill

Did you guys block me from like getting results?

hoan ton-that

I don't know nearly that.

kashmir hill

Because I was like, this doesn't make whatsoever sense.

kashmir hill

He said, oh yep, that was a software bug. But he laughed.

kashmir hill

I was like, I take ane,000 photos online. This tin't work as well as they say it works.

hoan ton-that

Yeah, well, information technology must accept been a issues in the software or something.

kashmir hill

[LAUGHS] Why did yous do that? It totally made me call up that —

hoan ton-that

Hey, maybe it doesn't work. You never know, right? This could be the long con.

kashmir hill

Ah, O.M.

hoan ton-that

I'm kidding, I'm kidding. It works.

annie brownish

What do you call up that was about?

kashmir loma

[LAUGHS] I don't think it was a software bug.

hoan ton-that

It's a bug. I don't know. I —

kashmir hill

Y'all take no idea, huh?

annie dark-brown

Huh.

kashmir hill

Aye. So he said the software bug is now stock-still.

hoan ton-that

Oh aye, then I'll show yous. This is the iPhone version.

kashmir hill

And he took a photo of me.

hoan ton-that

Oh, it does work.

kashmir hill

Oh, that's and then surprising.

hoan ton-that

I know.

kashmir hill

And there, the results included a agglomeration of photos of me online.

kashmir hill

Oh my god, I totally forgot.

hoan ton-that

Well, nosotros tin take —

kashmir hill

That's 10 years ago.

kashmir loma

Including some I had never seen earlier.

kashmir hill

Some of these photos I didn't know were online.

annie brown

So he's just brushing off this weird thing that happened to you lot. Just do you lot get the sense that he's thinking at all well-nigh privacy?

kashmir hill

And then I asked him, you know, this is a very powerful app. And I asked him what restrictions is he thinking about for it. And he said, i, that they were simply selling it to law enforcement right at present, though it does turn out that they're likewise selling it to a few private companies for security purposes. But he said they wouldn't sell it to bad actors or bad governments.

hoan ton-that

— and our philosophy is basically, if it's a U.Southward. based — or like a democracy or an ally of the U.South. — we will consider it. Only like, no China, no Russia or anything that wouldn't be good. So if it'due south a country where it's simply governed terribly or whatever, I don't know if we'd experience comfortable selling to certain countries.

annie chocolate-brown

So information technology doesn't sound like he has much of a rubric for deciding who to sell to. And it sounds like there's no i actually overseeing how he's making these decisions.

kashmir hill

At this betoken, it's simply up to Clearview to decide who they want to sell the app to.

hoan ton-that

No pressure, but when we talk to some venture capitalists, they're like, "Why don't y'all make this consumer? Law enforcement is such a pocket-sized market. You won't make that much money." And we've considered it, and nosotros're just like, what's the use case hither? And right at present, we take hold of, assistance catch pedophiles. What if a pedophile got admission to this, goes around the street, runs —

kashmir hill

Just when I was talking to one of their investors, he says, we want to dominate the law enforcement market, and so nosotros want to motility into other markets like hospitality, like real estate. And he predicted that ane day, all consumers will have access to this app.

hoan ton-that

Um, and —

kashmir hill

I tin can tell you that ane of your investors hopes that you guys are going to go into the consumer market.

hoan ton-that

Well, aye. He talks too much. But like, we're not — we're non going to do that. I just don't —

annie brown

Hoan seems to be saying, yep, at that place's pressure on us to sell to individual consumers, merely we're not going to practise that. And how reasonable is it to think that he has control or the visitor has command at this point over where this technology goes?

kashmir colina

I mean, one point that I made when I was talking to him is that often, the tools that law enforcement use stop upwards in the hands of the public.

kashmir hill

I but — I personally feel similar you guys have opened the door to at present this becoming more than normalized, just because a lot of tools that law enforcement take eventually make their style into public hands.

hoan ton-that

Not e'er. Not anybody has a gun. [LAUGHS] Right? That would be —

kashmir hill

Anyone who wants one can become one in the U.S. basically, just —

kashmir loma

His response was foreign. He said, well, look at guns. Law enforcement has guns, but not everybody has a gun. And I don't know if that's because he's from Australia?

annie dark-brown

Yep, he's proving your indicate, in a way.

kashmir loma

[LAUGHS] It did seem like he was proving my bespeak, rather than rebutting it.

[music]

We've been building the engineering to make this possible for years now. Facebook building this huge database of our photos with our names fastened to it, advances in prototype recognition and search technologies, it all led us here. But in that location's been no accompanying regulation or rules effectually how the technology should be used. There's no real law or regulation that makes this illegal. The scraping seems to be O.Thousand. We don't have a big ban on facial recognition. We don't need to requite consent for people to process our faces. And so in terms of holding this tool dorsum, we're just relying on the moral compasses of the companies that are making this technology and on the thoughtfulness of people similar Hoan Tan-That.

kashmir hill

But yeah, what do you think most that? Do you call back that this is as well dangerous a tool for everybody to have?

hoan ton-that

I take to think virtually that and actually go back to you on an answer, because it's a skillful question.

kashmir hill

Aye.

hoan ton-that

I've idea about it a piddling bit.

kashmir hill

You haven't thought well-nigh it? You lot have?

hoan ton-that

I have, I have. But I demand to really come up up with a good answer for that. Honestly like, yeah.

[music]

annie brown

Thanks, Kashmir.

kashmir colina

Give thanks yous.

michael barbaro

Since Kashmir began reporting on Clearview AI, several major social media companies including Facebook, Twitter and Venmo take demanded that the company stop using photos scraped from their websites. But it's unclear what, if whatever, power those social media companies take to force Clearview to comply. A few weeks ago, the state of New Jersey barred law enforcement from using Clearview's technology, but police remain gratis to do so in 49 other states.

We'll exist correct back.

Here's what else you demand to know today. President Trump has begun a campaign of retribution confronting witnesses in the impeachment enquiry, firing Gordon Sondland, his ambassador to the European Wedlock, who called the president'south actions toward Ukraine a quid pro quo. And Lieutenant Colonel Alexander Vindman, a member of the National Security Quango, who expressed alarm over the president's phone call with the leader of Ukraine. The Times reports that several Republican senators urged Trump not to burn down the witnesses, fearing it would send a dangerous bulletin, merely that the president ignored their advice. And the global decease toll from the coronavirus has reached more than 800, surpassing that of the SARS epidemic, which killed 774 in 2003. The number of confirmed infections from the coronavirus now stands at more than 37,000. Finally, new polling in New Hampshire, which will hold its primary tomorrow, shows Mayor Pete Buttigieg cervix-and-neck with Senator Bernie Sanders and former Vice President Joe Biden slipping into fourth place.

archived recording (george stephanopoulos)

Vice President Biden, the beginning question is for you. In the terminal few days, you've been maxim that Democrats will be taking too big a risk if they nominate Senator Sanders or Mayor Buttigieg, but they came out on top in Iowa. What risks did the Iowa Democrats miss?

michael barbaro

The poll, conducted by The Boston Globe, WBZ and Suffolk University propose Buttigieg is benefiting from a strong performance in the Iowa caucuses and that Biden may perform poorly for the second time in a row, a prediction Biden confirmed during Friday night'southward debate on ABC.

archived recording (joe biden)

Oh, they didn't miss anything. This is a long race. I took a hit in Iowa, and I'll probably take it here.

michael barbaro

That'due south information technology for "The Daily." I'k Michael Barbaro. Run across y'all tomorrow.

Paradigm

Credit... Charlotte Kesl for The New York Times

In Gainesville, Fla., Detective Sgt. Nick Ferrara heard about Clearview last summer when information technology advertised on CrimeDex, a list-serv for investigators who specialize in fiscal crimes. He said he had previously relied solely on a land-provided facial recognition tool, FACES, which draws from more than than 30 million Florida mug shots and Department of Motor Vehicle photos.

Sergeant Ferrara found Clearview's app superior, he said. Its nationwide database of images is much larger, and unlike FACES, Clearview'due south algorithm doesn't crave photos of people looking straight at the camera.

"With Clearview, yous can use photos that aren't perfect," Sergeant Ferrara said. "A person tin can be wearing a hat or glasses, or it can be a profile shot or fractional view of their confront."

He uploaded his own photo to the system, and information technology brought up his Venmo page. He ran photos from sometime, dead-finish cases and identified more than 30 suspects. In September, the Gainesville Police force Section paid $10,000 for an annual Clearview license.

Federal law enforcement, including the F.B.I. and the Department of Homeland Security, are trying information technology, as are Canadian law enforcement authorities, according to the company and government officials.

Despite its growing popularity, Clearview avoided public mention until the cease of 2019, when Florida prosecutors charged a adult female with grand theft after ii grills and a vacuum were stolen from an Ace Hardware shop in Clermont. She was identified when the police force ran a nonetheless from a surveillance video through Clearview, which led them to her Facebook page. A tattoo visible in the surveillance video and Facebook photos confirmed her identity, according to an affirmation in the case.

'Nosotros're All Screwed'

Mr. Ton-That said the tool does not always work. Most of the photos in Clearview's database are taken at eye level. Much of the material that the police upload is from surveillance cameras mounted on ceilings or high on walls.

"They put surveillance cameras also high," Mr. Ton-That lamented. "The angle is incorrect for adept face recognition."

Despite that, the company said, its tool finds matches upwardly to 75 percent of the time. But it is unclear how often the tool delivers false matches, considering it has not been tested by an independent political party such as the National Institute of Standards and Applied science, a federal agency that rates the operation of facial recognition algorithms.

"Nosotros accept no data to advise this tool is accurate," said Clare Garvie, a researcher at Georgetown University's Centre on Privacy and Technology, who has studied the government'south utilise of facial recognition. "The larger the database, the larger the risk of misidentification because of the doppelgänger effect. They're talking about a massive database of random people they've found on the cyberspace."

Only current and former police force enforcement officials say the app is constructive. "For u.s., the testing was whether information technology worked or not," said Mr. Cohen, the one-time Indiana Country Police captain.

One reason that Clearview is catching on is that its service is unique. That'south because Facebook and other social media sites prohibit people from scraping users' images — Clearview is violating the sites' terms of service.

"A lot of people are doing it," Mr. Ton-That shrugged. "Facebook knows."

Jay Nancarrow, a Facebook spokesman, said the company was reviewing the state of affairs with Clearview and "will take advisable action if we find they are violating our rules."

Mr. Thiel, the Clearview investor, sits on Facebook's lath. Mr. Nancarrow declined to comment on Mr. Thiel'south personal investments.

Some constabulary enforcement officials said they didn't realize the photos they uploaded were being sent to and stored on Clearview'south servers. Clearview tries to pre-empt concerns with an F.A.Q. document given to would-be clients that says its customer-support employees won't wait at the photos that the law upload.

Clearview also hired Paul D. Clement, a Us solicitor general under President George Due west. Bush, to assuage concerns about the app's legality.

In an August memo that Clearview provided to potential customers, including the Atlanta Constabulary Department and the Pinellas County Sheriff's Office in Florida, Mr. Clement said law enforcement agencies "do not violate the federal Constitution or relevant existing land biometric and privacy laws when using Clearview for its intended purpose."

Mr. Clement, now a partner at Kirkland & Ellis, wrote that the regime don't accept to tell defendants that they were identified via Clearview, as long every bit it isn't the sole basis for getting a warrant to abort them. Mr. Clement did not answer to multiple requests for comment.

The memo appeared to be effective; the Atlanta police force and Pinellas County Sheriff's Office presently started using Clearview.

Because the police force upload photos of people they're trying to identify, Clearview possesses a growing database of individuals who have attracted attending from law enforcement. The visitor also has the ability to manipulate the results that the constabulary see. After the company realized I was asking officers to run my photograph through the app, my face up was flagged by Clearview'due south systems and for a while showed no matches. When asked nigh this, Mr. Ton-That laughed and called it a "software bug."

"It's creepy what they're doing, only in that location will be many more of these companies. In that location is no monopoly on math," said Al Gidari, a privacy professor at Stanford Police force School. "Absent a very strong federal privacy law, we're all screwed."

Mr. Ton-That said his company used only publicly available images. If you change a privacy setting in Facebook and so that search engines can't link to your contour, your Facebook photos won't be included in the database, he said.

Simply if your profile has already been scraped, information technology is too late. The company keeps all the images it has scraped even if they are later deleted or taken down, though Mr. Ton-That said the visitor was working on a tool that would let people request that images be removed if they had been taken down from the website of origin.

Woodrow Hartzog, a professor of constabulary and computer science at Northeastern University in Boston, sees Clearview as the latest proof that facial recognition should be banned in the The states.

"We've relied on industry efforts to self-constabulary and not embrace such a risky applied science, only now those dams are breaking considering there is and so much money on the table," Mr. Hartzog said. "I don't see a future where we harness the benefits of face recognition technology without the crippling abuse of the surveillance that comes with it. The only way to stop information technology is to ban it."

Where Everybody Knows Your Name

During a contempo interview at Clearview's offices in a WeWork location in Manhattan'south Chelsea neighborhood, Mr. Ton-That demonstrated the app on himself. He took a selfie and uploaded it. The app pulled up 23 photos of him. In ane, he is shirtless and lighting a cigarette while covered in what looks like claret.

Mr. Ton-That then took my photo with the app. The "software issues" had been stock-still, and now my photo returned numerous results, dating back a decade, including photos of myself that I had never seen earlier. When I used my hand to cover my nose and the bottom of my face, the app still returned vii correct matches for me.

Police officers and Clearview's investors predict that its app will eventually be bachelor to the public.

Mr. Ton-That said he was reluctant. "At that place's always going to be a customs of bad people who will misuse it," he said.

Even if Clearview doesn't make its app publicly available, a copycat company might, now that the taboo is cleaved. Searching someone by face could become equally easy equally Googling a proper noun. Strangers would be able to listen in on sensitive conversations, take photos of the participants and know personal secrets. Someone walking down the street would be immediately identifiable — and his or her abode accost would be just a few clicks abroad. It would herald the end of public anonymity.

Asked about the implications of bringing such a ability into the earth, Mr. Ton-That seemed taken aback.

"I take to recall nigh that," he said. "Our conventionalities is that this is the best utilize of the technology."

Jennifer Valentino-DeVries, Gabriel J.X. Dance and Aaron Krolik contributed reporting. Kitty Bennett contributed research.

eubankswich1985.blogspot.com

Source: https://www.nytimes.com/2020/01/18/technology/clearview-privacy-facial-recognition.html

Share this post