Face Rec Coming to Chicago Transit

From Bob Roberts at WBBM:

Pickpockets and muggers beware -- soon, every turnstile at a CTA 'L' station will have airport-style face recognition cameras, and the transit agency is planning much more comprehensive cameras at all 144 of its 'L' stations. [...] CTA President Richard Rodriguez said the agency's goal is to fully outfit all 'L' stations, but said it is dependent on federal funding, and as a result chose the 29 Green and Red Line stations first.

Rodriguez said all of the cameras will be linked not just to the CTA Control Center but to the city's 911 call center and police, as well. At suburban CTA 'L' stations, CTA has established or intends to establish a similar link with local police agencies and Cook County Sheriff's Police.

CTA's board Wednesday approved a $4.3 million contract with Teleste Corp., of Georgetown, Tex., to install the cameras and related equipment. It is financing the high-resolution turnstile cameras, which will be capable of showing facial details, through a $17.9 million U.S. Dept. of Homeland Security grant.

This type of public deployment of facial recognition technology is extremely difficult. So far, 3VR Security is the only organization to come close to providing adequate technology of this task as showcased in its deployment in South Korea.

US/Canada Radio Frequency IDs Set to Go Live

A lot of outlets are covering this piece of news, Homeland Security to scan fingerprints of travelers exiting the US, but I think a far more interesting tidbit is buried deep in the story:

The DHS is also set to go-live with a trial of RF-enabled biometric identity cards on June 1, aimed at securing the border between the United States and Canada.

[...]

As of June 1, travelers crossing the border will be asked to carry an RF-enabled card which transmits a passport photo image and information about the traveler to border control staff systems, who can then check that photograph against the physical appearance of the traveler or their vehicle.

I think this is a sign of what's to come at the border. For now, these kinds of trials will likely remain limited. Also, given recent problems with more automated verification programs, like in Manchester, these trials will almost certainly require a good deal of manual monitoring and human review for the foreseeable future.

South Korean Study Delivers a Face Rec First (90%+ Accuracy Using Surveillance Video)

Those following the South Korean government’s most recent biometric initiative know it to be one of the most ambitious facial recognition testing and deployment projects ever conceived. The project, in its various phases, has been featured quite extensively on Korean television here and here.

Under the project, the Korean National Police (NPA) sought to deploy facial recognition technology against Seoul’s plentiful video surveillance cameras, both to generate alerts to the presence of targeted individuals, as well as provide a mechanism to quickly search city surveillance archives for persons identified after the fact.

Though the NPA has been quite public on its facial recognition project in general, details regarding the specific technologies and testing results have been closely guarded. However, today after much anticipation, the NPA has finally made its initial findings public. According to the NPA and SK Networks spokesmen, 3VR's facial recognition platform demonstrated the highest percentage of accuracy of any of the eight tested technologies in trials run by the NPA and its testing partner, South Korea's IT giant SK Networks.

Indeed the results, at around 90% accuracy, would seem far better than those achieved in any previous public video facial recognition study. After a grueling multi-year testing process, in 3VR SmartRecorders and SmartCams provided between 85 percent and 92 percent accuracy in recognizing and matching faces in a few crowded, highly-trafficked public train stations in Seoul. In each case, the images analyzed were of fast-moving groups of commuters entering or exiting various transit areas en masse.

Said Sung-Ho Kong of SK Networks:

“In 2008, we performed a live, uncontrolled test of 3VR's facial recognition technology in Seoul subway stations, where the solution was an impressive 85-92 percent accurate, depending on conditions. No other solution approached this level of accuracy, vastly improving our ability to track, find and thwart crime in subways and other highly populated areas, which had previously proven extremely difficult to monitor.”

To better appreciate just how impressive the NPA’s results are, we need to compare them to the best previous facial recognition study. In 2006-2007, a similar facial recognition-use case was evaluated by the German Federal Police. After months of testing in a German subway with lighting and traffic much more controlled than in South Korea, a mediocre 60 percent accuracy rating was achieved.

The improvements in accuracy afforded by 3VR are the result of a unique approach to video facial recognition. While most face rec vendors offer technologies optimized for the comparison of flat, normally lit, passport-style photos, only 3VR's software was built from the ground up to address many of the unique challenges presented by real-time surveillance video.

According to Tim Frederick, director of engineering at 3VR:

“This study demonstrates the breakthrough power of 3VR’s patented facial surveillance technology. Unlike other attempts at high-volume face surveillance, which re-purposed still-image face recognition algorithms, the South Korean study benefited from 3VR’s end-to-end video analysis system, specifically designed for this type of demanding video application.”

Expect more details from 3VR and the NPA shortly, but in the meantime, please check out SDN's Rhianna Daniels' feature story on the announcement, as well as a quick diagram of how the technology works and even some up-close and personal screen grabs of the UI.

UPDATE June 22, 2009:

Here is a translation of the South Korean National Police's Phase 1 testing results:

Face.com Brings Face Rec to Facebook

I was just accepted to the Face.com Alpha release and getting set up with the new service was pretty simple:

1) Face.com sent me an email with a Facebook link to their application.

2) I clicked the link and was asked to log on to my Facebook account.

3) The application then asked for permission to access my photos...and my friends photos.

4) Processing

5) Processing

6) Processing

7) I was presented with bunch of pictures of me that the software had found.



All-in-all not a bad experience and the facial recognition accuracy seems pretty good so far. I'll keep digging and check back in later with what I find out.

You are being watched...in 3D

I just got back from the International Security Show, ISCWest. I’ll be writing up some more thoughts on that in a bit, but in the meantime, I thought I’d tide you over with you a few cool videos I found from one of the exhibitors there.

Feeling Software got its start in 2005 building 3D technologies for video games and the entertainment industry. So, it’s no surprise that their offerings to the world of high-tech surveillance look a lot like a 3D shooter.





The demos are still a little rough around the edges, but in them, I think, one can see the beginnings of a new interface paradigm for surveillance monitoring that succeeds in providing both detail and context in one interface. It's huge potential advance over the brain-numbing eyeball-burning video walls that sit at the center of modern security rooms today.

And in addition to offering better situational awareness, the interface abstraction might be used to provide other features from privacy to integration, as well. 3D tracked models might be highlighted, tagged, or even obscured or removed from monitoring model based on security policies, privacy law, or real-time information from other systems.

One more note: For those interested in facial recognition, tracking, and modeling, you should also check out Feeling Software’s Face Flow demo. Though not quite up to the standard of the Benjamin Button tech I commented on the other week, it's pretty cool none-the-less.

How to Create a Digital Human in 18 Minutes

An detailed explanation of how state-of-the-art facial analysis technology transformed Brad Pitt in to an 87 year old man...then aged him backward.



Wow!

Now the Ads Watch You

From Fox4kc.com:

Watch an advertisement on a video screen in a mall, health club, or grocery store and there's a growing chance the ad is watching you too. Small cameras can now be embedded in the screen or hidden around it to track who looks at the screen and for how long. The makers say the software can determine the viewer's gender, approximate age range, and, in some cases, ethnicity, and can change the ads accordingly.

Tru-Media , the makers of the technology, argue that its next generation advertising systems don’t have privacy implications in that they only enable ads to become more targeted. Actual images and identity information are not transmitted back to their servers; only general counts and other statistics relating to ad viewership are retained.

Still, I think that elderly shoppers might not welcome the idea of ad monitors suddenly switching to commercials for Boniva, Depends, and Quaker Oatmeal as they walk by.

Korean ATMs Exploring Face Rec Options

According to JoongAng Daily, one of three English-language daily newspapers in South Korea, the Korean National Police Agency is currently looking into new facial recognition technologies as part of a comprehensive public security plan to reduce crimes in the country.

Bank machines have been involved in several notable crimes in South Korea, including the highly-publicized murders by confessed serial killer Kang Ho-sun earlier this year. In those murders, an ATM was used to withdraw money from the victims' bank accounts before they were kidnapped and killed.

In the JoongAng Daily article discussing new attempts to further secure the nation's ATMs, the author discusses concerns around the implementation of facial recognition technologies, including problems with complicated instructions, cost and the technical headaches often involved:

"But ATMs with such advanced features still come with a litany of problems. Facial recognition machines were once test-run by several banks in 2005. Users were confused by a string of instructions such as "Raise your head, please" or "Step up to the machine, please," before being able to access their accounts. The disabled had even more trouble."

All of these arguments point to an interesting issue, and ultimately the resounding truth that facial recognition should not be implemented solely in the manner criticized above. Instead, government agencies should focus on using face finding technologies to help police procure better evidence, while also implementing face searching technologies in order to help track down and identify suspects.

Using these technologies, alerts can still be generated if someone with a mask or disguise attempts to use an ATM, but that notice would go directly to a monitoring center for evaluation, rather than immediately freezing all activity on the machine. As we know, when you adversely affect a customer's experience too much they become upset, and when too many people become upset the offending technology must be removed -- in this case, the removal wouldn't help anyone, and thus the complete freezing of ATM machines should be uniformly avoided.

3VR has created its own technology to address this problem by combining face finding, face matching and face searching. It is this synthesis of capabilities that ultimately makes the technology effective and leads to the best, least intrusive and most efficient results for all users -- from the smallest businesses looking to manage access control for a single door to multifaceted government agencies and airports.

Protecting Citizens and Foiling Criminals: California DMV Looks to Biometrics Technology to Prevent Identity Fraud

Some consumer rights groups in California are up in arms around plans for the state Department of Motor Vehicles to use biometric and facial recognition technology to identify citizens and control identity fraud.

The California DMV has applied to use biometric technology to create a database that would contain facial and fingerprint information for all California citizens over the age of sixteen. The technology would be used to verify the identities of applicants for driver's licenses and ID cards and would additionally have the ability to compare new photos against older records, thus confirming the individual's identity and thwarting potential identity thieves.

The DMV claims the new system will reduce fraudulent driver's licenses -- pointing out that more than 1,200 identification cards are linked to the wrong individual each year. They are asking for roughly $63 million over the next five years to institute the system and begin creation of the database. States including New Mexico, Texas, Oregon and Georgia have already implemented biometrics technology and experienced success.

On the other hand, groups including the California American Civil Liberties Union (ACLU), the World Privacy Forum, and the Consumer Federation of California are protesting the proposal and calling on legislators to halt the contract's progress. Because the information contained in the DMV's database would be accessible to government agencies, some claim that it could be used for other purposes besides basic identity recognition. Privacy group leaders also worry about data falling into the wrong hands, and insist that safeguards must be put in place to ensure that the implications surrounding biometric data are fully discussed and understood by the public.

While the privacy concerns of innocent citizens are always valid and it certainly is necessary for people to be cognizant of data being held by government agencies, this case is in fact an excellent use of facial recognition technology. To apply for and receive a DMV identification card, citizens are required by prove their identities often with 2 or 3 other pieces of information. Therefore, the only people who have real privacy in this situation are the fraudsters, criminals and identity thieves who resort to using fake information and documents and never reveal their true identity in official records.

Ultimately, using facial recognition technology to prevent the acquisition of fake IDs in scenarios like this one helps everyone. While concerns that this type of biometric comparison may ultimately be used to track the general public in ways that genuinely impinge on privacy are legitimate, this is why sensible, balanced surveillance policy and technology is so important.

Image via Mercury News

Cheerleaders Demo Face Rec

From InfoSync:

"Face Recognition" is the next wave of visage-geared technology, and who better to demonstrate Panasonic's latest phenomenon than three Miami Dolphins cheerleaders? Though it was a nearly insurmountable task to remain focused, we were able to extract the general concept behind Face Recognition.

Somehow, I don't think Ariana has any trouble with people remembering her name.

iPhoto Offers Face Rec for Cats...Not Dogs

Via Maclife:

A huge disappointment for pet owners, was word that iPhoto's newest feature,
Faces, wouldn't recognize animals according to Apple employees on the show
floor. We even did a few tests at Macworld Expo with images they had of the dog
Luce. Well, we are happy to say that we were able to get iPhoto to recognize
Robbie's Cat, Lola.

Poncho will not be happy.

Faces: The New Fingerprints?

As we learned from the International Biometric Group's recent report, fingerprint scanning is now the biometrics option of choice for law enforcement agencies.

But one sheriff's department in Washington State has gotten a jumpstart in taking things a bit further-- the Pierce County Sheriff's Department was the first in the country to try out the MorphoFace Investigate system, a facial recognition system by Sagem Morpho, and their new technology recently made its inaugural criminal catch.

This past fall, the department caught on to an alleged bank fraud ring, and they were able to break up an organized effort to steal ATM cards and make fraudulent bank withdrawals by doing a database search around the ATM photograph. Two potential mugshot matches came up in less than 5 seconds, and it turned out the two matches were the same person, who happened to have a documented history of identify theft.

Police were able to move quickly, get a search warrant, and find the evidence they needed to make their arrest. Posthaste.

What's critical to note here is that facial recognition technology is adding a new layer to the biometric tools already in place. With this layer, we have access to new levels of speed and accuracy, and as the Pierce County forensic investigations manager says, the facial recognition system "eliminates 80 to 85 percent of the work in booking repeat offenders."

Maybe this year Santa will be making his list and checking it just once?

Security in 3D: Coming Soon to an Airport Near You?

Security engineers left and right have been attempting to crack the code on technology to match CCTV photos of faces with image archives in order to capture known criminals and thieves. Recent research out of Arizona State and Michigan State show that 3D laser scans could be eventually utilized to alleviate problems with lighting and photos angles in public areas with fast moving crowds.


Using a newly developed program, Dirk Colbry from ASU and his MSU colleague, George Stockman, performed 300 laser scans of 111 different faces using a commercial scanner to store images, while a horizontal plane of laser light passed over the subject's face. These images were then manipulated to create 3D models of each person's face.

The results were superb-- with the new system, different scans of the same face were matched even when lighting was unusual or the angle from which the images were taken was off by as much as 30 degrees. The error rate was a startling one percent.

This research demonstrates a new step in the advancement of surveillance technology, although its implementation will need to overcome the high price, slow scan speed and short-range sensors that may serve as roadblocks. The current scanner price of $50,000 needs to drop to about a tenth of that price to encourage widespread deployment. Additionally, scans currently take 2 to 5 seconds, making the technique obsolete in large crowds and only functional at choke points, such as airport and train station security checkpoints, where passengers are forced to wait in queues and in close range of scanning devices.

Being able to accurately identify someone who has paused, short-range, at a checkpoint and presented thier face, or iris, or fingers, or hands to the confidence levels demonstrated here has been possible for quite some time. But sometimes it's hard for me to get too excited about most of those approaches. That's because what we would all like to get out of any new biometric systems that get deployed at the airport are shorter lines and fewer checkpoints not more!

At 3VR, we're constantly looking for new algorithms to improve our facial recognition analysis features to combat the disparities in expression, lighting and angle. But to date, we have limited ourselves to writing algorithims for use in conjunction with conventional video cameras. That's because CCTV, and even new IP camers, are cheap, prolific, and offer something that laser scanning systems can't, the ability to work at a distance with uncooperative subjects.

Though current facial surveillance approaches fall somewhat short of 99%+ accuracy of 3D laser scanners, I can say that some of the techniques we are pioneering today hold strong promise of closing that gap. By processing streams of facial data from standard CCTV video feeds, it's possible to create an extremely accurate facial model; maybe one that will someday rival 3D scans. When will facial surveillance catch up? I can't say exactly. But, I do know that technologies like what we deploying at 3VR will get there long before a $50,000 laser scanner becomes as cheap as a video camera.

Face Rec Hacked! Needs "Liveness" Test

Very interesting article in CNet that highlighted an exciting new trend, but also pointed out that it may not be ready for prime time. Many new laptops, including new models from Lenovo, Asus and Toshiba, have started using facial recognition scans as the primary security mechanism for accessing their devices, rather than fingerprints or passwords. Definitely a cool use of new technologies, but as CNet points out, companies need to be sure they get it right before they introduce it to consumers, who would have no way to know their security was compromised.

In this test, security firm Vietnamese Internetwork Security Center (VISC) demonstrated vulnerabilities in laptops' face recognition-based authentication mechanisms that let anyone log in to a computer easily with a "special" photo of the legit owner, even at the highest authentication level. VISC was able to almost instantly produce a photo of CNet Editor Dong Ngo, taken over the laptop's webcam during a Skype chat, that fooled the computer's facial recognition software and successfully logged into a computer registered to Ngo.

Here's how Ngo described the offending photo:

About five minutes later, the technician produced a rather unflattering picture of me on a piece of letter-size paper. I could hardly agree that it was my mug on the photo. Nonetheless, when used in front of the laptop's camera, the Y430's authentication software was happy enough with the photo and logged in within a second. Pretty scary.

This type of hack is going to be very difficult for taditional facial recognition vendors to overcome. Early algorithms in this biometric field all focus exclusively on comparing one single image to another single image. Even if that image is being extracted from a laptop web camera. There is zero concept of context or "liveness" in this approach, and so it is easily spoofed. 3VR Security, is the only company I know of with a facial recognition platform built from the ground up to analyze streams of faces, like those in a video feed, rather than just single images. With this type of approach, subtle changes in motion, expression, pose, and other varialbles unique to a "live" 3D person can be analyzed at the same time a biomtric match is taking place and the kind of spoofing demonstrated here simply would not work. Maybe it's time for laptop vedors to upgrade their algorithms.

CyberExtruder Gets 2D to 3D Face Patent

At the end of November, CyberExtruder announced that the company had been granted a new patent on their process for creating reliable 3D models of a person’s face from a single or series of 2D images.




Though the enforceability of this patent has yet to be tested, CyberExtruder’s 2D-to-3D conversion is certainly an important innovation to the security and biometrics technology industries. Why? It enables better matching between offline photographs and surveillance video. While the matching of "watch list" images against surveillance video has traditionally achieved mixed results as a result of inadequate lighting, angle, expression, etc., this patent could signify a leap forward in terms of the quality and value of 3D facial images.

In June of 2007, I wrote on how XID was using a similar technology in the “world’s largest” facial recognition access control project. In that instance, XID literally generated hundreds of thousands of variants of an enrolled employee’s face rather than using just a single 2D photo converted to a 3D model. Each day when an employee arrived for work, his or her photo would be taken and compared to the database of generated images rather than a single original. Interestingly, this approach generated huge improvements in the performance and accuracy of the Thailand access control system.



We’ll see if these two companies come into conflict over the new patent, but I don’t think they will. XID’s approach to 2D-to-3D is very different than CyberExtruder's-- almost quick and dirty by comparison. CyberExtruder, on the other hand, has become famous for its hyper-realistic…if sometimes creepy…generated floating 3D heads that lend themselves to applications well beyond security including gaming and movies, and even boast a fan in Phillip Rosedale of SecondLife.

Face Swapping 2.0 - Now Mostly Prettier

In July, I made a post regarding new technology from Columbia University that swapped faces in photos and videos with generic amalgems drived from a photo database. The technique, which had potential privacy benefits...and even certain Hollywood applications...suffered from one major drawback. The modified faces were really really really ugly.

See below (From Columbia University Research Paper):



Tough to stomach, I know, but these results were bound to be approved someday, and thankfully that day is here. The New York Times is reporting that computer scientists in Israel have taken a more beauty-centric approach to face swapping that simply replaces faces in images and video with more idealized representations of the same faces. Their software applies an algorithm covering over 200 different facial measurements that brings a target face closer to the ideal. In most cases, this approach goes well beyond simple airbrushing, dramatically altering a person's appearance and perceived identity.

See below (From Lars Klove for the NYT):



I am not sure either technology is perfectly suitable for privacy protection in video surveillance just yet, but given the choice between the two approaches above, for aesthetic reasons, I'd go with the latter.

2 Years Post Neven Deal...Google Launches Face Rec

Via TechCrunch:

In the anticipated release of Google’s new and improved Picasa, the company will offer facial recognition technology to help you identify friends and family in your pictures without requiring you to tag them by-hand each time you see them.

Launching at noon PDT today, Picasa’s facial recognition technology will ask you to identify people in your pictures that you haven’t tagged yet. Once you do and start uploading more pictures, Picasa starts suggesting tags for people based on the similarity between their face in the picture and the tags you already put in place for them.

The facial recognition technology comes to Picasa thanks to an acquisition Google made in 2006 of Neven Vision, a company that specialized in matching facial detail with images already found in a centralized database. Picasa’a facial recognition technology works in much the same way.

More from the Folks at Image Metrics

This video is impressive...and a bit creepy:

Evansville Airport Gets Face Rec

From the Evansville Courier & Press:

Security is the paramount issue among air travelers these days, but Evansville Regional Airport — like other commercial airports throughout the world — rely on videocassette recorder security systems whose nonspecific tapes can take hours, days and sometimes weeks to decipher when suspicious concerns arise.

That, however, is about to change at the local airport.

The Evansville airport is the first in the nation to get a new technological security system, known as a 3VR (Third-Generation Video Recorder), said Bob Working, the airport manager. The searchable surveillance system uses a Google-like analytical search engine for spewing out valuable information instantly. For example, it displays in an instant on a computer monitor clear images of thousands of faces for identification purposes.

Eric Moss, vice president/director of data services for Gaither Technologies/STC, demonstrates a 3VR security system as company President Steve Rudolph is recorded on video, in the screen at left. The unit features searchable facial recognition and motion-changing events.

Face Swapping Still Not Pretty

Not since John Travolta and Nicolas Cage swapped mugs in the 1997 non-hit Face/Off, have I been so intrigued/disturbed by the prospect of changing faces. Researchers at Columbia University have developed a system capable of quickly replacing and/or obscuring faces in photographs with new composite or synthetically generated images. Unlike our technology at 3VR and that at Google, which only blurs faces, this technology actually changes them...with mixed results.

Here is an example:

The authors see several applications to this technology from protecting privacy in public images to Hollywood special effects. But, as you can see above, the researchers' favorite application seems to focus on making famous faces less attractive.

Penguins Under Surveillance

Yet another new and creative use of surveillance and biometric technologies. At the University of Bristol, scientists have designed an unobtrusive surveillance system that can be subtly integrated into the environment of various endangered species. The systems will help scientists observe and track endangered animals, starting with the African penguin, of which there are only 170,000 remaining. Though the project was initially designed with penguins in mind (it's called the Penguin Recognition Project), the scientists are confident it could have wide-ranging benefits for the conservation of other species.

Biometrics Are Breaking Through

We're starting to see some pretty powerful and simple uses of biometrics in the security industry. In this case, government agencies were able to access fingerprint info through a shared database, which led to keeping 365 suspected terrorists out of the country. Now granted, there has been a LOT revealed about what constitutes a terrorist suspect these days, but my point here is on the technology. It works, it's easily shared, and it's effective at connecting the dots. Of course, I believe that facial biometrics hold even more promise as mug shots and surveillance images are often much more readily available than finger print dossiers. 3VR CrimeDex is the best example of next generation biometric infrastructure that makes this kind of sharing possible. Mug shots are uploaded to a shared database that all members can access; plus, if a member has a 3VR system, they can even set alerts associated with a CrimeDex mugshot. So far the system has been used to catch hundreds of fraudsters at banks; we're pretty confident it will have similar success in other realms.

Face-Rec Continues to Go Mainstream - Can DVRs Keep Up?

Five years ago no one had ever heard of facial surveillance, or if they had it was mostly in a negative light. Now it’s a must-have feature and one of many must have new video analysis algorithms. The jury is still out, however, on whether DVR and security companies in general can adapt to this quickly evolving software-dominated surveillance market.

Face Rec Speeds Airport Lines

Adding to the existent iris and fingerprint "trusted traveler" enrollment scheme, now face rec is being added to the package at U.S. and U.K. airports to both improve security and speed frequent international travelers through security lanes. The British and American governments have launched a partnership to allow enrolled "trusted travelers" who fly frequently between their respective countries, to bypass security lines after a quick face scan. The "new" program basically just adds face rec to the existing iris enrollment program for trusted travelers, which to me says that 1)face rec is working a helluva lot better than it did ten years ago and 2) iris and fingerprint recognition might not be working well enough if officials feel the need to add face rec technology to the existing program.

Hacking Face Rec

This is why we are constantly evolving our technology and developing new video algorithms. Granted, some people viewing this will follow the creator's advice, "Don't go out and rob a bank!" but there are probably a few who will attempt exactly that.



Invisible Mask - video powered by Metacafe

Face Expression Rec Gets Us One Step Closer to Robot Teachers

Here's an interesting new use of face rec technology. Jacob Whitehill, a PhD student at UC San Diego is essentially turning faces into remote controls in the hopes that some day robot teachers will automatically assess when students are puzzled or confused and slow the class down accordingly. I know what you're thinking and yes, it's true, this could help build the perfect robot girlfriend some day, too.

Race Rec Face Rec

In most research and law enforcement circles, the prospect of using advanced facial recognition technology to determine an individual’s sex or race is an understandable gray area. Though census takers, demographers, and police reports have always concerned themselves with these details, the idea that a computer might reliably judge the racial background of each and every city passerby is more than a little creepy to some and raises the specter of racial profiling.

Recently Benjamin Wales, a graduating student at the Royale College of Art, set out to test peoples sensitivities on the subject by deploying his own "race detection" camera apperatuses [Spy Blog via Rajiv Shah] on the streets of London. And though he published no scientific data about the accuracy of his systems, it was clear that many who came across his art project shared Mr. Wales' mixed feelings on the technology.

Had scientific results actually been published in this project, however, I doubt they would have been terribly good. It turns out that this kind of image classification is awfully difficult to accomplish. To understand why, take a look at this research project at Mitsubishi Electric Research Laboratories (MERL).




In this research project, MERL succeeds in getting what you might consider "pretty good" results using two different image classification routines, Male vs. Female & Asian vs. Non-Asian, on excellent video footage of various faces. On thier own, these overly simple binary classifiers work well enough to justify further research, not well enough for any real world real-time law enforcement or profiling function.

Further, if MERL had added additional race classification outcomes, like black, or Korean, or Latino, etc., the level of accuracy they might be expected to achieve would decline considerably. And if they added enough racial classifications to approximate the actual diversity found in major cities, the algorithm would likely cease to provide any meaningful data at all.

One area where racial classification does show some promise, however, is in the the area of video search. The day is not very far away when a police official might be able to query a city surveillance system for an "Asian women with a red purse" when attempting to track down a suspected kidnaper and her victim. There would be a number of false matches, of course, and a human might still need to review a lot of video to ultimately close in on their intended suspect, but the search would be faster, more focused and sweep up fewer innocent bystanders if some face/person search algorithm was used.

I think this is a more realistic and reasonable use case for face classification...and one that might actually do some good, as well.

Face Rec Just Went Mobile!

Sorry I haven't posted for a few days, we've been busy at 3VR releasing our M-Series Rugged, a.k.a. the 3VR Mini, a portable, compact video management system with built-in advanced face recognition and motion analytics. As you can probably tell, we're really excited about it. It's the first system of its kind to support high-volume facial recogntion AND license plate recognition AND other object-based video analytics in a super-small, super-rugged form factor.

The M-Series is small enough to be deployed at ATMs and on buses and light rail, all three of which are areas where video surveillance is sorely needed. The new anti-shock, anti-vibration and fanless design of the M-Series, as well as its ability to withstand high temperatures (up to 113 degrees Fahrenheit) are design adjustments we made to make it more suitable for transit deployments. Other than making it smaller and tougher, everything else is the same - optimized storage, cross-analytic searching in seconds, case management, watch lists, a link to Crime Dex, it's all there.


Here are some some more pics:




Stay tuned for a live unboxing early next week!

Surveillance and Mapping Software Together at Last

It seems like a lot of cities are announcing large scale facial recognition systems these days, which will definitely require an increase in data storage capabilities to store more and better footage. What I find more interesting than the storage angle in this piece, is mention of combining Microsoft’s Virtual Earth (which could just as easily be Google Earth, for that matter) with surveillance video--presumably this will/could be used to map in real time the location of bad guys and dispatch law enforcement as appropriate. It’s an interesting approach and no doubt going to be much more useful than the giant video walls normally deployed.

The Costs and Benefits of Face Recognition

Most people think face recognition is great when it's catching baddies and weird or invasive when it's tracking innocent people. The technology has sparked debate after debate over whether privacy or security is more important. Honestly? I think we can have both.

First off, people need to understand what facial recognition can and can't do. This ABC story about a face rec system analyzing someone's face and telling a clerk that they're underage is bogus. If the kid had been in the store previously and been busted for buying liquor then yes, the system could alert the clerk, but no one is suggesting that face rec can tell the difference between 17 and 18 any more than a human can.

One thing face rec can do, which hasn't gotten a ton of press, is include simple privacy measures. We have been working on this at 3VR and I wrote recently about a team in Canada that's working on a similar project. Basically, software engineers can write a password-protected program that blurs faces and when an incident occurs, an investigator can unblur faces in particular pieces of video. This way, while people may still feel uncomfortable about being on camera, at least they will not really be watched unless they happen to be present during a robbery or some other incident, in which case they'll typically be glad the cameras were there to help catch the bad guys. Also, because this type of application can also have auditing capabilities written into it, it provides a crucial and often overlooked capability: a way to "watch the watchers," if you will.

The same could feasibly work in the grocery store situation described in the ABC story - if someone was a match with a suspect in the database, then the system could alert the clerk. For everyone else, faces could be blurred, and if someone is caught buying liquor or cigarettes underage, then the store manager could unblur the face and save it to the suspect list.

There has also been a lot of press recently about the rise of surveillance that risks privacy without actually improving security. I agree. Thing is, the bulk of new cameras installed are meant to catch traffic violators and raise money for municipal governments, not improve security. These cameras misfire fairly often, sometimes costing a city more than they're worth, and invading citizens' privacy for no good reason.

During the bombings in London, however, investigators were able to use video footage to find their suspect. If they had had face recognition and video search capabilities, that investigation would have been far shorter. And as banks have begun installing surveillance systems, they have seen a marked increase in the number of fraud cases they're able to solve.

Surveillance clearly has a place in modern society, but I do think that the industry needs to continue to work towards securing both people and their privacy.

UK Tests Face Rec at Airports

The UK government is set to be the latest to test facial recognition technology on a large scale. British border officials plan to test a new facial recognition system at airports this summer to see if it can boost security and reduce overcrowding.

American and French airports were the first to test face rec and both are now testing second generation technologies. Similarly the London Olympics committee and the Nashville School District are both testing face recognition software, but the most ambitious project to date seems to be the nationwide deployment of face recognition in South Korea.

Privacy-Enhancing Video Technology

It's great to see people trying to address the privacy vs. security question with new solutions instead of complaints on either side. Today, video surveillance is not particularly private nor effective, and we really need both. A new technology developed by researchers at the University of Toronto adds a sort of face blur filter to security video that can be reversed with a decryption key. So, the faces of the innocent remain fuzzy and out-of-focus, while investigators zero in on suspects.

The University of Toronto tech was most likely at least partially motivated by changes in Canadian privacy laws. Even without such laws, technology like this is important and certainly helps to address the privacy issue. At 3VR we are currently working on our own privacy enhancing technologies that also incorporate advanced facial recognition techniques. The technology helps to hide the faces of the innocent while at the same time identifying bad guys, providing a solution that gives equal weight to privacy and security, which is something I think should be at the center of any city surveillance deployment.

The Science of Face Generation

Advancements in facial recognition are not the only interesting face-related technologies making some astounding leaps these days. Driven by demand out of Hollywood, start-ups like Image Metrics are now taking the first steps to generate near perfect artificial faces that can be animated in real-time. The technology gives actors the ability to effortlessly bring digital characters to life with astounding realism; transferring every muscle movement and twitch to their digital puppet-selves.

For a look at what’s possible, check out this demo reel:



Hollywood and the gaming industry won’t have a monopoly on this technology though. I fear that these same tools might be used to undermine the evidentiary value of traditional surveillance video and even perpetrate new kinds of identity theft. When faces and bodies can be swapped in an out of video clips with this level of ease, and interactions that never existed in the real world can be recorded to video using only a laptop, it will be come even more critical that law enforcement find ways to separate face-fact from face-fiction.

'Biometrics' Used to Identify Terrorists

Computer scientists and engineers are working with intelligence agencies on an international collaborative database that will use biometrics or unique individual facial, voice and physical "signatures", to identify and trace terrorists and persons of interest.

The combination of a facial expression, a spoken word, or a fingerprint could be enough to stop a terrorist attack.

While tenable results for accuracy, speed and timing are five and 10 years out, the goal is to combine biometrics with artificial intelligence to prevent terrorist attacks—an ambitious, expensive, but worthwhile endeavor.

Next Gen Banking

http://blogs.zdnet.com/BTL/?p=7641

As banks build more branches to take advantage of customer demand for in-person experiences, they are looking to create more value by adding a variety of technologies designed for customer identification, screening, data collection and sales support.

As you enter the branch, a system identifies you by scanning your RFID loyalty card and then displays a personalized welcome message on the wall. At the same time, a sales representative is informed of your arrival and supplied with a quick summary of your financial situation. If you’re judged to be worth the effort, the sales rep will approach, shake hands, and gently inform you that if you maintain your current savings rate, you’ll be spending your Golden Years in a refrigerator carton.

RFID? Really? I am going to put my money on face recognition, possibly iris recognition, as the technologies that drive these kinds of changes.

The New Face of Photography

Stephen Shankland/CNET Networks is reporting on increasingly successful attempts to make cutting edge technologies like “face tracking” and “facial recognition” a common…and very useful…tool in amateur photography. While cameras that can adjust to keep faces clear and in focus have been around for a while, the conventional wisdom has been that actual face recognition is simply too computationally expensive to do at any scale on diminutive platforms like digital cameras.

Face recognition requires computational horsepower that is hard to fit into the confines of a digital camera, but one company likely to help make it a reality is Fotonation, which already supplies face-detection software for dozens of camera models from Samsung, Pentax, and others. The computational challenge is reduced by the fact that most folks tend to photograph the same set of 25 or 30 people, Eric Zarakov, Fotonation's vice president of marketing, said in an interview here at the 6sight digital imaging conference. A camera could be "trained" to recognize just those particular people.

I am not sure the technical constraint described here is still genuine, however. For instance, even though we have no immediate plans to do so, the facial recognition technology that we’ve built at 3VR could be adapted to provide real-time recognition of thousands of individuals...even on devices as underpowered as consumer digital cameras. Amateur photography might benefit from the same approaches.

Shoplifting on Steroids

http://www.azcentral.com/arizonarepublic/business/articles/1202biz-ev-secruity1203.html

Shoplifting has always been bad news. It’s also BIG business. According to Jason Beckerman, a Target Investigator, if you added up burglaries, auto theft, and indeed every kind of theft outside of shoplifting, you’d still have a problem less than half as large as shoplifting itself. In the Arizona Republic he is quoted as saying, “The value of property lost in […] excluding shoplifting was $18 billion in 2006. Shoplifting losses totaled more than $40 billion that year.” Part of the problem more recently has been professionalization of retail theft. Shoplifting isn’t so much about teen-agers sneaking a pair of designer jeans past a checkout counter anymore. Instead, well organized gangs of criminals steal huge quantities of popular goods from stores and warehouses. Items targeted range from iPods, to Visine, to baby formula, with values per truckload often reaching into the millions.

And because the Internet offers so many ready-made outlets, like eBay, to quickly sell stolen merchandise, cashing in on stolen goods is easier than ever. It’s enough to make one long for the good old days when crooks could be found selling speakers out of the trunk of a car for 10 cents on the dollar.

Smarter and more organized criminals, along with this so-called “fast food fencing,” have lead retailers to adopt higher prices and new approaches to fighting crime. They’ve been forced to become smarter and more organized themselves, building in-house investigative teams armed with technology straight out of C.S.I. For example:

This fiscal year, Target has closed 30 investigations, made 60 arrests and is
working about 16 cases in Arizona and New Mexico. Target has three investigators
in Arizona to hunt down the biggest offenders here and in New Mexico. The chain
also operates two forensic labs in the United States, including one in Las
Vegas.

Facial recognition systems and other advanced surveillance technologies are also being deployed to investigate, and more importantly prevent, organized retail theft. Today, databases of known criminals and gang members can be shared across retail institutions and tied directly into participating stores’ camera networks. When a “bad guy” is spotted on camera, security personal are instantly alerted. And as importantly, the advanced search engines can be used instantly sift through company surveillance archives for any and all related evidence to a suspect.

Shoplifters may be getting smarter and more organized, but they aren’t the only ones. And in an arms race between the bad guys and the Targets, Walgreens, and Wal-marts of the world… I think the smart money is on Wal-mart.


Target investigator Jason Beckerman (left) and Dan Helmick, a security leader for Target, watch monitors in a security office at a store in Mesa.

Nashville Schools Face Rec System (Update)

http://www.wkrn.com/nashville/news/metro-schools-unveil-face-recognition-cameras/131643.htm

From WKRN:

Tuesday, Metro Nashville Schools unveiled its new security system, aimed at keeping unwanted people out of the school buildings. As a test, the face-recognition cameras will be installed in three Metro schools. The cameras will work 24 hours a day, seven days a week, taking snapshots of every individual, looking for people who may not be welcomed. Ralph Thompson, Metro Schools’ Assistant Superintendent, said, “For instance, everyone that entered the building is now in the data bank and all we have to do is go back and store that image and put it on either our watch list or an approved list to come into the building.”

Danger Will Robinson

The latest in a seemingly endless line of ever creepier face recognizing robots.

3VR on Fox News

Tim Ross, EVP and Co-founder of 3VR, was interviewed today on Fox News:

You Look Too Young to Smoke

A Japanese vending machine company is using facial recognition to verify the age of would-be smokers. Creators of the device claim 90% accuracy in separating children and adults. In the case the machine can't make a clear determination, it can also request picture identification as backup. I wonder if children will try to trick the machine and look older by wearing makeup, wigs, or putting a cigarette in their mouth.

Nashville Schools Get Security Facelift

The 75,000-student Nashville School District announced that they will be deploying face recognition systems at 3 different facilities this December. It is hoped that the specialized cameras and software will help protect the safety of students, as well as, generally monitor access of the traditionally open educational settings settings.

"This will give us an edge in providing safety for our students and teachers," Thompson said of the $30,000 camera system. Several intruders have entered
Nashville schools in the past year, he said. A successful test in Nashville could prod other schools to try the technology, said Peter Pochowski, executive director of the National Association of School Safety and Law Enforcement Officers. He said Nashville is the first to use face-recognition cameras. Nashville will take digital photos of students and workers at the three test schools and store them in the new camera system, Thompson said. When a camera spots a face in a school that it cannot match to a stored photo, it will alert security. The system also could detect suspended and expelled students and fired employees, Thompson said.

In 2004, the Phoenix School District made a similar announcement, but ultimately never deployed their facial recognition systems over accuracy and privacy concerns. Of course, significant advancements in both facial recognition and privacy technologies have been made in recent years. Here is one example that I am partial to.

For Those Who Like Their Robots Creepy

Engineer Max Denton has fabricated something that is sure be a big hit this holiday season. It's a six-legged face tracking robot he calls the 'Hexapod.'



It's reassuring to know that before they enslave us, our robot masters will at least be able to look us in the eyes.

Real Life James Bond

If you are into spy gadgets and other 007 goodness, look no further than the "CIA's Venture Fund" In-Q-Tel. The name is actually derived from the combination of the word "intel" and the character "Q" from the Bond books.

Since In-Q-Tel was founded in 1999, the firm has reviewed more than 6,300
business plans for everything from identity recognition software to nano-sized
electronic circuits. Many proposals come in via its Web site. In-Q-Tel has put
about $200 million into more than 100 companies, beating traditional VC
investors to technologies such as the mapping software that's become Google
Earth.

One of their investments is a particular favorite of mine.

Another part of In-Q-Tel's video strategy is the 3VR investment with Kleiner Perkins. After the London subway bombings on July 7, 2005, about 1,000 British police spent six weeks sifting through video from 6,000 surveillance cameras, says Russell, who founded the company in his basement in 2003....With 3VR, every time someone passes in front of a camera, the software assigns an identification number and establishes a profile based on the geometry of the person's face. When the face is captured from a different angle or in different light, the software creates another mathematical model. The system can be programmed to recognize faces and to issue alerts once a suspect is spotted.

Bad guys beware.

Bringing Down the House...No More

These days card-counters have more to worry about than just the the odds.

No one realizes it at this casino, but Aponte is a veteran of the card-counting
team of math whizzes from the Massachusetts Institute of Technology. The team reportedly took more than $10 million from casinos in its heyday from 1994 to 2000. Aponte has been barred from more than 100 casinos in the United States and a few overseas. In St. Kitts, he said, he was recognized by a Biometrica database, and now he avoids the biggest, most modern casinos. The team's No. 1 downfall, he said, was information sharing. Once the members' faces began showing up in databases, their days were numbered. (via The Washington Post)

Card-counting isn't a real criminal problem, as being exceptionally good at math isn’t exactly stealing. Still, it’s interesting to see how effective this technology can be when combined with industry-wide information sharing.

Search Engines Using Facial Recognition

Search engines adopting face recognition now include Google, Microsoft Live, Exalead.
Blog, Search Engines are getting better as everyday passes by, and we see new features added to give users the best experience and satisfaction ever. Face Recognition is a technology that has been adapted by search engines like Google, Live Search and Exalead in a move to tune up their image search results. Now you could restrict image searches to only human faces, rather than getting more of irrelevant images. (Via Nitesh Gautam’s Blog)

Reuters launches facial video search web site using Viewdle. Not bad.