"As of right now there is no arrest and no anticipated arrest"

Here's a tragic story hitting headlines everywhere -- however, no stories I've found have seriously discussed the surveillance technology at play nor the length of time this investigation has taken.

Late last week, a Yale pharmacology grad student's body was found hidden in a wall in the basement of a Yale medical research building after she had been missing for almost a week.

However, due to the access control restrictions on the building, investigating officers believe this not to be a random act, but rather one committed by someone in the Yale community. Yale University President Richard Levin was quoted on Monday as saying, "We know everyone that was in the basement. There were limited number of people in the basement and we passed that on to police. There is an abundance of evidence."

And as CNN has reported, security cameras registered Le entering the building, but after searching hours of surveillance tapes, had been unable to find images of her leaving the building. The NY Daily News even reported that more than 100 FBI investigators and three police departments spent over six days pouring through building blueprints and surveillance footage -- and even used bloodhounds to search the building. Six days is a long time.

What does this tragic event teach us? While we await the murder details (expected to be revealed today), the value of using analytics and more sophisticated surveillance tools to search and comb through footage may have reduced the time needed to come to the conclusions we reached in seven days to maybe only a couple of hours. While even the tightest access control restrictions and clearest surveillance cameras cannot prevent a human from taking another's life, technology has the ability to hasten investigations and also equips security personnel with the eyes and ears needed when the human equivalent is not an option.

Using a variety of facial recognition, color tracking and other analytics, we may have been able to identify the student upon entering the building and followed her whenever she appeared on camera. We also could have also identified each person's face that entered and exited the building that evening, as well as tracked articles of clothing by color. We might not have an answer for the cause of such a brutal and senseless attack, but we do know that more stringent access control and surveillance technology may have helped in the investigation process.

Face.com Launches New Photo Tagger

(Photo via ReadWriteWeb)

First Face.com brought us Photo Finder, which started the process of scanning through the 15 billion pictures on Facebook (9,000 images per minute!), and now ReadWriteWeb reports that the company is using the same facial recognition algorithm – the "hybrid descriptor-based funneled" model – to launch Photo Tagger. While very similar in nature to the original Photo Finder, Photo Tagger "...scans through select online albums to automate the tagging process" to speed up the monotonous process of tagging albums. It doesn't matter if the album is yours or a friend's – Photo Tagger will take care of the entire process and save you endless hours of manual tagging.

While I'm not sure how Photo Finder and Photo Tagger differ exactly – of course I have a few ideas – with more and more facial recognition technologies finding their ways to online photo services, I do know it will be tougher and tougher to hide your face online. Bad lighting, odd angling and unfocused photographs look to be no match for Face.com (and the plethora of other photo applications).

3VR Featured in New Scientist!

This just in! Check out 3VR's image-scrambling technology in the most recent issue of New Scientist out this week.

I spoke with Paul Marks, New Scientist's chief technology correspondent, a little while back around our new method of scrambling CCTV images to preserve the privacy of innocent persons, and he discusses this technology in the issue out this week. Just another example of how we're bringing structure to the world of surveillance to eliminate the 'all-or-nothing' nature of such data collection.

Check out a longer post about the piece on 3VR's blog here.

I Can't Help It -- More iPhone MacRumors

MacRumors is back at it. Today, they're reporting additional patent filings around object recognition and facial detection extensions, continuing to push forward some of these technologies that I've posted about recently. While these take time to come to fruition, I can't help being excited -- the possible adaptations of these ideas are endless. Let's take a look.

The object recognition capability in which an iPhone would be able to "detect an object via camera, RFID sensor or other means and have their device automatically identify and provide additional information on the object" looks to be potentially quite useful. In the patent background, Apple used the example of an art museum:

"...a user might take a photo of a piece of art and wish to have it automatically identified and additional information on it provided, or engage in an audio tour or podcast and wish to access additional content beyond that provided in the audio files."

Think of all the possible ways to leverage this technology -- other than trying to distinguish between two pieces of art, maybe you can use it to uncover the name of that actress in the recent blockbuster movie that you can't recall but swear looks familiar. Or perhaps you will be able to identify the name of a certain wallpaper color swatch -- and be able to access its brand, serial number and all retail locations withing a five mile radius. Well, both of those might be a bit far off -- not sure the iPhone camera can yet detect the subtle difference between eggshell and off-white or has facial recognition capabilities on par with those of 3VR, but you catch my drift. ZDNet also noted using the technology for price comparisons between retail products.

As far as facial detection developments, it seems that iPhone engineers are indirectly attempting to remedy the device's often woefully poor battery life. New patents look to "determine whether a user is passively interacting with the device" -- meaning not watching the TV show they've downloaded or listening to a song on iTunes -- and if so, turn on a screensaver of some type (similar to the setting on a normal laptop or desktop) to save energy. The iPhone would use its internal camera to detect a user's presence, and while the idea may be antiquated, bringing this technology to a mobile device will be welcomed with open arms.

ePassports get Floating Heads

Here is the first video that I have seen of Samsung's new ePassport prototype. It features a small back-lit screen as part of the ID card that, in this case, can be seen displaying a semi-creepy floating head. Encoding of this kind or "3D" or multi-image biometric information would go a long way to eliminate problems like those recently seen at Manchester airport.



I want one.

Sitting Down with Rob Jenkins

I recently connected with Rob Jenkins, a lecturer at the University at Glasgow and leading authority of facial recognition technology in the UK, to pick his brain about different topics in facial recognition, namely airport security, the future of biometric technology and privacy/related concerns.

Rob had some very insightful, innovative answers to my questions, and I'm excited to share them with the growing IHF readership. Going forward, I'm hoping to have other thought leaders and readers contribute content and commentary to this blog, as I'd like to make this more of a forum for biometric, facial recognition and other technology discussions rather than a one-sided conversation. Feel free to make comments on any of the responses or questions, and I will be sure to address them!

Also, check out Rob's departmental Web site for selected publications on gaze perception and other facial identification topics. Very interesting stuff.

In response to Manchester Airport lowering their matching thresholds, The Telegraph quoted you saying that lowering the passport match level to 30 percent would make the system almost worthless. Another perspective is that the previous levels were causing horrendous queues and customer dissatisfaction. Is there a middle ground here?

There is certainly a middle ground in the sense that we can choose where to strike a balance between rejecting genuine matches and accepting false matches. But reducing either type of error generally increases the other, so it’s a trade-off. There is no ‘sweet spot’ where both types of error are reined in.

Despite the advanced nature of this technology, do you believe that there should still be a human element involved in security checks? If so, do you believe we will ever reach a point where this will no longer be necessary?

The main problem with referring the difficult cases to humans is that humans cannot do the task reliably either - even if we’re trained and experienced. Humans are fantastic at matching familiar faces, but our performance with unfamiliar faces is very poor. If we can somehow incorporate the benefits of familiarity into the technology, then it could be transformed.

Facial recognition technologies are popping up all over -- club entrances, bathroom faucets, online photo services, using cameras in lieu of passwords to access computers -- have they hit the tipping point? Is it only time before we use the technology to unlock our front doors and open our car trunks? What trajectory do you see it taking? Staying in security-based deployments, infiltrating everyday life or a balance between the two?

To some extent I think a tipping point is being ushered in, mostly by people who have something to sell. And it is an idea that some sectors are keen to buy into. So in that sense there is a lot of good will wishing the technology to work. I don’t find the gadget market especially troubling, provided that errors are of relatively little consequence. The real danger is in rushing to large-scale security deployments. For applications such as passport control or forensic face recognition the stakes can be much higher, and we know that the available technology is not yet up to the task.

In the same vein, has facial recognition reached a point where accuracy and reliability now line up with the media's expectations?

In my experience, identification errors tend not to go down well with the public. I often ask audiences how often they would be prepared to be the subject of a misidentification. The answers are in the order of once a decade, even when the imagined consequences are minor. That’s a tall order, given the number of identity checks that some proposals entail. It comes as something of a shock when these demands are compared against current capability. As far as media expectations are concerned, I think there has been a change in tone. Traditionally, the emphasis has been on the implications face recognition for privacy, with the unspoken assumption that it is reliable. These days there is more of an awareness that the technology simply is being phased in, whether it works or not. That changes the focus of the debate.

The "Big Brother" argument -- that citizens are losing their individual privacy rights due to increased public security efforts -- is always present in a discussion about surveillance. Is there a point at which facial recognition and biometric technology infringe on personal freedoms and the right to privacy? Is blurring faces enough? Are there places where surveillance should not be allowed?

I don’t think facial recognition and biometric technology necessarily infringe on privacy. It is certainly possible to imagine applications where privacy concerns don’t arise. However, for the security and surveillance applications that have been at the forefront of public discussion, the tension with privacy is fundamental. The whole purpose of identifying someone is to connect them with some other information, and the nature of that information is a major issue. We can think of face recognition as a key to identity. But focusing on the key tends to distract us from other questions, like What’s behind the lock? As more and more information is stored behind the lock, the reliability of the key becomes increasingly important. As does the question of who has access to the key.

The practice of blurring or pixellating faces to protect identity (as in Google Streetview) is often poorly informed. Although such manipulations can make it more difficult for observers to identify people, this is only the case when the observer is unfamiliar with the faces concerned. When the observer is familiar with the face, blurring or pixellating the image does surprisingly little to impede identification.

People have very different ideas about where surveillance should be allowed, and which places should be out of bounds. I don’t really foresee any wide agreement on the extent of coverage that is desirable or acceptable. The general trend is for rapid expansion, especially in the US and the UK, but my impression is that this trend is not driven by public demand.

The UK has over 4 million cameras -- that's one for every 14 people in the country and 200,000 in London alone. Chicago is working to improve its 'Virtual Shield' and include the entire metropolitan area in its surveillance grid to cut down crime. Yet, criminals still often get away with murder -- literally. Are expectations set too high? Are surveillance grids more of a scare tactic in preventing crime from happening rather than proactive in catching criminals in the act?

It has been known for some time that the unprecedented CCTV coverage in the UK has had little or no effect on crime rates. A recent Home Office report revealed that only 3% of crimes were solved using CCTV footage, and suggests that simple improvements to street lighting would be more effective. Part of the problem is that it is unrealistic for police to monitor CCTV footage on the scale that it is produced. But more importantly, little thought has gone into the use of CCTV evidence in court. It has only recently become clear how poor humans are at matching unfamiliar faces, even when the images are far higher quality than could be obtained from CCTV. We’ve already looked at machine performance in this context. Establishing a match that will stand up in court is very difficult indeed.

The deterrent argument is interesting because the figures imply little or no deterrent value in CCTV. The standard explanation for this is that people assume the cameras are not working, which is a reasonable inference to make if they are not reducing crime. However, I wonder if there is also a paradoxical effect of increasing coverage. After all, the more cameras there are, the less likely it is that any particular camera is being monitored.

For the Music Lovers in All of Us

Looks like Sony Ericsson is becoming psychic for the on-the-go music fan -- the company recently announced a patent that enables mobile music applications to not only be controlled by facial recognition according to 'visual mood analysis', but also create playlists based on these readings.

As a recent SlashGear post explains, "The system could either recognize moods by spotting particular facial characteristics or by comparing the whole face against presets; alternatively, a hybrid of the two might be applied."

If I stub my toe, my phone is ready to commiserate and play some Sarah MacLachlan. If I'm ready to celebrate the great weather expected in San Francisco next week and the camera catches my grinning face, perhaps U2's "Beautiful Day" will pop on. I like these possibilities.

Not sure that will tear me away from my love for the iPhone (or my excitement for its upcoming rebirth), but awesome technology nonetheless.

Daily Headers: May 22, 2009

Report: CCTV Schemes in City and Town Centers Have Little Effect on Crime
The Guardian
Alan Travis

• The use of closed-circuit television in cities, town centers and public housing estates does not have a significant effect on crime, according to Home Office-funded research to be distributed to all police forces in England and Wales this summer.
  
• The review of 44 research studies on CCTV schemes by the Campbell Collaboration found that they do have a modest impact on crime overall, but are at their most effective in cutting vehicle crime in car parks, especially when used alongside improved lighting and the introduction of security guards.
  
• The Campbell Collaboration report says that CCTV is now the single most heavily-funded crime prevention measure operating outside the criminal justice system and its rapid growth has come with a huge price tag. It adds that £170m was spent on CCTV schemes in town and city centerse, car parks and residential areas between 1999 and 2001 alone. "Over the last decade, CCTV accounted for more than three-quarters of total spending on crime prevention by the British Home Office," the report says.
  
• Am going to look into this one a bit further. Definitely some additional factors involved in the effectiveness of CCTV grids, including camera placement and image quality. Interested to dig a bit deeper and read the actual report. I'll report back.

Swiss Narrowly Accept Biometric Passport
Associated Press
Eliane Engeler

• By a narrow margin, Swiss voters accepted an overhaul of the country's passport system to include travel documents equipped with biometric data -- a change needed for Switzerland to stay on the United States' visa waiver program.
  
• The biometric passport was approved by 50.1 percent to 49.9 percent margin, reflecting widespread concern over government intrusion into people's privacy in a country that does not belong to the European Union and has long valued its independence.
  
• Switzerland joined Europe's control-free travel zone last year, which requires countries to register citizens' facial and fingerprint images on an electronic chip in the passport.
  
• Most of the 27 European Union members have issued biometric passports since 2006. But Switzerland has until March 2010 to put in place the new travel document, according to European law.
  
• Great to see this widespread deployment continue. DHS recently announced initiatives to improve current passport technology -- it looks like it's taking off worldwide.
  

Some People Really 'Never Forget A Face'
California Science & Technology News

• A new study suggests that skill in facial recognition might vary widely among humans. Previous research has identified as much as 2 percent of the population as having "face-blindness," or prosopagnosia, a condition characterized by great difficulty in recognizing faces. For the first time, this new research shows that others excel in face recognition, indicating that the trait could be on a spectrum, with prosopagnosics on the low end and super-recognizers at the high end.
  
• The research involved administering standardized facial recognition tests. The super-recognizers scored far above average on these tests—higher than any of the normal control subjects.
  
• One woman in the study said she had identified another woman on the street who served as her as a waitress five years earlier in a different city. Critically, she was able to confirm that the other woman had in fact been a waitress in the different city. Often, super-recognizers are able to recognize another person despite significant changes in appearance, such as aging or a different hair color.
  
• The human mind never fails to amaze me -- and disappoint at the same time.

Good Luck Hiding Your Face From Face.com

I wrote about this new site about a month ago and exciting new statistics from the alpha launch are detailed on Mashable today.

Although only a few thousand people have the application installed (myself included), Face.com has already tagged over 700,000 faces across 400 million (public) photos.

With the 15 billion pictures already on Facebook, Face.com is already fast at work going through the many fraternity party, wedding reception and family vacation pictures on the social networking site. In fact, it is tagging more than 9,000 images a minute! And if reports are correct, the tagging capabilities seem to be mostly accurate.

Exciting to see facial recognition coming into its own -- Face.com on Facebook, Flickr's use of Polar Rose, iPhoto's new Faces application...

Who knows where it will pop up next?

IHF Roundup: Retail Crime Rates Continue to Climb, Robotic Animals End Poaching Dreams & Other Top Headlines This Week

At their conference in Orlando earlier this week, RILA announced the continuing trend of crime in retail institutions at least partially blamed on the dismal economic conditions we've been experiencing. The survey included some of America's largest retailers in all market segments and inquired about measured or perceived changes in retail crimes over the last four months (on the heels of the December 2008 Crime Trends Survey). Here's a quick rundown of the statistics. Looks like we've still got a ways to go to turn this ship around:

• 61% of retailers surveyed report having experienced an increase in amateur/opportunistic shoplifting in the last 4 months.
• 55% have experienced an increase in financial fraud.
• 72% of respondents report that they continue to see an increase in organized retail crime (ORC).
• No retailers reported a decrease at all in amateur/opportunistic shoplifting since last surveyed.
  

Not exactly encouraging numbers, but they definitely show room for improvement and a growing market for surveillance deployments in the retail sector.

While some are resorting to stealing merchandise to make ends meet, others are betting on Lady Luck for a few extra bucks in this recession. Many casinos have seen increased gambling tendencies -- that is, of already-problematic gamblers -- in the past few months. Should security systems be preventing such detrimental habits? A Toronto Sun writer seems to think so.

With an estimated one-third of gambling revenue coming from problem gamblers in casinos, she puts forth the valid argument as to why they aren't stopped before hitting the tables. As she notes in the article, "In Holland, for instance, all casino visitors have to show their national ID card or a passport and are entered into a computer database that tracks their gambling behavior."

So, why aren't other countries monitoring such negative behavior? Should there be some sort of biometric database in gambling facilities? Good question. Although, if you think about it, isn't that almost like employing biometrics at a fast-food joint and prohibiting those with high blood pressure from ordering certain fatty foods? Definitely something to be considered.

Biometrics (or the lack thereof) aren't only in hot water in casinos -- that's hardly the case.

In England, the Home Office announced plans for organizations (including the Post Office and National Pharmacy Association) to collect and transmit biometric information for ID card enrollment. That would mean fingerprints and facial photographs would be stored on ID cards and in a central database.

However, the big question remains the security of the data itself. With such a massive high-street database, there must be a maximum security standard for the implementation to be viable. Additionally, who would be liable for a breach in the system?

That's an awfully high level of risk with the value this type of highly-sensitive information presents to identity thieves and the like. Still lots of kinks to be worked out here. Plus, really not sure how I'd feel about needing to give a set of prints to receive my prescription either.

Biometrics scans are popping up all over -- even in journalism. Fingerprints and retina scans are now required for all journalists covering the war in Afghanistan before being accredited to travel with NATO units or visit military bases. It's drawing some red flags from legal experts who have called the new produced "strange and offensive" and I'm not surprised. According to the article, "The data, including fingerprints and a retina scan, are used to verify identity and are apparently checked against an archive of known terrorists." I've heard journalists called bad things, but being checked against terrorist images -- not quite. At least yet.

Wrote about this quickly earlier in the week -- robotics out to save Mother Nature. Well, something like that. Researchers have designed robotic animals to catch animal poachers. Turkeys, swimming moose, white-tailed deer, black bears -- you name it, there's a robot that resembles it. These robots are saving species one at a time -- literally. With prices ranging from $500 for turkey to $5,500 for a grizzly bear, these decoys do not come cheap and with special reflective eyes glow at night, I'm not sure I'd want to be out in the wild with either species.

Also, don't forget to check out John Honovich's post about our 3VR Korea study on IPVideoMarket.info (along with the commentary from our CEO Al Shipp and myself) here. Gets into the nitty-gritty of the what the study results really mean. Definitely an interesting read.

Daily Headers: May 5, 2009

Retail Survey Shows Crime Continues to Rise Amid Troubled Economy
PRNewswire

• Today, RILA released a survey conducted of America's largest retailers that indicates an increase in retail crime associated with an economy in distress over the past four months.
• Respondents included 32 of the largest and most successful retailers in the U.S., from all segments including grocery, mass merchant, specialty store, apparel, electronics and appliances, and fabric/craft retail.
• Of the reported statistics:
• 61% of retailers surveyed report having experienced an increase in amateur/opportunistic shoplifting in the last 4 months.
• 55% have experienced an increase in financial fraud.
• 72% of respondents report that they continue to see an increase in organized retail crime (ORC).
• No retailers reported a decrease at all in amateur/opportunistic shoplifting since last surveyed.
• Looks like the holiday trend I blogged about in December has kept its momentum into the late winter and early spring months.
  

Uproar over face scan for foreign students
WAToday.com
Cameron Houston

• Melbourne colleges are considering the introduction of facial recognition technology at classroom doors to curb abuse of international student visas.
• The proposal has angered civil libertarians and overseas student organizations, which said the new measure has discriminated against foreign students and could threaten Victoria's $3.9 billion international education industry.
• Three Melbourne colleges or TAFE schools said they would consider the software, which requires overseas students to have their faces scanned. Cameras would then identify students entering or leaving classrooms and automatically record attendance.
• Looks like Aussies are following in Mother England's footsteps in implementing biometrics for attendance-taking purposes?
  

Robot Animals Snare U.S. Poachers
National Geographic
Maryann Mott

• Researchers have designed turkeys that are actually robotic decoys designed to catch such outdoor outlaws. Other robots include a swimming moose, white-tailed deer and black bear.
• For nearly 20 years, the Oregon State Police Department's Fish and Wildlife Division has run a decoy operation targeting violators who hunt off-season from their cars and roadways or at night with the aid of a spotlight.
• Molded-fiberglass animals are wrapped in genuine hides obtained by government officers through donations or illegal kills.
• Inside the bodies are radio-controlled motors—the same type found in toy cars or planes—allowing wildlife officers to remotely move a decoy's head, ears, and tail. Special reflective eyes glow at night when light is shined on them.
• The robots don't come cheap: Prices range from $500 for turkey to $5,500 for a grizzly bear.
• Not sure which I'm more afraid of -- the actual mammals or the robotic version. Do know I won't be caught poaching either of them anytime soon.

IHF Roundup: Facial Recognition Infiltrates Household Items, Facial Scanners Find the Bar Scene & Other Top Headlines This Week

Been a busy week of travel for me -- great chance to get caught up on the news of the world. Seems that every week, new ideas are being brought to the table in terms of where surveillance systems are being deployed and for what reasons -- Korea's most recent biometric initiative, for example. Take a look at other stories from around the world!

Facial recognition technologies are continuing to take online photo programs by storm. Announced this week, Flickr users can now import their photos to Polar Rose. This Swedish start-up identifies individual faces and names show up next to faces in the photos once the user has identified the faces based on the matching of 3D models. Yes, Google Picasa and Apple's iPhoto already do something similar to this, but Flickr's version of this technique demonstrates the depth of the technology's infiltration into popular culture and its growing momentum. Plus, these new improvements will save hours of time individually marking faces to names. I'm thrilled that Facebook might be next to jump onboard with Faces.com (still in alpha)!

Facial recognition isn't only on your computer, but also in your bathroom. Introduced this week, the SmartFaucet is capable of facial recognition to enable tempered water to avoid scalding hot temperatures -- and even enable users to access email, see weather predictions and update calendar events -- all while washing your hands! Although, I'm not quite sure wet skin and electronics mix?

And from faucet to front row at your favorite club, a Melbourne nightclub recently hit headlines with its new facial recognition deployment, the world's first nightclub installation. With Chasers nightclub already equipped with metal detectors, this new software is meant to fight violence common at the venue and keep out troublemakers. According to the article, "patrons' faces are scanned by a camera and the image and driver's license details are stored on a computer for 28 days." Also, police have access to the biometric data should an incident occur -- interesting collaboration between public and private institutions to keep the drinks flowing and glass un-shattered. I like it.

As I wrote yesterday, airports are well on their way to creating a common security system across the entire US aviation community -- and Singaporean IT is jumping onboard as well. While currently operating five individuals databases, the Immigration and Checkpoints Authority (ICA) will be creating the 'Bio-Hub' to integrate all facial and fingerprint images for Singaporeans, permanent residents and foreigners. Plans are also in the works, "to introduce a non-intrusive facial capture system at checkpoints, to deter imposters from finding their way into the country." Interesting initiative. Be interested to see if any other countries lobby for similar projects.

Manchester Airport Downgrades Matching Threshold on Facial Scanners

Those customer complaints about excessive wait times and congestion going through security in the UK must be piling up these days -- in a leaked memo, Manchester Airport allegedly re-calibrated its facial scanner machines from alerting security personnel if the passenger had less than a 80% likeness to their passport photograph to only calling out matches at less than 30%.

Deemed an "unacceptable" security risk by some, this change was prompted by an increased number of "false positives" not recognizing law-abiding passengers as the person pictured in their passport.

There has been no official explanation as to why the change occurred, and some facial recognition experts are up in arms, claiming that using a 30% match threshold is essentially useless. A memo noted, "[The fact that] the machines do not operate at 100% is unacceptable. In addition it would be interesting to know why the acceptance level has been allowed to decrease."

As another article also describes, with such low calibration levels, Kevin Spacey and Winona Ryder would easily pass through holding Osama Bin Laden's passport. With these odds, I bet could pass through security as Colin Firth, no problem.

I'll be interested to see if an official response to this "leaked memo" is released that explains the drastic drop in standards and their reason for the large number of false-positives. It could be the result of their technology choice, or just the result of poor, old, and varied photos common to passports. Likely both.

IHF Roundup: Robots Invading the Job Site, Surveillance Cameras Busting Fuzz and Other Top Headlines This Week

Robots were all over the place this week -- construction sites, ORs and warzone terrain. Engineers at Virginia Tech have designed 'Hydras' robots capable of rolling up and down poles and columns with the goal of deploying them at construction sites in the near future. Going forward, they hope to outfit these robots with ultrasound scanners and cameras to inspect bridges and buildings for flaws, tasks that kill over 1,000 workers each year. Brings new meaning to the term, 'pole dancing', don't you think?

In the Japanese healthcare sector, researchers plan to implement new safety rules for robo-nurses, which they expect to be a long-term solution to the lack of caregivers for the growing elderly demographic in the country. And from hospitals to front lines in Afghanistan, BigDog robots -- four-legged robotic creations capable of navigating the rocky terrain -- are being tested as potential danger alert monitors in battle zones. They can walk, lie down, have a great sense of smell and can even climb inclines. Grab them a collar and teach them to 'play dead' and they could make a great pet. Although probably not quite the friendliest or furriest companion I've come across.

As Geoff Kohl of SecurityInfoWatch.com wrote earlier today, biometrics may have reached the tipping point. When something becomes as mainstream as an iPhone, you know it's infiltrating popular culture -- and biometrics are doing just that. The Apple Insider blog reported today that Apple filed for a patent for a biometric reader to install in either an iPhone or Apple computer to secure the devices with either fingerprint or facial recognition. No passcodes required. Still a ways off, but I'm excited to think that instead of typing in a four-number passcode to make a phone call, my iPhone might someday be able to recognize the shape of my ear or authenticate my fingerprint on the screen instead.

Also, a fascinating look at how surveillance footage isn't just busting criminals, but also being used as evidence against cops lying on the witness stand in the New York Times. Instead of implicating suspected criminals, footage has established cases against the police officers committing perjury about their actions instead. The camera never lies.

Taking off for ISC West this weekend. Shoot @TheSteveRussell a direct message on Twitter if you want to meet up or just stop by the 3VR booth (#12081) -- hope to see you there!

Mark of the Penguins

They say no two adult African penguins have the same pattern of black spots on their chests. Until recently, however, scientists and researchers had yet to leverage this natural biometric phenomenon and long struggled with error-prone methods of tracking colonies with metal flipper bands.

With recent developments in the surveillance technologies used in wildlife tracking, researchers are now able to identify these unique marks and monitor individual birds in real-time over long periods.

First deployed on Robben Island in South Africa, a habitat for more than 20,000 birds, the software has not only been trained to recognize if there are any penguins in the camera's direct field of vision, but also to determine unique spot patterns to identify penguins and record the the bird's ID, in addition to the date, time and location of the sighting.

Researchers say the technology is adaptable to species other than penguins, too -- including cheetahs and sharks.

Bristol University Professor Barham, who developed the Penguin Recognition System, said, "For any species with patterned plumage, cheetahs or whale sharks for example, then the same technology could use the patterns as individual identifiers.

Exciting example of how biometric technology can be adapted to alternative habitats to eliminate the traditional manpower required and improve tracking accuracy, all the while drastically lowering costs. Think of it as facial recognition technology for the animal kingdom.

Look forward to seeing this technology adopted into other extreme environments, particularly species close to extinction. Maybe we go one step further past simple numbers tracking to learning more about daily habits and why some species are thriving and others dwindling (before they go extinct).

Biometrics: The New Timecard Verifier?

Will biometrics soon find be finding its way into office cubicles? That's the question biometrics experts, privacy advocates and academic researchers discussed in Westminster, England this week while discussing the future of biometric technology in the workplace.

In addition to augmenting security controls in private companies, experts realize that utilizing biometrics could have an economic benefit as well. Hugh Carr Archer, chief executive of biometric firm Aurora stated that a customer company utilizing facial biometrics to ensure employees were working when they claimed to be saved six per cent of its wage bill by defeating the typical strategy of employees clocking in and out for each other.

“Fred, for example, clocks on work [for Bill] when the foreman’s not looking, while Bill’s at home putting his feet up. The foreman creates three ghost workers because he can then get their pay packets," Archer said.

Particularly in this economic climate, ensuring the efficiency of your staff is crucial. As a result, with facial recognition and other biometrics solutions finding its way into airports, schools, ATMs and even DMVs, the workplace may be the next spot on the horizon.

However, being cognizant of privacy implications here is important. While infringements aren't an immediate concern to many watchdogs, going forward, some are worried they could play a role. "They are a lighting rod to privacy issues," Toby Stevens, director of the Enterprise Privacy Group, said. "This is because of personal sensitivities – my face, my fingers, my eyes, my voice – and they may carry information to you that I may not wish to convey.”

Some people worry about revealing passport and social security numbers when starting a new position -- not sure how they'll feel about having their irises scanned prior to entering the office or surveillance cameras looking over their shoulder while they're on the clock. Definitely still some particulars to work out, but should be interesting to watch.

IHF Roundup: New iPhone App Aids Gamblers, Photoshop Trumps Laptops, and Other Top Stories This Week

News from all corners of the country this week. Let's get started.

Casinos across the country were warned about an iPhone/iPod Touch application that counts cards, and according to the Nevada Gaming Commission, helps players beat the house in blackjack. If caught, I wouldn't be surprised if gamblers use this to fight off security guards. It must be more successful than trying to make a run for it.

At Black Hat DC this week, Vietnamese researchers showed the vulnerabilities of facial recognition software as a secure authentication method for laptops. With digital images and tweaked angles and lighting, they were able to bypass the biometrics on Lenovo, Asus and Toshiba laptops and gain access to the computer. Looks like traditional typed log-ins and passwords will hold the edge, at least for now.

Back in California, last week's DMV controversy has not died down in the least. A key legislative committee blocked the DMV's request to utilize facial recognition software to match up driver's license photographs with the entire DMV database of headshots. Fears that this system could make its way into police hands to monitor people at public gatherings are still rampant. So far, privacy groups: 1 DMV: 0. However, I'm sure this battle is far from over.

Did I mention Einstein was reborn? In a robot's body. The resemblance is uncanny! The researchers are still working on improving the IQ score.

Facial Recognition Authentication Software on Laptops Still Vulnerable to Hackers

At Black Hat DC yesterday, researchers demonstrated the vulnerability of current biometric facial recognition technology as a log-in security measure. Entitled "Your Face is NOT Your Password: Face Authentication Bypassing Lenovo – Asus – Toshiba," the report describes how the software -- a successor to traditional, typed passwords -- is still in its very early stages.

Lenovo's Veriface III, ASUS' SmartLogon V1.0.0005, and Toshiba's Face Recognition 2.0.2.32 systems all use webcams and algorithms to match a stored image with the user's face in order to log them into the system. However, as the researchers demonstrated, there remain several fairly simple ways for users to bypass this authentication process.

For example, Nguyen Minh Duc, manager of the application security department at the Bach Khoa Internetwork Security Center, a Hanoi-based security organization, described how using an image bearing a slight resemblance to the one stored by the system can allow a user to bypass the software. With millions of photos made accessible online by Facebook, MySpace and other social networking sites, hackers can easily manipulate images in terms of angle and lighting until one works -- even an intermediate PhotoShop user could make the requisite changes.

The team also showed a secondary spoofing technique called "fake face bruteforce," in which hackers generate multiple random faces to eventually gain access to the system. As one reporter recognized, this process parallels the conventional mechanism of trying hundreds, or possibly thousands, of text passwords before the correct combination is found.

Taking into account all of these concerns, it looks like the typed password will remain the preferred authentication method for laptops worldwide -- at least for now.

There are approaches, however, that can address both of these issues. First, dealing with the brute force attack vulnerability is the easiest. Since it's relatively easy for a laptop to know when it is being bombarded with hundreds or thousands of randomly generated and incorrect faces, that kind of activity burst can trigger a response from the facial recogntion software itself. It might shut down in response to the attack, or possibly just slow down. By suspending for longer and longer intervals with each failed access attempt beyond the acceptable threshold, the brute force attack is dragged out over such a long time period that the attack becomes impractical.

As for detecting the difference between live faces and pictures, a so called 'liveness' test can be performed to deter hacking, but is a much more difficult problem. However, I assure you that too can be done, as well. As to how? I need to keep that to myself for now, but let's just say that not all facial recognition technology is so easily hacked.

Einstein Reincarnated -- in a Robot!

Scientists at the University of California-San Diego's California Institute for Telecommunications and Information Technology (Calit2) recently installed specialized facial recognition software into a new robot -- the "Einstein Robot" -- which is designed to interact with humans "in a relatively natural, conversational way."

Why Einstein? "He's an icon of creativity, intelligence and science; he's emotionally accessible; he's lovable and visually recognizable very easily around the world," said David Hanson, the robot's primary designer and owner of Hanson Robotics.

Originally designed by Hanson Robotics of Dallas, Texas, the white-haired, bushy-mustached, and Frubber-faced robot publicly debuted last week at the Technology, Entertainment and Design (TED) conference in Long Beach, Florida. With software enabling Einstein to interpret and respond to "perceptual primitives" such as human facial expressions including sadness, fear and confusion, he is one of the first robots that can recognize facial cues suggesting different ages and genders of humans.

"When a robot interacts in a way we feel is human, we can't help but react. Developing a robot like this one teaches us how sensitive we are to biological movement and facial expressions, and when we get it right, it's really astonishing," said Javier Movellan, a research scientist in the Calit2-based UCSD Machine Perception Laboratory (MPL).

While still dealing with cost issues and what is known as Japanese roboticist Masahiro Mori's "uncanny valley theory," -- essentially that the more a robot looks like a human, the more repulsed we are -- large-scale manufacturing is still out of the question. However, this potential platform for future education purposes, entertainment or even cognitive therapy is another example of how facial recognition technologies are making their way out of the physical security and surveillance arenas and into other fields.

Very exciting stuff. I'm looking forward to seeing what comes of this research going forward.

Fool Me Once, Shame on You...

As reported last month, a South Korean woman fooled a fingerprint scanner at a Japanese airport to gain entry into the country. How did she do it? No need for an elaborate scheme -- instead, she used a special kind of tape purchased from a broker to easily bypass the state-of-the-art biometric system.

The 51-year-old woman used "special tape" to alter her fingerprints to trick the technology, which in November 2007 was installed at 30 airports around the country at a cost of around $45 million dollars. Designed to prevent entry by "international terrorists and other undesirables," reports say it is possible that a large number of South Koreans may have illegally entered the country using similar methods.

While the woman was eventually arrested in South Korea after attempting to purchase a fake passport, this incident raises a red flag about how effective border security really is and if expensive installments are worth the cost. In fact, last month, the DHS announced an update to its biometric technology in the US-VISIT program, which records biographic information to conduct security checks and verify the identities of international visitors to the United States.

However, why spend millions of tax dollars deploying systems that don't perform?

While biometric systems certainly complicate methods traditionally used by criminals to bypass the law, these tools will always exist -- ultimately, what customs and airport security checkpoints need is the deployment of an all-encompassing solution to stop those "special tape" users and repeat offenders in their tracks.

Integrating access control and facial recognition systems are two critical deployments needed to transform border security infrastructure and ensure a comprehensive solution to deter criminal acts. Designed to improve access control, passenger screening and liability protection, our deployments in airports (including Evansville Regional) monitor for criminal suspects and support both passenger and general airport visitor screening procedures with facial recognition technology, which stores profile pictures regardless of angle, shadow or lighting.

What does that mean in terms of catching criminals? We'll let's just say that no amount of "special tape" will get a person past an access control gate that requires an ID varified using facial recognition.

As the age-old proverb goes, "Fool me once, shame on you; fool me twice, shame on me." Multiple levels of security technology ensure the validity of this saying, and are ultimately crucial components of a successful surveillance program in the public places that need it most.

Opt-In Video Surveillance "Cloak" Aims to Calm Privacy Fears

The "Big Brother" argument -- that citizens are losing their individual privacy rights due to increased public security efforts -- is found in nearly all articles and reporting that cover the growing ubiquity of surveillance cameras worldwide.

Hewlett-Packard computer scientist, Jack Brassil, is hoping to address and resolve such criticisms with a new opt-in surveillance technology, Cloak, that he hopes will limit the privacy invasions presented by massive surveillance networks. According to Brassil, "Rather than prohibit surveillance, our system seeks to discourage surveillers distributing video without the authorization of the surveilled."

Cloak enables those who wish not to be identified in video footage to "opt-in" to the system, essentially making a "do-not-call" list for the surveillance grid. However, with this selection, the person must instead carry a 'privacy enabling device' that enables the image processing software to blur them out of any corresponding surveillance video, yet still allows the system to locate them on the grid.

Brassil is off to a great start in addressing the "loss of anonymity" argument that large-scale surveillance deployments bring to the forefront. However, the system still has a few kinks that need to be addressed before installation.

From a crime prevention standpoint, those that have the most to gain by blurring themselves out are those committing the crimes. In Brassil's proposal, it is impossible to 'un-blur' someone into focus for evidence in a possible investigation. Furthermore, requiring people to register GPS devices that tracks their location as a means to prevent surveillance systems from capturing their image seems to be just trading one modest privacy violation for another (perhaps even more) severe one.

As privacy analyst Ian Brown of the Oxford Internet Institute in the UK comments, "People shouldn't have to opt in to get privacy protection. And this system actively invades your privacy because it tells the service where you are at all times."

I commend Brassil in his efforts to resolve some of the privacy repercussions advanced surveillance deployments continue to introduce to society and have a couple suggestions to further improve the developing technology.

In an ideal system, every tracked face and motion activity should be blurred using a "reversible encryption" to enable future investigations if needed. That way, subject to policy (and perhaps subpoena), a person's anonymity can be reversed in a narrowly-focused way to solve crime and/or prove innocence.

Furthermore, the blurring functionality should be tailored to enable monitoring by police without revealing identity. For instance, security officials would be able to see if someone engages in violent or publicly-unacceptable behavior without identifying the individual. However, in an emergency, or if the system identifies a criminal, officials should have the option to disable the blurring feature in order to speed up the necessary response.

With these adjustments, Cloak is a much more workable proposal and more congruent with privacy, legal and societal concerns. Brassil is definitely a visionary in the space, and I hope to hear more about Cloak's progression in the near future.

Colin Firth as My Celebrity Twin?

While facial recognition technology has been available for personal pictures on desktop computers for years (for example, MyHeritage allows you to look up your celebrity twin), Gizmodo just announced that the functionality is now migrating to the iPhone via an application called FaceDouble. Not sure I see the resemblance, but my Twitter profile picture came up at 73% Colin Firth.


Once again, Poncho was not recognized.

Facial Recognition Training To Prevent Racial Bias?

(Credit: VizCogLab/University of Victoria)

Researchers from Brown University and the University of Victoria recently completed a study that suggests training people to recognize different facial features of individuals of a different race may reduce racial biases displayed unconsciously.

The landmark study involved 20 Caucasian participants, all of which were shown a series of black and white pictures of African-American faces in what was called an 'implicit association test.' Following each photo was either nonsense or a real word, of which the real ones had either a positive or negative connotation. The individuals responded immediately whether the word was real or nonsensical.

Initially, participants responded more quickly when a negative word followed the image and more slowly if the word was positive. However, in the same tests following 10 hours of training in which the first half learned how to distinguish among African-American faces and the other half learned to identify simply whether the faces were African American or not, individuals in the first group showed an increase in positive associations with the pictures and a decrease in negative ones.

"As soon as you can tell those people apart better and you can really tell that they're different individuals, then you'd be less likely to make an automatic generalization," said Michael Tarr, a professor of brain and cognitive science at Brown University.

Previous studies in this area have confirmed this 'learned aspect' and show that it is less difficult for participants to distinguish between faces of people in the same racial group in which they were raised. In fact, one project suggested that African-American children raised by white adoptive parents were more easily able to differentiate between white faces than black ones.

Although limited in scope, researchers would like to think this idea of recognition training could potentially be universally successful and signal a possible end to racial bias. Researchers are optimistic that these results could have implications in the real world, particularly for police officers, social workers and immigration officials looking to improve their differentiation of members of a racial group other than their own.

"The idea is this that this sort of perceptual training gives you a new tool to address the kinds of biases people show unconsciously and may not even be aware they have," said Tarr.

In fact, co-author Jim Tanaka believes the Obama administration taking office might be a perfect real world example of this study in action, which was actually released the day Obama was inaugurated.

He thinks that for the small segment of Caucasian folks with little interaction with those of African-American ethnicity, the tight media coverage of the First Family and the president's network will allow them to learn the features that make each person's face distinct and allow them to adapt this knowledge to the general population.

"I think clearly anytime you have positive examples to help break stereotypes is good," Tarr said. "President Obama does that."

While studies show that humans (and bees!) are born with innate, complex abilities to determine facial recognition, harnessing these skills to more precisely process features looks to be a developed talent. I'm thrilled to see the advances we're making in the research field and excited to see how they play out going forward. Perhaps seeing "Facial Recognition Training 101" in orientation programs for various public-facing career fields is not such a bizarre thought after all.

Bees -- Good For More Than Just Honey

There's some interesting facial recognition research just out of Monash University in Australia -- and I'm not talking about research on human faces. Instead, scientists have proven that honeybees can distinguish between two faces. Yes, honeybees.

In the study, performed over two years in Australia and Germany, researchers trained different groups of bees to fly toward human faces printed on a vertical screen. Bees that made the correct choice were rewarded with sugar or a bitter solution for incorrect decisions, and over the course of a day or so, bees had learned the task.

The groundbreaking discovery came later when only the bees that had experienced multiple views of a face at both 0 and 60 degrees were able to correctly identify the same face at the rotational angle of 30 degrees in non-rewarded tasks. According to Dyer, this demonstrates a bee's tendency to solve rotational problems by either averaging previously-seen images or mentally rotating previously learnt views.

"Bee brains clearly use image interpolation to solve the problem. In other words, bees that had learnt what a particular face looked like from two different viewpoints could then recognize a novel view of this target face. However, bees that had only learnt a single view could not recognize novel views," Dr. Dyer said.

This is exciting news considering most artificial intelligence recognition systems have major difficulties in reliably recognizing faces from different viewpoints.

"What we have shown is that the bee brain, which contains less than one million neurons, is actually very good at learning to master complex tasks. Computer and imaging technology programmers who are working on solving complex visual recognition tasks using minimal hardware resources will find this research useful," he added.

Turns out honeybees may be useful for more than honey and flower pollination (and of course, their tremendous dance moves).

Men are from Mars, and Women are Green

The next big breakthrough in facial recognition might be closer to home than we imagined. According to an article posted on DailyTech this week, the key lies within the neural workings of the human brain. However, what remains to be seen is how exactly the brain processes distinct facial features to determine identity. Going forward, understanding this internal facial recognition functioning could lead to a whole new wave of facial recognition technology.

Facial recognition is a topic of great debate and research, and the article presents two main schools of thought: those who believe the human brain is hardwired to recognize faces and those who think it's something we learn, and there are numerous subsets within these two camps.

Some have focused on facial tones and found men's faces to be "redder" than those of women's, which are more of a greenish hue. Others believe that the nose is the first place we look to identify a face, while another group suggests that the eyebrows play central role in recognition. Another MIT researcher believes that we process faces through a specialized part of the brain devoted to recognition called the fusiform face area, while many other scientists wholeheartedly disagree that we're born with any face recognition hardware at all. A hot topic indeed.

As science continues to catch up with innate abilities, facial recognition technology continues to adapt and improve. Who would have thought that eyebrows might be the real windows to the soul?

What’s Working and What’s Not in Facial Surveillance

Facial recognition technology has certainly been a hot topic in 2008 – and one that will likely only continue to make headlines as more companies and organizations recognize its unique value within their security infrastructures.

When most people think of facial surveillance, however, what often comes to mind is the technology’s very public failure at the Super Bowl and other large-scale public venues over the past several years. Though the technology has certainly improved, we are still not much closer to being able to spot a bad guy in a crowd than we were then, and this points to the fact that it is facial recognition’s role within the larger security platform that allows the technology to function most effectively, and ultimately most accurately.

Nonetheless, there have been some important facial rec successes over the past year that deserve mentioning – let’s take a look at what’s working, what’s not, and most importantly, why.

Identifying Suspects: Using Facial Rec to Compare Captured Images to Police Database Mugshots

The Sagem Morpho MorphoFace Investigate (MFI) system has scored its first arrest in Pierce County, Washington. The Pierce County Sheriff's Department used the MFI biometric facial recognition application to identify a suspect by comparing an automatic teller machine (ATM) photograph against the department's digital database of 350,000 mug shots.

In the past, the only way police could conduct these types of investigations was to endlessly flip through mugshot books -- this is a major improvement and compelling evidence of the next generation of crimefighting taking place.

Identifying a Suspect Using Commercial Surveillance and Transaction Records

Just the other month I learned about the following incident from a 3VR customer. A man had been followed home from a large African bank and subsequently robbed after making a large withdrawl. While the victim didn't recognize the thief, he was able to point him out in bank surveillance footage leaving the bank after the victim. Normally, this is where the investigation would have become difficult; actually identifying the suspect. However, in this instance, the bank was able to perform a facial search against it's own surveillance archives using its 3VR system. The thief, it turns out, was actually a bank customer. Even though he was not in any police database, they were able to identify him using bank transaction records. You see, this robber's tactic was to spend a lot of time in the bank doing small transactions and other petty business while he watched and waited for his victims to withdraw a large amount of money. But once even a single image was of him was captured, the robber's face led bank officials and law enforcement right back to those trasactions...and the theif's real identity.

Alerting Security When the Bad Guys Arrive

Though an uncontrolled venue such as the Super Bowl may provide too difficult a context to do real-time facial alerting, in more controlled venues like banks and some retail establishments and using relatively targeted top-quality watch lists, it is now possible to use facial alerting successfully.

Only two days after pilot installation of the 3VR platform at another large international bank, a person wanted for check fraud entered the bank accompanied by an accomplice, approached the teller and began a transaction. The 3VR system being utilized in the branch recognized the person and immediately sent an alert to the bank’s security personnel, who compared the image to photos in order to confirm that it was indeed the suspected fraudster.

The bank’s security personnel were able to quickly contact the police, apprehend the woman at the branch and question her. She ultimately admitted to the fraud — case closed.

Looking Ahead

Facial surveillance has developed significantly with new technology and new approaches making up for many past failures. And while still certainly not perfect, modern "facial surveillance" represents a quantum leap forward from they days when all police had to go on were “WANTED” posters, mug shot books, and their own eyes and energy.

Retail Fraud Gravy Train Could Be Derailed

With the new year of 2009 upon us, we're on the tail end of the holiday season and that means those customer service lines are packed with folks trying to return unwanted gifts for cash or store credit. However, what you may not know is that some of those customers are returning goods they never bought-- and cashiers may even be in on the tricks as well.

We blogged a few weeks ago about the growing trend of retail fraud in retail stores nationwide, and as we said, it's as simple as walking into a store, picking up an item and returning it for a full refund. Return fraud will cost retailers an estimated $11.8 billion in 2008 — $3.54 billion during the holiday season alone.

3VR was featured last week in a segment on CBS 5 San Francisco about our proactive approach to eliminating retail fraud with our surveillance software. The piece focused on our unique facial recognition and surveillance technology, which immediately sends alerts to security personnel the moment suspected fraudsters enter the store. Using the 3VR solution, retailers have access to everything they need to apprehend fraudsters before they can make their escape into a crowded mall or parking lot and before employees clock out for the night.

Research is showing that due in large part to the ailing economic conditions we're facing, retail thefts are no longer mostly limited to customers in urban areas. Both customer and employee hands are stickier this holiday season in more rural regions, demonstrated in the spike in crime in states such as Arizona, Colorado and New Mexico. Coupled with smaller budgets that have forced stores to cut prevention costs, retailers are stuck with tremendous financial losses. Organized Retail Crime (ORC), or "sophisticated crime rings that steal and stockpile huge quantities of merchandise that is sold later to unwitting buyers" in flea markets, pawn shop and other outlets, has also increased in these tough times.

"It's not pure need, but times are tough and the economy triggers people making bad decisions," said Joseph LaRocca, vice president of loss prevention for the National Retail Federation (NRF). An October NRF survey revealed 79 percent of retailers polled have experienced an increase in employee theft compared with the same period in the previous year, and according to LaRocca, retailers lost $34.8 billion in 2007 from employee thefts and shoplifting combined.

Ranging from simple shoplifting tricks and cash siphoning from registers to colluding with customers in sweethearting (when the store cashier works with the customer and intentionally undercharges by skipping items when checking out), retailers are fighting an uphill battle against employee fraud this year. As recent statistics and coverage is showing, proactive solutions like 3VR will be ever more critical in 2009 for retailers looking to prevent these criminal acts from both sides of the transaction and survive the slower revenue streams expected in the year ahead.

Advertising Gets Face Lift

Times Square. Piccadilly Circus. Even the turnstiles of the MTA subways in Manhattan.

These places are all highly trafficked areas with little advertising space to spare, and in such congested areas, marketers have tremendous difficulty determining the types of billboards that draw the most attention from audiences.

New technology out of Tokyo offers a solution to better measure the billboard's effectiveness – by utilizing surveillance technology. Slated to debut in a Tokyo railway station in January, NTT Communications is creating a billboard that stares back.

According to IDG News Service, the technology is able to measure how many people look at the billboard in order to determine the ad's ability to catch a consumer's attention.

Mounted just above the billboard, a digital camera coupled with basic facial recognition software will scan crowds and track if a person is looking directly at the advertisement. A second camera will be mounted underneath the billboard to track general crowd size in proximity to the ad. For privacy pundits, NTT is fast to stress that the cameras do not retain personal information about those tracked.

Capital or lowercase letters? Bold or neon color? Video or static image? This innovative approach will better measure how style, location and size aspects affect consumer attention and is the next step in adapting surveillance technology for other purposes.

Dunkin' Donuts actually performed trials a few months back to use facial scanning technology to target advertising to your age, gender and demographic group. Noted by the author, this was the "the first time it has been used by a mainstream advertiser in the US and works in the same way as systems used by law enforcement and emigration agencies to spot criminals in crowds."

Like the Tokyo billboard set-up, Dunkin' Donuts used cameras mounted above a screen to capture a customer's face and analyze facial features such as eye distance, jawline and cheekbone structure with complex algorithms. This information was then used to select ads more attune to the person's characteristics – a process about 85% accurate in tests.

Both Dunkin Donuts and NTT Communications show how surveillance technology can be modified to work for sectors other than security. Whether it is the donut shop around the corne or the train stop nearby, organizations are taking a page from Minority Report to operate more efficiently.

No More Smiling: Indiana Cracks Down on Happy Driver's License Photos

Latest news out of Indiana is that the state's Bureau of Motor Vehicles is instituting several new and somewhat interesting requirements for driver's license photos.

In an effort to aid facial recognition software and technology in its attempt to accurately identify residents, people will now be restricted from wearing scarves, hats and glasses, among other accessories.

Furthermore, drivers will need to refrain from smiling.

According to BMV spokesperson Dennis Rosebrough, it has been determined that smiles can inhibit the technology from accurate identifications and that the recognition software functions far better when given more standard images.

While this development may be a step in the right direction in terms of working to effectively leverage the technology at hand, a better approach would to to film a short video of the person, rather than take one single photo to rely upon for all identification purposes. While a single image is what always ends up on your license, there is actually no real reason that should be the only image the DMV can use when conducting a fraud search.

What we've found at 3VR is that comparing a collection of images of a person to other collection is always more accurate than working with a single passport-style photo-- regardless of facial expression, dress or accessories.

Multiple photos or even video would be a far more effective alternative, in our opinion, and what more, would even allow Hoosiers to resume their smiling at the DMV-- everybody wins.

Why So Serious?

New biometric technology research out of Concordia University may prove useful in recognizing mood via facial features in public and highly-trafficked areas, such as airports and train stations. The system may also have implications on homeland security technology used to observe suspicious behaviors and characters.

Prabir Bhattacharya, a professor at Concordia's Institute for Information Systems Engineering, "has developed a computer image processing system capable of classifying human facial expressions and identifying what emotion a person is conveying."

"Whether certain points like the eyebrows or the lips are expanding or contracting, you can know what sort of emotion they are conveying," Bhattacharya said. In fact, his system is capable of identifying all ten expressions below. While 18 facial features exist to determine mood, his system only requires seven.


As the ITBusiness article states, "The system analyzes a facial expression by first measuring the distance between the eyes. Based on that, it is able to map out other regions of the face and set a template. Then, it can process different markers that give away a person's mood. By focusing on specific groups of muscles near the eyes, nose and mouth, the system determines mood without requiring a full facial profile. That means less data is needed to determine a profile than other types of facial recognition systems."


The big question remains -- what is the value of mood recognition technology in identifying suspicious individuals? While some critics maintain that surveillance is more focused on recognizing one individual versus another and continue to discount the value of identifying a person's mood, others point to its many applications in the security field. According to ITBusiness, the concept has already been utilized by Israeli security forces trained in psychological methods to evaluate mood in order to maintain order in large crowds of people.

Whether deployed at crowded train stations, endless airport security lines, or local banks across the country, mood recognition technology certainly has the potential to supplement current facial recognition technology and make surveillance systems all the more comprehensive.

However, it may be some time before this technology makes it out of the lab and into a video camera near you. Frankly, there just aren't many surveillance cameras that create the kinds of straight-on talking-head-style videos required for this kind of algorithm. Maybe we will first see mood detection in something like the self-check-in kiosks at airports so the airlines can know how we are feeling when we travel. Just remember to smile.