Sitting Down with Rajiv Shah

It has been a busy past few months, but I recently had the opportunity to chat with Rajiv Shah, an adjunct assistant professor in the Department of Communication at the University of Illinois at Chicago, and the author of the Smart Cameras blog. Below are some of my answers to his questions around privacy, vendor comparisons, industry connections to academia and the future of the smart camera market.

Again, very excited to share them with the growing IHF readership. Feel free to make comments on any of the responses or questions, and I will be sure to address them.

Also, check out Rajiv's Smart Cameras blog -- it focuses on developments in Chicago's use of video surveillance, as well as other "smart cameras" that utilize additional sensors and/or computer processing techniques. Cool read and a staple on my blogroll.

1. Privacy: What should the industry approach be towards privacy? Should they incorporate features that protect privacy? Should they have default settings that protect privacy or delete information? Or should we not worry about this? Is there a need for an industry-wide approach to this issue?

Conventional wisdom presents ‘privacy vs. security’ as a zero-sum game, one in which gains in one arena necessitate sacrifices in the other. And while there is certainly much truth in this, it is also equally true that in a modern society neither principle can exist without the other. There can be no security without privacy, and no privacy without security.

Today, any meaningful national security failure could create a response that curtails our civil liberties quite broadly. And conversely, next-generation security technologies deployed without adequate privacy and civil liberties protections likely face the crippling backlash of a concerned public. As a result, the security industry needs to take issues of privacy VERY seriously.

For me, an approach to privacy in the context of surveillance starts with a few key principles designed (1) to narrowly tailor a system’s use and (2) to ensure that system access is adequately controlled and audited. Today, conventional “dumb” surveillance systems offer none of these benefits. A large video wall in a security room or command center does nothing to distinguish between security threats and the average person; these systems cast an unnecessarily wide net, relying on human expertise and interest to filter down to focus in on actual security threats.

The issue of what to delete or keep in terms of surveillance becomes much less important on systems where this kind of ‘all-or-nothing’ approach to data access doesn’t exist. For instance, on a 3VR, an investigator might search through many months worth of video information looking for matches or clues relating to the kidnapping of a little girl. However, because this query is done algorithmically using facial recognition, and because the search request is logged and audited, there is ultimately much less concern about the overall retention of video data. The public generally has very little problem with legitimate surveillance investigation that doesn’t subject them to what they feel is needless voyeurism.

Retention of video also becomes less of a concern in the context of new blurring and encryption algorithms designed to protect individual privacy. These new technologies prevent generally tracking and identification of the pubic using surveillance, while preserving the ability of law enforcement and security officials to detect and investigate crime. To better understand what I mean by this, you should take a look at the recent article in New Scientist on some of what we are working on in 3VR labs right now.

In any case, issues of data retention, encryption, access control and the like are often more policy issues than industry issues. Instead, our focus should be enabling decision and policy makers to make, monitor and enforce these choices themselves. Our solutions should present options to do all of this...and more. Today, most security solutions don’t include any privacy protections whatsoever. That needs to change; asking someone to chose between security and privacy isn’t much choice at all.

2. Comparing Vendor Solutions: What can be done to make it simpler for end users to compare and contrast different solutions? It's very confusing now for end users to sort through claims by tens of companies on effectiveness, costs, technology, etc.

Normally, I would say that the answer to this question solely involves the emergence of various standards groups, independent testing and analysis organizations -- that is because the best response to confusion is nearly always more good information. And, I do think there is some good news on the horizon in both of those areas with new security analysts, bloggers and agencies entering the marketplace of ideas every day.

However, because many new solutions’ claims today are so specific and require real-world deployment for actual evaluation, the only way for end-users to fully educate themselves may be through pilot and testing projects that they conduct themselves. New technologies being offered today represent a quantum leap over previous generations of security and surveillance solutions, and end users will ultimately need to make a very significant investment in time and money to educate themselves on their benefits.

3. Connections to Academia: Explain if anything needs to be done to expand the connection between industry and academia. After all, much of the engineering talent has come directly from universities. Are there any suggestions you have for universities and their research?

The disconnect between commercial markets and academia is a classic problem seen across many industries, but I have noticed is a particular problem in the security industry. And as a partial result, there has been comparatively little innovation at the core of this market in recent decades. The surveillance methods used to catch criminals hasn't changed drastically with investigators still found staring at video walls or fast-forwarding through video stores looking for needles in haystacks. Plus, the innovations responsible for rapid productivity gains in knowledge workers in other industry segments seems to have largely passed this industry by. Who are security’s Googles, Microsofts, and Oracles?

To begin to address this issue, I think that most importantly security needs to become the province of innovative and interesting companies again. Only by tackling big, tough and important problems can the security industry hope to lure academia’s best and brightest, or focus them on its problems.

As for universities and their research, there is one problem faced by the security industry today greater than all others…and that is a crisis of our own making. It’s “information overload.” There are quite simply too many cameras and sensors today generating way too much information today, and the resulting torrent of data threatens to overrun our entire industry. Identify ways to process and sort and make meaningful this flood, and you will have done us all a great service…and there is probably a job waiting for you at 3VR, as well.

4. Future Growth of Smart Cameras: Have cameras hit a period of steady growth or do you foresee a potential boom ahead? If so, what are the crucial factors that you see that are limiting growth of that will cause growth to increase? Do we need to improve technology, better end-user experience, etc.

Cameras have seen explosive growth already -- sales worldwide are booming. Not only that, but the general sense of a 'camera' is evolving dramatically; dumb cameras, smart cameras, cameras that record at 200 frames per second, cameras integrated with iPods – they're popping up all over the place and exploding in ways that people could not have anticipated. Not only are the types of cameras available growing exponentially, but the data being collected by cameras has increased by a geometric factor far beyond that. New cameras have higher resolution, higher frame rates. More of just about everything!

As a result, we're stuck drinking from the firehose for the time being. We're inundated with data and have no idea what to do with it due to the sheer volume we're faced with. It's coming in too quickly to comprehend, and as a result, we've discovered that it's not the volume of data you collect, but what you can do with that video (and how quickly) that matters.

The modality of staring at a wall of video screens broadcasting camera streams broke down a long time ago – and we're better off for it. However, as camera volume, quality and speed explodes, we need to figure out how to comprehend and process this volume of data. If we're going to manage the growth of cameras, they don't need to be smarter – we're already capturing more data than we need – but rather more searchable and enable efficient retrieval of vital information.

Daily Headers: May 13, 2009

Facial recognition gates trialled at Stansted Airport
Saffron Walder Reporter

• Facial recognition gates are being trialed at Stansted Airport in a bid to cut queues and improve border security. The facial recognition gates work by using scanners to compare the faces of passengers to their biometric passports.
• The gates can be used by any British or European passenger who has a new e-passport with an electronic chip.
• The system measures points on a person's face and compares them with the digital passport photograph. People who have changed their appearance since their passport pictures were taken will not pose problems, because the system will still be able to compare them accurately.
• Let's hope Stansted gets it right and isn't forced to downgrade matching thresholds like Manchester Airport allegedly did.

Robot teacher conducts first class in Tokyo school
Telegraph UK
Danielle Demetriou

• Saya, the female humanoid robot that I mentioned about a month ago, taught a science and technology lesson to a class of 10-year-old pupils at Kudan Elementary School in Tokyo.
• While Saya's creator Professor Hiroshi Kobayashi said the robot's main purpose was to highlight the joys of technology to children, he also said it would benefit schools suffering from a shortage of human teachers.

Brain scanning may be used in security checks
Guardian UK
Owen Bowcott

• Distinctive brain patterns could become the latest subject of biometric scanning after EU researchers successfully tested technology to verify ­identities for security checks.
• The experiments, which also examined the potential of heart rhythms to authenticate individuals, were conducted under an EU-funded inquiry into biometric systems that could be deployed at airports, borders and in sensitive locations to screen out terrorist suspects.
• The Foreign Office plans to spend up to £15M on fixed and mobile security devices that use methods including "facial recognition (two and/or three dimensional), fingerprint recognition, iris recognition and vein imaging palm recognition".

Manchester Airport Downgrades Matching Threshold on Facial Scanners

Those customer complaints about excessive wait times and congestion going through security in the UK must be piling up these days -- in a leaked memo, Manchester Airport allegedly re-calibrated its facial scanner machines from alerting security personnel if the passenger had less than a 80% likeness to their passport photograph to only calling out matches at less than 30%.

Deemed an "unacceptable" security risk by some, this change was prompted by an increased number of "false positives" not recognizing law-abiding passengers as the person pictured in their passport.

There has been no official explanation as to why the change occurred, and some facial recognition experts are up in arms, claiming that using a 30% match threshold is essentially useless. A memo noted, "[The fact that] the machines do not operate at 100% is unacceptable. In addition it would be interesting to know why the acceptance level has been allowed to decrease."

As another article also describes, with such low calibration levels, Kevin Spacey and Winona Ryder would easily pass through holding Osama Bin Laden's passport. With these odds, I bet could pass through security as Colin Firth, no problem.

I'll be interested to see if an official response to this "leaked memo" is released that explains the drastic drop in standards and their reason for the large number of false-positives. It could be the result of their technology choice, or just the result of poor, old, and varied photos common to passports. Likely both.

Beverly Hills 48025

It seems that my old high school in Beverly Hills, Michigan has found itself embroiled in a debate over the use of surveillance in its hallways after cash, MP3 players and cell phones were stolen from several lockers. While this may signal a loss in trust in the community, it could also be reflective of the expensive nature of many items commonly found in lockers nowadays.

"It's one more sign that times are different today," Jim Ballard, executive director of the Michigan Association of Principals, said. "Years ago, we trusted each other more than we do now.”

I can also tell you firsthand that years ago, we students didn’t keep a lot of cash, MP3 players and cell phones in our lockers. Heck, if you broke into mine, you would have been lucky to find some old textbooks, gym socks and a Daisy Duke poster. Not quite a treasure trove of goods compared to what's readily available today.

But whether or not what students put into their lockers is valuable, the students themselves are pretty valuable, and it seems to me they deserve at least the same level of protection afforded to the malls they hang out in.

However, not all the students think new security measures are such a good idea.

Seaholm student Jake Drutchas wrote the following comment on the wall of a Facebook group called, "Seaholm and Groves Students Against Security Cameras," which has more than 870 members, "For now, let's start with a little student responsibility. Lock your stuff up. Don't leave it out in the open."

He has a point, but it’s also true that school security deployments have nearly always proven a smart development. I think Jake's argument is more founded on the fear that the same surveillance technology used to fight locker thefts will be turned on him for reasons other than safety.

Will the cameras be monitored all the time or just used to investigate incidents? Who will have access to the cameras and for what purposes? Lots and lots of very good questions.

If Beverly Hills Groves follows the national trend, they too will get security cameras installed -- if not this year, then sometime in the not-too-distant future. However, Jake Drutchas and his Facebook posse can continue to put their passion and energy on the issue to good use by helping the district draft a clear and responsible policy with respect to the use of surveillance inside their hallways.

There certainly are ways to balance security and privacy inside of educational institutions, and I think you will find the hippie baby-boomers who run school districts these days are more than open to them.

Biometrics: The New Timecard Verifier?

Will biometrics soon find be finding its way into office cubicles? That's the question biometrics experts, privacy advocates and academic researchers discussed in Westminster, England this week while discussing the future of biometric technology in the workplace.

In addition to augmenting security controls in private companies, experts realize that utilizing biometrics could have an economic benefit as well. Hugh Carr Archer, chief executive of biometric firm Aurora stated that a customer company utilizing facial biometrics to ensure employees were working when they claimed to be saved six per cent of its wage bill by defeating the typical strategy of employees clocking in and out for each other.

“Fred, for example, clocks on work [for Bill] when the foreman’s not looking, while Bill’s at home putting his feet up. The foreman creates three ghost workers because he can then get their pay packets," Archer said.

Particularly in this economic climate, ensuring the efficiency of your staff is crucial. As a result, with facial recognition and other biometrics solutions finding its way into airports, schools, ATMs and even DMVs, the workplace may be the next spot on the horizon.

However, being cognizant of privacy implications here is important. While infringements aren't an immediate concern to many watchdogs, going forward, some are worried they could play a role. "They are a lighting rod to privacy issues," Toby Stevens, director of the Enterprise Privacy Group, said. "This is because of personal sensitivities – my face, my fingers, my eyes, my voice – and they may carry information to you that I may not wish to convey.”

Some people worry about revealing passport and social security numbers when starting a new position -- not sure how they'll feel about having their irises scanned prior to entering the office or surveillance cameras looking over their shoulder while they're on the clock. Definitely still some particulars to work out, but should be interesting to watch.

Biometrics Finding Its Way into Customs Checkpoints

It's already been an active new year for customs checkpoints.

Earlier this week, European Parliament approved the use of biometric data in EU passports, and yesterday the U.S. Department of Homeland Security (DHS) announced that upgraded biometric technology has been installed at every major port of entry, and that most visitors should expect to use the new technology upon entering the country.

The European Parliament will amend a 2004 regulation to include fingerprints and facial patterns in passports issued in the EU, thus making it more difficult for criminals to forge identification documents and/or travel under stolen passports. Additionally, the new EU regulations will set the minimum age to take fingerprints at twelve, as fingerprints change as children grow. Instead of being included in their parent's passports, all children will be required to carry their own in an effort to make child trafficking more difficult.

Back in the U.S., the DHS announced that it has updated biometric technology in its US-VISIT program, which records biographic information to conduct security checks and verify identities of international visitors to the United States. The program is also expanding the categories of non-U.S. citizens required to provide digital fingerprints and a photograph upon entry to the U.S. in December, and requiring 10 fingerprint collections rather than two. Many experts agree that collecting a full set increases matching accuracy and also reduces the chance of misidentification.

With increased wait times expected and obstacles associated with integrating the various Extended Access Control (EAC) systems now in use across the globe, it will take years for travelers and customs officials to get acclimated to the new procedures and documents. In the meantime, bring a magazine for that next trip through customs.

Face Rec Hacked! Needs "Liveness" Test

Very interesting article in CNet that highlighted an exciting new trend, but also pointed out that it may not be ready for prime time. Many new laptops, including new models from Lenovo, Asus and Toshiba, have started using facial recognition scans as the primary security mechanism for accessing their devices, rather than fingerprints or passwords. Definitely a cool use of new technologies, but as CNet points out, companies need to be sure they get it right before they introduce it to consumers, who would have no way to know their security was compromised.

In this test, security firm Vietnamese Internetwork Security Center (VISC) demonstrated vulnerabilities in laptops' face recognition-based authentication mechanisms that let anyone log in to a computer easily with a "special" photo of the legit owner, even at the highest authentication level. VISC was able to almost instantly produce a photo of CNet Editor Dong Ngo, taken over the laptop's webcam during a Skype chat, that fooled the computer's facial recognition software and successfully logged into a computer registered to Ngo.

Here's how Ngo described the offending photo:

About five minutes later, the technician produced a rather unflattering picture of me on a piece of letter-size paper. I could hardly agree that it was my mug on the photo. Nonetheless, when used in front of the laptop's camera, the Y430's authentication software was happy enough with the photo and logged in within a second. Pretty scary.

This type of hack is going to be very difficult for taditional facial recognition vendors to overcome. Early algorithms in this biometric field all focus exclusively on comparing one single image to another single image. Even if that image is being extracted from a laptop web camera. There is zero concept of context or "liveness" in this approach, and so it is easily spoofed. 3VR Security, is the only company I know of with a facial recognition platform built from the ground up to analyze streams of faces, like those in a video feed, rather than just single images. With this type of approach, subtle changes in motion, expression, pose, and other varialbles unique to a "live" 3D person can be analyzed at the same time a biomtric match is taking place and the kind of spoofing demonstrated here simply would not work. Maybe it's time for laptop vedors to upgrade their algorithms.

Adobe Video-Object Manipulation Project Holds Significant Promise

Video Object Manipulation may be going mainstream.

Once the exclusive domain of computer vision geeks in the Department of Defense, and more recently Hollywood, Adobe has now got its hands on these algorithms and every YouTuber with an HD camera will soon have the tools to not only mark up video in fantastic new ways, but literally to bend reality.

What does it mean for security when just about anyone can add, remove or alter people and objects within a video stream to create a perfectly realistic video of something that never happened? Well, in my opinion it’s mostly not good.

But, there is a bright side -- the entrance of consumer-focused companies like Adobe into this industry is likely to help the security professionals as much as the criminals. More experts, new approaches, better tools and easier-to-use interfaces are a welcome addition to security’s video analytic offerings, and can certainly help security personnel fulfill the promise of improved surveillance.

CyberExtruder Gets 2D to 3D Face Patent

At the end of November, CyberExtruder announced that the company had been granted a new patent on their process for creating reliable 3D models of a person’s face from a single or series of 2D images.




Though the enforceability of this patent has yet to be tested, CyberExtruder’s 2D-to-3D conversion is certainly an important innovation to the security and biometrics technology industries. Why? It enables better matching between offline photographs and surveillance video. While the matching of "watch list" images against surveillance video has traditionally achieved mixed results as a result of inadequate lighting, angle, expression, etc., this patent could signify a leap forward in terms of the quality and value of 3D facial images.

In June of 2007, I wrote on how XID was using a similar technology in the “world’s largest” facial recognition access control project. In that instance, XID literally generated hundreds of thousands of variants of an enrolled employee’s face rather than using just a single 2D photo converted to a 3D model. Each day when an employee arrived for work, his or her photo would be taken and compared to the database of generated images rather than a single original. Interestingly, this approach generated huge improvements in the performance and accuracy of the Thailand access control system.



We’ll see if these two companies come into conflict over the new patent, but I don’t think they will. XID’s approach to 2D-to-3D is very different than CyberExtruder's-- almost quick and dirty by comparison. CyberExtruder, on the other hand, has become famous for its hyper-realistic…if sometimes creepy…generated floating 3D heads that lend themselves to applications well beyond security including gaming and movies, and even boast a fan in Phillip Rosedale of SecondLife.

The Costs and Benefits of Face Recognition

Most people think face recognition is great when it's catching baddies and weird or invasive when it's tracking innocent people. The technology has sparked debate after debate over whether privacy or security is more important. Honestly? I think we can have both.

First off, people need to understand what facial recognition can and can't do. This ABC story about a face rec system analyzing someone's face and telling a clerk that they're underage is bogus. If the kid had been in the store previously and been busted for buying liquor then yes, the system could alert the clerk, but no one is suggesting that face rec can tell the difference between 17 and 18 any more than a human can.

One thing face rec can do, which hasn't gotten a ton of press, is include simple privacy measures. We have been working on this at 3VR and I wrote recently about a team in Canada that's working on a similar project. Basically, software engineers can write a password-protected program that blurs faces and when an incident occurs, an investigator can unblur faces in particular pieces of video. This way, while people may still feel uncomfortable about being on camera, at least they will not really be watched unless they happen to be present during a robbery or some other incident, in which case they'll typically be glad the cameras were there to help catch the bad guys. Also, because this type of application can also have auditing capabilities written into it, it provides a crucial and often overlooked capability: a way to "watch the watchers," if you will.

The same could feasibly work in the grocery store situation described in the ABC story - if someone was a match with a suspect in the database, then the system could alert the clerk. For everyone else, faces could be blurred, and if someone is caught buying liquor or cigarettes underage, then the store manager could unblur the face and save it to the suspect list.

There has also been a lot of press recently about the rise of surveillance that risks privacy without actually improving security. I agree. Thing is, the bulk of new cameras installed are meant to catch traffic violators and raise money for municipal governments, not improve security. These cameras misfire fairly often, sometimes costing a city more than they're worth, and invading citizens' privacy for no good reason.

During the bombings in London, however, investigators were able to use video footage to find their suspect. If they had had face recognition and video search capabilities, that investigation would have been far shorter. And as banks have begun installing surveillance systems, they have seen a marked increase in the number of fraud cases they're able to solve.

Surveillance clearly has a place in modern society, but I do think that the industry needs to continue to work towards securing both people and their privacy.

Video surveillance gets smarter in Verbania, Italy

Verbania, the capital of the province of Verbano-Cusio-Ossola, Italy, was created when the towns of Intra and Pallanza merged. As one of the most idyllic and famous tourist destinations on Lake Maggiore, the town relies heavily on holidaymakers to fill its streets and generate income. To ensure the safety of visitors and citizens, the town council decided to launch a community surveillance project based on a system of network cameras.

Monitors linked to the ten Sony SSNC-RX550 network cameras are installed in the Verbania Municipal Police control room, which is currently undergoing restructuring. Even so, the system does not require the intervention of dedicated security operatives. The human element only comes into play when real-time monitoring is needed (for example during a major event), or when an automatic alarm is triggered.

Today the Municipality of Verbania can be assured that those who commit crime will be identified, thanks to technology which directly recognizes objects and reads vehicle registration plates. In fact, the town's network cameras have been positioned so that they can monitor all arrival and escape points in every area of the town. Intelligent image analysis functions now enable allow the city to keep special areas such as no-stopping zones under control. Using these features, live images can be monitored in unattended mode until suspicious activity occurs, at which point the operator is proactively alerted to the threat by means of an appropriate alarm.

Biometric Powered ‘Medicine’ Dispenser…Dude

Biometric technology stands ready to change forever the way we visit the pharmacy…potentially rendering a lot of people in white lab coats obsolete in the process. Anytime Vending Machines or AVMs went into operation in California today and employ a variety security features designed to make the automated vending of controlled and prescription medicines possible without any human intervention. The first product to benefit from AVM technology, however, might catch you a little by surprise. It’s marijuana, and here is how the process works:

Patients have to present a prescription and be fingerprinted before they are issued with a pre-paid credit card that stores the dosage and type of drug prescribed. […] Vince Mehdizadeh, owner of the Herbal Nutrition Centre in Los Angeles, where one of the first machines is based, said patients could get access to prescribed drugs after hours. "They'll be greeted by a security guard right there. They'll slide the card in and they'll fingerprint in to verify that it's them,'' he said. "A camera takes a picture of them, verifying that they're actually at the machine. And they get the medicine and they move on.''

AVMs are a very cool device, though I think they might end up with an interesting reputation given their market entry strategy. In any case, I would I am not sure the necessity of a full-time security guard standing next to the AVM fits terribly well with the automation concept being pitched.

Top 10 Asian Telecom Predictions for 2008

IDC has revealed its top 10 telecommunications predictions for the Asia-Pacific region excluding Japan (APEJ) in 2008. These include greater demand for Web 2.0 and unified communications applications such as collaboration tools within the enterprise, as well as increasing awareness in IP-surveillance products.

8. IP surveillance With IP-surveillance tools currently being deployed in Beijing, China, as part of the 2008 Olympics security plan, 2008 is set to be a "big" year for the technology. "Businesses will be receptive to IP surveillance due to increased security concerns, as well as IP surveillance's cost effectiveness," IDC said.

IP surveillance technology will become popular in the
gaming, financial services, and retail industries, the study noted, citing
physical security as a primary concern for these vertical industries. I would
add banking to this list.