3D Chip Company Creates Gesture Video Interface

I looked at this company, Canesta, and a competing company, 3DV, a few years back. (3DV was recently acquired by Microsoft.) Both companies use special 3D sensors to cleanly and perfectly track objects that are relatively close by. Unlike conventional video analytics that must use computationally expensive algorithms to track and extract objects from video feeds, these companies use rays of light to directly measure those object. The result is pretty cool. Here is what that looks like:



Canesta recently released the following demonstration of their technology tied into what looks like the user interface from Cooliris:



Microsoft also announced its own gesture control system called Project Natal at E3. The interface is no doubt based on the new technology they acquired from 3DV.

So, why am I writing about this here? Well, the mass production of these kinds of 3D sensors holds significant potential to shape and change the biometrics and video analytics industries as we know them. For instance, complicated algorithms designed to extract and identify faces from video might not be needed if the camera chip itself was already producing perfectly accurate measurements of the faces crossing it’s path. Similarly, 3D video cameras might obviate the need for expensive video analytic software.

For now, however, there is a problem with that prediction. The range of these sensors is just not quite long enough to be used in any kind of surveillance context. Today, one really needs to be standing quite close for the 3D analysis to work.

But, like all things in the world of technology...that will change.

You are being watched...in 3D

I just got back from the International Security Show, ISCWest. I’ll be writing up some more thoughts on that in a bit, but in the meantime, I thought I’d tide you over with you a few cool videos I found from one of the exhibitors there.

Feeling Software got its start in 2005 building 3D technologies for video games and the entertainment industry. So, it’s no surprise that their offerings to the world of high-tech surveillance look a lot like a 3D shooter.





The demos are still a little rough around the edges, but in them, I think, one can see the beginnings of a new interface paradigm for surveillance monitoring that succeeds in providing both detail and context in one interface. It's huge potential advance over the brain-numbing eyeball-burning video walls that sit at the center of modern security rooms today.

And in addition to offering better situational awareness, the interface abstraction might be used to provide other features from privacy to integration, as well. 3D tracked models might be highlighted, tagged, or even obscured or removed from monitoring model based on security policies, privacy law, or real-time information from other systems.

One more note: For those interested in facial recognition, tracking, and modeling, you should also check out Feeling Software’s Face Flow demo. Though not quite up to the standard of the Benjamin Button tech I commented on the other week, it's pretty cool none-the-less.

Security in 3D: Coming Soon to an Airport Near You?

Security engineers left and right have been attempting to crack the code on technology to match CCTV photos of faces with image archives in order to capture known criminals and thieves. Recent research out of Arizona State and Michigan State show that 3D laser scans could be eventually utilized to alleviate problems with lighting and photos angles in public areas with fast moving crowds.


Using a newly developed program, Dirk Colbry from ASU and his MSU colleague, George Stockman, performed 300 laser scans of 111 different faces using a commercial scanner to store images, while a horizontal plane of laser light passed over the subject's face. These images were then manipulated to create 3D models of each person's face.

The results were superb-- with the new system, different scans of the same face were matched even when lighting was unusual or the angle from which the images were taken was off by as much as 30 degrees. The error rate was a startling one percent.

This research demonstrates a new step in the advancement of surveillance technology, although its implementation will need to overcome the high price, slow scan speed and short-range sensors that may serve as roadblocks. The current scanner price of $50,000 needs to drop to about a tenth of that price to encourage widespread deployment. Additionally, scans currently take 2 to 5 seconds, making the technique obsolete in large crowds and only functional at choke points, such as airport and train station security checkpoints, where passengers are forced to wait in queues and in close range of scanning devices.

Being able to accurately identify someone who has paused, short-range, at a checkpoint and presented thier face, or iris, or fingers, or hands to the confidence levels demonstrated here has been possible for quite some time. But sometimes it's hard for me to get too excited about most of those approaches. That's because what we would all like to get out of any new biometric systems that get deployed at the airport are shorter lines and fewer checkpoints not more!

At 3VR, we're constantly looking for new algorithms to improve our facial recognition analysis features to combat the disparities in expression, lighting and angle. But to date, we have limited ourselves to writing algorithims for use in conjunction with conventional video cameras. That's because CCTV, and even new IP camers, are cheap, prolific, and offer something that laser scanning systems can't, the ability to work at a distance with uncooperative subjects.

Though current facial surveillance approaches fall somewhat short of 99%+ accuracy of 3D laser scanners, I can say that some of the techniques we are pioneering today hold strong promise of closing that gap. By processing streams of facial data from standard CCTV video feeds, it's possible to create an extremely accurate facial model; maybe one that will someday rival 3D scans. When will facial surveillance catch up? I can't say exactly. But, I do know that technologies like what we deploying at 3VR will get there long before a $50,000 laser scanner becomes as cheap as a video camera.

CyberExtruder Gets 2D to 3D Face Patent

At the end of November, CyberExtruder announced that the company had been granted a new patent on their process for creating reliable 3D models of a person’s face from a single or series of 2D images.




Though the enforceability of this patent has yet to be tested, CyberExtruder’s 2D-to-3D conversion is certainly an important innovation to the security and biometrics technology industries. Why? It enables better matching between offline photographs and surveillance video. While the matching of "watch list" images against surveillance video has traditionally achieved mixed results as a result of inadequate lighting, angle, expression, etc., this patent could signify a leap forward in terms of the quality and value of 3D facial images.

In June of 2007, I wrote on how XID was using a similar technology in the “world’s largest” facial recognition access control project. In that instance, XID literally generated hundreds of thousands of variants of an enrolled employee’s face rather than using just a single 2D photo converted to a 3D model. Each day when an employee arrived for work, his or her photo would be taken and compared to the database of generated images rather than a single original. Interestingly, this approach generated huge improvements in the performance and accuracy of the Thailand access control system.



We’ll see if these two companies come into conflict over the new patent, but I don’t think they will. XID’s approach to 2D-to-3D is very different than CyberExtruder's-- almost quick and dirty by comparison. CyberExtruder, on the other hand, has become famous for its hyper-realistic…if sometimes creepy…generated floating 3D heads that lend themselves to applications well beyond security including gaming and movies, and even boast a fan in Phillip Rosedale of SecondLife.

Trace & Extract 3D Objects from Video

From Australian Centre fro Visual Technologies:

VideoTrace is a system for interactively generating realistic 3D models of objects from video—models that might be inserted into a video game, a simulation environment, or another video sequence. The user interacts with VideoTrace by tracing the shape of the object to be modelled over one or more frames of the video. By interpreting the sketch drawn by the user in light of 3D information obtained from computer vision techniques, a small number of simple 2D interactions can be used to generate a realistic 3D model. Each of the sketching operations in VideoTrace provides an intuitive and powerful means of modelling shape from video, and executes quickly enough to be used interactively. Immediate feedback allows the user to model rapidly those parts of the scene which are of interest and to the level of detail required. The combination of automated and manual reconstruction allows VideoTrace to model parts of the scene not visible, and to succeed in cases where purely automated approaches would fail.