In the movies, the bad guys are somehow always able to slip past the guards watching hundreds of security cameras. And no wonder. Who could watch video feeds of hundreds of empty hallways for hours on end without getting bored and distracted? That’s why the security community has put such an emphasis on video analytics, letting computers monitor the video feeds and send an alert if they see anything worth investigating.
But video analytics hasn’t always been up to the task. First generation video analytics was limited to watching for changes in individual pixels, or perhaps a significant portion of the pixels. If a camera was outdoors, trees waving in the wind or even clouds moving in front of the sun might trigger a false alarm. Most of the research and development in video analytics has therefore gone into reducing the number of false alarms. One way they’ve done this is to program the analytics to view the changes as objects rather than pixels: That’s just a rabbit, that’s rain, that’s snow, that’s a spider making a web on the front of the camera. The technology is continuously improving the accuracy of categorizing what the camera is observing according to size, shape, aspect ratio, speed, motion behavior, and color among other parameters.
That first generation of video analytics has evolved in two ways: In-camera or edge analytics, where the specialized camera has the intelligence to interpret what it sees, are getting much more sophisticated at a very low price point, sometimes to the extent of being bundled with the camera itself. In contrast, Server-based analytics integrate information from a large number of cameras – no matter the brand or model - to give security professionals a greater situational understanding, and they do so by leveraging the latest in IT-class processors that continue to rapidly evolve.
While first generation video analytics focused on real-time alerts and minimizing false alarms, second generation technology focused on providing forensic tools to search the video post-event. If there had been a robbery and someone saw the suspects escape in a white van, you might search a camera for all white vans that drove past your location the day prior. Or if the situation involved an assault by a person wearing red, you might search a camera for the past three hours for people wearing red. The second generation of video analytics helped to reconstruct what happened; however, it was designed more as a forensic tool rather than a tool that exists to tell you where the van or person is right now, just moments after you heard about them. That sense of immediacy is what makes all the difference in response and mitigation.
The third generation of video analytics is purpose-built to offer real-time insight, based on searching for someone or something across multiple cameras, during a time frame typically leading up to the current time. It answers the simple question ‘Where is he now?’ and has to be fast, accurate, and easy to use in a high stress real-time situation.
Taking Server Analytics to the Next Level
Using first generation analytics as a sensor is useful and inexpensive, but it’s not revolutionary, and the technology is bound to produce false alarms. Second generation analytics can perform detailed forensics, which certainly beats manually searching through hours of video, but it does not give value as an incident is unfolding, and that is when it is most needed in order to prevent the situation from escalating. Third generation analytics give first responders the situational awareness they need by delivering real-time forensics, and follows the natural evolution of video analytics.
Imagine a scenario where you receive a call from someone who found a man lying unconscious in a corridor inside a building on campus. After calling the medics, you pull up the security video of that corridor to find out what happened. It turns out the man was attacked, and your video footage provides a picture of the assailant.
Real-time forensic analytics enables you to use the picture of the assailant or, if the suspect wasn’t captured on-camera, even a general description of the suspect from an eye witness, to quickly search through a database. This database includes full-body images previously extracted from the recorded footage of all video feeds in the network to track the assailant’s movements after the attack and, if he’s still in the building, find his current location. The key is that the analytics can find a person in almost real-time. They do not have the limitations of facial recognition, which demands that cameras be carefully positioned; instead the analytics considers the person as a whole. The system needs to be able to spot the suspect from behind, as well as from the front, and from a camera mounted at an angle typically associated with standard video surveillance, which is not ideal for facial recognition.
As important as the suspect’s current location is for apprehension purposes, so too are his whereabouts in the time leading up to the incident. How did the assailant get in the building? When did he enter? Which door did he come through? Which fence did he climb over? By following the individual’s past movements, you can identify the answers to these questions and potentially uncover security vulnerabilities that need to be plugged. In this case, video analytics acts as an intrusion detector, but ironically does so after the fact.
What’s the point of having an intrusion detection system that alerts you to an open door half an hour after someone walked through it? Post-incident video analytics allows you to pinpoint the weak spots in your infrastructure to help avoid future security incidents.
In the previous example, video analytics revealed that an insider cooperated with the assailant, and two other people carrying guns entered at the same time. The victim in the hallway didn’t have a gunshot wound, and no one in the building reported gunfire, so the analytics provided a valuable piece of knowledge that could aid the authorities in apprehending the suspect. We need to now search for the other two armed accomplices and find out where they are right now. Video analytics that follow suspects as they move from camera to camera - forwards and backwards in time - provide a much greater situational awareness and expedite investigations so authorities can apprehend the suspects before a situation escalates.
Statistically, most crimes are committed by people you know, not by random strangers. Law enforcement experts in Europe and the U.S. have commented that over 50 percent of breaches were caused by employees misusing access privileges, whether maliciously or unwittingly. Ideally, an intrusion detection system puts the pieces of the puzzle together, including how it began, not just what’s happening now. Certainly, you first want to know the current location of the bad guy, but there’s no point in finding out a week later while doing post-event video forensics that he came in with two other people and received help from the inside.
Using analytics in this manner enables organizations to be proactive rather than reactive and leverage their existing investment in what is often a sizeable video system. Unleashing this new power justifies an expansion of the existing camera count because now the argument shifts away from the need for a visual record of as many things as possible in order to reduce liability to the possibility of stopping incidents before they happen. Security personnel are used to intrusion detection systems that focus on random strangers trying to get through a secured area, but today’s video analytics look at an intrusion in a completely different way. It doesn’t matter that the suspects came in without triggering an intrusion alarm because the analytics can still find where they are, how they got in, and what they did.
There are many ways to get around an intrusion system or access control system, and that’s why real-time forensics video analytics system can be so valuable. The system doesn’t care how the intruder got in. It’s just going to find the suspect. The fact that someone was able to trick the intrusion detection system doesn’t matter. Real-time forensics video analytics provide the tools needed to keep on top of unfolding events, in real time.
About the Author
Dr. Bob Banerjee is Senior Director of Training and Development for NICE Systems' Security Division. A recognized PSIM expert, Banerjee has more than a decade of experience, having held senior marketing and global product management positions with several IT and software firms. He is a frequent speaker at security industry conferences and has published numerous articles on PSIM and other security topics. He holds a Ph.D. in Artificial Intelligence from the Advanced Research Center at the University of Bristol, England.
