The Transformational New World of Data-Rich Video Systems

Since the advent of video surveillance technology, CCTV cameras have been an essential aspect of any comprehensive physical security plan. The ability to monitor entrances and exits, high traffic areas, points of sale, and other important areas has obvious value to a business, not only from a loss prevention perspective but from a safety and risk mitigation perspective as well. For decades, businesses and organizations have availed themselves of this important technology to keep their premises secure, their valuables safe, and their people protected.

That has changed considerably in recent days—and promises to change even more in the near future. Thanks to breakthroughs in artificial intelligence (AI), machine learning, and deep learning technology, a camera is no longer just a camera. In fact, today’s cameras and their wide-ranging capabilities bear little resemblance to the cameras of the past. Able to integrate with advanced analytics software and detecting things invisible to the naked eye, today’s surveillance technology has been supercharged. With this evolution of technology, the video surveillance landscape has changed dramatically: no longer is it about spotting a wrongdoer and identifying them after the fact. Soon, AI-enabled security devices may even be able to detect potential incidents before they even happen.

There Is Always Room for Improvement

Thanks to movies, television shows, and portrayals in other media, the general public has a very specific image of video surveillance: CCTV cameras scattered throughout a property, each feeding video to a bank of television screens monitored by one or two security guards. And while Hollywood does like to take liberties with this sort of thing, in general, that image is fairly representative of traditional video surveillance setups. The camera captures the video, but it is up to the security personnel on duty to monitor that video for anything that might be out of the ordinary.

The security guards monitoring these feeds need more support from the businesses they are working hard to protect. Using traditional analog security cameras, even the clearest, sharpest video or the maximum possible camera coverage is not enough. Instead, the success of the entire system depends on the human being monitoring those cameras—and even the most conscientious security guard is not immune to mistakes. Simply put, human attention has its limits, and a human being cannot be relied upon to notice or fully register everything that happens on a video screen.

Harvard University performed a famous study demonstrating this very thing in the late 1990s. Cheekily titled “Gorillas in Our Midst,” (1)  it centered around a video that you can still watch here (2). The video featured a group of people passing a basketball back and forth to each other and participants in the study were asked to count the number of passes made. What the scientists failed to mention, however, was that a man in a gorilla costume wandered through the center of the group about halfway through the video. Shockingly, they found that while the participants were occupied with counting the number of passes, less than half of them noticed the gorilla. This phenomenon, dubbed “inattention blindness,” serves as an effective reminder of human fallibility—and, therefore, of the limits of human-monitored surveillance.

It is critical to note that the Harvard study was conducted by asking people to monitor a single screen. Security personnel facing a wall of monitors are being asked to do much more. It is simply impossible to manually process the information being projected onto a dozen or more screens, and the practical result of this is that a massive percentage of footage captured by traditional security cameras is never seen by human eyes. In effect, surveillance becomes less about catching wrongdoers in the act, and more about identifying the after the fact. A valuable service, but one with clear room for improvement.

In today’s world, this reactive approach to security is no longer sufficient. As artificial intelligence has become more advanced, it has allowed security teams to greatly increase their effectiveness—for example, by more efficiently monitoring video surveillance feeds without the drawbacks associated with human error and distraction. Artificial intelligence has enabled vast new capabilities for devices that formerly did little more than record video for later review, and it is important to understand what this means for both the present and future of the security and surveillance industries.

What Is AI in the Context of Video Surveillance?

What is artificial intelligence? Perhaps more importantly, what is the difference between AI, machine learning, and deep learning? If you’re looking for an in-depth answer to that question, there are plenty of primers available (3)—but for our purposes, think of it as a hierarchy. AI is an overarching term that refers to any demonstrated machine intelligence. Machine learning is a more complex subset of AI, and deep learning is an even more complex subset of machine learning. Hollywood has conditioned us to think of “AI” as a superintelligent machine, but, in reality, AI might be little more than what controls enemy units in your favorite mobile game. Machine learning (and especially deep learning) is where machines begin to get really smart.

AI begins to fall into the machine learning subset when it begins to demonstrate basic problem-solving skills and can be “trained” to serve a specific function. Deep learning goes one step further and is essentially modeled on the human brain. Deep learning technology can label and classify large amounts of information, identifying patterns, and using that information to make decisions. Within the security industry alone, deep learning can take many forms. It might be trained to look for signs of fraud at a credit card company or to identify suspicious activity after business hours. It might also be trained to recognize things like trespassers, car accidents, gunshots, or even signs of aggressive behavior.

Understanding this, AI and its subsets have obvious and significant value to the video surveillance industry. And before even bringing artificial intelligence into the conversation, it’s important to note that the simple availability of tools like thermal cameras and radar detectors today has already vastly increased the capability of surveillance deployments, allowing cameras to pick up things invisible to the naked eye and automatically alert the necessary authorities. If there is a courtyard where no one should be present after hours, a thermal camera can detect an intruder, no matter how stealthy they may think they’re being. Likewise, radar detectors are great at sensing movement.

AI Provides Major Improvements to Existing Surveillance Tools

Tools like thermal cameras and radar detectors are of particular note because not only do they provide significant added value to any video surveillance deployment, but they are also prime candidates for AI-enabled improvements. A common complaint with regard to thermal cameras is that they occasionally produce false positives—if, for example, a deer passes through the camera’s field of vision, it might reasonably assume that an intruder is present. But when armed with deep learning capabilities and trained on a sufficient amount of data, a thermal camera can learn to tell the difference between a human and a deer and react accordingly. A radar detector might likewise be trained to tell the difference between a flag flapping in the wind and a trespasser moving with intention.

This is all in addition to the obvious benefits associated with incorporating machine learning and deep learning into basic visual surveillance deployments, where the possibilities are nearly endless. As the technology grows more advanced, it will make its way into countless industries: it may be deployed in hospitals to detect patients in distress, at busy intersections to identify vehicle crashes and near misses, in government buildings to detect suspicious behavior and more. Data gathered by these cameras might save patient lives, let municipal planners know that a “no turn on red” sign might be needed, or even recognize an individual banned from the premises and stop a potential incident before it happens. It can help with loss prevention, people counting, and even risk mitigation. Today, even simple applications like identifying loiterers or tracking the movement of customers through retail stores to improve the flow of traffic have helped demonstrate the everyday use of technology.

This risk mitigation point is particularly important. Surveillance tools are about more than just keeping a location secure. In many cases, they can help address notoriously difficult-to-address problems, such as slip, trip, and fall cases. Estimates vary, but the average slip, trip, and fall case generally costs tens of thousands of dollars to settle today, which represents an exorbitant expenditure—especially to smaller businesses. High-quality surveillance cameras can help: Businesses and organizations naturally want to protect their employees and customers from unnecessary injuries, so identifying areas where slips and falls are common—or training them to detect places where people almost fall—can help prevent both injuries and liability.

Training cameras to recognize these cases and automatically alert the appropriate authorities can dramatically decrease response time, potentially mitigating the seriousness of any resulting injury and easing the associated liability, and of course, the ability to detect fraudulent cases. Faced with a potentially lucrative payout, it is not unheard of for customers (or even employees) to fake slip, trip, and fall cases in the interest of securing a large settlement. Effective video surveillance has the potential to save businesses and their insurer's large fraudulent settlements. Just last year the Department of Justice convicted three members of a New York City-based “trip-and-fall scheme” of defrauding area businesses and their insurance companies out of more than $31 million. AI-enabled surveillance has the potential to put a major dent in this problem.

AI, Machine Learning, Deep Learning Grow in Both Complexity and Utility

As our understanding of AI, machine learning, and deep learning have grown, so too has our ability to effectively apply the capabilities they enable to the video surveillance space. The more complex these tools grow, the better they are able to learn, identify patterns, and make decisions. While today’s technology can be trained to perform tasks like facial recognition, pattern analysis, incident detection, and other tasks, those capabilities are always growing and expanding as users discover new ways to apply them.

AI is only beginning to make its impact felt, but its potential to significantly improve countless aspects of the video surveillance industry is tantalizing. As security teams continue searching for new and better ways to keep people safe, property protected, and risks under control, AI-enabled devices will soon make their task significantly easier. Even in their relative infancy, AI applications have demonstrated their usefulness to security teams across the globe. As they continue to grow more advanced, these tools have the potential to change the video surveillance landscape in new and exciting ways.

About the Author: Ryan Zatolokin is a Senior Technologist with Axis Communications, Inc.

Reference:

(1)  https://www.ncbi.nlm.nih.gov/pubmed/10694957

(2) https://www.youtube.com/watch?v=vJG698U2Mvo&feature=emb_title

(3) https://www.forbes.com/sites/forbestechcouncil/2020/03/25/artificial-intelligence-machine-learning-and-deep-learning-whats-the-difference/#3b7a64667bd0