Editor’s note: This is the 54th article in the “Real Words or Buzzwords?” series from SecurityInfoWatch.com contributor about how real words can become empty words and stifle technology progress.
Just a few weeks ago, Amazon Web Services (AWS) announced their release of a new technology for video cameras called Panorama, which surprisingly has nothing to do with panoramic cameras (i.e. 360-degree cameras). AWS refers to Panorama as computer vision (CV) technology, and right away I asked myself: Is there actually a technical basis for calling a security video analytics technology “computer vision” or is this just another marketing hype term that is going to add more confusion into the world of security video analytics?
About a year ago, in a Real Words or Buzzwords? article titled, “Artificial Intelligence, Machine Learning and Deep Learning,” I said that future articles will discuss more aspects of AI, ML and DL as products using them continue to emerge. This is one such article. In that article I mentioned that the history of AI research goes like this:
- Artificial Intelligence – 1950s to 1980s and still ongoing
- – –> Machine Learning – 1980s to 2010s and still ongoing
- – –> – –> Deep Learning – 2010s to now and still ongoing
As Wikipedia explains: In the late 1960s, computer vision began at universities which were pioneering artificial intelligence. It was meant to mimic the human visual system, as a stepping stone to endowing robots with intelligent behavior. In 1966, it was believed that this could be achieved through a summer project, by attaching a camera to a computer and having it "describe what it saw”. What distinguished computer vision from the prevalent field of digital image processing at that time was a desire to extract three-dimensional structure from images with the goal of achieving full scene understanding.
“Full scene understanding” is another way of saying “situational awareness,” an important and also broad topic covered in a series of seven Real Words or Buzzwords? articles (Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, and Part 7). Situational awareness applies in a great many fields, including sports, for which there is sport-specific situational awareness training. It is critically important to autonomous driving, including for anti-collision and other vehicle safety features, for which a combination of sensors, including radar and video analytics, are applied.
Security Video Analytics Hype Factor
It is the recent advances in deep learning that have attracted billions of dollars of annual investment in video analytics and has resulted in very significant improvements in video analytics capabilities, which has been good news for the security video industry. However, advances in video analytics have historically been a good-news/bad-news situation for the security industry, which has already experienced two over-hyped phases of video analytics, with a third new phase already under way.
These three phases evolved from the three phases of computer vision research listed above: early artificial intelligence, machine learning, and now deep learning. The previous two phases of security video analytics were both over-hyped, and so security industry end user customers, as well as security service providers, are now doubly cautious about current security video analytics claims.
From Hype to Reality
There are several major differences today from the earlier decades of security video analytics, including the fact that there is now very good reason for referring to video analytics applications as computer vision applications, as AWS is now doing. First, security video in general is benefiting from the related simultaneous IT megatrends of continuing technology cost reduction and continuing exponential increase in technology capabilities. These are explained in the short but meaty article by Pivot 3 in this month’s Security Business magazine, that highlights the modern technologies that are simplifying the design, deployment and scaling of intelligent video analytics deployments.
Second – as explained in another Security Business article, “AI and Specialty Analytics are Changing Video Surveillance” – various categories of specialty analytics now make offer tremendous value both for security and safety situational awareness and for business operations purposes. The emergence of specialty analytics is made possible by advances in computing and data storage technology for devices, on-site equipment, and in the cloud.
Third, advances in free open source and licensed proprietary deep learning software have enabled the development of a wide range of specialty analytics – and this is what AWS, Microsoft Azure, Bosch and others now have a strong focus on.
Computer Vision and Video Analytics
As explained in the whitepaper that I wrote for the Security Industry Association titled, “The State of Video Security Analytics (PDF),” the basic definition of video analytics, also known as video content analysis (VCA) in the academic world, is the extraction of meaningful and relevant information from digital video. Figure 1 below shows how the world of academics – where most of the AI/ML/DL research is done – views computer vision and video content analysis (what we call video analytics). This is how areas of research are labeled, not products.
Figure 1. Artificial Intelligence and Computer Vision
When it comes to products, it’s easy to understand why the video security industry used the “video analytics” label as opposed to “computer vision” for its products. At the time that security video analytics were introduced, machine vision’s widely in use in manufacturing, where automated production lines were able to run at high speed through the use of computer vision. Initially simple light sensors were used, for example, to detect the presence or absence of boxes and other objects on a production line. Computer vision provided optical character recognition whereby part and product labels could be read automatically, and computers could manage production line activity and log the tracked results.
These and other manufacturing applications were in the general public’s thinking when it came to computer vision. Thus, it made sense to call video security’s use of computer vision “video analytics.” At that time, simple pattern recognition was used for video analytics, and machine learning and deep learning were not involved.
As AI advanced and evolved to the state shown in Figure 1, the term video analytics continued to apply to the security video industry products and there was no reason to change it. However, AWS is very strongly involved in providing software tools and cloud computing platforms for machine learning, deep learning and the broader field of computer vision, and so has very good reasons to label its Panorama technology in the computer vision category, as it is narrowly focused use of the broader AI tools and computing platforms it offers.
Additionally, Panorama technology consist of much more than what security video analytics offerings have provided in the past. Video analytics are a part of the larger Panorama technology offering. And AWS’s purpose for Panorama is not to provide video analytics products per se, but to provide the capabilities for developing them and the technology that enables running them on devices (cameras), computers on site and in the cloud.
This is a similar purpose to Security & Safety Things (S&ST) initiative from Bosch, with a significantly greater cloud technology aspect, as the Panorama appliance contains hardware elements specifically designed to run deep learning applications, and its software has built-in connectivity to the AWS and Azure cloud environments. More will be said about such specialized hardware capabilities in an upcoming Real Words or Buzzwords? article.
What Is AWS Panorama?
The Panorama technology consists of firmware, software, a software development kit (SDK) and an appliance for running what AWS calls “computer vision applications,” and is intended to enable the running of Panorama-based machine learning (ML) video analytics on cameras that have the Panorama technology embedded in them, or on the Panorama appliance on site - close to where the cameras, or in the cloud hosted on AWS in Microsoft Azure.
Since Panorama’s purpose is to foster the development of high-value niche video applications and their deployment, it’s perfectly fine to categorize the technology category as “video analytics.” But at the same time, it’s important to realize that Panorama provides much more than what we have seen in the past from security video industry video analytics vendors.
For more details about the AWS Panorama technology, see the SecurityInfoWatch.com article titled, “AWS Panorama: A potential game-changer for video analytics.”
An Expanding Category of End-User Applications
An important final point is that today’s deep learning-based analytics are model-based. It’s not the old days where applications were simply software code. Most of the AI software code already exists, and the important work is in gathering and selecting the data for training, and in selecting the right training models.
Students in the field have a wealth of open source resources available to them that give them knowledge and experience directly applicable to real world applications. Many students work on real works applications prior to graduation. Although much of the academic work available online is steeped in the AI domain’s vocabulary, the technology is very understandable, especially at a high level.
Application development involves selecting the right machine learning and deep learning models, and then selecting and/or developing the right training sets so that the initial deep learning training enables the analytics applications to continue leaning in the directions intended.
More information about this is available on the Machine Learning Mastery website, which has a section titled, "Start Machine Learning." It contains a wealth of information of the kind formerly never available to interested learners. One article is titled, "How to Go From Working in a Bank To Hired as Senior Data Scientist at Target." Another is, "14 Different Types of Learning in Machine Learning" – which is a roughly 7,000-word introduction to the subject with references to more advanced information.
I’m not suggesting that readers dive right into the technical details. The point is that AI is a field of study that is growing exponentially in popularity and has a wealth of available online knowledge and tools. Students skilled in deep learning application development are a new kind of resource that is now available to security integrators and to large end-user companies who have an internal technical team.
For example, Amazon Protective Services (high-level corporate security group) has their own software development team that has been working with AWS AI technology for years. For the earlier in-person AWS re:Invent event, the group developed vehicle tracking analytics that used people and vehicle counting, and GPS information, to determine how many seats were still available on which shuttles and buses. This enables their event’s mobile app to help attendees at their choose the best mode of transportation to local venues.
AWS Panorama and S&ST platforms and technology, coupled with a new generation of AI technologists, provide integrators with a way to address customer-specific important analytics use cases. These are the kinds of high-value analytics that provide continuing benefits to end-user customers who are happy to subscribe to them as an incremental expansion of their security system capabilities.
About the Author:
Ray Bernard, PSP CHS-III, is the principal consultant for Ray Bernard Consulting Services (RBCS), a firm that provides security consulting services for public and private facilities (www.go-rbcs.com). In 2018 IFSEC Global listed Ray as #12 in the world’s Top 30 Security Thought Leaders. He is the author of the Elsevier book Security Technology Convergence Insights available on Amazon. Mr. Bernard is a Subject Matter Expert Faculty of the Security Executive Council (SEC) and an active member of the ASIS International member councils for Physical Security and IT Security. Follow Ray on Twitter: @RayBernardRBCS.
