A word of caution on AI

Over the past several years, the security industry has been inundated with products espousing the benefits of artificial intelligence (AI). As many are quick to point out, however; true AI, which is the ability of machines to think and reason on their own without humans in the decision-making loop, is non-existent at the moment and that the analytics products being leveraged across the industry currently are really machine learning solutions that are designed to help with the arduous and somewhat monotonous task of detecting and classifying objects – people, vehicles, animals, etc.

But even these machine learning products are only as good at the human-developed algorithms that drive them. Such was the cautionary messaged shared by Genetec President Pierre Racz during the company’s “Connect'DX” virtual tradeshow that was held earlier this week.

“I’m not knocking the technology, but we need proper engineering and proper engineering requires knowledge of these limits of the technology,” Racz said. “We have to avoid wishful thinking and anthropomorphizing and we have to be open and honest about what these limits are. We do not want to build systems that provide a false sense of security.”

With today’s machine learning solutions, Racz said it is paramount to define the problem you are trying to solve.

“Artificial intelligence does not exist, but real stupidity exists. You mustn’t confuse the appearance of intelligence with actual intelligences,” he added. “Intelligent automation keeps the human in the loop and the human can provide you with the intuition and creativity while the machine does the heavy lifting.”   

At Genetec, Racz said the company is using intelligent automation (IA) tools and statistical machine learning but not what much of the market is referring to today as AI. Having been in the software business for 40 years, Racz said he’s already been through five AI “winters” which occurred after the technology appeared on the scene and then quickly fizzled after failing to meet what the Gartner Hype Cycle refers to as the “peak of inflated expectations.”

Problems with AI

One of the telltale signs that this is occurring with a technology is when negative press headlines start to appear about it in publications and this is already happening with regards to AI this go-round. In fact, one of the most prominent AI technology’s on the market, IBM’s Watson supercomputer, which famously beat two of the most successful Jeopardy champions of all time, was recently called “a piece of shit” by a doctor at a Florida hospital after recommending the wrong treatment for cancer patients.

“(IBM) thought they could apply this to the medical system and Watson learned quickly how to scan (medical) articles but what Watson didn’t learn was how to read the articles the way that doctors do,” Racz explained. “The information that physicians extract is more about therapy and there are small details and footnotes and they extract more than the major point of the study. Watson, who’s reasoning is purely statistical-based, only looks at the first order of the paper but doctors don’t work that way.”  

Racz also pointed to the problems currently being experienced with regards to autonomous vehicles and their inability to respond appropriately when they encounter atypical data during a drive as another prominent example of the learning pains of AI. In particular, he said a recent crash in Arizona involving a self-driving Uber vehicle that struck and killed a pedestrian walking her bicycle across the road is a prime example of this.

“The biggest failure was that the AI-engine driving that car had only been trained on humans crossing the road at a crosswalk. To see a human cross the road where there was no crosswalk lowered its confidence score,” he said. “It was also trained to see humans on a bicycle but a human walking beside the bicycle was something it wasn’t familiar with and so that also confused the AI. And I say AI as a pejorative.”      

In fact, Racz said the AI we have today has reasoning power of an earthworm. “The danger is not that AI is going to take over the world and enslave us, the danger is they do exactly what we ask it to,” Racz added.

Real World Implications for Security

Racz also presented a series of photos in which a small distortion was introduced to vastly change the confidence of an AI algorithm in how it classified the object in a photo. In one case, a lifeboat that was identified with 89% accuracy confidence was subsequently classified as a Scottish Terrier with more than a 99% accuracy confidence after this distortion was inserted in the image.

“Think about this: If you’re using facial recognition as a single factor for access control, I could walk in front of your access control camera with my smartphone generating an adversarial image and I could get the system to recognize me as anybody in your database,” he said. “Of course, this is not a cheap attack. It would most likely require me to break into your system and find out what is your training set, what is the algorithm you’re using but this is what you have to do; you have to decide if you’re going to use such technology as a single factor for access control. What is the impact of failure?”

Though many organizations are now buying into intelligent video analysis and other so-called AI security tools, Racz cautions that they really need to do their research first.

“A lot of companies have jumped on the bandwagon and most end users don’t really know the limits of the technology and they don’t know if it was actually well-engineered. They don’t know if the training set has bias, so my word of caution is caveat emptor: buyer beware,” Racz added. “You can design these systems and they can look really good in a demonstration during a sales pitch, but you have to make sure you’ve properly engineered for false positives and the false negatives because there will be those. You also have to make sure there is not unanticipated things in your training set. Remember, the adversarial system only needs to get it right once, you have to get it right every time.”

Other Highlights from Connect'DX

In addition to Racz’s keynote, Genetec held a wide range of sessions during its three-day online even this week. Here are few of the highlights.

Technology Roadmap

According to Christian Morin, VP of Integrations and Cloud Services at Genetec, the company has really been focused on four main areas when it comes to their product strategy, which include delivering mass scalability, transition their core business to Software-as-a-Service (SaaS), placing a larger emphasis on access control, and developing tools to protect the rights of citizens in liberal democracies.

As the company has experienced rapid growth in recent years, Leon Langlais, Genetec’s VP of Infrastructure Product Groups, said one of the side effects of that is that the size of systems they are helping customers deploy have increased exponentially. In fact, the company recently won a project to help secure a large, multinational corporation that has more than 450,000 cameras on its network.

“You may be thinking, ‘ok, fine 160,000 to 450,000, that’s not a big leap, at least not as big as you’ve done in the past.’ But this company will be leveraging our Privacy Protector (module) on the vast majority of those connections, which basically means two feeds per camera,” Langlais explained. “They also add in their system 15,000 to 20,000 new cameras per year just as a regular, ongoing business as they acquire new companies, expand in some regions and things of that nature. So, if I project myself five years in the future, I could have a system that has a million cameras feeding back into Security Center. Not only that, this company is also leveraging the vast majority of the Genetec portfolio, so they are going to reuse a lot of the key analytics – from people counting to intrusion detection and the list goes on and on. The platform we have today will have some bottlenecks that will not allow us to do this full project at scale today. But the good news is we have a few years to get ready and we have already started.”

In addition, Langlais said that company is working to massively scale-up the number of card holders that its access control platform can support. The company recently expanded its limit from 250,000 to 500,000 card holders and they are now working to scale that into the millions.

“What our engineers are working on right now is to increase this limit to a million card holders. And even then, this is not enough because we have one customer in India that we believe is going to be at 1.2 million card holders within the next 18 months,” he added. “So, ultimately, the goal I’ve given the engineering team is to increase the number of active card holders managed by our system to 10 million.” 

Move to the Cloud

Another highlight from Connect'DX was a panel discussion on the evolution of the cloud in physical security and why many organizations are now taking a so-called “cloud first” approach to system implementation. Among the panelists included Daniel Lanecki, Head of Technology for Barclays Bank; Traegon Hon, Director of Global Physical Security, Starbucks; and Joel Hosino, Physical Security Systems Manager, Uber.

According to Hon, there were a couple of main drivers that led Starbucks to moving its VMS to the cloud. The first driver, according to Hon, had to do with the availability of systems.

“We have a fairly distributed environment, supporting a little over 10,000 locations in our security systems today and we do experience failures and most of those are related to hardware being installed in those locations that we can’t always react to fast enough,” he said. “What we want do is remove some of those failure points from our security technology to be able to create a more available environment.”

Secondly, Hon added that they also wanted to be able to make this security data available for other use cases in the organization. “What we find is that when we distribute this data out to the edge, it becomes harder to process in a way to provide some of that intelligence that we want to,” he said.

For Barclays, Lanecki said that in evaluating their legacy products, many of which were on standalone platforms, it had become a challenge to maintain those systems logistically. “As we started looking at how much effort it takes and the value that we could get out of the legacy system, we started looking at what other options were available for us and as we started looking at on-prem solutions and how much that would cost to upgrade or replace,” he said.  "We started comparing that to cloud platforms and I think the driver or decider for us was maintaining security from a cyber perspective but also from a manageability perspective,” he said.

At Uber, Hosino said they had similar concerns relative to information security and ongoing maintenance, however; what differed for their organization was a lack of available network infrastructure at many of their locations.

“Being able to deploy a technology that no longer requires corporate network backbone, allowing me to push video straight to the cloud… and encrypt the data, run analytics against the data, but without that requirement of the corporate network infrastructure, that’s one of the big driving forces for us,” he said.

To learn more about Connect'DX or to watch on-demand sessions from the event, click here.