The evolution of machine learning solutions that has resulted in the current generation of intelligent video analytics would not have been possible without the advent of other technologies that have greatly aided the ability of algorithms to detect and classify objects. For example, were it not for advancements in the image resolution capabilities of cameras and the additional processing power provided by GPUs (graphic processing units), today’s so-called artificial intelligence (AI)-powered analytics would be no more advanced than their traditional, rules-based counterparts that were introduced in the market more than a decade ago.
However, if the industry ever wishes to achieve the level of decision-making autonomy that many people think analytics will one day be capable of providing, it will require the development of technologies that can ingest and process images at even greater levels of detail. Enter Xfuse, a California-based developer of High Dynamic Range (HDR) Image Signal Processing (ISP) technology that made its official debut in the AI vision market late last month.
According to Alfred Zee, President and CEO of Xfuse, the company’s Phoenix ISP processor sends the footage captured by a camera through a “pipeline of corrections” to render a more accurate image of a scene.
“We take the raw image from the sensor, but most cameras today don’t do that. It goes through a de-mosaic process, which basically puts in the color, and then they process the corrections – whether it is gamma correction, color correction and that type of thing,” Zee explains. “What makes our technology unique is that we do all of the color processing at the end. We first take care of all the image correction in black and white and once all the dynamic range has been corrected for things like exposure and other types of requirements, then it goes through the de-mosaic and produces a much higher quality, much better color quality image than most. We can create a very high quality and very realistic type of dynamic range image in a video format.”
Currently, Zee says that many processors are only tuned to work with a specific set of sensors and that adding a new sensor to that system-on-chip (SoC) is simply too difficult, if not impossible. “What makes (Phoenix) flexible is that you have a downloadable ISP that you can tune and have available at any time for different types of sensors – whether it is for 4K (resolution), 2K or many other types of applications,” he adds.
For those tasked with upgrading an existing surveillance system to accommodate new analytic features or those types of things, Zee says use of the Phoenix ISP will aid them tremendously.
“If they want a new sensor or feature on the camera, that requires tweaking on the processor side. Typically, unless you are a huge company, you are not going to get that kind of service. This is FPGAs (field programmable gate arrays), so everything can be changed or tweaked (on the fly), and it makes it a very robust and flexible type of system,” Zee adds. “By having our ISP in a downloadable form, that would really make security developers’ lives much easier.”
While they are targeting their ISP tech for use in conjunction with a full range of commercial video surveillance deployments, Zee says the company has recently been in talks with a video doorbell maker about how it could be used with their hardware. “On that type of device, the processing is very limited and there are a lot of limitations as to what we can do in terms of memory and all of the available specs, but we are targeting the broad spectrum of surveillance products,” he explains.
Applications for Drones
Additionally, Zee believes their technology could be widely adopted in drones – both aerial and ground-based robotic units – to greatly improve their capabilities.
“Those kinds of applications require multiple cameras, they require lidar and other types of sensors, such as thermal to detect live versus inanimate objects or millimeter wave for weapons detection, for example,” he says. “For drones, that type of aerial surveillance is getting more popular in areas like agriculture (monitoring) and things like that.”
Zee expects the Phoenix ISP sensor to be deployed in its first video surveillance applications within the next several months and says they will continue to roll out new sensor configurations in the months ahead.
“In a year and a half or two, we will start introducing some lidar units along with some software for mapping and navigation,” he explains. “Our main targets are robotic and surveillance-type applications and really the sweet spot is the Knightscope-type devices, which we believe will be more prevalent in the years to come.”
Joel Griffin is the Editor of SecurityInfoWatch.com and a veteran security journalist. You can reach him at [email protected].
