In the spring of 1997, a group of engineers and security consultants from Wells Fargo Alarm Services (now part of ADT), including myself, surveyed the Waste Isolation Pilot Plant in Carlsbad, N.M. under the jurisdiction of the Defense Nuclear Facilities Safety Board (DNFSB).
WIPP is the world's first underground repository licensed to safely and permanently dispose of transuranic radioactive waste left from the research and production of nuclear weapons. Our team had just concluded the safety briefing required before our descent, in breathing apparatus, to survey areas like disposal rooms mined 2,150 feet underground in a 2,000-foot-thick salt formation that has been stable for more than 200 million years.
Under the Atomic Energy Act of 1954, DNFSB is charged with providing independent safety oversight of the DFFSB, a U.S. Department of Energy a complex that designs, manufactures, tests, maintains and decommissions nuclear weapons.
Before the survey could start, there was sudden a delay. “We’re on hold for a while, there seems to be an aircraft doing some illegal surveillance,” said a member of the U.S. Dept. of Energy Carlsbad Field Office.
It turned out to be a false alarm, as drug traffickers -- unaware of the facility -- are often heard near the perimeter looking for suitable drop locations. The Department of Energy’s contractors, Nuclear Waste Partnership, are tasked with keeping access to the multiple underground levels of the pilot plant safe and secure.
A quarter century later, teams tasked with securing several of the nation’s long-range intercontinental ballistic missile facilities -- complete with under- and above-ground logistics pathways -- were once again under direct aerial surveillance from a long range, persistent, high-altitude surveillance vehicle (PHASV), also known as the “spy balloon” from China (see figure 1).
On Wednesday, the U.S. intelligence community linked the Chinese surveillance balloon shot down last Saturday to a vast surveillance program run by China’s People’s Liberation Army. U.S. officials have begun to brief allies and partners who have been similarly targeted, according to a report from the Washington Post.
The vehicle is a potential example of sensor fusion, combined with a positioning platform (item 7), powered by a solar array (item 10) and communicating via radio masts (item 6). As in the case of drone platforms, the sensor array contains high resolution, long-range visible light (item 8) and thermal imaging devices (item 9).
Here is one example of this technology: NASA plans to send an 8.4-foot telescope into the upper stratosphere aboard a “football stadium” sized balloon. The ASTHROS (astrophysics stratospheric telescope for high spectral resolution observations at submillimeter-wavelengths) “balloon” is similar to the PHASV in that both are designed to reach an altitude of 24.6 miles, or about 130,000 feet, requiring a helium-inflated balloon around 400 feet wide.
Both use far infrared to deliver thermal imagery from great distances. The NASA telescope and sensor array looks up to view “star killers,” while the PHASV payload has multiple sensors able to be aimed via moveable platform or turret.
Explaining Sensor Fusion
Visual intelligence of incredible detail can be obtained by analyzing and “fusing” thermal and visible-light still images at similar focal lengths. If the launch facility does not use buried thermal elements to mask long-range thermal imaging, underground paths may appear “cooler.”
Visible light camera images of areas above underground structures may also reveal terrain changes. Combine the two, and use algorithms with Convolutional Neural Networks, and the most probable locations of hidden structures may be revealed.
But how can visible light sensors on board a high-altitude vehicle obtains sharp images? The camera uses the integration of two overlapping sensors and two lenses. For example, the software integration of two overlapping images from 150-megapixel (MP) imagers can generate a single, sharp, 280 MP image (see Figure “Camera Array Over Target)
To obtain long-range detection at high altitude in wide-ranging weather conditions, a camera with a cooled thermal imager is paired with a high-focal-length lens, integrated into an electro-optical infrared tracking system. Compare this with the equipment typically present on Vertical Takeoff and Lift (VTOL) drones (see figure 1, item 3) that typically flies at a maximum altitude of 400 feet.
Costing about $50,000 and 15-20 feet long, VTOL drones use a combination of fuel and electric propulsion and are designed for carrying heavy payloads up to 100 pounds, which is extremely useful in search-and-rescue missions.
Both this craft and smaller, fully electric hexacopter rotor drones (see item 4) use single-image HD, 4K and 8K visible light cameras and lower-cost, uncooled thermal imaging sensors mounted in a gimbal. The combination of a 2-axis equalizer, anti-vibration mount and camera gimbal virtually eliminate image degradation, and the technology is used by both cinematographers and terrorists alike.
The solution provider that most closely fits this process from multiple cameras is Phase One, with their 4-Band (RGB + NIR), or 4-band red-green-blue + near-infrared imaging solution.
The most likely workflow is as follows:
1. Move the aerial vehicle to a position as close to the vertical axis of the target in question.
2. Adjust the “gondola’s” imager array.
3. Verify imager array is within ground speed and altitude constraints.
4. Capture & Store RGB and NIR images.
5. Transmit images to nearest collection point; relay to processing location.
6. Run capture software, match NIR to RGB images, apply computer vision algorithms.*
7. Verify location, altitude and ground speed data integrity.*
8. Select most accurate, integrated image.*
9. Apply various terrain, structure algorithms to identify scene characteristics.*
10. “Stitch” new known location “patch” to existing digital mapping (cartography).*
*Function not performed on aerial vehicle.
Heavy Lifting
Heavy-lift drones are large drones with the ability to carry and operate 10 pounds or more of heavy payloads used to transport cargo, food, medical supplies, weapons and, in some cases, IEDs.
Multi-rotor drones, such as heavy quadcopters, hexacopters and octocopters, are characterized by their simplicity of assembly and deployment, while VTOL drones have a larger payload capacity and longer flight times.
To generate the massive lift required, counter-rotating drone motors are generally used, along longer, carbon fiber blades for greater stability.
LiDAR as a Tool
At these altitudes, aerial LiDAR sensors are typically included and are popular at crime scenes, archaeology, agriculture, inspections, irrigation, terrain modeling/mapping and delivery of arms, ammunition, IEDs, and chemical weapons.
The advancements of LiDAR in traffic, pedestrian safety and perimeter security have been reported by SecurityInfoWatch.com in the website’s CES 2023 Part 2 coverage. The adaptability of LiDAR solutions to “heavy lift” drones is a matter of time, power requirements and packaging.
AEye is a total-solution provider, from their 4Sight computer vision AI software to software-defined LiDAR. In a recent test at the American Center for Mobility (ACM) test track in Ypsilanti, Mich., their LiDAR sensor achieved detection at a 1,000-meter range in rain behind windshield glass.
LiDAR also works in darkness, allowing it to see through objects in the ground and overgrown areas of farmland or shallow water.
LiDAR produces an incredibly detailed point cloud. Since the flight altitude affects the density of the point cloud, different use cases may require different point-cloud densities. Tactical-range sensors are perfect for flying below 150 feet above ground level. For higher altitudes, mid- or long-range LiDAR models are selected.
From Search and Rescue
to Contraband Delivery
Heavy-lift drones equipped with LiDAR can be used to locate survivors and wreckage in flooded areas, and they are especially helpful in mountainous regions or complex terrain.
A LiDAR drone can be used to survey terrain and perform terrain-modeling tasks, such as assessing the vegetation of an area. Areas around the southern U.S. and northern border crossings with Canada have the most complex terrain and harshest environments, making the requirements of a tactical-use drone by U.S. Customs and Border Patrol similar to the VTOL heavy lift vehicle.
In the final day of its flight, the PHASV vehicle from China left landfall in the southeastern U.S. In its first air to air sortie of this type, an F-22 fighter jet flying out Langley Airforce Baser shot down the balloon using single sidewinder missile.
There were some questions if there was enough of a heat signature with the balloon, although the mission was a success as it was neutralized at an altitude of 58,000 feet. The debris plunged into the Atlantic Ocean off the coast of South Carolina and is being recovered and hauled to the FBI’s lab in Quantico, Va. for analysis.
Steve Surfaro is Chairman of the Public Safety Working Group for the Security Industry Association (SIA) and has more than 30 years of security industry experience. Follow him on Twitter, @stevesurf.
