Bringing real autonomy to battlefield drone swarms

Conflicts in Ukraine and Iran have shown how drones have carved out a role on the modern battlefield, which will only grow as technology advances and less expensive options proliferate.

Regardless of the role drones are filling on the battlefield today, everything from information gathering to facilitating logistics and combat operations, how effectively they carry out that role in the future will come down to how much cognitive load these machines place on our warfighters in the field. Palladyne AI describes the technology that enables machines to sense, reason, and act in the physical world as decentralized embodied collaborative AI, or DECA. In DECA-equipped drones, the intelligence resides on each aircraft, with each drone able to dynamically react in real time to changing circumstances without human direction.

The first thing to understand is that the words “unmanned” and “autonomous” are not synonymous, according to Ben Wolff, president and CEO of Palladyne AI .

“A lot of folks in the industry refer to waypoint following or coordinated pre-programmed flight as autonomy,” Wolff said. “I take issue with that because autonomy means the machine is making real-time decisions, not merely following a pre-programmed script where human operators have made all of the decisions in advance. That’s a critical distinction when we are talking about the unpredictable nature of the battlefield.”

Real autonomy goes well beyond predetermined routes or checklists of tasks for drones to carry out. The ability to reduce the cognitive load on operators and to deliver a true force-multiplier capability requires real autonomy, not simply unmanned capabilities. But just like with autonomous or self-driving cars, there are different levels of autonomy and it is critical that the industry gets on the same page about what level is being required and offered by various solutions.

Wolff points out that the five levels of UAV autonomy closely track the commonly adopted levels of autonomy for self-driving vehicles.

At the most basic level, most modern automobiles will at least have cruise control. The next step beyond that would be adaptive cruise control, where a vehicle slows down or speeds up based on traffic flow and proximity of other vehicles. At the other end of the spectrum would be full self-driving, where the driver hits a button and the car makes every decision along the way, though the driver still has the ability to intervene.

“We see the same thing in drones,” said Wolff. “Level 0 drone autonomy is where the drone is being directed with its every movement by the human operator by remote control. But let’s say there’s a big wind gust and the drone self-rights in the wind gust. It has an automatic stabilization capability. That’s the equivalent of that adaptive speed control in a car. It can right itself automatically. It’s all preprogrammed.”

“At the other end of the spectrum, you’ve got a drone where the soldier launches the drone and gives it a high-level mission. Let’s say the mission is canvass this 20-square-mile area for a vehicle that is emitting a certain type of signature, meaning an RF signature or an energy signature, acoustic signature, whatever it might be. You hit the button after giving it that high-level mission profile and it just goes and does its job entirely.”

Between Level 0 and Level 5 are levels that reflect less and less input required by the human in order for the drone to complete its mission.

Regardless of the level of autonomy, both now and in the future, Wolff believes autonomous drones should always enable human intervention. In his view, critical decisions such as authorization to impact a target with a kinetic device should always require a human in the loop, although he acknowledges that our adversaries might not always agree.

“The ability to abort or to proceed with an attack should, in my view, always reside with a human,” said Wolff. “I fear that someday, we may have an adversary that doesn’t share my perspective.”

The latest advantage of drones on the battlefield is deploying them in swarms to develop better situational awareness. With most low cost drones having limited computer and sensors on board, and considering that a single drone can only be in one place at one time, Wolff suggests that each drone possesses a unique perspective and that by sharing information among all drones, each drone in the swarm can leverage the intelligence of the whole. “This concept is no different than the concept of a human team,” says Wolff. “The value and capabilities of the collaborative team is much greater than the sum of its parts.”

Palladyne AI developed its SwarmOS software to make collaborative intelligence in a heterogenous drone swarm a reality by enabling disparate drones to share high-value curated information. Similar to a squad of soldiers in the field, drones can carry out their individual mission while sharing information with other drones, enabling better situational awareness and mission effectiveness. The company’s algorithms reside on the individual drone, so it can act autonomously with no centralized control.

“What’s cool about that is it doesn’t have to be seven, eight, 10 or 20 drones all from the same manufacturer,” said Wolff. “In the most recent Army PC-C6 (Project Convergence Capstone 6) exercise, we had drones from multiple manufacturers, all with our SwarmOS software on them, and the drones were collaborating together to complete the mission using limited amounts of information from the sensors on each of those drones.”

The software stack on each drone enables it to make decisions and react quickly in real time without having to wait for instructions. Wolff uses the example of playing catch: if someone throws a baseball to you, you aren’t going to consciously think about your arm and hand movement. Rather, you will simply react by lining up your glove to catch the ball.

“You just do it in a split second,” he said. “That’s what we’re embedding in these drones. They can act in real time responding to what’s going on in the battlefield. The fact that it’s on the edge, not centralized, the fact that it is a closed-loop sense, reason and act system; and then the fact that it can collaborate with other drones using that same sense, reason, and act system, that’s what makes it unique.”

Palladyne AI is developing what Wolff describes as “Oracle-Class Wolf Pack Swarming.” “Our algorithms are predictive in nature,” he said. “We can actually leverage the sum of knowledge across all of the drones to be able to use our algorithms to predict what the target might be doing next or what our friendly colleagues on the team might be doing next.” He gave an example of a swarm tracking a convoy: “The convoy went under a canopy of trees. Where is it likely to come out based on all the information we have about it was going when it went under the canopy?” “Oracle class to us is a fancy way of saying predictive,” Wolff said. “I think that is the holy grail. When we can start predicting what the enemy is going to do… that adds a whole other element of capability.”

While building better intelligence in drones means pilots will have less of a role, they will still be needed for decisions such as defining the mission parameter and whether to attack a target. As drone swarms grow, it is not practical, from either a personnel or an economic standpoint, to have one pilot for each drone. With demand signals from the Department of War focused on increasing the volume of drones in the arsenal, that creates a need for both pilots and solutions that will lessen the load on those pilots and enable them to manage large numbers of drones at once.

“Who’s going to fly all of them and how are they going to be flown as part of a collaborative tactical effort?” asked Wolff. “If you think about wanting to launch a thousand drones on the battlefield all at once, do we have a thousand pilots that are trained on that particular type of drone to be able to fly them? The answer is probably not.”

Palladyne AI’s software aims to solve that challenge by making large fleets of drones manageable by a much smaller number of pilots, operating from a controller rather than juggling separate controls for each drone. Because drones can share information and make decisions by themselves, pilots are not needed to approve every deviation or decision by a drone to meet its mission objectives. Operators can then focus on higher-level decisions, like confirming a target or redirecting a mission, while the fleet’s shared information gives them the clearest possible picture to make those decisions.

The key is leveraging collective sensor information from all drones in a particular region of interest to give pilots the best situational awareness possible. To do that, drones must work together effectively. “That doesn’t mean they have to be from the same vendor or model. Through SwarmOS, Palladyne AI lets different kinds of drones, sensor payloads and assets share what they see with each other and with the pilot, building a fuller picture together,” said Wolff.

“Using the drones in the sky much as the way we use teams of soldiers on the field, each of them is able to observe what’s going on in the battlefield,” he said. “The drones share information through the mesh network created by the drones and the team as a whole is able to be more effective.”

Airborne drones are currently the most common application for uncrewed vehicles, in large part because operating on the ground or in the water is mechanically more difficult, given challenges such as terrain avoidance. Airborne drones can operate along all three axes, X, Y, and Z, allowing them to change altitude to avoid obstacles.

But according to Wolff, the next big thing is extending that kind of autonomy to every domain, whether it’s land, sea, space or air. He sees the real challenge ahead as integrating and bringing systems across those domains together into one coordinated, unified battlefield picture that gives commanders and warfighters a far clearer sense of what’s happening and how to respond.

“There will be a day in the not-too-distant future where we’re not only incorporating data from all domains, but we’re actually causing embodied AI to manage all of these machines in a collaborative function, frankly, in a way that even we humans today are challenged by.”