In a new and rapidly evolving era of warfare, the value of autonomous and uncrewed assets – across air, land, sea and subsea – in the modern battlespace is evident. This has been particularly apparent in Ukraine, where the use of low-cost, commercially available drones is widespread on both sides of the conflict.
However, using these types of platforms at scale comes with challenges. Specifically, manpower is proving to be a bottleneck, with militaries struggling to meet the growing requirement for specialised and highly trained operators. Indeed, it’s typically much faster to manufacture drones or most other uncrewed assets than it is to train people to effectively control them.
This highlights a real need for a solution that reduces the burden of traditional manual operation, which typically places high cognitive, visual and manual workload on operators. A solution that allows a wider range of personnel to interact with an asset or multiple assets simultaneously, without having to meet the high training thresholds associated with traditional, manual control of such assets. This is what will enable the full value of autonomy – and it’s exactly what we’ve been developing here in our Digital Intelligence business.
Unleashing intelligent autonomy at scale and speed
Using the latest advances in AI, like Large Language Models (LLMs), we’ve developed a flexible natural language interface that makes it easier than ever for military personnel to leverage autonomous and uncrewed assets for operational effect at speed and scale.
We’re developing a drone controller solution that uses LLMs and other machine learning models running at the edge. LLMs were specifically chosen for their proficiency at refined tasks like reasoning, including spatial and contextual, and code generation. We developed a bespoke evaluation framework for assessing an LLM’s suitability to a given use case, and the means to fine-tune or refine them if needed. The LLMs can understand the context of tasks and perceive changes in their environment. They are then supported by machine learning models that can recognise and classify objects in the environment in real-time.
These capabilities are combined with a user-friendly interface that allows operators to interact with the system through natural language, either by speaking or typing to suit different operational environments. Operators give an instruction and the system translates that natural human language into an actionable command that the asset understands based on the software’s reasoning of the operator’s requirements and contextual information.
The platform then executes that command autonomously – for example, performing a search of an area of interest – thereby freeing up the operator to carry out other activities or tasks. In the event that a potential threat is detected, the software will alert the operator and suggest an action. The operator can then ask for further detail or clarification, dynamically re-task the asset to perform another task, or simply confirm the action suggested by the system – thereby ensuring that humans stay in the loop throughout the process.
As well as overcoming technical and cost training barriers, our cutting-edge AI controller solution gives operators the situational awareness required to accelerate decision making in complex missions. It offers a tangible example of human-machine teaming in action, all designed to reach an end objective in the most effective way possible.
As far as we know, we are the first to demonstrate this novel human-machine teaming concept in live trails with multiple low-cost commercial-off-the-shelf uncrewed air and ground vehicles across several defence and security scenarios. Indeed, we went from concept, to demonstration, to live trials in less than nine months.
And our solution, fully developed in-house by our engineers, adopts an open architecture approach underpinned by leveraging commercially available and open source software in some cases. This means any LLM can be used and the solution can be applied to virtually any drone or uncrewed asset that allows interaction through API calls. There’s no requirement on the platform to be proprietary or bespoke, just an open design and philosophy that removes potential issues such as vendor lock-in or platform-specific incompatibilities.
Building in assurance
As with any AI-based solution, implementing various guardrails and assurance frameworks formed an important part of the development process.
For example, at BAE Systems we’ve been developing suitable assurance frameworks that support AI-enabled autonomy across all domains and are workings with SMEs that specialise in assuring autonomy. We also ensured that the system can only operate within specific operational parameters, such as a determined area of surveillance, and the assets themselves feature low-level safety mechanisms (e.g. geofencing and collision avoidance) that the solution can’t override or access.
The AI drone controller solution also can’t go beyond the language of the platform it’s connected to and declared to it. It can only call on a platform’s existing capabilities – for example take-off, land, search, hover etc. – and operators always have access to the AI’s decision making to ensure full transparency.
Finally, the solution itself is secure by design. It operates on a completely private network that is locked down behind a firewall with no internet access, ensuring no unauthorised access or interference by third parties, including with the operated platforms or assets.
What’s next?
Our robotics, AI and software engineering specialists have worked incredibly hard over the last few months to get to this point – fusing our hardware and software expertise as part of a multidisciplinary team. But this is just the beginning.
Over the coming months, we’ll conduct further testing and verification to mature the technology, before we start to explore integrations with BAE Systems’ range of autonomous and uncrewed platforms, assets and offerings. We’re also looking to extend the capability to drone swarms to amplify the capacity of the UK and allied armed forces to collect intelligence and deliver effect without putting humans in harm’s way. This will build on existing work we’ve done in swarming, including novel algorithms that can be seamlessly integrated with our natural language interface.
Ultimately, our focus is on helping the UK Government meet its ambition to get AI innovations out to the front line faster. The fact that we went from concept, to demonstration, to live trials in less than nine months showcases the speed of innovation – along with our continued dedication to meeting the UK’s mission requirements of today and tomorrow.