End-to-end wide-baseline drone localisation using camera pose

Why Edge AI

Running a wide-baseline relative-pose network is compute-intensive — yet a drone that has just lost GNSS is exactly where cloud inference breaks down: the uplink may be jammed, unavailable or too slow, and offloading perception to a server adds latency a flight controller cannot tolerate. The answer is Edge AI: executing the trained model directly on the aircraft, at the point where the images are captured. Bringing the intelligence to the sensor — instead of the sensor data to a server — delivers: 

• Deterministic, low latency — perception and pose estimation run in a fixed time budget, inside the control loop
• Full autonomy — no dependence on connectivity, cloud or ground station
• Energy- and weight-efficiency — the entire inference pipeline fits the power envelope of a small UAV
• Data stays on the drone — nothing has to be streamed off the platform to be understood 

To meet this edge-AI requirement, our group designed and built a dedicated on-board compute platform in-house — the bridge between our DirectionNet / E2ENet models and real flight in GNSS-denied airspace. 

A custom Edge-AI system-on-module

The platform is a compact system-on-module built around an AMD/Xilinx Zynq-7010 (XC7Z010) — a heterogeneous SoC that pairs a dual-core ARM Cortex-A9 with FPGA programmable logic on a single, low-power device. The ARM cores run a full embedded Linux stack and the high-level localisation software, while the FPGA fabric provides hardware acceleration for the AI model itself (detailed below). The result is real-time, on-device inference — no external computer, no ground station. 

Everything needed for autonomous, self-contained operation is integrated onto one board that mates to any drone carrier through its board-to-board connectors: 

• AMD/Xilinx Zynq-7010 SoC — dual-core ARM Cortex-A9 (PS) + FPGA fabric (PL) for on-device AI acceleration
• 1 GB DDR3L memory, 16 MB QSPI and 4 GB eMMC for fully standalone boot
• Gigabit Ethernet, USB 2.0 (ULPI) and integrated UART/JTAG debug (FTDI)
• On-board DC/DC supplies generating all rails (+3V3, +1V8, +1V35, +1V0) from a single input
• Compact SoM form factor with dual mezzanine connectors — designed, routed and validated in-house at THI (hardware design: Emre Tsaliskan) 

Designing the platform ourselves means the compute, the sensors and the edge-AI models are developed as one integrated system, rather than being forced to fit off-the-shelf constraints. 

FPGA-based AI hardware acceleration

What makes real-time inference possible within a drone's power budget is hardware acceleration in the FPGA fabric. Rather than relying on the ARM cores alone, the compute-intensive parts of the perception network are mapped directly onto the Zynq's programmable logic, where they execute as dedicated, massively parallel hardware. This gives the platform: 

• Deterministic, real-time inference — a fixed, predictable latency, in the control loop
• High throughput-per-watt — far more efficient than a CPU-only solution
• A datapath tailored to the model — custom precision, quantisation and pipelining 

In effect, the FPGA turns the module from a general-purpose computer into a purpose-built AI accelerator sized for a small UAV — the hardware foundation of the whole edge-AI approach. 

Onboard deployment

For flight, the module is integrated into a compact, in-house sensor pod with an onboard monocular camera and LiDAR — the deployable, flight-ready form of the platform. 

All perception and localisation run entirely on the module inside the pod. 

From research to real GNSS-denied flight

Together with our DirectionNet / E2ENet models, the in-house module brings wide-baseline relative-pose estimation out of the dataset and onto the aircraft — running as edge AI, accelerated in the FPGA, in real time, precisely in the jammed and GNSS-denied environments that motivate this work.