FPGA Accelerator for Multiscale ORB Keypoint Extraction With Adaptive Nonmaximum Suppression

Keypoint extraction is a central stage in real‐time vision pipelines, but its computational cost can limit CPU‐only deployment. This paper presents a hardware/software codesign for ORB keypoint generation on AMD/Xilinx Alveo FPGA accelerators using Vitis HLS and the OpenCL heterogeneous programming model. The pipeline is implemented with two hardware kernels: an eight‐level multiscale Harris nonmaximum suppression (NMS) merge stage and an adaptive nonmaximum suppression (ANMS) stage. Host‐side nonblocking execution, pipeline slots, and compute‐unit replication are used to overlap transfers, FPGA execution, and CPU descriptor computation. The design is evaluated on the complete 48‐image Oxford Visual Geometry Group benchmark at the native sequence resolutions, ranging from 765 × 512 to 1000 × 700 pixels. In single‐lane execution, the accelerator reaches measured wall‐clock runtimes of 13.486 ms per image on Alveo U50 and 11.103 ms per image on Alveo U55C, corresponding to 11.01 × and 13.37 × speedups over the algorithm‐equivalent CPU reference. With compute‐unit replication, throughput reaches 206.764 frames per second on the U50 with three lanes and 297.524 frames per second on the U55C with six lanes. The evaluation further shows that the proposed detector preserves favorable keypoint localization and spatial distribution characteristics while supporting scalable real‐time processing.