HRL-Det: Hierarchical Reinforcement Learning for Sequential Object Detection in Aerial Imagery.

Overview

1. College of Geography and Environment, Xianyang Normal University, Xianyang 712000, China
2. School of Computer Science, National University of Defense Technology (NUDT), Changsha 410073, China

Abstract

Object detection in unmanned aerial vehicle (UAV) imagery suffers from severe scale variation, dense object packing, and prohibitive computational cost when conventional detectors exhaustively evaluate high-resolution frames. Reinforcement learning (RL)-based sequential detectors offer a promising alternative by formulating localization as an active search process, yet existing methods are limited by discrete-time state transitions, sparse reward signals, and premature policy collapse. In this paper, we propose HRL-Det, a hierarchical reinforcement learning framework that addresses these challenges through two tightly coupled innovations. First, a Neural ODE-driven Continuous-Time Bellman State Evolution module models the agent’s state dynamics as a stochastic differential equation governed by the Hamilton–Jacobi–Bellman equation, enabling fine-grained temporal reasoning with memory-efficient adjoint-based backpropagation. Second, a Lyapunov-Guided Entropy-Regularized Reward Shaping mechanism constructs convergence-promoting dense rewards informed by Lyapunov stability analysis while maintaining exploration diversity through maximum entropy optimization. Extensive experiments on VisDrone2019, DroneVehicle, and MS COCO 2017 show that HRL-Det achieves of 0.412, 0.812, and 0.735, respectively, outperforming existing RL-based detectors and achieving competitive accuracy relative to representative non-RL detectors under the same COCO metric, while requiring only 17.3 M parameters and an average of 6.3 search steps per object.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

The paper links to its data, not to its authors' code: see the Data section.

The paper's code and data availability statement is in the Data section.

Tracing map

A tracing map links a paper to the code its authors published: this paper has none, so it has no map.

Data

Datasets cited

• github.com/visdrone/visdrone-dataset — at github.com; found in “Data Availability Statement”

Data Availability Statement

All datasets used in this study are publicly available. VisDrone2019 is available at https://github.com/VisDrone/VisDrone-Dataset (accessed on 23 June 2026). DroneVehicle is available at https://github.com/VisDrone/DroneVehicle (accessed on 23 June 2026). MS COCO 2017 is available at https://cocodataset.org (accessed on 23 June 2026).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 2 authors, 8 keywords, 13 references.

Cite

This paper

Li, M., & Hu, Y. (2026). HRL-Det: Hierarchical Reinforcement Learning for Sequential Object Detection in Aerial Imagery. Sensors (Basel, Switzerland), 26(13), 4232. https://doi.org/10.3390/s26134232

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1162/imag.a.1249 [code]

Global search metaheuristics for neural mass model calibration.

Journal: Imaging neuroscience (Cambridge, Mass.)

In common: computational, 1 reference

[2] doi:10.1038/s41467-026-73825-3 [code]

Volatile self-selective memristive neuron for millisecond-latency neuromorphic object detection at the edge.

Journal: Nature communications

In common: 1 reference

[3] doi:10.1038/s44321-026-00488-4 [code]

DeepPlaque: a scalable multimodal platform for Aβ pathology and cell analysis in Alzheimer's disease.

Journal: EMBO molecular medicine

In common: 1 reference

[4] doi:10.1364/boe.605322 [code]

Generalized plaque digitization framework for multi-dimensional mesoscopic images.

Journal: Biomedical optics express

In common: 1 reference

[5] doi:10.3390/diagnostics16111705

Cross-Dataset Generalization of Deep Learning-Based Detectors for Intracranial Hemorrhage Subtype Localization on Noncontrast Head CT: A Comparative Study.

Journal: Diagnostics (Basel, Switzerland)

In common: 1 reference

[6] doi:10.1038/s41598-026-50240-8

High-precision brain tumor segmentation with switchable normalization in faster R-CNN architecture.

Journal: Scientific reports

In common: 1 reference

[7] doi: [code]

Going deeper with morphologically detailed neural networks by simulation-based gradient propagation

Journal: Frontiers in computational neuroscience

In common: computational

[8] doi:10.1523/eneuro.0283-26.2026 [code]

A Cortico-Basal Ganglia-Thalamic Network Model Linking Intermittent Postural Control to Sway-Related Beta-Band Oscillations.

Journal: eNeuro

In common: computational

[9] doi:10.1007/s00422-026-01063-3 [code]

Increased firing rates monotonically expand neuronal coding bandwidth.

Journal: Biological cybernetics

In common: computational

[10] doi:10.1007/s12021-026-09784-3 [code]

Estimating Mutual Information and Pearson Correlation on Neural Evoked Responses.

Journal: Neuroinformatics

In common: computational

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.