Navigating Environmental Stochasticity in UAV Autonomous Flight: A Curriculum-Enhanced Deep Reinforcement Learning Framework and Sim-to-Real Considerations

Abstract

The deployment of Unmanned Aerial Vehicles in highly nonlinear and dynamic environments is subject to the profound structural limitations inherent in traditional heuristic-based obstacle avoidance paradigms; specifically, these conventional methods inevitably encounter severe computational bottlenecks when processing high-dimensional perceptual data streams. In an effort to transcend these deterministic constraints, this study investigates the efficacy of incorporating "soft action evaluation" into reinforcement learning architectures. During the initial training phase, the extreme sparsity of feasible trajectories within obstacle-dense scenarios led to severe policy collapse. To mitigate this algorithmic stagnation, we embedded a multi-modal feature extraction network, integrated with a curriculum learning mechanism designed to deliberately grade environmental complexity to stabilize the gradients of the value network. Given these observed empirical discrepancies, it becomes imperative to move beyond the mere practice of injecting superficial noise into sensor inputs. This necessitates a fundamental redefinition of robust control representations—implying, furthermore, that to fully validate these data-driven navigation behaviors within the adversarial context of the real physical world, further research into asynchronous action execution is required.