Development of a method for localizing objects in a closed and saturated environment

The purpose of the work is to study and develop methods for localizing an ultra-light unmanned aerial vehicle (UAV) in a closed environment saturated with objects, based on semantic and topological data obtained from the environment. The purpose of the work is also to develop software and select a hardware complex for launching and testing the developed solution.

Methods. To achieve this goal, a review and comparison of existing solutions were conducted. Optimization of the neural network architecture for detecting objects. Development of an algorithm for compiling a graph of objects reflecting their relationships. Development of an algorithm for comparing graphs to determine the position of the UAV. Implementation of a solution to improve the accuracy of determining the geometric center of detected objects. Use of keypoint detection methods (SIFT, SURF) to solve the problem of identifying objects of the same class.

Results. The result of the work is a developed localization method based on semantic and topological data obtained from the environment. A software package based on the ROS2 humble platform and implemented on the hardware based on the Rockchip 3588 board was also developed. The experiments were conducted on ready-made datasets (KUM dataset) and using UAVs indoors.

Conclusion. The developed localization system is a promising step towards creating efficient and flexible systems capable of operating in complex conditions. In the future, it is planned to integrate this method with other sensors to improve robustness in dynamic conditions, add visual odometry algorithms to improve the accuracy of UAV localization, and expand the application of the system to UAVs used in other industries (infrastructure inspection, search and rescue).

1. Candiago S., Remondino F., De Giglio M., Dubbini M., Gattelli M. Evaluating Multispectral Images and Vegetation Indices for Precision Farming Applications from UAV Images. Remote Sensing. 2015. https://doi.org/10.3390/rs70404026.

2. Duan T., Chapman S.C. Dynamic monitoring of NDVI in wheat agronomy and breeding trials using an unmanned aerial vehicle. Field Crops Research. 2017. Available at: https://www.researchgate.net/publication/317423059_Dynamic_monitoring_of_NDVI_in_wheat_agronomy_and_breeding_trials_using_an_unmanned_aerial_vehicle

3. Tan Yi., Li S. Automatic inspection data collection of building surface based on BIM and UAV. Automation in Construction. 2021. Available at: https://www.sciencedirect.com/science/article/abs/pii/S0926580521003320?via%3Dihub

4. Mader D., Blaskow R., Westfeld P. Potential of uav-based laser scanner and multispectral camera data in building inspection. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2016. Available at: https://isprsarchives.copernicus.org/articles/XLI-B1/1135/2016/

5. Ibrahim Motawa, Alexandra Kardakou. Unmanned aerial vehicles (UAVs) for inspection in construction and building industry. The 16th International Operation & Maintenance Conference. 2018. Available at: https://pure.ulster.ac.uk/en/publications/unmannedaerial-vehicles-uavs-for-inspection-in-construction-and-

6. Motawa I., Kardakou A. Unmanned aerial vehicles (UAVs) for inspection in construction and building industry. The 16th International Operation & Maintenance Conference. 2018. Available at: https://pure.ulster.ac.uk/en/publications/unmanned-aerial-vehiclesuavs-for-inspection-in-construction-and-

7. Handley D.D., Yu P.L., Wechsler S.P. GPS-Denied UAV Localization Using Satellite Navigation and Inertial Sensors. International Conference on Robotics and Autonomous Systems. 2019. Available at: https://www.researchgate.net/publication/335140533_GPS-Denied_UAV_Localization_using_Pre-existing_Satellite_Imagery

8. Hacohen S., Medina O. Improved GNSS Localization and Byzantine Detection in UAV Swarms. Sensors. 2020. https://doi.org/10.3390/s20247239.

9. Marais J., Ambellouis S., Flancquart A., Lefebvre S., Meurie C., Ruichek Y. Accurate Localisation Based on GNSS and Propagation Knowledge for Safe Applications in Guided Transport. Procedia - Social and Behavioral Sciences. Available at: https://www.sciencedirect.com/science/article/pii/S1877042812027930

10. Cadena C., Carlone L., Carrillo H., Latif Y., Scaramuzza D., Neira J., Reid I., Leonard J.J. Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Trans. Robot. 2016. https://doi.org/10.1109/TRO.2016.2528477

11. Khairuddin A.R., Haron H. Review on simultaneous localization and mapping. 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE). Available at: https://ieeexplore.ieee.org/document/7482163

12. Sumikura S., Shibuya M., Sakurada K. OpenVSLAM: A versatile visual SLAM framework. Proceedings of the 27th ACM International Conference on Multimedia. 2019.

13. Labb´e M., Michaud F. RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics.

14. Kinnari J., Verdoja F., Kyrki V. GNSS-denied geolocalization of UAVs by visual matching of onboard camera images with orthophotos. 20th International Conference on Advanced Robotics (ICAR). 2021. Available at: https://arxiv.org/pdf/2103.14381.pdf

15. Mao J., Zhang L., He X., Qu H., Hu X. Precise Visual-Inertial Localization for UAV with the Aid of A 2D Georeferenced Map. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Available at: https://arxiv.org/ftp/arxiv/papers/2107/2107.05851.pdf

16. Klein L.C., Klein L.C., Klein L.C. A Machine Learning Approach to Robot Localization Using Fiducial Markers in RobotAtFactory 4.0 Competition. Sensors. 2023. Available at: https://www.mdpi.com/1424-8220/23/6/3128

17. Cao Q., Zhang Y., Liu Y., Lv G. Semantic loop closure detection based on graph matching in multi-objects scenes. Journal of Visual Communication and Image Representation. 2021. https://doi.org/10.1016/j.jvcir.2021.103072

18. Gawel A., Del Don C., Siegwart R., Nieto J., Cadena C. [X-View: Graph-Based Semantic Multi-View Localization. IEEE Robotics and Automation Letters. 2019. Available at: https://ieeexplore.ieee.org/document/8281068

19. Object Detection on COCO test-dev Leaderboard]. Available at: https://paperswithcode.com/sota/object-detection-on-coco