Method and Algorithm for Recognition Dynamic Objects from a Mobile Platform, from Images Obtained in Different Spectral Ranges and Lidar Data

Purpose of research. In modern automatic information collection systems, autonomous mobile devices are increasingly used, data from which can be obtained in conditions hazardous to human health, from geographically remote places, in difficult meteorological conditions and in round-the-clock observation mode. For the autonomous operation of such devices, it is necessary to use methods and algorithms that allow you to build a map of the area, link a mobile platform to it, determine a route to a target point, highlight obstacles along the route and correct the route taking into account detected obstacles.
Methods. The article proposes a method and an algorithm for the selection of dynamic objects from a mobile platform, based on the analysis of data obtained from a multispectral camera, which allows the selection of obstacles, such as water, plant origin, technogenic nature, etc. with reduced computational complexity. To improve the accuracy of determining the coordinates of detected objects, a laser rangefinder is used.
Results. We consider the well-known methods of multispectral images recognition and present their comparative analysis. A method and an algorithm for recognition dynamic objects from a mobile platform, from images obtained in different spectral ranges and lidar data are proposed. Experimental studies were carried out to confirm the adequacy of the mathematical substantiation of the method, to reduce the error in calculating the coordinates of the object, at a distance of up to 100 meters to the object, RMSE - 0.447%, MAPE - 0.397, to increase the performance, it took 0, 04 seconds to select the object and determine its coordinates.
Conclusion. The article analyzes modern methods for recognizing multispectral images, presents the principles on which each method is based, gives advantages and disadvantages. The authors have developed a method and an algorithm that make it possible to identify static and dynamic obstacles along the route of a mobile platform, based on a sequence of images obtained in different spectral ranges. In the course of experimental studies, the performance of the proposed solutions and compliance with the specified requirements for accuracy and reliability were confirmed.

1. Hafizov R. G., Okhotnikov S. A. Raspoznavanie nepreryvnykh kompleksnoznachnykh konturov izobrazhenii [Recognition of continuous complex-valued image contours]. Izvestiya vysshikh uchebnykh zavedenii. Priborostroenie = News of higher educational institutions. Priborostroenie, 2012, vol. 55, no. 5 (In Russ.).

2. Degtyarev S. V., Spevakov A. G., Tipikin A. P. Opredelenie koordinat dvizhushchikhsya ob"ektov stereoskopicheskoi sistemoi tekhnicheskogo zreniya [Determination of coordinates of moving objects by the stereoscopic system of technical vision]. Telekommunikatsii = Telecommunications, 2004, no. 8, pp. 35 -36 (In Russ.).

3. Shirabakina T. A., Spevakov A. G. Stereoskopicheskaya optiko-elektronnaya si-stema opredeleniya parametrov dinamicheskikh ob"ektov v real'nom vremeni [Stereoscopic opticalelectronic system for determining parameters of dynamic objects in real time]. Datchiki i sistemy = Sensors and Systems, 2004, no. 6, pp. 65-67 (In Russ.).

4. Spevakova S. V. [The route of a mobile robot based on the analysis of multispectral data, in the collection: intelligent and information systems]. Intellektual'nye i informatsionnye sistemy. Intellekt-2019 [Intellect-2019 Proceedings of the all-Russian scientific and technical conference]. Tula, 2019, pp. 334-337 (In Russ.).

5. Spevakov A. G., Rubanov A. F. Stereoskopicheskaya optiko-elektronnaya sistema slezheniya [Stereoscopic optical-electronic tracking system]. Izvestiya vysshikh uchebnykh zavedenii. Priborostroenie = News of Higher Educational Institutions. Priborostroenie, 2005, vol. 48, no. 2, pp. 62-67 (In Russ.).

6. Spevakov A. G. Metod vydeleniya dvizhushchikhsya ob"ektov [Method of selection of moving objects]. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravlenie, vychislitel'naya tekhnika, informatika. Meditsinskoe priborostroenie = Proceedings of the Southwest State University. Series: Control, Computing Engineering, Information Science. Medical Instruments Engineering, 2013, no. 1, pp. 233-237 (In Russ.).

7. Davison A. J. et al. MonoSLAM: Real-time single camera SLAM. IEEE transactions on pattern analysis and machine intelligence, 2007, vol. 29, no. 6, pp. 1052-1067.

8. Newcombe R. A. et al. KinectFusion: Real-time dense surface mapping and tracking. 10th IEEE International Symposium on Mixed and Augmented Reality, IEEE, 2011, pp. 127-136.

9. Kerl C., Sturm J., Cremers D. Dense visual SLAM for RGB-D cameras. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014, pp. 2100–2106.

10. Jiang G. et al. A simultaneous localization and mapping (SLAM) framework for 2.5 D map building based on low-cost LiDAR and vision fusion. Applied Sciences, 2019, vol. 9, no. 10, pp. 2105.

11. Antyufeev V. I., Bykov V. N. Sravnitel'nyi analiz algoritmov sovmeshcheniya izobrazhenii v korrelyatsionno-ekstremal'nykh sistemakh navigatsii letatel'nykh apparatov [Comparative analysis of image matching algorithms in correlation-extreme navigation systems of aircraft]. Aviatsionno-kosmicheskaya tekhnika i tekhnologiya = Aviation and Space Technology and Technology, 2008, no. 1, pp. 70-78 (In Russ.).

12. Zubarev I. B., Sagdullaev I. S., Sagdullaev T. I. Spektrozonal'nye metody i sistemy v kosmicheskom televidenii [Spectrosonal methods and systems in space television]. Voprosy radioelektroniki. Seriya tekhnika televideniya = Questions of Radio Electronics. TV Technique Series, 2009, is. 1, pp. 47-64 (In Russ.).

13. Bondarenko M. A., Drinkin V. N. Otsenka informativnosti kombinirovannykh izobrazhenii v mul'tispektral'nykh sistemakh tekhnicheskogo zreniya [Evaluation of the information content of combined images in multispectral systems of technical vision]. Programmnye sistemy i vychislitel'nye metody = Software Systems and Computational Methods, 2016, no. 1, pp. 64-79 (In Russ.).

14. Krishnamoorthy S., Soman K. P. Implementation and comparative study of image fusion algorithms. International Journal of Computer Applications, 2010, vol. 9, no. 2, pp. 25-35 (In Russ.).

15. Kaluutsky I. V., Matyushin Yu. S., Spevakova S. V. Analysis of Modern Static Methods of Biometric Identification. Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta = Proceedings of the Southwest State University. 2019; 23 (1): pp. 84-94. (In Russ.). https://doi.org/10.21869/2223-1560-2019-23-1-84-94.

16. Bekhtin Yu. S., Emelyanov S. G., Titov D. V. Teoreticheskie osnovy tsifrovoi obrabotki izobrazhenii vstraivaemykh optiko-elektronnykh sistem [Theoretical bases of digital image processing of embedded optical-electronic systems]. Kursk, 2016 (In Russ.).

17. Spevakova S. V., Kalutsky I. V. Podvizhnoe stereoskopicheskoe ustroistvo vydeleniya dinamicheskikh ob"ektov [Mobile stereoscopic device for selecting dynamic objects]. Patent RF, no. 2018147106, 23.01.2020. (In Russ.).

18. Spevakova S. V., Matiushin I. S., Spevakov A. G. Detecting objects moving in space from a mobile vision system. Radio Electronics, Computer Science, Control, 2019, no. 4, pp. 103-110.

19. Kalutskiy I., Spevakova S., Matiushin I. Method of Moving Object Detection from Mobile Vision System. International Russian Automation Conference (RusAutoCon). IEEE, 2019, pp. 1-5.