Skip to main content
SpringerLink
Log in

End-to-End Vision-Based Cooperative Target Geo-Localization for Multiple Micro UAVs

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Compared to monocular target localization, cooperative target geo-localization can obtain the target’s three-dimensional geographic position in real-time without forwarding requirements. But precision-restricted sensors carried by micro Unmanned Aerial Vehicles (UAVs) lead to measurement error and sensor noise, the localization accuracy under this condition needs to be further improved. The random movement of the non-cooperative target further increases the difficulty of accurate localization. In this paper, considering observation noise and error, a multi-view vision-based cooperative target geo-localization method is proposed. This method estimates the target’s position end-to-end using Interactive Multi-Model Unscented Kalman Filter (IMM-UKF) through the multi-view observation of the target from UAVs. The initial state of the filter is calculated by intersection localization algorithm. Compared with other state-of-the-art cooperative localization methods, simulation and flight experiments show that the proposed cooperative localization method can effectively improve the accuracy of vision-based target geo-localization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhu, K., Han, B., Zhang, T.: Multi-UAV distributed collaborative coverage for target search using heuristic strategy. Guidance, Navigation and Control 1(1), 2150002 (2021). https://doi.org/10.1142/S2737480721500023

    Article  Google Scholar 

  2. Zhang, J., Zhang, Y.: A method for UAV reconnaissance and surveillance in complex environments. In: 2020 6Th International Conference on Control, Automation and Robotics (ICCAR), pp. 482–485. IEEE (2020), https://doi.org/10.1109/ICCAR49639.2020.9107972

  3. Chen, X., Liu, Y., Yin, L., Qi, L.: Cooperative task assignment and track planning for multi-UAV attack mobile targets. Journal of Intelligent & Robotic Systems 100, 1383–1400 (2020). https://doi.org/10.1007/s10846-020-01241-w

    Article  Google Scholar 

  4. Lee, S.J., Lee, D., Kim, H.J.: Cargo transportation strategy using T 3-Multirotor UAV. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 4168–4173. IEEE (2019), https://doi.org/10.1109/ICRA.2019.8794203

  5. Wang, A., Ji, X., Wu, D., Bai, X., Ding, N., Pang, J., Chen, S., Chen, X., Fang, D.: Guideloc: UAV-assisted multitarget localization system for disaster rescue. Mob. Inf. Syst. 2017(4), 1–13 (2017). https://doi.org/10.1155/2017/1267608

    Article  Google Scholar 

  6. Tang, D., Fang, Q., Shen, L., Hu, T.: Onboard Detection-Tracking-Localization. IEEE/ASME Transactions on Mechatronics 25(3), 1555–1565 (2020). https://doi.org/10.1109/TMECH.2020.2976794

    Google Scholar 

  7. Kyristsis, S., Antonopoulos, A., Chanialakis, T., Stefanakis, E., Linardos, C., Tripolitsiotis, A., Partsinevelos, P.: Towards autonomous modular UAV missions: The detection, geo-location and landing paradigm. Sensors 16, 1844 (2016). https://doi.org/10.3390/s16111844

    Article  Google Scholar 

  8. Qi, G.Q., Li, Y.Y., Sheng, A.D.: Virtual intersecting location based UAV circumnavigation and bearings-only target-tracking techniques. Inform. Sci. 505, 571–585 (2019). https://doi.org/10.1016/j.ins.2019.07.080

    Article  MathSciNet  Google Scholar 

  9. Pai, P., Naidu, V.: Target geo-localization based on camera vision simulation of UAV. J. Opt. 46, 425–435 (2017). https://doi.org/10.1007/s12596-017-0395-0

    Article  Google Scholar 

  10. Turner, D., Lucieer, A., Wallace, L.: Direct georeferencing of ultrahigh-resolution UAV imagery. IEEE Trans. Geosci. Remote Sens. 52, 2738–2745 (2013). https://doi.org/10.1109/TGRS.2013.2265295

    Article  Google Scholar 

  11. Kukreti, S.R., Kumar, M., Cohen, K.: Detection and localization using unmanned aerial systems for firefighting applications. In: AIAA Infotech @ Aerospace, 2016, 1903. https://doi.org/10.2514/6.2018-2136 (2016)

  12. Hinas, A., Roberts, J.M., Gonzalez, F.: Vision-based target finding and inspection of a ground target using a multirotor UAV system. Sensors 17, 2929 (2017). https://doi.org/10.3390/s17122929

    Article  Google Scholar 

  13. Sheng, X., Yafei, L., Zhongxi, H., Shangqiu, S., Zheng, G.: Passive geo-location for ground target with multiple measurements using fixed wing UAV. In: 2016 IEEE Chinese Guidance, Navigation and Control Conference (CGNCC), pp. 2477–2482. IEEE. https://doi.org/10.1109/CGNCC.2016.7829182 (2016)

  14. Gao, F., Ma, X., Gu, J., Li, Y.: An active target localization with monocular vision. In: 11Th IEEE International Conference on Control & Automation (ICCA), pp. 1381–1386. IEEE. https://doi.org/10.1109/ICCA.2014.6871125 (2014)

  15. Sun, S.Y., Zhu, H.M., Chen, J.C.: Research on target location method of UAV based on aided beacon in different field of view. In: 2018 IEEE 4Th Information Technology and Mechatronics Engineering Conference (ITOEC), pp.1767–1774. IEEE (2018), https://doi.org/10.1109/ITOEC.2018.8740659

  16. Zhang, R., Liu, H.H.: Vision-based relative altitude estimation of small unmanned aerial vehicles in target localization. In: Proceedings of the 2011 American Control Conference, pp. 4622–4627. IEEE (2011), https://doi.org/10.1109/ACC.2011.5991109

  17. Zhuo, J., Sun, L., Yang, Y., Zhao, X.: A target localization method for UAV image sequences based on DEM matching. In: 2016 9Th International Symposium on Computational Intelligence and Design (ISCID), pp. 215–218. IEEE (2016), https://doi.org/10.1109/ISCID.2016.2058

  18. Conte, G., Hempel, M., Rudol, P., Lundstrom, D., Duranti, S., Wzorek, M., Doherty, P.: High accuracy ground target geo-location using autonomous micro aerial vehicle platforms. In: AIAA Guidance, Navigation and Control Conference and Exhibit, pp. 6668. https://doi.org/10.2514/6.2008-6668 (2008)

  19. Cheng, H., Lin, L., Zheng, Z., Guan, Y., Liu, Z.: An autonomous vision-based target tracking system for rotorcraft unmanned aerial vehicles. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1732–1738. IEEE. https://doi.org/10.1109/IROS.2017.8205986 (2017)

  20. Zhou, Y., Tang, D., Zhou, H., Xiang, X., Hu, T.: Vision-based online localization and trajectory smoothing for fixed-wing UAV tracking a moving target. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 153–160. https://doi.org/10.1109/ICCVW.2019.00024 (2019)

  21. Xu, C., Huang, D.Q., Liu, J.Y.: Target location of unmanned aerial vehicles based on the electro-optical stabilization and tracking platform. Measurement 147, 1–13 (2019). https://doi.org/10.1016/j.measurement.2019.07.076

    Article  Google Scholar 

  22. Barber, D.B., Redding, J.D., Mclain, T.W., Beard, R.W., Taylor, C.N.: Vision-based target geo-location using a fixed-wing miniature air vehicle. Journal of Intelligent & Robotic Systems Theory & Applications 47, 361–382 (2006). https://doi.org/10.1007/s10846-006-9088-7

    Article  Google Scholar 

  23. Zhang, L., Deng, F., Chen, J., Bi, Y., Phang, S.K., Chen, X., Chen, B.M.: Vision-based target three-dimensional geolocation using unmanned aerial vehicles. IEEE Trans. Ind. Electron. 65, 8052–8061 (2018). https://doi.org/10.1109/TIE.2018.2807401

    Article  Google Scholar 

  24. Cheng, X., Chenglong, H., Daqing, H.: Geo-location for ground target with multiple observations using unmanned aerial vehicle. Transactions of Nanjing University of Aeronautics and Astronautics 35, 829–837 (2018). https://doi.org/10.16356/j.1005-1120.2018.05.829

    Article  Google Scholar 

  25. Frew, E.W.: Sensitivity of cooperative target geolocalization to orbit coordination. Journal of Guidance, Control, and Dynamics 31, 1028–1040 (2008). https://doi.org/10.2514/1.32810

    Article  Google Scholar 

  26. Fu, X., Bi, H., Gao, X.: Multi-UAVs cooperative localization algorithms with communication constraints. Math. Probl. Eng. 2017, 1–8 (2017). https://doi.org/10.1155/2017/1943539

    Article  MathSciNet  MATH  Google Scholar 

  27. Che, F., Niu, Y., Li, J., Wu, L.: Cooperative standoff tracking of moving targets using modified lyapunov vector field guidance. Appl. Sci. 10, 3709 (2020). https://doi.org/10.2514/1.30507

    Article  Google Scholar 

  28. Kwon, H., Pack, D.J.: Cooperative target localization by multiple unmanned aircraft systems using sensor fusion quality. Optim. Lett. 6, 1707–1717 (2012). https://doi.org/10.1007/s11590-011-0360-9

    Article  MathSciNet  Google Scholar 

  29. Bethke, B., Valenti, M., How, J.: Cooperative vision based estimation and tracking using multiple UAVs. Advances in cooperative control and optimization, pp. 179–189. Springer. https://doi.org/10.1007/978-3-540-74356-9_11 (2007)

  30. Bai, G., Liu, J., Song, Y., Zuo, Y.: Two-UAV intersection localization system based on the airborne optoelectronic platform. Sensors 17, 98 (2017). https://doi.org/10.3390/s17010098

    Article  Google Scholar 

  31. Xu, C., Yin, C., Huang, D., Han, W., Wang, D.: 3D target localization based on multi–unmanned aerial vehicle cooperation. Measurement and Control 2020, 1–13 (2020). https://doi.org/10.1177/0020294020922268

    Article  Google Scholar 

  32. Julier, S., Uhlmann, J.K.: A General Method for Approximating Nonlinear Transformations of Probability Distributions. Technical Report, University of Oxford, Oxford (1996)

  33. Zhu, P., Wen, L., Du, D., Bian, X., Hu, Q., Ling, H.: Vision meets drones: Past, present and future. arXiv:2001.06303 (2020)

  34. Bochkovskiy, A., Wang, C., Liao, H.: Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934 (2020)

  35. Tan, M., Le, Q.: Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)

  36. Alexey, B.: Darknet: Open Source Neural Networks in Python. In: GitHub. https://github.com/AlexeyAB/darknet (2022) Accessed on 2 Jan 2022

Download references

Acknowledgements

The authors would like to thank Zhaowei Ma, Helu Zhou from National University of Defense Technology for their assistance and efforts on onboard target detection. Thanks are also expressed to Shengde Jia, Tianqing Liu, Tengxiang Li and Huan Wang for their assistance in the flight experiments.

Funding

This work was supported by National Natural Science Foundation of China (grant numbers 61876187).

Author information

Authors and Affiliations

  1. College of Intelligence Science and Technology, National University of Defense Technology, Changsha, 410073, China

    Bosen Lin, Lizhen Wu & Yifeng Niu

Authors
  1. Bosen Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lizhen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifeng Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Bosen Lin, Lizhen Wu and Yifeng Niu conceived and designed the research; Bosen Lin and Lizhen Wu performed the experiments. Bosen Lin performed the programming and data analyses. Bosen Lin and Lizhen Wu edited and reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lizhen Wu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, B., Wu, L. & Niu, Y. End-to-End Vision-Based Cooperative Target Geo-Localization for Multiple Micro UAVs. J Intell Robot Syst 106, 13 (2022). https://doi.org/10.1007/s10846-022-01639-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01639-8

Keywords

Search

Navigation