Automatic recognition of UML diagrams in images: Approaches, trends, and challenges

Abstract

The purpose of the study was to analyse and generalise modern methods for recognising UML diagrams in images. The main focus was on automated extraction of text and graphic elements to further reproduce models in text formats. The research methodology covered the analysis of scientific publications, which included 23 papers available in open sources. The study focused on exploring existing approaches to recognising UML diagrams in images. Analysis of scientific publications has shown what modern methods of UML diagram recognition allow achieving more than 90% accuracy in recognising UML diagrams in images. The advantages, limitations, and effectiveness of classical algorithms for computer vision, machine learning, and deep neural networks were investigated. It was found that the best results in classification were provided by deep neural networks, while classical algorithms remain effective for interpreting and extracting elements of UML diagrams. It was found that the main areas in the field of UML diagram recognition are classification of UML diagram types, and interpretation and conversion of UML images to text formats. The main problems were identified: poor image quality, limited training data, and format variability. Possible areas of further research are presented, such as creating large annotated sets of UML diagrams to improve accuracy, and summarising modern approaches to support recognition of more chart types. The findings will contribute to improving the automation processes for working with UML diagrams, and provide an understanding of the current state of the information technology and software development industry, opening up new prospects for development