Ibrahim, Yahya and Szulovszky, Péter and Benedek, Csaba (2023) Masonry Structure Analysis, Completion and Style Transfer Using a Deep Neural Network. In: ICPR 2022: Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenge. Lecture Notes in Computer Science (13645). Springer, pp. 155-168.

Text ICPR_2022_workshop.pdf Download (13MB)

Abstract

In this paper, we present a novel deep learning-based fully automatic method for masonry wall analysis in digital images. The proposed approach is able to automatically detect and virtually complete occluded or damaged wall regions, it segments brick and mortar areas leading to an accurate model of the wall structure, and it can also perform wall-to-wall style transfer as well. Our method involves numerous sequential phases. Initially, a U-Net-based network is used to segment the wall images into brick, mortar, and occluded/damaged regions. Thereafter the hidden wall regions are predicted by a two-stage adversarial inpainting model: first, a schematic mortar-brick pattern is predicted, then the second network component adds color information to these areas, providing a realistic visual experience to the observer. Next, a watershed transform-based segmentation step produces accurate outlines of individual bricks in both the visible and the inpainted wall segments. Furthermore, we show that the second adversarial network can also be used for texture transfer: one can change the texture style of a given wall image, based on another wall image, and we can artificially color a schematic wall sketch map, based on the style of a sample wall image. Experiments revealed that the proposed method produces realistic results for various masonry wall types in terms of inpainting the occluded regions or style transfer.

Item Type:	Book Section
Subjects:	Q Science > QA Mathematics and Computer Science > QA75 Electronic computers. Computer science / számítástechnika, számítógéptudomány
Divisions:	Distributed Events Analysis Research Laboratory
Depositing User:	Csaba Benedek
Date Deposited:	15 Oct 2023 13:03
Last Modified:	15 Oct 2023 13:03
URI:	https://eprints.sztaki.hu/id/eprint/10584

Update Item