Deep learning for BIPV segmentation on facades: Comparison with human annotations across facade designs

Ayca Duran1, Pedram Mirabian1, Panagotis Karapiperis1, Christoph Waibel12, Bernd Bickel3, Arno Schlueter1
1Architecture and Building Systems ETH Zurich, 2Flemish Institute for Technological Research (VITO), 3Computational Design Laboratory ETH Zurich
Building and Environment 2026

Abstract

Building-integrated photovoltaics (BIPV) on facades are a significant but underutilized source of solar energy in urban environments. Automating the recognition of BIPV on facades through vision based systems can help guide design recommendations and extend solar asset maps. Unlike rooftop photovoltaic (PV) systems, facade BIPV recognition is difficult due to limited visibility in overhead imagery, high visual variability, and the absence of structured datasets. This study proposes a method based on deep learning (DL) for automated segmentation of BIPV panels on building facades using street-level and web images. A new dataset comprising 400 annotated BIPV projects was created, including detailed pixel-level masks and project attributes. Two model architectures, Mask Region-based Convolutional Neural Network (Mask R-CNN) and SegFormer, as well as human baselines are evaluated. The SegFormer model outperforms Mask R-CNN in pixel-level metrics. A user study conducted with human annotators without domain-specific expertise provides comparative insight into human performance, revealing common challenges in recognizing facade BIPV. The results demonstrate that DL models, trained specifically for this task, can segment BIPV panels more accurately than mean of human annotators, with SegFormer achieving an IoU of 0.69 compared to 0.42. The user study suggests that BIPV with satin finishes, invisible cells, and a PV-to-facade ratio of more than half challenge human recognition and therefore can be prioritized in visually sensitive areas. The outputs of the segmentation model are also utilized to estimate the BIPV energy yield. The annotated dataset and models are made available to facilitate future research.

Citation

				
					@article{duran2026deep,
  title={Deep learning for BIPV segmentation on facades: Comparison with human annotations across facade designs},
  author={Duran, Ayca and Mirabian, Pedram and Karapiperis, Panagiotis and Waibel, Christoph and Bickel, Bernd and Schlueter, Arno},
  journal={Building and Environment},
  pages={114292},
  year={2026},
  publisher={Elsevier}
}
Paper