TransRAUNet: A Deep Neural Network with Reverse Attention Module Using HU Windowing Augmentation for Robust Liver Vessel Segmentation in Full Resolution of CT Images

Background: Liver cancer has a high mortality rate worldwide, and clinicians segment liver vessels in CT images before surgical procedures. However, liver vessels have a complex structure, and the segmentation process is conducted manually, so it is time-consuming and labor-intensive. Consequently, it would be extremely useful to develop a deep learning-based automatic liver vessel segmentation method. Method: As a segmentation method, UNet is widely used as a baseline, and a multi-scale block or attention module has been introduced to extract context information. In recent machine learning efforts, not only has the global context extraction been improved by introducing Transformer, but a method to reinforce the edge area has been proposed. However, the data preprocessing step still commonly uses general augmentation methods, such as flip, rotation, and mirroring, so it does not perform robustly on images of varying brightness or contrast levels. We propose a method of applying image augmentation with different HU windowing values. In addition, to minimize the false negative area, we propose TransRAUNet, which introduces a reverse attention module (RAM) that can focus edge information to the baseline TransUNet. The proposed architecture solves context loss for small vessels by applying edge module (RAM) in the upsampling phase. It can also generate semantic feature maps that allows it to learn edge, global context, and detail location by combining high-level edge and low-level context features. Results: In the 3Dricadb dataset, the proposed model achieved a DSC of 0.948 and a sensitivity of 0.944 in liver vessel segmentation. This study demonstrated that the proposed augmentation method is effective and robust by comparisons with the model without augmentation and with the general augmentation method. Additionally, an ablation study showed that RAM has improved segmentation performance compared to TransUNet. Compared to prevailing state-of-the-art methods, the proposed model showed the best performance for liver vessel segmentation. Conclusions: TransRAUnet is expected to serve as a navigation aid for liver resection surgery through accurate liver vessel and tumor segmentation.