A significant task in remote sensing is context-aware segmentation for high-resolution aerial images, which supports applications such as land cover classification, urban analysis, and environmental monitoring. In this paper, we propose a new framework directed to enhance the segmentation accuracy of aerial images by combining a split-depthwise convolution layer to handle high-resolution aerial images with lower computational complexity and a focus-based feature modulation module to emphasize spatial regions of interest. Our model was trained and tested on aerial images collected from geographically distinct areas within Alexandria. The results show that the model with our proposed framework exceeded the original U-net model in terms of accuracy and IoU value. Our framework provides an efficient solution for processing large-scale aerial image analysis and can be adapted for remote sensing applications, such as infrastructure development in Egypt, urban planning, and environmental monitoring.