As an important basis for lesion determination and diagnosis, medical image segmentation has become one of the most important and hot research fields in the biomedical field, among which medical image segmentation algorithms based on full convolutional neural network and U-Net neural network have attracted more and more attention by researchers. At present, there are few reports on the application of medical image segmentation algorithms in the diagnosis of rectal cancer, and the accuracy of the segmentation results of rectal cancer is not high. In this paper, a convolutional network model of encoding and decoding combined with image clipping and pre-processing is proposed. On the basis of U-Net, this model replaced the traditional convolution block with the residual block, which effectively avoided the problem of gradient disappearance. In addition, the image enlargement method is also used to improve the generalization ability of the model. The test results on the data set provided by the "Teddy Cup" Data Mining Challenge showed that the residual block-based improved U-Net model proposed in this paper, combined with image clipping and preprocessing, could greatly improve the segmentation accuracy of rectal cancer, and the Dice coefficient obtained reached 0.97 on the verification set.