In the field of artificial intelligence (AI) medical imaging, data annotation is a key factor in all AI development. In the traditional manual annotation process, there are prominent problems such as difficult data acquisition, high manual labor intensity, strong professionalism and low labeling quality. Therefore, an intelligent multimodal medical image annotation system is urgently needed to meet the requirements of labeling. Based on the image cloud, West China Hospital of Sichuan University collected the multimodal image data of hospital and allied hospitals, and designed a multi-modal image annotation system through information technology, which integrated various image processing algorithms and AI models to simplify the image data annotation. With the construction of annotation system, the efficiency of data labeling in the hospitals is improved, which provides necessary data support for the AI image research and related industry construction in the hospital, so as to promote the implementation of artificial intelligence industry related to medical images in the hospital.