Application of mixed reality technology in orthopedic field_West China Medical Journal

Authors：

CHEN Lin , LIU Bailian , PU Qi , WANG Daxing ,  YANG Huiqin

Department of Orthopedics, Yan’an Hospital Affiliated to Kunming Medical University, Kunming, Yunnan 650051, P. R. China;

Corresponding author：

YANG Huiqin, Email: doctoryhq@163.com

Keywords：

Orthopedics; Mixed reality; Precision medicine; Digital orthopedics

DOI：

10.7507/1002-0179.202005101

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Mixed reality technology is new digital holographic imaging technology that generates three-dimensional simulation images through computers and anchors the virtual images to the real world. Compared with traditional imaging diagnosis and treatment methods, mixed reality technology is more conducive to the advantages of precision medicine, helps to promote the development of medical clinical application, teaching and scientific research in the field of orthopedics, and will further promote the progress of clinical orthopedics toward standardization, digitization and precision. This article briefly introduces the mixed reality technology, reviews its application in the perioperative period, teaching and diagnosis and treatment standardization and dataization in the field of orthopedics, and discusses its technical advantages, aiming to provide a reference for the better use of mixed reality technology in orthopedics.

Citation： CHEN Lin, LIU Bailian, PU Qi, WANG Daxing, YANG Huiqin. Application of mixed reality technology in orthopedic field. West China Medical Journal, 2021, 36(8): 1126-1131. doi: 10.7507/1002-0179.202005101 Copy

1.	Hauze SW, Hoyt HH, Frazee JP, et al. Enhancing nursing education through affordable and realistic holographic mixed reality: the virtual standardized patient for clinical simulation. Adv Exp Med Biol, 2019, 1120: 1-13.
2.	方驰华, 陈康, 张鹏. 智能化诊疗技术在普通外科中应用现状及前景. 中华外科杂志, 2019, 57(1): 1-5.
3.	Halic T, Kockara S, Bayrak C, et al. Mixed reality simulation of rasping procedure in artificial cervical disc replacement (ACDR) surgery. BMC Bioinformatics, 2010, 11(Suppl 6): S11.
4.	Gao Y, Tan K, Sun J, et al. Application of mixed reality technology in visualization of medical operations. Chin Med Sci J, 2019, 34(2): 103-109.
5.	Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
6.	Goo HW, Park SJ, Yoo SJ. Advanced medical use of three-dimensional imaging in congenital heart disease: augmented reality, mixed reality, virtual reality, and three-dimensional printing. Korean J Radiol, 2020, 21(2): 133-145.
7.	叶哲伟, 吴星火. 混合现实技术在骨科的最新应用进展. 临床外科杂志, 2018, 26(1): 13-14.
8.	Tuladhar S, AlSallami N, Alsadoon A, et al. A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality. Int J Med Robot, 2020, 16(5): 1-22.
9.	Salmas M, Chronopoulos E, Chytas D. Comment on: “A novel evaluation model for a mixed-reality surgical navigation system: where microsoft hololens meets the operating room”. Surg Innov, 2020, 27(6): 702-703.
10.	Checcucci E, Amparore D, Pecoraro A, et al. 3D mixed reality holograms for preoperative surgical planning of nephron-sparing surgery: evaluation of surgeons’ perception. Minerva Urol Nephrol, 2021, 73(3): 367-375.
11.	万磊, 庾伟中, 王建华, 等. 基于混合现实的三维测量技术辅助诊断颅底凹陷症. 中国骨科临床与基础研究杂志, 2018, 10(5): 280-284.
12.	Chakravarthy V, Sheikh S, Schmidt E, et al. Imaging technologies in spine surgery. Neurosurg Clin N Am, 2020, 31(1): 93-101.
13.	Wu X, Liu R, Yu J, et al. Mixed reality technology launches in orthopedic surgery for comprehensive preoperative management of complicated cervical fractures. Surg Innov, 2018, 25(4): 421-422.
14.	Yu H, Zhou Z, Lei X, et al. Mixed reality-based preoperative planning for training of percutaneous transforaminal endoscopic discectomy: a feasibility study. World Neurosurg, 2019, 129: e767-e775.
15.	Wu X, Liu R, Yu J, et al. Mixed reality technology-assisted orthopedics surgery navigation. Surg Innov, 2018, 25(3): 304-305.
16.	Chu Y, Li X, Yang X, et al. Perception enhancement using importance-driven hybrid rendering for augmented reality based endoscopic surgical navigation. Biomed Opt Express, 2018, 9(11): 5205-5226.
17.	Wei P, Yao Q, Xu Y, et al. Percutaneous kyphoplasty assisted with/without mixed reality technology in treatment of OVCF with IVC: a prospective study. J Orthop Surg Res, 2019, 14(1): 255.
18.	陶星光. 计算机辅助技术在胫骨平台骨折分期治疗中的应用研究. 江苏: 南京医科大学, 2019: 26-39.
19.	Wu X, Liu R, Xu S, et al. Feasibility of mixed reality-based intraoperative three-dimensional image-guided navigation for atlanto-axial pedicle screw placement. Proc Inst Mech Eng H, 2019, 233(12): 1310-1317.
20.	万磊, 梁笃, 李保林, 等. 基于 CAD-MR 技术混合现实重建骨盆与髋臼骨折及血管网辅助手术临床研究. 实用医学杂志, 2019, 35(7): 1019-1022, 1027.
21.	Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
22.	Lindeman DA, Kim KK, Gladstone C, et al. Technology and caregiving: emerging interventions and directions for research. Gerontologist, 2020, 60(Suppl 1): S41-S49.
23.	王慧文, 高春红, 胡甜. 股骨颈骨折青少年患者混合现实技术引导下微创手术的快速康复护理. 护理学杂志, 2019, 34(10): 23-25.
24.	Negrillo-Cárdenas J, Jiménez-Pérez JR, Feito FR. The role of virtual and augmented reality in orthopedic trauma surgery: from diagnosis to rehabilitation. Comput Methods Programs Biomed, 2020, 191: 105407.
25.	沈剑辉, 高兴莲, 鄢利芳, 等. 混合现实技术引导下行复杂骨折手术的护理配合. 中华护理杂志, 2018, 53(9): 1050-1054.
26.	Gerup J, Soerensen CB, Dieckmann P. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ, 2020, 11: 1-18.
27.	Flores A, Linehan MM, Todd SR, et al. The use of virtual reality to facilitate mindfulness skills training in dialectical behavioral therapy for spinal cord injury: a case study. Front Psychol, 2018, 9: 531.
28.	Stefan P, Habert S, Winkler A, et al. A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery. Int J Comput Assist Radiol Surg, 2018, 13(9): 1335-1344.
29.	McJunkin JL, Jiramongkolchai P, Chung W, et al. Development of a mixed reality platform for lateral skull base anatomy. Otol Neurotol, 2018, 39(10): e1137-e1142.
30.	Wainman B, Pukas G, Wolak L, et al. The critical role of stereopsis in virtual and mixed reality learning environments. Anat Sci Educ, 2020, 13(3): 401-412.
31.	Ruthberg JS, Tingle G, Tan L, et al. Mixed reality as a time-efficient alternative to cadaveric dissection. Med Teach, 2020, 42(8): 896-901.
32.	张里程, 雷明星, 张浩, 等. 基于混合现实技术骨科教学模式的构建及应用. 中国继续医学教育, 2018, 10(36): 13-16.
33.	Moro C, Gregory S. Utilising anatomical and physiological visualisations to enhance the face-to-face student learning experience in biomedical sciences and medicine. Adv Exp Med Biol, 2019, 1156: 41-48.
34.	Sappenfield JW, Smith WB, Cooper LA, et al. Visualization improves supraclavicular access to the subclavian vein in a mixed reality simulator. Anesth Analg, 2018, 127(1): 83-89.
35.	Hochman JB, Sepehri N, Rampersad V, et al. Mixed reality temporal bone surgical dissector: mechanical design. J Otolaryngol Head Neck Surg, 2014, 43(1): 23.
36.	Maniam P, Schnell P, Dan L, et al. Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype. J Vis Commun Med, 2020, 43(1): 17-26.
37.	Condino S, Turini G, Parchi PD, et al. How to build a patient-specific hybrid simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the microsoft HoloLens. J Healthc Eng, 2018, 2018: 5435097.
38.	Coelho G, Defino HLA. The role of mixed reality simulation for surgical training in spine: phase 1 validation. Spine (Phila Pa 1976), 2018, 43(22): 1609-1616.
39.	Stefan P, Pfandler M, Wucherer P, et al. Team training and assessment in mixed reality-based simulated operating room: current state of research in the field of simulation in spine surgery exemplified by the ATMEOS project. Unfallchirurg, 2018, 121(4): 271-277.
40.	马春阳, 周益龙, 李怀亮, 等. 混合现实术中导航技术联合三维可视化技术在复杂性肝癌切除术中的应用. 中华医学杂志, 2019(4): 279-283.
41.	Kanevsky J, Safran T, Zammit D, et al. Making augmented and virtual reality work for the plastic surgeon. Ann Plast Surg, 2019, 82(4): 363-368.
42.	朱捷, 沈诞, 刘启明, 等. 混合现实平台远程协作机器人微创手术 1 例报告. 微创泌尿外科杂志, 2018, 7(4): 278-281.
43.	Rojas-Muñoz E, Cabrera ME, Lin C, et al. Telementoring in leg fasciotomies via mixed-reality: clinical evaluation of the STAR platform. Mil Med, 2020, 185(Suppl 1): 513-520.
44.	Wang C, Daniel BK, Asil M, et al. A randomised control trial and comparative analysis of multi-dimensional learning tools in anatomy. Sci Rep, 2020, 10(1): 6120.
45.	He Y, Yang G, Yang J, et al. Dense biased networks with deep priori anatomy and hard region adaptation: semi-supervised learning for fine renal artery segmentation. Med Image Anal, 2020, 63: 101722.
46.	Lei PF, Su SL, Kong LY, et al. Mixed reality combined with three-dimensional printing technology in total hip arthroplasty: an updated review with a preliminary case presentation. Orthop Surg, 2019, 11(5): 914-920.
47.	Talaat S, Ghoneima A, Kaboudan A, et al. Three-dimensional evaluation of the holographic projection in digital dental model superimposition using HoloLens device. Orthod Craniofac Res, 2019, 22(Suppl 1): 62-68.
48.	Perkins SL, Krajancich B, Yang CJ, et al. A patient-specific mixed-reality visualization tool for thoracic surgical planning. Ann Thorac Surg, 2020, 110(1): 290-295.
49.	Coelho G, Figueiredo EG, Rabelo NN, et al. Development and evaluation of pediatric mixed-reality model for neuroendoscopic surgical training. World Neurosurg, 2020, 139: e189-e202.
50.	Huber T, Hadzijusufovic E, Hansen C, et al. Head-mounted mixed-reality technology during robotic-assisted transanal total mesorectal excision. Dis Colon Rectum, 2019, 62(2): 258-261.
51.	Vávra P, Roman J, Zonč a P, et al. Recent development of augmented reality in surgery: a review. J Healthc Eng, 2017, 2017: 4574172.

1. Hauze SW, Hoyt HH, Frazee JP, et al. Enhancing nursing education through affordable and realistic holographic mixed reality: the virtual standardized patient for clinical simulation. Adv Exp Med Biol, 2019, 1120: 1-13.
2. 方驰华, 陈康, 张鹏. 智能化诊疗技术在普通外科中应用现状及前景. 中华外科杂志, 2019, 57(1): 1-5.
3. Halic T, Kockara S, Bayrak C, et al. Mixed reality simulation of rasping procedure in artificial cervical disc replacement (ACDR) surgery. BMC Bioinformatics, 2010, 11(Suppl 6): S11.
4. Gao Y, Tan K, Sun J, et al. Application of mixed reality technology in visualization of medical operations. Chin Med Sci J, 2019, 34(2): 103-109.
5. Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
6. Goo HW, Park SJ, Yoo SJ. Advanced medical use of three-dimensional imaging in congenital heart disease: augmented reality, mixed reality, virtual reality, and three-dimensional printing. Korean J Radiol, 2020, 21(2): 133-145.
7. 叶哲伟, 吴星火. 混合现实技术在骨科的最新应用进展. 临床外科杂志, 2018, 26(1): 13-14.
8. Tuladhar S, AlSallami N, Alsadoon A, et al. A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality. Int J Med Robot, 2020, 16(5): 1-22.
9. Salmas M, Chronopoulos E, Chytas D. Comment on: “A novel evaluation model for a mixed-reality surgical navigation system: where microsoft hololens meets the operating room”. Surg Innov, 2020, 27(6): 702-703.
10. Checcucci E, Amparore D, Pecoraro A, et al. 3D mixed reality holograms for preoperative surgical planning of nephron-sparing surgery: evaluation of surgeons’ perception. Minerva Urol Nephrol, 2021, 73(3): 367-375.
11. 万磊, 庾伟中, 王建华, 等. 基于混合现实的三维测量技术辅助诊断颅底凹陷症. 中国骨科临床与基础研究杂志, 2018, 10(5): 280-284.
12. Chakravarthy V, Sheikh S, Schmidt E, et al. Imaging technologies in spine surgery. Neurosurg Clin N Am, 2020, 31(1): 93-101.
13. Wu X, Liu R, Yu J, et al. Mixed reality technology launches in orthopedic surgery for comprehensive preoperative management of complicated cervical fractures. Surg Innov, 2018, 25(4): 421-422.
14. Yu H, Zhou Z, Lei X, et al. Mixed reality-based preoperative planning for training of percutaneous transforaminal endoscopic discectomy: a feasibility study. World Neurosurg, 2019, 129: e767-e775.
15. Wu X, Liu R, Yu J, et al. Mixed reality technology-assisted orthopedics surgery navigation. Surg Innov, 2018, 25(3): 304-305.
16. Chu Y, Li X, Yang X, et al. Perception enhancement using importance-driven hybrid rendering for augmented reality based endoscopic surgical navigation. Biomed Opt Express, 2018, 9(11): 5205-5226.
17. Wei P, Yao Q, Xu Y, et al. Percutaneous kyphoplasty assisted with/without mixed reality technology in treatment of OVCF with IVC: a prospective study. J Orthop Surg Res, 2019, 14(1): 255.
18. 陶星光. 计算机辅助技术在胫骨平台骨折分期治疗中的应用研究. 江苏: 南京医科大学, 2019: 26-39.
19. Wu X, Liu R, Xu S, et al. Feasibility of mixed reality-based intraoperative three-dimensional image-guided navigation for atlanto-axial pedicle screw placement. Proc Inst Mech Eng H, 2019, 233(12): 1310-1317.
20. 万磊, 梁笃, 李保林, 等. 基于 CAD-MR 技术混合现实重建骨盆与髋臼骨折及血管网辅助手术临床研究. 实用医学杂志, 2019, 35(7): 1019-1022, 1027.
21. Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
22. Lindeman DA, Kim KK, Gladstone C, et al. Technology and caregiving: emerging interventions and directions for research. Gerontologist, 2020, 60(Suppl 1): S41-S49.
23. 王慧文, 高春红, 胡甜. 股骨颈骨折青少年患者混合现实技术引导下微创手术的快速康复护理. 护理学杂志, 2019, 34(10): 23-25.
24. Negrillo-Cárdenas J, Jiménez-Pérez JR, Feito FR. The role of virtual and augmented reality in orthopedic trauma surgery: from diagnosis to rehabilitation. Comput Methods Programs Biomed, 2020, 191: 105407.
25. 沈剑辉, 高兴莲, 鄢利芳, 等. 混合现实技术引导下行复杂骨折手术的护理配合. 中华护理杂志, 2018, 53(9): 1050-1054.
26. Gerup J, Soerensen CB, Dieckmann P. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ, 2020, 11: 1-18.
27. Flores A, Linehan MM, Todd SR, et al. The use of virtual reality to facilitate mindfulness skills training in dialectical behavioral therapy for spinal cord injury: a case study. Front Psychol, 2018, 9: 531.
28. Stefan P, Habert S, Winkler A, et al. A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery. Int J Comput Assist Radiol Surg, 2018, 13(9): 1335-1344.
29. McJunkin JL, Jiramongkolchai P, Chung W, et al. Development of a mixed reality platform for lateral skull base anatomy. Otol Neurotol, 2018, 39(10): e1137-e1142.
30. Wainman B, Pukas G, Wolak L, et al. The critical role of stereopsis in virtual and mixed reality learning environments. Anat Sci Educ, 2020, 13(3): 401-412.
31. Ruthberg JS, Tingle G, Tan L, et al. Mixed reality as a time-efficient alternative to cadaveric dissection. Med Teach, 2020, 42(8): 896-901.
32. 张里程, 雷明星, 张浩, 等. 基于混合现实技术骨科教学模式的构建及应用. 中国继续医学教育, 2018, 10(36): 13-16.
33. Moro C, Gregory S. Utilising anatomical and physiological visualisations to enhance the face-to-face student learning experience in biomedical sciences and medicine. Adv Exp Med Biol, 2019, 1156: 41-48.
34. Sappenfield JW, Smith WB, Cooper LA, et al. Visualization improves supraclavicular access to the subclavian vein in a mixed reality simulator. Anesth Analg, 2018, 127(1): 83-89.
35. Hochman JB, Sepehri N, Rampersad V, et al. Mixed reality temporal bone surgical dissector: mechanical design. J Otolaryngol Head Neck Surg, 2014, 43(1): 23.
36. Maniam P, Schnell P, Dan L, et al. Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype. J Vis Commun Med, 2020, 43(1): 17-26.
37. Condino S, Turini G, Parchi PD, et al. How to build a patient-specific hybrid simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the microsoft HoloLens. J Healthc Eng, 2018, 2018: 5435097.
38. Coelho G, Defino HLA. The role of mixed reality simulation for surgical training in spine: phase 1 validation. Spine (Phila Pa 1976), 2018, 43(22): 1609-1616.
39. Stefan P, Pfandler M, Wucherer P, et al. Team training and assessment in mixed reality-based simulated operating room: current state of research in the field of simulation in spine surgery exemplified by the ATMEOS project. Unfallchirurg, 2018, 121(4): 271-277.
40. 马春阳, 周益龙, 李怀亮, 等. 混合现实术中导航技术联合三维可视化技术在复杂性肝癌切除术中的应用. 中华医学杂志, 2019(4): 279-283.
41. Kanevsky J, Safran T, Zammit D, et al. Making augmented and virtual reality work for the plastic surgeon. Ann Plast Surg, 2019, 82(4): 363-368.
42. 朱捷, 沈诞, 刘启明, 等. 混合现实平台远程协作机器人微创手术 1 例报告. 微创泌尿外科杂志, 2018, 7(4): 278-281.
43. Rojas-Muñoz E, Cabrera ME, Lin C, et al. Telementoring in leg fasciotomies via mixed-reality: clinical evaluation of the STAR platform. Mil Med, 2020, 185(Suppl 1): 513-520.
44. Wang C, Daniel BK, Asil M, et al. A randomised control trial and comparative analysis of multi-dimensional learning tools in anatomy. Sci Rep, 2020, 10(1): 6120.
45. He Y, Yang G, Yang J, et al. Dense biased networks with deep priori anatomy and hard region adaptation: semi-supervised learning for fine renal artery segmentation. Med Image Anal, 2020, 63: 101722.
46. Lei PF, Su SL, Kong LY, et al. Mixed reality combined with three-dimensional printing technology in total hip arthroplasty: an updated review with a preliminary case presentation. Orthop Surg, 2019, 11(5): 914-920.
47. Talaat S, Ghoneima A, Kaboudan A, et al. Three-dimensional evaluation of the holographic projection in digital dental model superimposition using HoloLens device. Orthod Craniofac Res, 2019, 22(Suppl 1): 62-68.
48. Perkins SL, Krajancich B, Yang CJ, et al. A patient-specific mixed-reality visualization tool for thoracic surgical planning. Ann Thorac Surg, 2020, 110(1): 290-295.
49. Coelho G, Figueiredo EG, Rabelo NN, et al. Development and evaluation of pediatric mixed-reality model for neuroendoscopic surgical training. World Neurosurg, 2020, 139: e189-e202.
50. Huber T, Hadzijusufovic E, Hansen C, et al. Head-mounted mixed-reality technology during robotic-assisted transanal total mesorectal excision. Dis Colon Rectum, 2019, 62(2): 258-261.
51. Vávra P, Roman J, Zonč a P, et al. Recent development of augmented reality in surgery: a review. J Healthc Eng, 2017, 2017: 4574172.

Previous Article
Virtual reality for the upper limb motor rehabilitation after stroke
Next Article
The role of alternative splicing in autoimmune diseases

West China Medical Journal

Application of mixed reality technology in orthopedic field

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content