An efficient and robust multi-scenario artificial intelligent medical model based on metaverse

Jiuwen ZHU; Yubing ZHOU; Hongbiao SI; Liang XU

doi:10.11959/j.issn.2096-0271.2023006

您当前的位置：

首页 >

文章列表页 >

An efficient and robust multi-scenario artificial intelligent medical model based on metaverse

STUDY | 更新时间：2024-06-03

- An efficient and robust multi-scenario artificial intelligent medical model based on metaverse
- Big data research Vol. 10, Issue 2, Pages: 122-139(2024)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2023006
  CLC： F49,F270.7,TP399
- Online First：2024-03，
  
  Published：15 March 2024
- 稿件说明：
移动端阅览
Jiuwen ZHU, Yubing ZHOU, Hongbiao SI, et al. An efficient and robust multi-scenario artificial intelligent medical model based on metaverse[J]. Big data research, 2024, 10(2): 122-139.
DOI：

Jiuwen ZHU, Yubing ZHOU, Hongbiao SI, et al. An efficient and robust multi-scenario artificial intelligent medical model based on metaverse[J]. Big data research, 2024, 10(2): 122-139. DOI： 10.11959/j.issn.2096-0271.2023006.

摘要

现今医疗行业普遍存在医疗资源和教育资源不均衡、医疗体系智能化水平低、手术操作依赖个体经验等问题，拥有沉浸、互动特点的元宇宙为以上问题提供了解决方案。但现有的解决方案多基于虚拟现实或人工智能中的一种技术，针对特定的手术或疾病诊断问题进行探索，少有针对多功能、多场景应用的医疗元宇宙进行的系统研究。基于此，提出了元宇宙环境下的多场景智能医疗模型框架（MetaMed），从接入层、数据层、技术层、应用层自底向上阐述了模型的框架设计。该框架在智能手术、线上会诊、医疗培训、机器人手术和门诊挂号5种应用场景中给出了数学描述，并为未来医疗元宇宙的构建提供参考。

Abstract

Unbalanced medical and educational resources

low intelligence of the medical system

and reliance on individual experience in surgical operations are common in medical trade.The metaverse with immersive and interactive features is an effective tool to solve the problem.However

most of the existing solutions are based on a specific technology of virtual reality or artificial intelligence or a specific operation

and there is little systematic research on the multifunctional and multi-scenario medical metaverse.Therefore

a multi-scenario artificial intelligent medical model based on metaverse (MetaMed) was proposed

which elaborated the bottom-up implementations from four layers

including the access layer

data layer

technology layer and application layer.MetaMed was mathematically applied in five medical scenarios

i.e.

intelligent surgery

online consultation

medical education

robotic surgery and outpatient registration scenarios

which provides references for the construction of medical metaverse in the future.

关键词

Keywords

references

LEE L H , BRAUD T , ZHOU P Y , et al . All one needs to know about metaverse:a complete survey on technological singularity,virtual ecosystem,and research agenda [EB ] . arXiv preprint , 2021 .

CHAPMAN J R , WANG J C , WIECHERT K . Into the Spine Metaverse:reflections on a future Metaspine (Uni-) verse [J ] . Global Spine Journal , 2022 , 12 ( 4 ): 545 - 547 .

RUIZ MEJIA J M , RAWAT D B . Recent advances in a medical domain metaverse:status,challenges,and perspective [C ] // Proceedings of the 2022 Thirteenth International Conference on Ubiquitous and Future Networks . Piscataway:IEEE Press , 2022 : 357 - 362 .

MOZUMDER M A I , SHEERAZ M M , ATHAR A , et al . Overview:technology roadmap of the future trend of metaverse based on IoT,blockchain,AI technique,and medical domain metaverse activity [C ] // Proceedings of the 2022 24th International Conference on Advanced Communication Technology . Piscataway:IEEE Press , 2022 : 256 - 261 .

TAN T F , LI Y , LIM J S , et al . Metaverse and virtual health care in ophthalmology:opportunities and challenges [J ] . AsiaPacific Journal of Ophthalmology , 2022 , 11 ( 3 ): 237 - 246 .

KALA D N . Revolutionizing medical education with metaverse [J ] . International Journal of Scientific Research in Computer Science,Engineering and Information Technology , 2022 : 26 - 32 .

BRUYNSEELS K , SANTONI DE SIO F , VAN DEN HOVEN J . Digital twins in health care:ethical implications of an emerging engineering paradigm [J ] . Frontiers in Genetics , 2018 ,9:31.

D’SOUZA M , GENDREAU J , FENG A , et al . Robotic-assisted spine surgery:history,efficacy,cost,and future trends [J ] . Robotic Surgery (Auckland) , 2019 , 6 : 9 - 23 .

侯颖 , 许威威 . 增强现实技术综述 [J ] . 计算机测量与控制 , 2017 , 25 ( 2 ): 1 - 7 , 22 .

HOU Y , XU W W . A survey of augmented reality technology [J ] . Computer Measurement＆ Control , 2017 , 25 ( 2 ): 1 - 7 , 22 .

MAAS M J , HUGHES J M . Virtual,augmented and mixed reality in K–12 education:a review of the literature [J ] . Technology,Pedagogy and Education , 2020 , 29 ( 2 ): 231 - 249 .

AULOGE P , CAZZATO R L , RAMAMURTHY N , et al . Augmented reality and artificial intelligencebased navigation during percutaneous vertebroplasty:a pilot randomised clinical trial [J ] . European Spine Journal , 2020 , 29 ( 7 ): 1580 - 1589 .

CASTELAN E , VINNIKOV M , ZHOU X A . Augmented reality anatomy visualization for surgery assistance with HoloLens:AR surgery assistance with HoloLens [C ] // Proceedings of the Proceedings of the 2021 ACM International Conference on Interactive Media Experiences . New York:ACM , 2021 : 329 - 331 .

LIU A , JIN Y K , COTTRILL E , et al . Clinical accuracy and initial experience with augmented reality-assisted pedicle screw placement:the first 205 screws [J ] . Journal of Neurosurgery Spine , 2021 , 36 ( 3 ): 351 - 357 .

HOANG T , REINOSO M , JOUKHADAR Z , et al . Augmented studio:projection mapping on moving body for physiotherapy education [C ] // Proceedings of the Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems . New York:ACM , 2017 .

KYE B , HAN N , KIM E , et al . Educational applications of metaverse:possibilities and limitations [J ] . Journal of Educational Evaluation for Health Professions , 2021 ,18:32.

陈浩磊 , 邹湘军 , 陈燕 , 等 . 虚拟现实技术的最新发展与展望 [J ] . 中国科技论文在线 , 2011 , 6 ( 1 ): 1 - 5 , 14 .

CHEN H L , ZOU X J , CHEN Y , et al . Overview of the advance in virtual reality technology [J ] . Sciencepaper Online , 2011 , 6 ( 1 ): 1 - 5 , 14 .

RIZZO A A . Bravemind:advancing the virtual iraq/afghanistan ptsd exposure therapy for MST [R ] . 2015 .

RIZZO A , ROY M J , HARTHOLT A , et al . Virtual reality applications for the assessment and treatment of PTSD [M ] // Handbook of Military Psychology . Cham : Springer , 2017 : 453 - 471 .

MOZGAI S , LEEDS A , KWOK D , et al . Building BRAVEMIND Vietnam:usercentered design for virtual reality exposure therapy [C ] // Proceedings of the 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality . Piscataway:IEEE Press , 2021 : 247 - 250 .

KWAN H Y , LIN L , FAHY C , et al . Designing VR training systems for children with attention deficit hyperactivity disorder (ADHD) [C ] // Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops . Piscataway:IEEE Press , 2022 : 88 - 89 .

BOEDECKER C , HUETTL F , SAALFELD P , et al . Using virtual 3D-models in surgical planning:workflow of an immersive virtual reality application in liver surgery [J ] . Langenbeck’s Archives of Surgery , 2021 , 406 ( 3 ): 911 - 915 .

ZHANG J , CHANG J , YANG X , et al . Virtual reality surgery simulation:a survey on patient specific solution [C ] // Proceedings of Next Generation Computer Animation Techniques . Cham:Springer , 2008 : 220 - 233 .

DINC F , OUMIMOUN K , KWABLA W , et al . Towards real-time bone drilling simulation for anchor placement in VR based arthroscopic rotator cuff surgery simulation [J ] . AMIA Joint Summits on Translational Science Proceedings AMIA Joint Summits on Translational Science , 2022 , 2022 : 178 - 185 .

MAYTIN M , DAILY T P , CARILLO R G . Virtual reality lead extraction as a method for training new physicians:a pilot study [J ] . Pacing and Clinical Electrophysiology:PACE , 2015 , 38 ( 3 ): 319 - 325 .

CREUTZFELDT J , HEDMAN L , FELLÄNDER-TSAI L . Cardiopulmonary resuscitation training by avatars:a qualitative study of medical students' experiences using a multiplayer virtual world [J ] . JMIR Serious Games , 2016 , 4 ( 2 ): e22 .

REAL F J , DEBLASIO D , BECK A F , et al . A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal [J ] . Academic Pediatrics , 2017 , 17 ( 4 ): 431 - 435 .

ZHAO G J , FAN M J , YUAN Y B , et al . The comparison of teaching efficiency between virtual reality and traditional education in medical education:a systematic review and meta-analysis [J ] . Annals of Translational Medicine , 2021 , 9 ( 3 ): 252 .

GUTIÉRREZ F , PIERCE J , VERGARA V M , et al . The effect of degree of immersion upon learning performance in virtual reality simulations for medical education [J ] . Studies in Health Technology and Informatics , 2007 , 125 : 155 - 160 .

RASHIDI H H , TRAN N , ALBAHRA S , et al . Machine learning in health care and laboratory medicine:general overview of supervised learning and Auto-ML [J ] . International Journal of Laboratory Hematology , 2021 , 43 ( Suppl 1 ): 15 - 22 .

KRATZ A , LEE S H , ZINI G , et al . Digital morphology analyzers in hematology:ICSH review and recommendations [J ] . International Journal of Laboratory Hematology , 2019 , 41 ( 4 ): 437 - 447 .

WANG D , ZHANG Y , ZHANG K X , et al . FocalMix:semi-supervised learning for 3D medical image detection [C ] // Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 3950 - 3959 .

YANG D , XU Z Y , LI W Q , et al . Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China,Italy,Japan [J ] . Medical Image Analysis , 2021 ,70:101992.

CHEN S , BORTSOVA G , GARCÍAUCEDA JUÁREZ A , et al . Multitask attention-based semi-supervised learning for medical image segmentation [C ] // International Conference on Medical Image Computing and Computer-Assisted Intervention . Cham:Springer , 2019 : 457 - 465 .

WANG N , HUANG Y Q , LIU H L , et al . Study on the semi-supervised learning-based patient similarity from heterogeneous electronic medical records [J ] . BMC Medical Informatics and Decision Making , 2021 , 21 ( Suppl 2 ): 58 .

RAZA K , SINGH N K . A tour of unsupervised deep learning for medical image analysis [J ] . Current Medical Imaging , 2021 , 17 ( 9 ): 1059 - 1077 .

IQBAL T , ELAHI A , WIJNS W , et al . Exploring unsupervised machine learning classification methods for physiological stress detection [J ] . Frontiers in Medical Technology , 2022 ,4:782756.

CHEN J Y , FREY E C . An unsupervised learning model for medical image segmentation [EB ] . arXiv preprint , 2020 ,arXiv:2001.10155.

XIE Q S , LI Y X , HE N J , et al . Unsupervised domain adaptation for medical image segmentation by disentanglement learning and selftraining [J ] . IEEE Transactions on Medical Imaging , 2024 , 43 ( 1 ): 4 - 14 .

SHERWANI M K , MARZULLO A , DE MOMI E , et al . Lesion segmentation in lung CT scans using unsupervised adversarial learning [J ] . Medical ＆Biological Engineering ＆ Computing , 2022 , 60 ( 11 ): 3203 - 3215 .

ZHANG Y M , LI H L , HOU Y W , et al . Consecutive knowledge meta-adaptation learning for unsupervised medical diagnosis [J ] . Knowledge-Based Systems , 2024 ,291:111573.

CHEN P H C , GADEPALLI K , MACDONALD R , et al . An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis [J ] . Nature Medicine , 2019 , 25 ( 9 ): 1453 - 1457 .

MORETA-MARTINEZ R , POSE-DÍEZDE-LA-LASTRA A , CALVO-HARO J A , et al . Combining augmented reality and 3D printing to improve surgical workflows in orthopedic oncology:smartphone application and clinical evaluation [J ] . Sensors , 2021 , 21 ( 4 ): 1370 .

COFANO F , DI PERNA G , BOZZARO M , et al . Augmented reality in medical practice:from spine surgery to remote assistance [J ] . Frontiers in Surgery , 2021 ,8:657901.

CAMPISI C A , LI E H , JIMENEZ D E , et al . Augmented reality in medical education and training:from physicians to patients [M ] . Augmented reality in education . Cham : Springer , 2020 :111138.

KOMAL KUMAR N , VIGNESWARI D . A drug recommendation system for multidisease in health care using machine learning [C ] // Proceedings of International Conference on Advanced Communication and Computational Technology . Singapore:Springer , 2021 : 1 - 12 .

DHAYNE H , HAQUE R , KILANY R , et al . In search of big medical data integration solutions - a comprehensive survey [J ] . IEEE Access , 2019 , 7 : 91265 - 91290 .

XIONG J H , HSIANG E L , HE Z Q , et al . Augmented reality and virtual reality displays:emerging technologies and future perspectives [J ] . Light,Science ＆Applications , 2021 , 10 ( 1 ): 216 .

NGUYEN H X , TRESTIAN R , TO D , et al . Digital twin for 5G and beyond [J ] . IEEE Communications Magazine , 2021 , 59 ( 2 ): 10 - 15 .

SAAD W , BENNIS M , CHEN M Z . A vision of 6G wireless systems:applications,trends,technologies,and open research problems [J ] . IEEE Network , 2020 , 34 ( 3 ): 134 - 142 .

SHEN S , WANG P Z , LI X Y , et al . Preoperative simulation using a threedimensional printing model for surgical treatment of old and complex tibial plateau fractures [J ] . Scientific Reports , 2020 , 10 ( 1 ): 6044 .

WAGNER J P , COCHRAN A L , JONES C , et al . Professional use of social media among surgeons:results of a multiinstitutional study [J ] . Journal of Surgical Education , 2018 , 75 ( 3 ): 804 - 810 .

WISE S , DUFFIELD C , FRY M , et al . A team mental model approach to understanding team effectiveness in an emergency department:a qualitative study [J ] . Journal of Health Services Research ＆ Policy , 2022 , 27 ( 1 ): 14 - 21 .

BASHKANOV O , SAALFELD P , GUNASEKARAN H , et al . VR multi-user conference room for surgery planning [C ] // Proceedings of Annual Meeting of the German Society of Computer and RobotAssisted Surgery .[S.l.:s.n. ] , 2019 : 264 - 268 .

Views

281

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Edge-intelligence-based immersive metaverse:key technologies and prospects

Metaverse and big data: data insight and value connection in spatio-temporal intelligence

A survey of voice conversion based on non-parallel data

Big data and computing models

Research on algorithm and application risk and its governance

Related Author

Zhi WANG

Shutao XIA

Rui MAO

Yang SHEN

Menglong YU

Pengcheng LI

Xulong ZHANG

Jianzong WANG

Related Institution

Shenzhen University

Tsinghua Shenzhen International Graduate School

University of Science and Technology of China

Ping An Technology (Shenzhen) Co., Ltd.

School of Electronic and Information Engineering, Beijing Jiaotong University

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273110 Visits Today：16

⁰