Evaluation of an Artificial Intelligence System for Retinopathy of Prematurity Screening in Nepal and Mongolia

Objective
The purpose of this study is to evaluate the performance of a deep learning algorithm for retinopathy of prematurity (ROP) screening in Nepal and Mongolia.

Design
This was a retrospective analysis of prospectively collected clinical data.

Subjects
Clinical information and fundus images were obtained from infants in two ROP screening programs in Nepal and Mongolia.

Methods
Fundus images were obtained using the Forus 3nethra neo in Nepal and RetCam® Portable in Mongolia. The overall severity of ROP was determined from the medical record using the International Classification of ROP (ICROP). The presence of plus disease was independently determined in each image using a reference standard diagnosis. The Imaging and Informatics for ROP (i-ROP) deep learning (DL) algorithm, which was trained on images from the RetCam® was used to classify plus disease, as well as assign a vascular severity score (VSS) from 1-9.

Main outcome measures
The main outcome measures were area under the receiver operating characteristic (AUC-ROC) and area under the precision recall curve (AUC-PR) for the presence of plus disease or type 1 ROP, and association between VSS and ICROP disease category.

Results
The prevalence of type 1 ROP was found to be higher in Mongolia (14.0%) than in Nepal (2.2%, p < 0.001) in these data sets. In Mongolia (Retcam images), the AUC-ROC for exam-level plus disease detection was 0.968 and AUC-PR was 0.823. In Nepal (Forus images), the AUC-ROC for exam-level plus disease detection was 0.999 and AUC-PR was 0.993. The ROP vascular severity score was associated with ICROP classification in both datasets (p < 0.001). At the population level, the median [interquartile range] VSS was found to be higher in Mongolia (2.7 [1.3–5.4]) as compared to Nepal (1.9 [1.2–3.4], p < 0.001). Conclusions These data provide preliminary evidence of the effectiveness of the i-ROP DL algorithm for ROP screening in neonatal populations in Nepal and Mongolia, using multiple camera systems, and provide useful data for consideration in future clinical implementation of AI-based ROP screening in low- and middle-income countries.

The role of telepathology in diagnosis of pre-malignant and malignant cervical lesions: Implementation at a tertiary hospital in Northern Tanzania

Introduction
Adequate and timely access to pathology services is a key to scale up cancer control, however, there is an extremely shortage of pathologists in Tanzania. Telepathology (scanned images microscopy) has the potential to increase access to pathology services and it is increasingly being employed for primary diagnosis and consultation services. However, the experience with the use of telepathology in Tanzania is limited. We aimed to investigate the feasibility of using scanned images for primary diagnosis of pre-malignant and malignant cervical lesions by assessing its equivalency to conventional (glass slide) microscopy in Tanzania.

Methods
In this laboratory-based study, assessment of hematoxylin and eosin stained glass slides of 175 cervical biopsies were initially performed conventionally by three pathologists independently. The slides were scanned at x 40 and one to three months later, the scanned images were reviewed by the pathologists in blinded fashion. The agreement between initial and review diagnoses across participating pathologists was described and measured using Cohen’s kappa coefficient (κ).

Results
The overall concordance of diagnoses established on conventional microscopy compared to scanned images across three pathologists was 87.7%; κ = 0.54; CI (0.49–0.57).The overall agreement of diagnoses established by local pathologist on conventional microscopy compared to scanned images was 87.4%; κ = 0.73; CI (0.65–0.79). The concordance of diagnoses established by senior pathologist compared to local pathologist on conventional microscopy and scanned images was 96% and 97.7% respectively. The inter-observer agreement (κ) value were 0.93, CI (0.87–1.00) and 0.94, CI (0.88–1.00) for conventional microscopy and scanned images respectively.

Conclusions
All κ coefficients expressed good intra- and inter-observer agreement, suggesting that telepathology is sufficiently accurate for primary diagnosis in surgical pathology. The discrepancies in interpretation of pre-malignant lesions highlights the importance of p16 immunohistochemistry in definitive diagnosis in these lesions. Sustainability factors including hardware and internet connectivity are essential components to be considered before telepathology may be deemed suitable for widely use in Tanzania.

Physicians’ Perceptions of and Satisfaction With Artificial Intelligence in Cancer Treatment: A Clinical Decision Support System Experience and Implications for Low-Middle–Income Countries

As technology continues to improve, health care systems have the opportunity to use a variety of innovative tools for decision-making, including artificial intelligence (AI) applications. However, there has been little research on the feasibility and efficacy of integrating AI systems into real-world clinical practice, especially from the perspectives of clinicians who use such tools. In this paper, we review physicians’ perceptions of and satisfaction with an AI tool, Watson for Oncology, which is used for the treatment of cancer. Watson for Oncology has been implemented in several different settings, including Brazil, China, India, South Korea, and Mexico. By focusing on the implementation of an AI-based clinical decision support system for oncology, we aim to demonstrate how AI can be both beneficial and challenging for cancer management globally and particularly for low-middle–income countries. By doing so, we hope to highlight the need for additional research on user experience and the unique social, cultural, and political barriers to the successful implementation of AI in low-middle–income countries for cancer care.

Artificial intelligence and machine learning for early detection and diagnosis of colorectal cancer in sub-Saharan Africa

Colorectal cancer (CRC) was once considered a rare disease in sub-Saharan Africa (SSA), but decades of globalisation has changed this narrative. Currently, CRC is the fifth most common cancer in SSA, and while CRC incidence and mortality are decreasing in some high-income countries, rates in SSA are on the rise.1 Because CRC develops from a benign precursor polyp over several years, early detection is critical to either prevent malignancy or detect it at an early stage when it is highly curable. Moreover, curative surgery has been shown to improve survival in a SSA setting.2 Unfortunately, more than 60% of patients in SSA present with stage 4 CRC with a <1% 5 year survival rate.3–5 In contrast, almost 40% of patients in the USA present with stage 1 CRC, resulting in a 5-year survival rate of 90%.6 7 Widespread population-based CRC screening programmes and tools (eg, faecal immunochemical test (FIT), colonoscopy) have improved early detection in high-income countries, but SSA-specific data, tools and screening programmes are currently lacking. There is an urgent need to develop more efficient approaches to CRC screening and early detection that do not rely heavily on trained healthcare personnel or specialised resources (eg, endoscopy, pathology), which are often scarce in low- and middle-income countries (LMICs).

Recent technological advances and developments in artificial intelligence (AI) and machine learning (ML) methods have the potential to transform global health, particularly for early detection and diagnosis of CRC in SSA. Researchers are collecting enormous volumes of data, and while data science applications are largely underdeveloped in Africa, many enabling factors are already in place. Developments in cloud computing, substantial investments in digitising health information, and robust mobile phone penetration have poised many places in SSA with the necessary basics to initiate meaningful AI/ML applications.8 Businesses in SSA …

Implementation and Usefulness of Telemedicine Services in the Healthcare System of Pakistan

Telemedicine, also known as telehealth or e-health, is the remote delivery of healthcare services over the telecommunications infrastructure. It allows healthcare providers to evaluate, diagnose and treat patients without the need for an in-person visit. Pakistan is a developing nation with its population majorly concentrated in rural areas which lack adequate high quality healthcare services and patient care. Telemedicine has the potential to surpass many barriers that hinder the healthcare delivery in these remote and rural areas. However, Pakistan hasn’t been able to achieve significant benefit from these e-health advancements due to lack of adequate guidelines, laws or policies needed for e-health to properly work here and due to a lack of government’s interest. There are many barriers to implementation of e-health in Pakistan with very low literacy rate being the major one. Other barriers include high budget requirements and limited access to internet and technology in remote and rural areas of the country.
Apart from these barriers, there are many challenges that the patients and doctors face dealing with telemedicine. In order to achieve significant benefit from these ehealth technologies in Pakistan, it is also necessary to increase the knowledge of it in both patients and doctors. More workshops and training programs should be arranged to teach doctors about the telemedicine technology and proper telemedicine guidelines should be made and regulated by higher authorities

Reducing Inappropriate Urinary Catheter Use by Involving Patients Through the Participatient App: Before-and-After Study

Background: The risk of urinary tract infections is increased by the inappropriate placement and unnecessary prolongation of the use of indwelling urinary catheters. Sustained behavior change in infection prevention could be promoted by empowering patients through a smartphone app.
Objective: The aim of this study is to assess the feasibility and efficacy of implementation actions on patients’ use of the Participatient app on a clinical ward and to compare 3 survey methods for urinary catheter use.
Methods: Participatient was introduced for all admitted patients at the surgical nursing ward in a university hospital in the Netherlands. Over a period of 3 months, the number of new app users, days of use, and sessions were recorded. In a comparison of urinary catheter use before and after the implementation of the app, 3 methods for point prevalence surveys of catheter use were tested. Surveys were conducted through manual parsing of the text in patients’ electronic medical records, parsing a survey of checkbox items, and parsing nursing notes.
Results: In all, 475 patients were admitted to the ward, 42 (8.8%) installed the app, with 1 to 5 new users per week. The actions with the most ensuing app use were the kick-off with the clinical lesson and recruiting of the intake nurse. Between the survey methods, there was considerable variation in catheter use prevalence. Therefore, we used the standard method of manual parsing in further analyses. Catheter use prevalence decreased from 38% (36/96) to 27% (23/86) after app introduction (OR 0.61, 95% CI 0.32-1.14).
Conclusions: The clinical application of Participatient, the infection prevention app for patients, could be feasible when implementation actions are also used. For surveying indwelling urinary catheter use prevalence, manual parsing is the best approach.

An e-learning pediatric cardiology curriculum for Pediatric Postgraduate trainees in Rwanda: implementation and evaluation

Background
Access to pediatric sub-specialty training is a critical unmet need in many resource-limited settings. In Rwanda, only two pediatric cardiologists are responsible for the country’s clinical care of a population of 12 million, along with the medical education of all pediatric trainees. To strengthen physician training opportunities, we developed an e-learning curriculum in pediatric cardiology. This curriculum aimed to “flip the classroom”, allowing residents to learn key pediatric cardiology concepts digitally before an in-person session with the specialist, thus efficiently utilizing the specialist for additional case based and bedside teaching.

Methods
We surveyed Rwandan and US faculty and residents using a modified Delphi approach to identify key topics in pediatric cardiology. Lead authors from Rwanda and the USA collaborated with OPENPediatrics™, a free digital knowledge-sharing platform, to produce ten core topics presented in structured videos spanning 4.5 h. A mixed methods evaluation was completed with Rwandan pediatric residents, including surveys assessing knowledge, utilization, and satisfaction. Qualitative analysis of structured interviews was conducted using NVivo.

Results
Among the 43 residents who participated in the OPENPediatrics™ cardiology curriculum, 33 (77%) completed the curriculum assessment. Residents reported using the curriculum for a median of 8 h. Thirty-eight (88%) reported viewing the curriculum on their personal or hospital computer via pre-downloaded materials on a USB flash drive, with another seven (16%) reporting viewing it online. Twenty-seven residents viewed the course during core lecture time (63%). Commonly reported barriers to utilization included lack of time (70%), access to internet (40%) and language (24%). Scores on knowledge assessment improved from 66.2% to 76.7% upon completion of the curriculum (p < 0.001) across all levels of training, with most significant improvement in scores for PGY-1 and PGY-2 residents. Residents reported high satisfaction with the visuals, engaging presentation, and organization of the curriculum. Residents opined the need for expanded training material in cardiac electrocardiogram and echocardiogram and requested for slower narration by foreign presenters.

Conclusion
Video-based e-learning via OPENPediatrics™ in a resource-limited setting was effective in improving resident’s knowledge in pediatric cardiology with high levels of utilization and satisfaction. Expanding access to digital curriculums for other pediatric sub-specialties may be both an effective and efficient strategy for improving training in settings with limited access to subspecialist faculty.

Real-time diabetic retinopathy screening by deep learning in a multisite national screening programme: a prospective interventional cohort study

Background
Diabetic retinopathy is a leading cause of preventable blindness, especially in low-income and middle-income countries (LMICs). Deep-learning systems have the potential to enhance diabetic retinopathy screenings in these settings, yet prospective studies assessing their usability and performance are scarce.

Methods
We did a prospective interventional cohort study to evaluate the real-world performance and feasibility of deploying a deep-learning system into the health-care system of Thailand. Patients with diabetes and listed on the national diabetes registry, aged 18 years or older, able to have their fundus photograph taken for at least one eye, and due for screening as per the Thai Ministry of Public Health guidelines were eligible for inclusion. Eligible patients were screened with the deep-learning system at nine primary care sites under Thailand’s national diabetic retinopathy screening programme. Patients with a previous diagnosis of diabetic macular oedema, severe non-proliferative diabetic retinopathy, or proliferative diabetic retinopathy; previous laser treatment of the retina or retinal surgery; other non-diabetic retinopathy eye disease requiring referral to an ophthalmologist; or inability to have fundus photograph taken of both eyes for any reason were excluded. Deep-learning system-based interpretations of patient fundus images and referral recommendations were provided in real time. As a safety mechanism, regional retina specialists over-read each image. Performance of the deep-learning system (accuracy, sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were measured against an adjudicated reference standard, provided by fellowship-trained retina specialists. This study is registered with the Thai national clinical trials registry, TCRT20190902002.

Findings
Between Dec 12, 2018, and March 29, 2020, 7940 patients were screened for inclusion. 7651 (96·3%) patients were eligible for study analysis, and 2412 (31·5%) patients were referred for diabetic retinopathy, diabetic macular oedema, ungradable images, or low visual acuity. For vision-threatening diabetic retinopathy, the deep-learning system had an accuracy of 94·7% (95% CI 93·0–96·2), sensitivity of 91·4% (87·1–95·0), and specificity of 95·4% (94·1–96·7). The retina specialist over-readers had an accuracy of 93·5 (91·7–95·0; p=0·17), a sensitivity of 84·8% (79·4–90·0; p=0·024), and specificity of 95·5% (94·1–96·7; p=0·98). The PPV for the deep-learning system was 79·2 (95% CI 73·8–84·3) compared with 75·6 (69·8–81·1) for the over-readers. The NPV for the deep-learning system was 95·5 (92·8–97·9) compared with 92·4 (89·3–95·5) for the over-readers.

Interpretation
A deep-learning system can deliver real-time diabetic retinopathy detection capability similar to retina specialists in community-based screening settings. Socioenvironmental factors and workflows must be taken into consideration when implementing a deep-learning system within a large-scale screening programme in LMICs.

A qualitative study of an undergraduate online emergency medicine education program at a teaching Hospital in Kampala, Uganda

Background
Globally, half of all years of life lost is due to emergency medical conditions, with low- and middle-income countries (LMICs) facing a disproportionate burden of these conditions. There is an urgent need to train the future physicians in LMICs in the identification and stabilization of patients with emergency medical conditions. Little research focuses on the development of effective emergency medicine (EM) medical education resources in LMICs and the perspectives of the students themselves. One emerging tool is the use of electronic learning (e-learning) and blended learning courses. We aimed to understand Uganda medical trainees’ use of learning materials, perception of current e-learning resources, and perceived needs regarding EM skills acquisition during participation in an app-based EM course.

Methods
We conducted semi-structured interviews and focus groups of medical students and EM residents. Participants were recruited using convenience sampling. All sessions were audio recorded and transcribed verbatim. The final codebook was approved by three separate investigators, transcripts were coded after reaching consensus by all members of the coding team, and coded data were thematically analyzed.

Results
Twenty-six medical trainees were included in the study. Analysis of the transcripts revealed three major themes: [1] medical trainees want education in EM and actively seek EM training opportunities; [2] although the e-learning course supplements knowledge acquisition, medical students are most interested in hands-on EM-related training experiences; and [3] medical students want increased time with local physician educators that blended courses provide.

Conclusions
Our findings show that while students lack access to structured EM education, they actively seek EM knowledge and practice experiences through self-identified, unstructured learning opportunities. Students value high quality, easily accessible EM education resources and employ e-learning resources to bridge gaps in their learning opportunities. However, students desire that these resources be complemented by in-person educational sessions and executed in collaboration with local EM experts who are able to contextualize materials, offer mentorship, and help students develop their interest in EM to continue the growth of the EM specialty.

Artificial intelligence for early diagnosis of lung cancer through incidental nodule detection in low- and middle-income countries-acceleration during the COVID-19 pandemic but here to stay

Although the coronavirus disease of 2019 (COVID-19) pandemic had profound pernicious effects, it revealed deficiencies in health systems, particularly among low- and middle-income countries (LMICs). With increasing uncertainty in healthcare, existing unmet needs such as poor outcomes of lung cancer (LC) patients in LMICs, mainly due to late stages at diagnosis, have been challenging-necessitating a shift in focus for judicious health resource utilization. Leveraging artificial intelligence (AI) for screening large volumes of pulmonary images performed for noncancerous reasons, such as health checks, immigration, tuberculosis screening, or other lung conditions, including but not limited to COVID-19, can facilitate easy and early identification of incidental pulmonary nodules (IPNs), which otherwise could have been missed. AI can review every chest X-ray or computed tomography scan through a trained pair of eyes, thus strengthening the infrastructure and enhancing capabilities of manpower for interpreting images in LMICs for streamlining accurate and early identification of IPNs. AI can be a catalyst for driving LC screening with enhanced efficiency, particularly in primary care settings, for timely referral and adequate management of coincidental IPN. AI can facilitate shift in the stage of LC diagnosis for improving survival, thus fostering optimal health-resource utilization and sustainable healthcare systems resilient to crisis. This article highlights the challenges for organized LC screening in LMICs and describes unique opportunities for leveraging AI. We present pilot initiatives from Asia, Latin America, and Russia illustrating AI-supported IPN identification from routine imaging to facilitate early diagnosis of LC at a potentially curable stage.