Abertay University
Dec 19, 2023
It can diagnose disease faster and more accurately than the most skilled experts, spot the signs of disease earlier and recognise patterns that the human eye cannot see. Computer vision is opening up some exciting new possibilities in healthcare. Sarah Harrop takes a closer look.
What is computer vision?
Computer vision is part of that notorious field of computer science which is never far from the headlines these days: artificial intelligence (AI). In brief, computer vision enables computers and systems to process, understand and extract meaningful information from visual data, such as digital images and video.
Just like human vision and perception, computer vision can distinguish between different objects, understand how near or far away they are, whether they are moving and whether there is an abnormality in an image.
But instead of relying on a retina, optic nerve and visual cortex along with a lifetime of context to make sense of what can be seen, as we do, computer vision trains machines to carry out these functions using cameras, data and algorithms.
“If AI enables computers to think, computer vision enables them to see, observe and understand. Because a system trained to inspect products or watch a production asset can analyze thousands of products or processes a minute, noticing imperceptible defects or issues, it can quickly surpass human capabilities,” explains a piece by IBM.
Computer vision technology is already used across a broad cross section of industries, from energy to automotive. According to IBM, it was worth USD48.6 billion in 2022 and looks set to grow further in coming years.
How is computer vision used in healthcare?
Over the past decade, there have been great strides in the use of computer vision that are of particular benefit to medicine and healthcare. In part, this is down to the breakthrough of a type of machine learning called deep learning. This allows computational models made up of multiple processing layers to learn representations of data with multiple levels of abstraction (ie generalisation of detail to extract ‘bigger picture’ information).
Deep learning has turbo powered speech recognition, object recognition, object detection, drug discovery and genomics. According to a review of the field in Nature, two other major factors that have led to uptake of computer vision within healthcare is the progress that has been made in localised compute power via Graphics Processing Units (GPUs) and the open-sourcing of large, labelled datasets which can be used to train the algorithms.
With these resources at their fingertips, the research community using computer vision in medical research and the healthcare industry has grown dramatically and they’ve made some impressive progress.
Computer vision’s strength is that it can identify speedily the type of object in an image, where those objects are and both the type and the location at the same time. This makes it well placed for improving the accuracy, speed and efficiency of many clinical tasks, including screening for diseases, diagnosing and detecting conditions and aiding clinicians’ decision-making. Provided that there is enough data, computer vision can be as accurate, or even more accurate than the level of expert doctors.
Computer vision can make light work of tricky tasks such as pathology image segmentation – for example, categorising the many different types of overlapping cells in tissues seen through a microscope. Key to this has been the development of a type of deep learning algorithm called a convolutional neural network (CNN), which can also pinpoint corresponding points across similar images, find similar images and reconstruct and enhance medical images. For example, converting low resolution computed tomography (CT scan) images into high-resolution versions.
CNNs can accurately diagnose many conditions, particularly relating to ophthalmology (eye diseases). For example, CNNs have consistently shown that they can grade diabetic retinopathy— ‘leaky’ blood vessels in the eyes of diabetic patients that can lead to blindness — at the same level of grading as doctors, which has led to FDA authorisation of an automated AI system for diagnosis of the disease.
Because computer vision can recognise visual patterns in order to spot lesions or make a diagnosis and the fact that there are now more highly structured medical images available than ever before, there are thousands of research publications which have applied its techniques to medical image recognition, particularly in the areas of radiology (eg for cardiovascular disease and lung cancer), cardiology, pathology and dermatology (eg skin cancer).
Some applications of computer vision in healthcare
Detecting eye disease: A deep learning model for accurate, automated, early detection of a disease called geographic atrophy (GA) – a leading cause of blindness – has been developed and proven to work by scientists at Apellis Pharmaceuticals. The research used hundreds of randomly selected standard patient retinal scan images from Moorfields Eye Hospital patients to train the algorithm. When tested on a set of scans from undiagnosed patients, the AI was able to outperform human graders and accurately segment three features of the disease. This raises the promising possibility of automation of analysis for GA from standard eye scans in future.
Phone-based ear diagnostics: Prototype software using a CNN developed by scientists at the University of Calgary was able to differentiate between a healthy eardrum, ears blocked with ear wax and a grommit (tube used to drain fluid from the ear) with 84.4% accuracy. With a large enough dataset to train the algorithm, this research suggests that computer vision could successfully support diagnosis of ear diseases through mountable devices attached to smartphones.
Blood test in a second: Complete blood cell counts are widely used tests in medical diagnoses for a range of health conditions. Traditionally, three types of blood cells – white blood cells, red blood cells and platelets – are counted manually; a time consuming and tedious task for lab staff. Researchers at Bangladesh Institute of Engineering and Technology have developed a machine learning approach for automatic identification and counting of three types of blood cells using an object detection and classification algorithm which allows blood cells to be accurately counted from smear images in less than a second.
Extending the use of endoscopy: Video-based CNNs can be integrated with endoscopes (a small camera in a tube passed down the throat and used by doctors to examine the digestive system) to guide the route taken by the scope and detect signs of disease in endoscope images. Some of the applications of this include faster detection of gastric cancer, infections by ulcer-causing bacteria that can lead to cancer and even finding intestinal parasites.
Smart loos: Stanford University researchers have taken computer vision analysis of the digestive system a step further by building smart toilets with built-in AI that can be used by patients in their own homes. Their state of the art loo is activated by pressure and motion sensors and then gets to work analysing the colour, flow rate and volume of the user’s urine using computer vision, and classifying their stools according to the Bristol Stool Scale, using deep learning. It performs at least as well as trained medics and it collects and stores data on each user securely, identifying them through fingerprints and their individual anatomy. Smart loos could be useful for screening, diagnosis and long-term monitoring of specific patient populations for research.
Choosing viable embryos in IVF: Life Whisperer AI brings together computer vision and deep learning techniques to analyse light microscope images of five-day-old human embryos. It is able to predict, with 70% sensitivity, which ones will go on to successful pregnancies when transferred to a woman’s womb in in vitro fertilisation (IVF). The AI was 24% better at predicting which embryos were viable or non-viable compared with embryologists using traditional grading techniques involving looking at the size, shape and appearance of the embryos. Life Whisperer could lead to better pregnancy success rates and cloud-based software could make it easy to roll out in IVF clinics across the world.
Revolutionising radiology: Computer vision is now used so widely in X-rays, magnetic resonance imaging (MRI) and CT scan image processing that it is a field of research in its own right which extends into all modalities. To give just one example, it has been used to analyse chest X-ray images for heart and lung diseases, with an algorithm trained to high levels of efficiency and accuracy through collecting over a million open-source images with annotation.
Rapid response to COVID: Disease classification, nodule detection, and region segmentation models have been developed for most conditions for which data can be collected, according to the Nature review of the field. This came into its own when the COVID-19 pandemic struck, enabling fast development and roll-out of COVID-19 detection models. A Cornell University ‘confidence-aware anomaly detection (CAAD)’ model can rapidly and accurately detect viral pneumonia using chest X-ray images for large-scale screening and epidemic prevention, for example.
Cancer detection and monitoring: Computer vision can be used to track images of the same patient over a period of time, for example image registration to track the growth of tumors over time to monitor how well cancer treatments are working. And in breast cancer, for example, CNN techniques have been used to count rapidly dividing cancer cells in breast cancer pathology images, replacing the time-intensive and labour-intensive process of manual mitosis counting by a pathologist under a microscope.
What does the future hold for computer vision in healthcare?
“As medical AI advances into the clinic, it will simultaneously have the power to do great good for society, and to potentially exacerbate long-standing inequalities and perpetuate errors in medicine. If done properly and ethically, medical AI can become a flywheel for more equitable care—the more it is used, the more data it acquires, the more accurate and general it becomes,” say the authors of a 2021 Nature review of the field.
However there are four key considerations when applying machine learning technologies in healthcare, they suggest:
Assessment of data – ensuring data quality, removal of any biases and transparency to make sure that failures of the system can be seen
Planning for model limitations – ML can struggle with data samples that are unlike any seen during model training so it is important to develop confidence intervals to detect anomalies and uncertainties.
Community participation – patients, physicians and computer scientists will all need to be involved in successful roll-out of computer vision in healthcare, for example to weed out racial and other biases and make sure that it is fit for real world use. Clinical trials and rigorous evaluation will be needed.
Building trust – Rigorous trials of AI algorithms in real world clinical environments will be essential to earn the trust of clinicians and patients. Computer vision cannot work without access to medical data, and lots of it. Patient trust in the use of their sensitive data must be earned. The ongoing development of privacy preserving techniques for sensitive data such as federated analytics and federated learning—in which centralized algorithms can be trained on distributed data that never leaves protected enclosures — will go a long way towards achieving this.
Computer science expertise for tomorrow’s world
Technology never stands still, and nor does business as it adapts to evolving computer science techniques. Study the University of Abertay’s MSc Computer Science and you’ll develop sought-after skills for the modern work environment – from data structure and algorithms to emerging technologies like drones, AI and cloud computing. The Abertay MSc is designed to equip mid-level professionals looking to take their next step with these up-to-date skills and knowledge to prepare themselves for a successful future. Furthermore, you can earn as you learn and fit the course around your life because it is 100% online and part-time.