New computer vision system designed to analyse cells in microscopy videos

15-Feb-2022 - Spain

Researchers at the Universidad Carlos III de Madrid (UC3M) have developed a system based on computer vision techniques that allows automatic analysis of biomedical videos captured by microscopy in order to characterise and describe the behaviour of the cells that appear in the images.

These new techniques developed by the UC3M engineering team have been used for measurements on living tissues, in research carried out with scientists from the National Centre for Cardiovascular Research (CNIC in its Spanish acronym). As a result, the team discovered that neutrophils (a type of immune cell) show different behaviours in the blood during inflammatory processes and have identified that one of them, caused by the Fgr molecule, is associated with the development of cardiovascular disease. This work, published in Nature, could allow the development of new treatments to minimise the consequences of heart attacks. Researchers from the Vithas Foundation, the University of Castilla-La Mancha, the Singapore Agency for Science, Technology and Research (ASTAR) and Harvard University (USA), among other centres, have participated in the study.

"Our contribution consists of the design and development of a fully automatic system, based on computer vision techniques, which allows us to characterise the cells under study by analysing videos captured by biologists using the intravital microscopy technique", says one of the authors of this work, Professor Fernando Díaz de María, head of the UC3M Multimedia Processing Group. Automatic measurements of the shape, size, movement and position relative to the blood vessel of a few thousand cells have been made, compared to traditional biological studies that are usually supported by analyses of a few hundred manually characterised cells. In this way, it has been possible to carry out a more advanced biological analysis with greater statistical significance.

This new system has several advantages, according to the researchers, in terms of time and precision. Generally speaking, "it is not feasible to keep an expert biologist segmenting and tracking cells on video for months. On the other hand, to provide an approximate idea (because it depends on the number of cells and 3D volume depth), our system only takes 15 minutes to analyse a 5-minute video", says another of the researchers, Ivan González Díaz, Associate Professor in the Signal Theory and Communications Department at UC3M.

Deep neural networks, the tools these engineers rely on for cell segmentation and detection, are basically algorithms that learn from examples, so in order to deploy the system in a new context, it is necessary to generate sufficient examples to enable their training. These networks are part of machine learning techniques, which in turn is a discipline within the field of Artificial Intelligence (AI). In addition, the system incorporates other types of statistical techniques and geometric models, all of which are described in another paper, recently published in the Medical Image Analysis journal.

The software that implements the system is versatile and can be adapted to other problems in a few weeks. "In fact, we are already applying it in other different scenarios, studying the immunological behaviour of T cells and dendritic cells in cancerous tissues. And the provisional results are promising", says another of the researchers from the UC3M team, Miguel Molina Moreno.

In any case, when researching in this field, researchers stress the importance of the work of an interdisciplinary team. "In this context, it is important to recognise the prior communication effort between biologists, mathematicians and engineers, required to understand the basic concepts of other disciplines, before real progress can be made", concludes Fernando Díaz de María.