On the Hunt for Patterns: from Hippocrates to Supercomputers

The time has come for years of work and research in the domain of healthcare to reap the benefits of building a multimodal knowledge graph. In this article, we want to present to you the ExaMode project - a research project that will put the capabilities of supercomputers and knowledge graphs in the hands of medical professionals for more efficient patient care through data-driven diagnoses.

May 18, 2020 8 mins. read Teodora Petkova

Ever since Hippocrates founded his school of medicine in ancient Greece some 2,500 years ago, writes Hannah Fry in her book Hello World: Being Human in the Age of Algorithms, what has been fundamental to healthcare (as she calls it “the fight to keep us healthy”) was observation, experimentation and the analysis of data. The entire history and practice of modern medicine, argues Fry, is built on finding patterns in data.

Today, to the hunt for finding patterns, we can add a mighty army of iron soldiers who can help us in the fight for health. These are the so-called supercomputers, led by a smart legion of researchers and practitioners in the fields of data-driven knowledge discovery. Thanks to their might, now scientists and practitioners can develop innovative ways of collecting, storing, processing, and, ultimately, finding patterns in data.

What are supercomputers and why do we need them?

If you, like me, have a penchant for cyberpunk, maybe all this brings to mind Wowbagger – Douglas Adam’s character who “after a period of total boredom, especially on Sunday afternoons, decided to insult everyone in the entire universe in alphabetical order”. Wowbagger used the supercomputer on his space ship to calculate the location of each and every soon-to-be insulted living creature. Alternatively, if you like the history of computing, another association that can come to mind is Licklinder’s Intergalactic Computer Network or the Galactic Network, envisioned in the 1960s.

Both associations are not far from the general idea behind supercomputers as they relate to connecting and working with massive amounts of data.

Supercomputers, also known under the umbrella term high-performance computing (HPC), are machines built to execute tasks that cannot be executed by general-purpose computers. Their main strength lies in the capability to solve a single large problem in the shortest possible time with the maximum computing power available. Such problems and the complexities related to such computationally-intensive tasks are essential in the fields of weather forecasting, molecular modeling, airplane and spacecraft aerodynamics, personalized medicine, self-driving cars.

To keep it simple, the idea behind HPC is to solve a problem by dividing it into chunks and planning how and in what sequence these chunks will be tackled in order to execute the task. The capacity and performance of supercomputers is measured with the so-called FLOPS (floating point operations per second). And since the early 1960s, when one of the first supercomputers – the Livermore Atomic Research Computer (LARC) – was built, until today when Facebook has already bought 26 supercomputers, the FLOPS have been rising.

As of 2017, the fastest computers have reached a speed of 93 PetaFLOPS, which is: 93×1015, or 93,000,000,000,000,000 operations per second. And just when we might have thought FLOPS had hit their limit, here’s another peak achieved at the U.S. Summit: 1.88×1018. This is where the so-called exascale computing enters the stage.

Exascale computing refers to systems capable of at least one exaFLOPS calculation per second and that is billion billion (or if you wish a quintillion) operations per second. Although still not very well-known, exascale supercomputers are poised to dramatically change the way we approach the solutions (at least their computational facet) of the world’s most vexing problems in the areas of climate, healthcare, national security, etc. Another significant application of exascale supercomputers are research projects.

One of them, the ExaMode project, is what we want to tell you about.

Discover more about Ontotext’s Healthcare and Life Sciences Industry Solutions!


The ExaMode project: driven by data, developed for patients

ExaMode, an acronym for Extreme-scale Analytics via Multimodal Ontology Discovery & Enhancement, is a project funded by the European Union, H2020 programme. It aims to solve the challenges that healthcare faces as a result of the heterogeneity and the volume of biomedical data (more than 2’000 exabytes of biomedical data are expected to be produced by 2020).

The ExaMode project aims to develop new architectures and tools for pathologists and medical researchers. These tools will allow them to effectively and efficiently handle extremely large volumes of disparate data – digitized histopathology slides from the visual and textual content of patient’s records, medical publications, diagnoses, etc.

The project is coordinated by the Institute of Information Systems of the HES-SO Valais-Wallis, Sierre, Switzerland. Besides the HES-SO, six other universities, hospitals and companies from Italy, the Netherlands, Poland and Bulgaria form the project consortium. These partners are: the Department of Information Engineering from the University of Padova (UNIPD), Radboud University Medical Center, MicroscopeIT, Cannizzaro Hospital (AOEC), SurfSara and Sirma AI, trading as Ontotext.

ExaMode’s main goal is to help professionals efficiently search and work with medical (and more specifically histopathology) data, developing a system for easy and fast knowledge discovery based on heterogeneous exascale data. With the help of such sophisticated tools, a physician will be able, for example, to access a constellation of information related to a patient’s case, including similar cases, the latest publications in the field, specific terms or visual features and related images in both scientific literature and hospital information systems.

Both the information inferred from the image analysis and from the raw textual data in the EHR records needs to be semantically normalized in order to be used for the generation of the multimodal knowledge graph.

And this is what Ontotext’s role in the project is about: knowledge graphs. More specifically, the semantically normalized annotation of images and textual resources that are further fed into a knowledge graph for easier search and discovery.

Behind the scenes of linking histopathology data and building a knowledge graph out of it

Together with the other partners, Ontotext will be leveraging text analysis in order to extract structured data from medical records and from annotated images related to histopathology information. Furthermore, the team will be working to normalize the data with established public medical ontologies to create a knowledge graph and thus enable knowledge discovery, identification of similar medical cases and referential cases described in the scientific literature.

Again, the overall aim is to extract knowledge from data and, through algorithms based on artificial intelligence, to assist medical professionals in routine diagnostics processes.

Text is a unique resource because medical reports carefully describe the diagnosis associated with multimodal and multimedia data and also the relevant image content. Moreover, text is also used in scientific papers and blog posts that describe specific images, thus making healthcare-related data potentially the biggest annotated data collection worldwide. [Information source: H2020 grant for Computational Pathology]

There are four types of data sources that the team will work with. The first type is metadata from images. The second is the brief texts doctors write to summarize each of the patient’s images and findings in them, the so-called synopses, where they describe each individual image case in short. The third type of data comes from the longer text forms such as discharge letters and EHRs (Electronic Health Records), the clinical record of the patient’s stay in the hospital information system that include the anamnesis, complaints, diagnoses, treatment, etc. The final one is research publications data from the PubMed repository – the repository of the US National Library of Medicine collected from quality sources. The relevant scientific publications in full-text format can be accessed there.

All these descriptions of images, the brief summaries of patient records and the articles from research databases (in various formats, be it a scientific publication, a description, a synopsis, etc.) will be processed. This will be done by mapping the extracted data to relevant ontologies  – already existing ones as well as specialized ontologies developed by UNIPD and AOEC that cover histopathological conditions. The training of the image processing algorithms requires massive computing power, which will be provided by the exascale computer hosted on one of the most energy-efficient data centers – SurfSara, in Amsterdam. It is called Cartesius and is one of the best high-performance computers in Europe.

GraphDB and the ExaMode tools will blend the data and provide a semantic layer to it. Ontotext’s signature RDF database will create a powerful knowledge graph where all pieces will be put together to serve computer-aided diagnosis systems. At the end of the day, enriching text (and images) with semantic metadata will allow for better knowledge discovery, which will help doctors with their diagnosis and decision making.

Epilogue: Will your next doctor be a supercomputer?

Certainly not!

But your doctor will definitely have a richly interlinked archive to consult. ExaMode and the objectives it sets are by no means about replacing the human doctor but rather about doing the heavy lifting of processing data.

A supercomputer-powered decision support system will allow physicians to use a highly interconnected architecture of medical records, histopathological images and scientific publications. This will level up the processes of observation, experimentation and analysis, which are fundamental for medicine. More importantly, it will take the patient diagnosis and care to the next level.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825292 (ExaMode, htttp://www.examode.eu/)

Discover how Ontotext’s Healthcare and Life Sciences solutions enable advanced analytics!

New call-to-action

Article's content

Marketing Expert at Ontotext

Teodora is a philologist fascinated by the metamorphoses of text on the Web. Curious about our networked lives, she explores how the Semantic Web vision unfolds, transforming the possibilities of the written word. From 2022 on, Teodora helps with the creation and curation of the Ontotext knowledge graph to foster information ecology out of marketing content that will enable relevant user experiences across Ontotext's universe.

GraphDB in Action: Smells Like Semantics Spirit

Read about a project called Odeuropa and a number of exciting applications delivered by it with our RDF graph database humming behind them

KGF 2023: Bikes To The Moon, Datastrophies, Abstract Art And A Knowledge Graph Forum To Embrace Them All

Read about this year’s Knowledge Graph Forum – a space where Ontotext and partners presented smart and linked ways to tame and thrive on complexity, rather than be drowned by it

Do Large Language Models Dream of Knowledge Graphs – Impressions from Day 2 At SEMANTiCS 2023

Read our report from Day 2 of SEMANTiCS 2023 to find out if ChatGPT is the killer app for the Semantic Web, how do we tame the genie of LLMs for Healthcare and more

Can LLMs Become Knowledgeable – Impressions from Day 1 At SEMANTiCS 2023

Read about the interplay between LLMs & KGs and how business and academia tackle them in our report from Day 1 at SEMANTiCS 2023

It’s Time We Give Each Other More Data Spaces: Impressions from the Pre-conference Day at SEMANTiCS 2023

Read about SEMANTiCS pre-conference day, which covered the topics of interoperability, ESG data, knowledge engineering, scholarly communication, and academia & industry collaboration.

GraphDB in Action: Navigating Knowledge About Living Spaces, Cyber-physical Environments and Skies 

Read about three inspiring GraphDB-powered use cases of connecting data in a meaningful way to enable smart buildings, interoperable design engineering and ontology-based air-traffic control

Your Knowledge Graph Journey In Three Simple Steps

A bird’s eye view on where to start in building a knowledge graph solution to help your business excel in a data-driven market

GraphDB in Action: Putting the Most Reliable RDF Database to Work for Better Human-machine Interaction

Read about the world of academia research projects that use GraphDB to meet the challenges of heterogeneous data across various domains

Knowledge Graphs for Retail – Connecting People, Products and Platforms

Read about how knowledge graphs can serve the retail industry’s growing need to connect, manage and utilize data efficiently, aligning it in a collaborative data ecosystem

Data Wants To Be Truly Sovereign: Designing Data Spaces with Linked Data Principles In Mind

Read about what data spaces are and how semantic technologies and Linked Data can make them a stronger and safer mechanism for commercial data exchange

GraphDB in Action: Powering State-of-the-Art Research

Read about how academia research projects use GraphDB to power innovative solutions to challenges in the fields of Accounting, Healthcare and Cultural Heritage

KGF22: Knowledge Graphs and The Not So Quiet Cognitive Revolution

Read about Ontotext’s KGF22 days dedicated to stories about knowledge graphs in the domains of Industry, Healthcare & Life Sciences and Financial Services

KGF22: Wittgenstein, Developers Empathy and Other Semantic Data Considerations

Read about our event report from Ontotext’s Knowledge Graph Forum 2022, highlighting expert insight on building knowledge graphs and designing enterprise-grade solutions with semantic technologies.

A Little SEMANTiCS Goes A Long Way

Take a sneak peek at some of the keynote speeches and tutorials throughout SEMANTiCS 2022

It Takes A Village To Raise An Enterprise Knowledge Graph

Read about the design processes behind crafting knowledge-graph enabled solutions and explore some of the stories of our partners.

Smart Buildings Are Built of Smart Data: Knowledge Graphs for Building Automation Systems

Read about how knowledge graphs offer a sustainable solution for harnessing and making sense of heterogeneous data in the building automation industry.

Metadata Moves: Knowledge Graph Technology for Logistics

Read about how the world of metadata humming behind the logistics and other supply chain processes can benefit from using knowledge graph technology.

Electrical Standards, Smart Grids and Your Air Conditioner

Read about how applying Linked Data principles and semantic technology to electricity data can make for a more efficient, reliable and sustainable electricity market.

The Semantic Web: 20 Years And a Handful of Enterprise Knowledge Graphs Later

Read about how the Semantic Web vision reincarnated in thousands of Linked Open Data datasets and millions of Schema.org tagged webpages. And how it enables knowledge graphs to smarten up enterprises data.

Metadata is Like Packaging: Seeing Beyond the Library Card Metaphor

Read about what metadata is, why it is important and how it can enhance the ways information flows across the enterprise.

From Fragmented Data to a Comprehensive Knowledge Graph: The Case for Building an R&D Repository

Read about how enterprise knowledge graphs can unlock meaning and thus create a smart future-proof living repository of scientific data and its relationships.

Texts Without Pages: Advancing Text Analytics with Content Enrichment

Read about how text analytics can be brought forward with content enrichment processes for better text authoring, delivery and navigation.

A Shield Built of Connected Data: Knowledge Graphs Meet Cybersecurity

Read about how a knowledge graph can help organizations stay vigilant of the increasing number of cyber threats, keeping malicious attacks at bay with the help of semantics.

Digital Twins: If It Sounds Like Cyberpunk, It’s Because It Is

Read about what digital twins are, what makes them attractive to companies and how digital twins relate to semantic technology and enable organizations to design, simulate and validate various scenarios virtually.

Eating the Knowledge Soup, Literally

Read about the fluid essence of knowledge and the capability of knowledge graphs to power an information-rich platform of diverse facts about anything, a broccoli soup included.

If Curiosity Cabinets Were Knowledge Graphs

Read about why and how knowledge graph technology can help build networks of interwoven digital objects in the world of cultural heritage.

On the Hunt for Patterns: from Hippocrates to Supercomputers

Read about the ExaMode project that will help medical professional use the power of supercomputers and knowledge graphs for more efficient patient care through data-driven diagnoses.

Crafting a Knowledge Graph: The Semantic Data Modeling Way

Read about how to build a knowledge graph the semantic data modeling way in 10 steps, provided by our knowledge graph technology experts.

A Graphful of Investment Opportunities

Read about the story of an algorithm that mines data to narrow down opportunities for investing.

Okay, You Got a Knowledge Graph Built with Semantic Technology… And Now What?

Read about how knowledge management can be made smarter using a knowledge graph built with semantic technology.

If Johnny Mnemonic Smuggled Linked Data

Read about how semantic technology and Linked Data can help enterprises benefit from smart data management and retrieval practices.

Data, Databases and Deeds: A SPARQL Query to the Rescue

Read about why and how SPARQL queries make for a better search in diverse datasets across an organization in an integrated way.

Semantic Technology and the Way We See the World

Read about how semantic technology can help you set the wheels for many processes related to еfficient data management and governance.

Telling Stories with an RDF Database

Read about the opportunities for authoring and publishing workflows opened by an RDF triplestore.

The Power of URI or Why Odysseus Called Himself Nobody

Read about URI and its power to enable the sharing and reuse of machine-readable data with minimum integration costs.

From Cultivating Nature to Cultivating Data: Semantic Technology and Viticulture

Learn how the potential that Big Data streams bring to grape and wine production can be harnessed with the right kind of technology.

The Knowledge Graph and the Enterprise

Read about the knowledge graph and about how many enterprises are already embracing the idea of benefiting from it.

It Don’t Mean a Thing If It Ain’t Got Semantics

Learn how you can turn data pieces into actionable knowledge and data-driven decisions with an RDF database.

The Bounties of Semantic Data Integration for the Enterprise

Learn about the potential semantic data integration carries for piecing massive amounts of data together.

Here’s a Graph, Go Figure! Coupling Text Analytics with a Knowledge Graph

Learn why and how a Knowledge Graph boosts significantly Text Analytics processes and practices and makes text work for us in a more meaningful way.

Cognitive Computing: Let’s Play an Awareness Game

Read about the new breed of computing is emerging before our eyes – cognitive computing and join us in our Awareness Game.

Machine Learning and Our (Insatiable) Penchant for Making Things Smarter

Read about how machines can be of great help with many tasks where fast and error-free computation over big amounts of data are required.

Staying In the Vanguard of Digital Transformation with Open Data

Learn about Open Data and its potential to be part of smart solutions to data problems and innovative products and services.

Whose Meaning? Which Ontology?

Read about how ontologies open up opportunities for a new class of tools to power information consumption and knowledge management.

Shiny Happy Data: A Praise for RDF

Learn how to choose the right solution for working with your data the conceptual framework of “happy connected people”.

Enterprise Metadata Matters: From Having Data to Acting Upon Them

Learn more about the importance of being metadata-driven today in our latest SlideShare presentation.

Data Daiquiri: The Power of Mixing Data

Learn how companies can tap into the power of the data coming their way by integrating the huge data flows with their proprietary data.

Migrating to GraphDB: Your Why and How in 20 slides

Learn about the steps you need to migrate your data to GraphDB to use it as a smart brain on top of your legacy systems.

Got meaning? Or Why an RDF Graph Database Is Good for Making Sense of Your Data

Read about how you can create systems capable of discovering relationships and detecting patterns within all kinds of data.

Brains Armored with Smart Data

Read our thoughts rising from questions such as “Will Giant Brains Rule the World?” and “Can a mechanical brain replace you?”

One Step Closer to Intertwingularity: Semantic Metadata

Learn about how semantic metadata allows us to add granularity to an object, interlink it to other objects and make it easy to search.

Exceptional User Experiences with Meaningful Content NOW

Content enrichment and semantic web technologies are key to efficient content management. Learn why and see these technologies in action.

Semantic Information Extraction: From Data Bits to Knowledge Bytes

Learn about semantic information extraction and how it pulls out meaningful data from textual sources, ready to be leveraged for insights, decisions and actions.

Weaving Data Into Texts: The Value of Semantic Annotation

Read about how semantic annotation links certain words to context and references that can be processed by an algorithm.

Exploring Linked Open Data with FactForge

Learn about FactForge and how you can turn the opportunities that data flows on the web can pour into our business into a real experience.

What is GraphDB and how can it help you run a smart data-driven business?

Learn about GraphDB in a simple and easy to understand way and see what Ontotext’s semantic graph database has to do with pasta making.

Linked Data for Libraries: Our New Librarians

Learn how semantic technologies can bring audiences back to libraries and make library archives and collections visible and accessible.

Working with Data Just Got Easier: Converting Tabular Data into RDF Within GraphDB

Read about OntoRefine – a new tool that allows you to do many ETL (extract, transform and load) tasks over tabular data.

GraphDB: Answers for Kids of All Ages

Read about how GraphDB can help you clean up messes of data (just like your room) – whether you are a kid or not.

The Knowledge Discovery Quest

Learn how by joining the dots, semantic search enhances the way we look for clues and compare correlations on our knowledge discovery quest.

Connectivity, Open Data and A Bag of Chips

Learn how LOD’s connectivity allows data to be shared seamlessly, used and reused freely. As simple as a bag of chips.

Data Integration: Joining the Data Pieces of Your Business Puzzle

Learn how to use information interconnectedness to integrate, interpret and ultimately make sense of data.

Cooking Up the Semantic Web

Read about the Semantic Web and what it takes to reach its potential and evolve from a Web of Documents to a Web of Data.

Semantic Search: The Paradigm Shift from Results to Relationships

Read about semantic search and how it takes information retrieval to the next level and puts information at our fingertips.

A Web of People and Machines: W3C Semantic Web Standards

Learn how and why Semantic Web Standards are to serve the Web of Data for better collaboration between people through computers.

Thinking Outside the Table

Learn how to manage highly connected data, working with complex queries and having readily available relationships, without the need to express them explicitly.

Our Networked Lives, Publishing and Semantic Technologies

Read about how semantic technology helps publishing handle data in an interconnected way, attaching machine-processable and readable meaning to them.

Why Graph Databases Make a Better Home for Interconnected Data Than the Relational Databases?

Read about how you can turn data into a resource, easily accessed and effectively used across the organization with a graph database.

Text, Data and the Roman Roads: Semantic Enrichment

Read about semantic enrichment and the unique opportunity it offers for interconnecting objects to facilitate knowledge discovery.

4 Things NOW Lets You Do With Content

Go beyond conventional publishing with Ontotext’s News On the Web and get the feel of how you can discover and consume content with semantic technology.