• Blog
  • Informational

From Data Silos to Data Fabric with Knowledge Graphs

Knowledge graphs (KGs) have significant advantages to the data architect trying to bring a Data Fabric to their organization. Data Fabric might feel like a buzzword but it underlines a set of architectural and design principles that address the issue of the complex, diverse and large-scale data ecosystem that has long outgrown traditional centralized approaches.

September 15, 2020 8 mins. read Jarred McGinnis

What Makes a Data Fabric?

There is only one thing that the IT industry likes more than a clever acronym. It is a buzzword. ‘Data Fabric’ has reached where ‘Cloud Computing’ and ‘Grid Computing’ once trod. Data Fabric hit the Gartner top ten in 2019. However, Data Fabric is not an application or software package but a set of design principles and strategies to deal with the very real and concrete truth that centralized data storage and control is gone.

Today’s organizations are dealing with data of unprecedented diversity in terms of type, location and use at equally unprecedented volumes and no one is proposing that it is ever going to simplify. This multiplicity of data leads to the growth silos, which in turns increases the cost of integration. Added to this is the increasing demands being made on our data from event-driven and real-time requirements, the rise of business-led use and understanding of data, and the move toward automation of data integration, data and service-level management.

The purpose of weaving a Data Fabric is to remove the friction and cost from accessing and sharing data in the distributed ICT environment that is the norm. It is the understanding that data management must be simplified and the seams between cloud-based storage and local one must be invisible. It must not attempt to force or predict how the data is to be used but to enable that decision to be made by today’s user (and tomorrow’s) while still maintaining organization appropriate access control and security. It is important to remember that in an age where new technologies can go from cult usage to widespread adoption with astonishing rapidity that a Data Fabric aims to orchestrate existing and future data services rather than replace existing infrastructure.

The Data Fabric paradigm combines design principles and methodologies for building efficient, flexible and reliable data management ecosystems. This means the creation of reusable data services, machine-readable semantic metadata and APIs that ensure the integration and orchestration of data across the organization and with third-party external data.

Knowledge Graphs are the Warp and Weft of a Data Fabric

To implement any Data Fabric approach, it is essential to be able to understand the context of data. This means having the ability to define and relate all types of metadata. There must be a representation of the low-level technical and operational metadata as well as the ‘real world’ metadata of the business model or ontologies.

There are several characteristics of the knowledge graph that make this possible:

  • The formal semantics of the RDF standards stack, and the use of ontologies, allows data and metadata of different types to be represented in a KG in such a way that they can be analyzed and interpreted together, without ambiguity, clashes or loss of information. This provides a solid foundation for efficient data integration.
  • Connecting the data in a graph allows concepts and entities to complement each other’s description. Given a critical mass of domain knowledge and good level of connectivity, KGs can serve as context that helps computers comprehend and manipulate data.
  • Even when structured data is integrated, often there are unstructured elements, e.g. free-text attribute values, which need further analysis before they can be accurately interpreted, aligned and combined with the rest of the data. KGs can be used to foster text analysis and make this easier, as in Ontotext Platform.

In other words, knowledge graphs solve the data silo problem by making it irrelevant. The use of knowledge graphs doesn’t try to enforce yet another format on the data but instead overlays a semantic data fabric, which virtualizes the data at a level of abstraction more closely to how the users want to make use of the data. The multiple and varying ‘views’ of the data are now possible without modifying the data at its source or the host system.

New call-to-action

 

The users are freed from having to negotiate the particularities of where the data is, how to get it and the effect of changes on that data for others. With knowledge graphs, users can create on-the-fly views of the data without duplication and without being beholden to the idiosyncrasies of the data’s origins and tailored to the user’s security privileges, technical ability and needs.

Ontotext’s Platform for Enterprise Knowledge Graphs

With its cloud-agnostic infrastructure, Ontotext Platform can operate in cloud and on-premise environments. It is an enterprise ready platform supporting LDAP integration, multi factor authentication and performance monitoring dashboards.

Ontotext Platform ensures data is accessible to the people in the organization that need the data rather than depending on a technical staff to package it and ferry it to them. Often software architects rely on customized APIs that require extra time and effort to develop. These additional software components need to be updated, tested and deployed, which goes counter to the Data Fabric goal of creating frictionless movement of data.

Ontotext uses an automatically generated GraphQL API to support efficient integration into presentation layers. The GraphQL API created in this process allows for the emerging changes made via the ontology management interface to smooth out the evolution of schema, and it also enables mashup generation for simple adaptation to changing integration requirements. Ontotext Platform provides efficient, consistent and easy access to the data of knowledge graphs via the GraphQL interfaces.

10 Steps toward a Data Fabric with Knowledge Graphs

In order to realize a Data Fabric, there are practical steps needed to bring knowledge graphs to enterprises.

  1. Clarify your business and expert requirements

Establish the goal behind collecting the data and define what questions you want to get answered. If needed, Ontotext’s consultants and partners can advise you on your data management strategy and plans.

  1. Gather and analyze relevant data

Discover what datasets, taxonomies and other information (proprietary, open or commercially available) would serve you best to achieve your goal in terms of domain, scope, provenance, maintenance, etc. Consider using data catalogs for this purpose.

  1. Clean data to ensure data quality

Correct any data quality issues to make the data most applicable to your task. This includes removing invalid or meaningless entries, adjusting data fields to accommodate multiple values, fixing inconsistencies, etc.

  1. Create a human AND machine-meaningful data model

Analyze thoroughly the different data schemata to prepare for harmonizing the data. Reuse or engineer ontologies, application profiles, RDF shapes or some other mechanism on how to use them together. Formalize your data model using standards like RDF Schema and OWL.

  1. Integrate data with ETL or virtualization

Apply ETL tools to convert your data to RDF or use data virtualization to access it via technologies such as NoETL, OBDA, GraphQL Federation, etc. Gather metadata from the underlying systems and generate semantic metadata to make the data easier to integrate, update, discover and reuse.

  1. Harmonize data via reconciliation, fusion and alignment

Match descriptions of one and the same entity across datasets with overlapping scope, handle their attributes to merge the information.

  1. Architect the data management and search layer

Merge different graphs flawlessly using the RDF data model. For locally stored data Ontotext Platform can efficiently enforce the semantics of the data model via reasoning, consistency checking and validation. It can scale in a cluster and synchronize with search engines like Elasticsearch to match the anticipated usage and performance requirements.

  1. Augment your data via reasoning, analytics and text analysis

Enrich your data extracting new entities and relationships from text. By applying inference and graph analytics to uncover new information. Now your graph has more data than the sum of its constituent datasets. It is also better interconnected, which brings more content and enables deeper analytics.

  1. Maximize the usability of your data

Start delivering the answers to your original questions through different knowledge discovery tools such as powerful SPARQL queries, easy to use GraphQL interface, semantic search, faceted search, data visualization, etc. Also, ensure that your data is FAIR (findable, accessible, interoperable and reusable).

  1. Make it easy to maintain and evolve your data fabric

Finally, after you have crafted your knowledge graph and people have started using it, keep it live by setting up your maintenance procedures – the way it would evolve and updates from the different sources will be consumed without compromising data quality.

Ontotext Can Help Realizing Your Company’s Data Fabric

Ontotext technology provides the continuous data operations, data management for analytics and the metadata management that a Data Fabric requires. Each organization has different requirements, challenges and goals when implementing a Data Fabric. Often, this requires a tailored solution to ensure the Data Fabric meets the enterprise’s specific requirements and data strategy.

There is no universal approach, but as more and more organizations in various industries have turned to Ontotext products and services for better enterprise knowledge management, data and content analytics, Ontotext has developed a methodology for transforming and interlinking huge amounts of data and providing intuitive and useful interfaces for the frictionless access to data that today’s organizations need.

Learn more about  enterprise knowledge graph!

 

New call-to-action

Article's content

Technical Author at Freelancer

Jarred McGinnis is a managing consultant in Semantic Technologies. Previously he was the Head of Research, Semantic Technologies, at the Press Association, investigating the role of technologies such as natural language processing and Linked Data in the news industry. Dr. McGinnis received his PhD in Informatics from the University of Edinburgh in 2006.

Human-computer Collaboration with Text Analysis for Content Management

Read about how knowledge-driven computing such as Ontotext’s content management solutions are essential for closing the semantic gap between humans and computers.

RDF-Star: Metadata Complexity Simplified

Read about how RDF-Star brings the simplicity and usability of property graphs without sacrificing the essential semantics that enables correct interpretation and diligent management of the data.

Knowledge Graphs for Open Science

Read about how knowledge graphs model the relationships within scientific data in an open and machine-understandable format for better science

Knowledge Graphs and Healthcare

Read about how industry leaders are using Ontotext knowledge graph technology to discover new treatments and test hypotheses.

Does Your Right Hand Know That Your Left Hand Just Lost You a Billion Dollars?

Read about how by automatically identifying and managing human, software and hardware related outages and exposures, Ontotext’s smart connected inventory solution allows banks to save much time and expenses.

Data Virtualization: From Graphs to Tables and Back

Read about how GraphDB’s data virtualization allows you to connect your data with the knowledge graph regardless of where that data lives on the internet or what format it happens to be in.

Throwing Your Data Into the Ocean

Read about how knowledge graphs help data preparation for analysis tasks and enables contextual awareness and smart search of data by virtue of formal semantics.

Ontotext Invents the Universe So You Don’t Need To

Read about the newest version of Ontotext Platform and how it brings the power of knowledge graphs to everyone to solve today’s complex business needs..

From Data Silos to Data Fabric with Knowledge Graphs

Read about the significant advantages that knowledge graphs can offer the data architect trying to bring a Data Fabric to their organization.

What Does 2000 Year Old Concrete Have to Do with Knowledge Graphs?

Read about how knowledge graphs provide a ‘human-centric’ solution to preserving institutional memory and avoiding operational mistakes and missed business opportunities.

Three’s Company Too: Metadata, Data and Text Analysis

Read about how metadata grew more expressive as user needs grew more complex and how text analysis made it possible to get metadata from our information and data.

The New Improved and Open GraphDB

Read about Ontotext’s GraphDB Version 9.0 and its most exciting new feature – open-sourcing the Workbench and the API Plugins.

It Takes Two to Tango: Knowledge Graphs and Text Analysis

Read about how Ontotext couples text analysis and knowledge graphs to better solve today’s content challenges.

Artificial Intelligence and the Knowledge Graph

Read about how knowledge graphs such as Ontotext’s GraphDB provide the context that enables many Artificial Intelligence applications.

Semantic Search or Knowing Your Customers So Well, You Can Finish Their Sentences For Them

Read about the benefits of semantic search and how it can determine the intent, concepts, meaning and context of the words for a search.

The Knowledge Graph and the Internet’s Memory Palace

Learn about the knowledge graph and how it tells you what it knows, how it knows it and why.

The Web as a CMS: How BBC joined Linked Open Data

Learn what convinced the skeptics on the editorial side of the BBC to try the simple but radical idea of ‘The Web as a CMS’.

Can Semantics be the Peacemaker between ECM and DAM?

Learn about how semantics (content metadata) can give peace a chance and resemble how humans understand and use the content.

The Future is NOW: Dynamic Semantic Publishing

Learn how semantically annotated texts enhance the delivery of content online with Ontotext’s News On the Web (NOW) demo.

Introducing NOW – Live Semantic Showcase by Ontotext

Discover interesting news, aggregated from various sources with Ontotext’s NOW and enjoy their enriched content with semantic annotation.