GraphDB Users Ask: Can GraphDB Infer Data Based on Values From a Virtualized Repository?

TESTED ON: GraphDB 9.10 Enterprise

October 29, 2021 4 mins. read GraphDB Q&As

ONTOTEXT ANSWER:

The answer to this question loops back to one of our previous posts – GraphDB is a forward chaining database. This means that inferred statements are changed – inserted or removed – during data updates. Virtualized repositories are, as the name suggests, virtual. No data is ingested for these repositories. You can think about them as you would think about a translator. You write a SPARQL query, they translate it to another language and give you the result triples. At no point do they store the triples.

So, if the triples are not stored, is there no way to perform inference on them?

There’s another issue here as well. Even if data was inferred for virtualized repositories, it would probably contain literals. GraphDB implements simple inference based on statement patterns. To give you an example of a statement pattern:

Inference on gift giving

That’s great. But what if we want to specify that the gift was non-trivial, i.e., it cost more than 1 euro? Unfortunately, that’s not possible. GraphDB performs inference directly on the identifiers that IRIs get assigned. Literals, like the value of the object, are not stored in the same data structure. This means that the data structure is kept as small and as fast as possible, but precludes us from doing inference with values.

So, is this a lost cause? Fortunately, it is not. Even before the advent of GraphDB 9.8, there was the notifications API, which listens to specific statement patterns and notifies the user that something’s changed. And although the notifications API is hard to scale, Kafka is not.

GraphDB 9.8 introduced a connector that allows you, when a triple is changed, to feed data about that change into a Kafka producer. This can include entity filters and complex triple patterns. Then this data can be consumed by a script, or, if it’s relatively simple, fed directly into a Smart Updates template. With this, you can use a SPARQL insert to trigger a SPARQL insert to trigger a SPARQL insert…

For a virtual repository, this would require configuring your own Kafka producer at the core database that you are virtualizing. Fortunately, most modern databases, just like GraphDB, have some sort of Kafka connector. What is great about Kafka is that you can do this recursive logic at scale. You may have as many partitions and as many consumers of Kafka data as you want.

def process_kafka_string(msg, graphdb_endpoint):
   if msg.value:  # Insertion
       lead_iri = msg.key.decode("utf-8")
       data = json.loads(msg.value.decode("utf-8"))
       qname = "queries/infer.rq"
       path = os.path.join(os.path.dirname(__file__), qname)
       with open(path, 'r') as request:
           update = request.read().replace("?template", "<" + lead_iri + ">")
           if "someValue" in data:
               update = update.replace("?someValueTemplate", str(data["someValue"]))
           _perform_update_request(graphdb_endpoint, lead_iri, update, "performing inference", "Successfully inferred data for")
   else:  # Truth maintenance.
       lead_iri = msg.key.decode("utf-8")
       qname = "queries/remove.rq"
       path = os.path.join(os.path.dirname(__file__), qname)
       with open(path, 'r') as request:
           update = request.read().replace("?template", "<" + lead_iri + ">")
           _perform_update_request(graphdb_endpoint, lead_iri, update, "removing inferred statements", "Successfully cleaned inferred statements for")

Sample Python-based Kafka consumer that infers statements

And what about retraction of assertions? Or, in other words, when inferred data is no longer relevant, how do we clean it up? If you look at the example above, there is a section on “Truth maintenance”. When a triple is deleted, this also produces a Kafka message. The key of that message is the subject of the triple, and there is no value. So, we can use this to know that we should remove the relevant inferred statements.

The caveat of this is that connectors only listen to typed instances – i.e., entities that have a rdf:type. So, as always, looking into data quality is crucial. Perhaps you want to use SHACL for it?

Did this help you solve your issue? Your opinion is important not only to us but also to your peers.

New call-to-action

 

Article's content

Ontotext answers questions from our GraphDB users. You can also check out the frequently asked questions on general topics about GraphDB. Or you can get quick answers on technical questions from the community as well as Ontotext experts using the graphdb tag on stack overflow.

GraphDB Users Ask: Where Can We Deploy GraphDB And What Are Some Best Practices?

In this blog, we answer questions from our GraphDB users. This question is about where can one deploy GraphDB and what are some best practices

GraphDB Users Ask: What Isolation Levels Does GraphDB Support?

In this blog, we answer questions from our GraphDB users. This question is about the the isolation levels GraphDB supports..

GraphDB Users Ask: What is the Most Important Hardware Attribute for Optimizing GraphDB Performance?

In this blog, we answer questions from our GraphDB users. This question is about the most important hardware attribute for optimizing GraphDB performance.

GraphDB Users Ask: What is the Best Way to Store the Triples’ History in the Database?

In this blog, we answer questions from our GraphDB users. This question is about the best way to store the triples’ history in the database

GraphDB Users Ask: Can I Use Nested Repositories to Introduce Logical Separation to GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about using nested repositories to introduce logical separation to GraphDB

GraphDB Users Ask: Can I Fine-tune Security on Some of the Endpoints in GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about fine-tuning securing on a GraphDB endpoint.

GraphDB Users Ask: What Are the Different Ways to Deploy GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about the different ways to deploy GraphDB.

GraphDB Users Ask: What is the best way to integrate JSON data in GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about the best ways to integrate JSON data in GraphDB.

GraphDB Users Ask: How Does GraphDB’s Security Work, Especially for Automated APIs?

In this feature, we answer questions from our GraphDB users. This question is about how about GraphDB security workds, especially for Automated APIs

GraphDB Users Ask: Is Kafka Only Used for Exporting Data, or for Importing, or Can We Do Both?

In this feature, we answer questions from our GraphDB users. This question is about if Kafka is used only for exporting or importing data or we can use for both

GraphDB Users Ask: How Do I Change the Configuration of an Existing Connector?

In this feature, we answer questions from our GraphDB users. Today’s question is about how to change the configuration of connector if you’ve made a mistake when creating it

GraphDB Users Ask: Are There Any Administration Differences to Operating a Cluster on GraphDB 10?

In this feature, we answer questions from our GraphDB users. Today’s question is about whether there are administration differences to operating a cluster in GraphDB 10

GraphDB Users Ask: Can I Scale GraphDB?

In this feature, we answer questions from our GraphDB users. Today’s question is if one can scale GraphDB.

GraphDB Users Ask: Can I Change My Inference At Runtime?

In this feature, we answer questions from our GraphDB users. Today’s question is if one can change inference at runtime.

GraphDB Users Ask: How To Mark Statements In A Query As Explicit Or Implicit?

In this feature, we answer questions from our GraphDB users. Today’s question is about how to mark statements in a query as explicit or implicit.

GraphDB Users Ask: Can I Use the Standard Ontop Configurations?

In this feature, we answer questions from our GraphDB users. Today’s question is if one can use the standard Onotp configurations.

GraphDB Users Ask: Should I Use a SPARQL Repository or a HTTP Repository?

In this feature, we answer questions from our GraphDB users. Today’s question us whether to use a SPARQL Repository or a HTTP Repository.

GraphDB Users Ask: Do You Have Any Advice on the Log4j Vulnerability for Different Versions of GraphDB?

In this feature, we answer questions from our GraphDB users. Today’s question is about the Log4j vulnerability for different versions of GraphDB.

GraphDB Users Ask 12 Very Short Questions

In this feature, we answer questions from our GraphDB users. Today, we answer 12 very short question from GraphDB users.

GraphDB Users Ask: Which of the GraphDB Logs Do I Need to Monitor for Problems?

In this feature, we answer questions from our GraphDB users. Today’s question is about GraphDB logs and how to monitor for problems.

GraphDB Users Ask: Can You Help Me Optimize My Queries?

In this feature, we answer questions from our GraphDB users. Today’s question is about how users can optimize their queries.

GraphDB Users Ask: What’s the Difference Between SPARQL and FedX Federation?

In this feature, we answer questions from our GraphDB users. Today’s question is about the difference between SPARQL and FedX federation.

GraphDB Users Ask: What Does The “Insufficient Free Heap Memory” Error Mean?

In this feature, we answer questions from our GraphDB users. Today’s question is about what the “Insufficient Free Heap memory” error means.

GraphDB Users Ask: How To Optimize My Inference?

In this feature, we answer questions from our GraphDB users. Today’s question is about how to optimize inference.

GraphDB Users Ask: Is RDF-Star The Best Choice For Reification?

In this feature, we answer questions from our GraphDB users. Today’s question is about whether RDF-star is the best choice for reification.

GraphDB Users Ask: Can GraphDB Infer Data Based on Values From a Virtualized Repository?

In this feature, we answer questions from our GraphDB users. Today’s question is about if GraphDB’s inference works with virtualized repositories.

GraphDB Users Ask: How Does SHACL Work on GraphDB?

In this feature, we answer questions from our GraphDB users. Today’s question is about how SHACL works on GraphDB.

GraphDB Users Ask: Does GraphDB Support ABAC?

In this feature, we answer questions from our GraphDB users. Today’s question is about if GraphDB supports ABAC.

GraphDB Users Ask: Why Do I Get Errors About GraphDB Being “Unable to Find Valid Certification Path to Requested Target”?

In this feature, we answer questions from our GraphDB users. Today’s question is about getting errors about GraphDB being “unable to find valid certification path to requested target”.

GraphDB Users Ask: How Can I Break Up My Data to Control Access To It?

In this feature on our blog, we answer questions from our GraphDB users. Today’s question is about GraphDB security and access control.

GraphDB Users Ask: Why does My Import Start Really Fast But Then Starts Losing Speed After a While?

In this feature on our blog, we answer questions from our GraphDB users. Today’s question is about GraphDB import speed.

GraphDB Users Ask: Can You Help Me Understand The Built-in GraphDB Security?

In this feature on our blog, we answer questions from our GraphDB users. Today’s question is about GraphDB security.

GraphDB Users Ask: How Many Repositories Can I Have in GraphDB and How Can I Unite the Disparate Data Between Them?

In this feature, we answer questions from our GraphDB users. Today’s question is about the number of repos in GraphDB and accessing the data.