GraphDB Users Ask: Why does My Import Start Really Fast But Then Starts Losing Speed After a While?

TESTED ON: GraphDB 9.9

August 3, 2021 3 mins. read GraphDB Q&As

ONTOTEXT ANSWER:

Unless you are already running an “empty” ruleset repository, that’s the consequence of inference. The short explanation is that there are two types of inference – forward and backward chaining. Forward chaining starts at the data and infers all possible statements you can get. Backward chaining starts at the query and infers only the statements you can get given a particular query. Forward chaining happens at load time. Backward chaining happens at query time. This means that forward chaining has slower loads and faster queries, plus it is easier to manipulate the materialized triples. Backward chaining leads to fast loads and slower queries.

GraphDB is a forward chaining database. This means that all your statements are materialized during load time. The more statements there are, the harder it is to infer new data. This is why imports get slower over time.

If you want your import to go faster, you can always try to fine-tune the inference:

  • Use a simpler ruleset – GraphDB comes with several flavors of rulesets. You can even disable inference altogether if you don’t need it.
  • Use a custom ruleset – often, you will only need to use a specific thing from a given ruleset, and not all of the rules. For example, for you it may be important only that if some triple has the predicate example:address, then its subject must have the class example:Location. So, your ruleset may only state that rule. In general, it is good to start with a minimum viable ruleset and go from there. You can even debug any custom rulesets.
  • Import now, infer later – in GraphDB, you can import with Preload or disable inference for a particular transaction. Then you can trigger the reinfer action. This will make your partial data available for reads faster. You can also use system statements to change inference rulesets.
  • Scale vertically – inference is a CPU-intensive task that can be parallelized. Therefore, adding more computational power would speed up your imports.

Remember that there’s always a trade off in inference. Sure, the data import may be going slower than expected but, in return, queries are consistently fast and you can easily access these inferred triples programmatically.

Did this help you solve your issue? Your opinion is important not only to us but also to your peers.

New call-to-action

 

Article's content

Ontotext answers questions from our GraphDB users. You can also check out the frequently asked questions on general topics about GraphDB. Or you can get quick answers on technical questions from the community as well as Ontotext experts using the graphdb tag on stack overflow.

GraphDB Users Ask: Where Can We Deploy GraphDB And What Are Some Best Practices?

In this blog, we answer questions from our GraphDB users. This question is about where can one deploy GraphDB and what are some best practices

GraphDB Users Ask: What Isolation Levels Does GraphDB Support?

In this blog, we answer questions from our GraphDB users. This question is about the the isolation levels GraphDB supports..

GraphDB Users Ask: What is the Most Important Hardware Attribute for Optimizing GraphDB Performance?

In this blog, we answer questions from our GraphDB users. This question is about the most important hardware attribute for optimizing GraphDB performance.

GraphDB Users Ask: What is the Best Way to Store the Triples’ History in the Database?

In this blog, we answer questions from our GraphDB users. This question is about the best way to store the triples’ history in the database

GraphDB Users Ask: Can I Use Nested Repositories to Introduce Logical Separation to GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about using nested repositories to introduce logical separation to GraphDB

GraphDB Users Ask: Can I Fine-tune Security on Some of the Endpoints in GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about fine-tuning securing on a GraphDB endpoint.

GraphDB Users Ask: What Are the Different Ways to Deploy GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about the different ways to deploy GraphDB.

GraphDB Users Ask: What is the best way to integrate JSON data in GraphDB?

In this blog, we answer questions from our GraphDB users. This question is about the best ways to integrate JSON data in GraphDB.

GraphDB Users Ask: How Does GraphDB’s Security Work, Especially for Automated APIs?

In this feature, we answer questions from our GraphDB users. This question is about how about GraphDB security workds, especially for Automated APIs

GraphDB Users Ask: Is Kafka Only Used for Exporting Data, or for Importing, or Can We Do Both?

In this feature, we answer questions from our GraphDB users. This question is about if Kafka is used only for exporting or importing data or we can use for both

GraphDB Users Ask: How Do I Change the Configuration of an Existing Connector?

In this feature, we answer questions from our GraphDB users. Today’s question is about how to change the configuration of connector if you’ve made a mistake when creating it

GraphDB Users Ask: Are There Any Administration Differences to Operating a Cluster on GraphDB 10?

In this feature, we answer questions from our GraphDB users. Today’s question is about whether there are administration differences to operating a cluster in GraphDB 10

GraphDB Users Ask: Can I Scale GraphDB?

In this feature, we answer questions from our GraphDB users. Today’s question is if one can scale GraphDB.

GraphDB Users Ask: Can I Change My Inference At Runtime?

In this feature, we answer questions from our GraphDB users. Today’s question is if one can change inference at runtime.

GraphDB Users Ask: How To Mark Statements In A Query As Explicit Or Implicit?

In this feature, we answer questions from our GraphDB users. Today’s question is about how to mark statements in a query as explicit or implicit.

GraphDB Users Ask: Can I Use the Standard Ontop Configurations?

In this feature, we answer questions from our GraphDB users. Today’s question is if one can use the standard Onotp configurations.

GraphDB Users Ask: Should I Use a SPARQL Repository or a HTTP Repository?

In this feature, we answer questions from our GraphDB users. Today’s question us whether to use a SPARQL Repository or a HTTP Repository.

GraphDB Users Ask: Do You Have Any Advice on the Log4j Vulnerability for Different Versions of GraphDB?

In this feature, we answer questions from our GraphDB users. Today’s question is about the Log4j vulnerability for different versions of GraphDB.

GraphDB Users Ask 12 Very Short Questions

In this feature, we answer questions from our GraphDB users. Today, we answer 12 very short question from GraphDB users.

GraphDB Users Ask: Which of the GraphDB Logs Do I Need to Monitor for Problems?

In this feature, we answer questions from our GraphDB users. Today’s question is about GraphDB logs and how to monitor for problems.

GraphDB Users Ask: Can You Help Me Optimize My Queries?

In this feature, we answer questions from our GraphDB users. Today’s question is about how users can optimize their queries.

GraphDB Users Ask: What’s the Difference Between SPARQL and FedX Federation?

In this feature, we answer questions from our GraphDB users. Today’s question is about the difference between SPARQL and FedX federation.

GraphDB Users Ask: What Does The “Insufficient Free Heap Memory” Error Mean?

In this feature, we answer questions from our GraphDB users. Today’s question is about what the “Insufficient Free Heap memory” error means.

GraphDB Users Ask: How To Optimize My Inference?

In this feature, we answer questions from our GraphDB users. Today’s question is about how to optimize inference.

GraphDB Users Ask: Is RDF-Star The Best Choice For Reification?

In this feature, we answer questions from our GraphDB users. Today’s question is about whether RDF-star is the best choice for reification.

GraphDB Users Ask: Can GraphDB Infer Data Based on Values From a Virtualized Repository?

In this feature, we answer questions from our GraphDB users. Today’s question is about if GraphDB’s inference works with virtualized repositories.

GraphDB Users Ask: How Does SHACL Work on GraphDB?

In this feature, we answer questions from our GraphDB users. Today’s question is about how SHACL works on GraphDB.

GraphDB Users Ask: Does GraphDB Support ABAC?

In this feature, we answer questions from our GraphDB users. Today’s question is about if GraphDB supports ABAC.

GraphDB Users Ask: Why Do I Get Errors About GraphDB Being “Unable to Find Valid Certification Path to Requested Target”?

In this feature, we answer questions from our GraphDB users. Today’s question is about getting errors about GraphDB being “unable to find valid certification path to requested target”.

GraphDB Users Ask: How Can I Break Up My Data to Control Access To It?

In this feature on our blog, we answer questions from our GraphDB users. Today’s question is about GraphDB security and access control.

GraphDB Users Ask: Why does My Import Start Really Fast But Then Starts Losing Speed After a While?

In this feature on our blog, we answer questions from our GraphDB users. Today’s question is about GraphDB import speed.

GraphDB Users Ask: Can You Help Me Understand The Built-in GraphDB Security?

In this feature on our blog, we answer questions from our GraphDB users. Today’s question is about GraphDB security.

GraphDB Users Ask: How Many Repositories Can I Have in GraphDB and How Can I Unite the Disparate Data Between Them?

In this feature, we answer questions from our GraphDB users. Today’s question is about the number of repos in GraphDB and accessing the data.