GraphDB Cluster Deployment Strategies

Ensuring High Performance and Reliability in Critical Systems.

February 21, 2024 6 mins. read Ivo Rusev

Ontotext GraphDB stands out as a high-performing and scalable graph database, boasting robust RDF and SPARQL capabilities. However, once the decision is made to integrate GraphDB into your system, the crucial question arises:

What is the proper deployment strategy?

The answer to this question hinges on many variables, and while there is no one-size-fits-all solution, there are general guidelines that can be followed based on your specific requirements. And because often GraphDB is deployed as a business-critical system, such a scenario requires you to set up a high-availability cluster.

This article will focus on the details of setting up a GraphDB cluster, shedding light on the different options based on the intended deployment environment. While the official GraphDB documentation provides comprehensive coverage of this topic, this article aims to offer additional insights and practical hints to improve your deployment experience.

GraphDB Cluster

The GraphDB cluster ensures high availability through data replication among its nodes. A minimum of three nodes is required for a high availability cluster. Scaling beyond three nodes enhances fault tolerance, making the cluster more resilient to potential disruptions and also performance.

To better understand how many nodes we should use, we need to get to know some basic GraphDB cluster terminology. We will start with the types of the cluster nodes.

Server Nodes

Within the GraphDB high availability cluster, there are two types of nodes: a leader and multiple followers. Both types function as regular GraphDB instances, allowing users to use the GraphDB Workbench or API for tasks like querying, modifying data, and adjusting settings. Despite their external similarity, it’s important to note that they operate with slight internal differences.

Leader Node

The leader node plays a key role as a load balancer for the client requests. So when a client request reaches a follower it gets redirected to the leader. The leader handles incoming requests differently where its approach varies based on the type of client request — whether it’s a read or a write. Read requests are redirected to and executed by the node with the lowest load. Write requests are executed only on the leader, because only it is allowed to write.

To sum it up- if a client contacts a follower, the follower redirects it to the leader by allowing the leader to forward the request to the client with the lowest load. The leader can execute both read and write queries. However, the leader is the only one allowed to write data. This means that if there is a high volume of write queries it does not make sense to increase the number of nodes (scale horizontally). Vertical scaling is required by increasing CPU, memory and disk throughput.

Consequently, if you’re dealing with a substantial number of write queries, it doesn’t make sense to increase the number of nodes (scale horizontally). Instead, vertical scaling is necessary, achieved by enhancing CPU, memory, and disk throughput.

Follower Node

All other nodes from the cluster besides the leader are followers. They are passive, meaning that they issue no requests on their own but simply respond to requests from the leader. The followers can execute only read queries. Data from write queries is replicated to the followers after the leader verifies that it is valid. If there are a lot of parallel read queries or even slow-running queries, one option to scale the system would be horizontally by adding more nodes. Remember, more followers equals more parallel reads.

GraphDB Proxies

In this section we will briefly describe what is a cluster proxy, their types and purpose.

Internal Cluster proxy

This is an integrated proxy in the GraphDB instance on a different port that will redirect all requests (besides the cluster-specific communication) to the leader node. The leader will then load balance read queries. This ensures that regardless of which cluster node is reached, it can accept all user requests.

External Server Proxy

The proxy can be deployed separately on its own URL. This is called an external server proxy. This way, you do not need to know where all cluster nodes are. Instead, there is a single URL that will always point to the leader node. The proxy keeps track of the cluster status and will switch the leader if a leader change happens. The externally deployed proxy will behave like a regular GraphDB instance, including opening and using the Workbench.

Setup Recommendations

What would a good setup look like? You need to decide on the number of nodes, where the minimum for a high availability cluster is three. More followers will only be needed if the system needs to handle a lot of parallel queries. For our example, let’s assume a three-node cluster will be enough.

It’s important to correctly choose the hard disks on which the data is stored. GraphDB works best with local disks with high IO throughput. Using Network-Attached Storage (NAS) or even distributed storage will result in performance penalties.

We recommend that you configure an external proxy deployed as a separate GraphDB instance but with low CPU and memory resources. Using an external proxy has the benefit that even if one node fails for some reason you will still have a functioning cluster because the proxy will just redirect the requests to the nodes that are running. However, if we have an internal proxy it is bound to the node and if the node goes down so will the proxy. So we get the following picture:

This setup has one weak spot, a single point of failure: the GraphDB external proxy. If it fails, communication to the cluster will be possible only if we change the URL on which we access the cluster to one of the actual nodes. This is why a better setup would be to use several proxies and a load balancer in front of them, as shown in the diagram below. 

 

This way, the external GraphDB proxy server would not be a single source of failure and the system would be more resilient and highly available. This is the setup that we at Ontotext use for internal projects but also it is what we recommend to our customers. Our cloud offerings are based on similar deployment configurations.

The needed CPU and memory resources for each instance can be found on the GraphDB documentation’s Requirements page. 

Environment-Specific Deployments

The strategies outlined above provide a simplified overview of deployment, as the specifics may vary depending on the infrastructure used. Our customers deploy the GraphDB high availability cluster in various environments such as AWS, Azure, and Kubernetes. Since each infrastructure is unique, you can always get in touch with us and use Ontotext’s solution architects’ and DevOps knowledge to help you design the best possible approach. 

Stay tuned for our follow-up posts on deploying the GraphDB high availability cluster in the Cloud.

Choose your proper deployment strategy!

New call-to-action

 

 

Article's content

Solution/System Architect at Ontotext

Ivo Rusev is a Solution Architect at Ontotext with over 15 years of experience in the IT field. He specialises in Java-related technologies and has expertise in developing back-end and middleware components for complex enterprise applications. Ivo is dedicated to implementing the best development practices to ensure high code quality and easy product maintenance. Alongside his work, he is pursuing a Ph.D. at the University of Ruse, where he teaches Java and Computer Graphics.