The Goal

A leading provider of information, benchmark prices and analytics for the energy and commodities markets needed to standardize and automate the process of making better use of the unstructured data coming from price reporters. The goal was to facilitate their work with accurate, up to date information, and enable clients to quickly obtain market information that was more accurate and easier to use. 

Beyond that, the solution also needed to include:

• modeling the knowledge of price reporters using an ontology
• dynamic text analysis adapting to changes in the ontology and training data
• continuous evaluation and improvement of the data extraction quality

The Challenge

The market information produced by the company’s price reporters is extremely time sensitive, vital for investors and other market agents, and is the de facto industry standard and reference. It is very important that this information is as accurate as possible and utilized in downstream products as soon as it becomes available.

Although price reporters follow strict guidelines and methodologies when gathering and validating market information, they input the raw text in the system manually. This made the process prone to human error and differences in writing styles. The information was then sent to clients in unstructured form and it was up to each client to turn it into machine-readable structured data.

The Solution: Ontology-Driven Text Analysis

Ontotext developed a solution that automatically structures the price reporters’ input, based on an ontology that describes the commodity market reported. Once the data is manually input into the system, it goes through the bespoke text analysis (TA) service Ontotext developed for this use case. It turns the raw texts into a structured form, which is then used in various end-user products. By better accessing and understanding this textual data, the solution can contribute to accurate predictive models for price movements.

The solution was built around Ontotext products and TA components – GraphDB, Concept Extraction Service, and Ontotext Metadata Studio. It was further extended with custom components to fulfill the company’s requirements for their specific use case.

Each component brings the following advantages:

• GraphDB:
  • Allows performant and reliable access to the ontology, the documents, and their assigned annotations
  • Provides continuous updates of the data via the Kafka Sink connector and the Kafka GraphDB connector
  • Enables hassle-free integration with the TA service via the GraphDB Text Mining plugin

• Concept Extraction Service:
  • Enables scalability of the text analysis
  • Provides synchronization of the knowledge graph and training data updates with the machine learning model for the text analysis
  • Facilitates explainability and troubleshooting of the text analysis
• Ontotext Metadata Studio:
  • Simplifies the process of monitoring the quality of text extraction to allow Price reporters without any particular knowledge of text analysis tools and techniques to easily do it
  • Enables continuous learning by automatically synchronizing the newly added human data with the TA service, leveraging the input of price reporters to improve the overall quality
  • Makes it easy to evaluate any changes in the behavior of the TA service (including any new versions) against the Ground Truth baseline

Business Benefits

• Delivering a consistent and timely output, regardless of location or the type of commodity market reported
• Minimizing potential negative business and reputational impact on the company as a result of human error or incorrect extraction
• Significant decrease in time and resources for structuring content manually
• Facilitating work of price reporters, enabling better decision support, and adding more value through their core competencies

Why Choose Ontotext?

Thanks to Ontotext’s significant text analysis expertise, the solution helps the company use the price reporters more efficiently and enables them to properly structure market information before it reaches their clients, making it less error-prone and easier to use.

By simplifying the process of creating and managing complex metadata, Ontotext Metadata Studio enables the company’s team to extend and fully control the scope of their text analysis extraction. It removes them from the complexities of the text analysis domain and provides easy-to-use quality insights, which ensure that their downstream products are based on a solid metadata foundation.

Do you think this case resembles your particular business needs?