Administration :: Configuration :: CORE Module

CORE Module

KIM version [1.6.2.23] officially introduces the CORE module of KIM Server. CORE stands for Co-Occurrence and Ranking of Entities and the power it adds to KIM is timeline analysis and a novel incremental search based on both full text search and co-occurrance of entities. The essence of the approach is a specific indexing, performed on the basis of semantic annotation of text with respect to named entities and key-phrases. These entities form a reduced dimension feature space, where the documents are characterized by the occurrence of the entities. The frequency of occurrence of an entity in a document indicates its level of association with it, which can also be considered as “popularity”. The frequency of co-occurrence of entities is evidence of the existence and strength of an associative relationship between them, although the exact relation type might not be known and this can be used for rich metadata extraction. The dates of the documents are used to provide temporal extent to the context space and allow for analysis of trends of popularity or of association.

CoreDbAPI and TimelinesAPI are now available for doing both tasks - inscremental search and timeline analysis. The first one also provides all the basic routines for storing and retrieving documents, entities, aliases, etc. from/to the database. Although the APIs themselves allow for multiple CORE implementations, there is only one available at the moment and it is based on Oracle since scale ans speed are main issues.

The KIM WebUI has two new sections called respectively CORE Search and Timelines that demostrate the potential of the CORE module for incremental search and for calculating trends. Both are being automatically enabled/disabled depending on whether there is an enabled and correctly configured CORE module. More configuration details are provided below.

Finaly, a tool is provided for the initial creation and population of tables and creating of indices (it can also be used to clear and pre-populate all the tables and re-create the indices). To do so, use bin/toolCoreDb.bat (for Windows) or ./bin/toolCoreDb.sh (for Linux). But before that make sure you have Sesame, KIM and the database configured properly and running. It is a console tool and status messages will be printed as different actions are being performed. Note that the tool will re-create and clear all existing tables and will populate all the entities currently hold by Sesame but not any documents. It will also create but will not update the FTS (full text search) indices as this is done automatically by the CORE module in the process of inserting documents through the API provided.

Configure :

To generally enable/disable the usage of the CORE module edit the %KIM_HOME%/config/install.properties file :
```
	# enables/disables usage of CoreDbAPI
	com.ontotext.kim.KIMConstants.USE_CORE_DB = false
	
```
It is disabled by default, as it demands a database to be expicitly created and configured.
CORE Properties are configured in %KIM_HOME%/config/coredb.properties
```
	# CoreDbAPI parameters for connecting
	com.ontotext.kim.KIMConstants.COREDB_CONNSTRING=jdbc:oracle:thin:@//localhost:1521/orcl
	com.ontotext.kim.KIMConstants.COREDB_USER=user
	com.ontotext.kim.KIMConstants.COREDB_PASS=pass
	com.ontotext.kim.KIMConstants.COREDB_SYNCHRONIZE_COUNT=100
	com.ontotext.kim.KIMConstants.COREDB_OPTIMIZE_COUNT=1000
	
```
- COREDB_CONNSTRING
  The connection string is used to connect to a database runing on certain server and accessible at certain port with the given service name. As long as currently only Oracle 10g databases are supported the usage of the above connection string notation is recommended. At server side there must be a database created, also a database service and a listener must be running in order to access the database.
- COREDB_USER, COREDB_PASS
  These parameters set the user and the password to be used when connecting. The main idea is that you may have several users created which have different default tablespaces, this would allow for changing to another set of tables by just changing the user. Note that the users must have DBA rights on their own tablespace, this is required by the CORE module for the initial creation and population of CoreDb tables and creating of indices.
- COREDB_SYNCHRONIZE_COUNT, COREDB_OPTIMIZE_COUNT
  These parameters determine the count of documents after which automatic sync or optimize of FTS (full text search) indices will fire. The CoreDbAPI provides methods that take these parameters into account, ignore them, or explicitly force sync or optimize operation. What is important about those operations is that sync simply updates the index so that the content of the new entries is included and optimize rather rewrites it from scratch which may take a long time.

Administration :: Configuration :: CORE Module