From kelebelloze at gmail.com  Tue Nov  8 09:10:57 2011
From: kelebelloze at gmail.com (Kele Belloze)
Date: Tue, 8 Nov 2011 12:10:57 -0200
Subject: [Kim-discussion] Annotation with KimPlugin
Message-ID: <CAN6u3=ATYuVu0RmkUPmYC-=mLLCkHFoXGo=m6-haqszUVF9uGA@mail.gmail.com>

Hi,
i'm a student in the Oswaldo Cruz Institute (Brazil) and I'm trying KIM for
semantic annotation of documents. I installed it. But, If the Kim Plugin is
no longer supported, how to perform these annotations? It is not clear in
the documentation.

Can you help me?

Thanks,
Kele
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111108/a9d8a129/attachment.html>

From philip.alexiev at ontotext.com  Wed Nov  9 04:10:13 2011
From: philip.alexiev at ontotext.com (Philip Alexiev)
Date: Wed, 9 Nov 2011 11:10:13 +0200
Subject: [Kim-discussion] Annotation with KimPlugin
In-Reply-To: <CAN6u3=ATYuVu0RmkUPmYC-=mLLCkHFoXGo=m6-haqszUVF9uGA@mail.gmail.com>
References: <CAN6u3=ATYuVu0RmkUPmYC-=mLLCkHFoXGo=m6-haqszUVF9uGA@mail.gmail.com>
Message-ID: <31D4691C-938A-4892-B95F-188F11F5536D@ontotext.com>

Hi Kele,

Thank you for your interest in KIM.

The easiest way to start is with the Quick Start Guide from the documentation.  I will provide the direct link for convenience: Quick Start Guide .

All the best,
Philip Alexiev
Software Engineer, KIM team

On 8 Nov 2011, at 4:10 PM, Kele Belloze wrote:

> Hi,
> i'm a student in the Oswaldo Cruz Institute (Brazil) and I'm trying KIM for semantic annotation of documents. I installed it. But, If the Kim Plugin is no longer supported, how to perform these annotations? It is not clear in the documentation.
> 
> Can you help me?
> 
> Thanks,
> Kele _______________________________________________
> Kim-discussion mailing list
> Kim-discussion at ontotext.com
> http://ontotext.com/mailman/listinfo/kim-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111109/2a1d0dc9/attachment.html>

From borislav at sirma.bg  Fri Nov 11 08:45:11 2011
From: borislav at sirma.bg (borislav popov)
Date: Fri, 11 Nov 2011 15:45:11 +0200
Subject: [Kim-discussion] Fwd: Annotation with KimPlugin
References: <CAN6u3=AGL86CKkgSsqjh_BFUuCLjcH4ZGq4XecTm5GbRcGQW9w@mail.gmail.com>
Message-ID: <422B2A0E-DCF7-4064-84D6-E79F0C10B5D8@sirma.bg>



Begin forwarded message:

> From: Kele Belloze <kelebelloze at gmail.com>
> Subject: Annotation with KimPlugin
> Date: November 8, 2011 4:09:02 PM GMT+02:00
> To: kim-info at ontotext.com
> 
> Hi,
> i'm a student in the Oswaldo Cruz Institute (Brazil) and I'm trying KIM for semantic annotation of documents. I installed it. But, If the Kim Plugin is no longer supported, how to perform these annotations? It is not clear in the documentation.
> 
> Can you help me?
> 
> Thanks,
> Kele

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111111/02724713/attachment.html>

From kelebelloze at gmail.com  Wed Nov 16 09:09:40 2011
From: kelebelloze at gmail.com (Kele Belloze)
Date: Wed, 16 Nov 2011 12:09:40 -0200
Subject: [Kim-discussion] Annotation with KimPlugin
In-Reply-To: <31D4691C-938A-4892-B95F-188F11F5536D@ontotext.com>
References: <CAN6u3=ATYuVu0RmkUPmYC-=mLLCkHFoXGo=m6-haqszUVF9uGA@mail.gmail.com>
	<31D4691C-938A-4892-B95F-188F11F5536D@ontotext.com>
Message-ID: <CAN6u3=DkxQEobGTkuFqGvo4aPPoynLwB5FCBxV=fkByu-S3wSA@mail.gmail.com>

Hi Philip,
sorry for the delay. Thank you for contact.
I was trying to run KIM again. I followed the Quick Start Guide and I
visualized the annotated documents in the KIM Web UI.

But, I have two questions:
1- Where do I access (which folder) the annotated documents? What format
are they stored?
2 - Can I load an arbitray ontology?

Regards,
Kele
2011/11/9 Philip Alexiev <philip.alexiev at ontotext.com>

> Hi Kele,
>
> Thank you for your interest in KIM.
>
> The easiest way to start is with the Quick Start Guide from the
> documentation.  I will provide the direct link for convenience: Quick
> Start Guide<http://www.ontotext.com/sites/default/files/kim/KIM_Getting_Started_Guide_0.pdf>
>  .
>
> All the best,
> Philip Alexiev
> Software Engineer, KIM team
>
>   On 8 Nov 2011, at 4:10 PM, Kele Belloze wrote:
>
>   Hi,
> i'm a student in the Oswaldo Cruz Institute (Brazil) and I'm trying KIM
> for semantic annotation of documents. I installed it. But, If the Kim
> Plugin is no longer supported, how to perform these annotations? It is not
> clear in the documentation.
>
> Can you help me?
>
> Thanks,
> Kele _______________________________________________
> Kim-discussion mailing list
> Kim-discussion at ontotext.com
> http://ontotext.com/mailman/listinfo/kim-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111116/2b5b9fef/attachment.html>

From philip.alexiev at ontotext.com  Wed Nov 16 09:21:12 2011
From: philip.alexiev at ontotext.com (Philip Alexiev)
Date: Wed, 16 Nov 2011 16:21:12 +0200
Subject: [Kim-discussion] Annotation with KimPlugin
In-Reply-To: <CAN6u3=DkxQEobGTkuFqGvo4aPPoynLwB5FCBxV=fkByu-S3wSA@mail.gmail.com>
References: <CAN6u3=ATYuVu0RmkUPmYC-=mLLCkHFoXGo=m6-haqszUVF9uGA@mail.gmail.com>
	<31D4691C-938A-4892-B95F-188F11F5536D@ontotext.com>
	<CAN6u3=DkxQEobGTkuFqGvo4aPPoynLwB5FCBxV=fkByu-S3wSA@mail.gmail.com>
Message-ID: <6AE77A28-9605-4C8A-8452-D6982A242017@ontotext.com>

Hello Kele,

You can not access the stored documents directly.  Only though the KIM interfaces.

The documents are stored in 2 separate places:
 - The content of the document and the annotations are stored in a Lucene index inside KIM. It is used for full text search and  visualization.
 - Information about occurrances  (which entities are mentioned in which documents) are stored in a separate Owlim index. It is used for the more complicated semantic search.

You can however observe the result of the annotation process by running the Gate interface of KIM and annotating a document. Do this by entering  './kim  gate'  at the command line when inside KIM/bin .  This will load the configuration of resources used in the annotation phase of KIM.  Then add a document of your choice and execute the pipeline (additional information on how to work with gate can be found at the official site:   http://gate.ac.uk ).  Please note that annotating a document this way will not  insert it into KIM. That is - you will not see it the next time you start KIM and browse the visual interfaces.

I hope this answers your question. If not, please be more specific as to what you want to achieve.

All the best,

Philip Alexiev
Software Engineer, KIM team 
 

On 16 Nov 2011, at 4:09 PM, Kele Belloze wrote:

> Hi Philip,
> sorry for the delay. Thank you for contact.
> I was trying to run KIM again. I followed the Quick Start Guide and I visualized the annotated documents in the KIM Web UI.
> 
> But, I have two questions:
> 1- Where do I access (which folder) the annotated documents? What format are they stored?
> 2 - Can I load an arbitray ontology?
>  
> Regards,
> Kele
> 2011/11/9 Philip Alexiev <philip.alexiev at ontotext.com>
> Hi Kele,
> 
> Thank you for your interest in KIM.
> 
> The easiest way to start is with the Quick Start Guide from the documentation.  I will provide the direct link for convenience: Quick Start Guide .
> 
> All the best,
> Philip Alexiev
> Software Engineer, KIM team
> 
> On 8 Nov 2011, at 4:10 PM, Kele Belloze wrote:
> 
>> Hi,
>> i'm a student in the Oswaldo Cruz Institute (Brazil) and I'm trying KIM for semantic annotation of documents. I installed it. But, If the Kim Plugin is no longer supported, how to perform these annotations? It is not clear in the documentation.
>> 
>> Can you help me?
>> 
>> Thanks,
>> Kele _______________________________________________
>> Kim-discussion mailing list
>> Kim-discussion at ontotext.com
>> http://ontotext.com/mailman/listinfo/kim-discussion
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111116/4a7bd25f/attachment.html>

From reneta.popova at ontotext.com  Tue Nov 22 07:08:39 2011
From: reneta.popova at ontotext.com (Reneta Popova)
Date: Tue, 22 Nov 2011 14:08:39 +0200
Subject: [Kim-discussion] Fwd: Dictionary life cycle for the Large KB
	Gazetteer
References: <CA+zNo+EV0KXKAo4LyMDmFBVRPkkB2GqMMuPcAjqHb7-kSx5uYQ@mail.gmail.com>
Message-ID: <9F97A94F-A977-464C-9EA6-7BBEFB6DCF86@ontotext.com>



Begin forwarded message:

> From: Mihaela Olteanu <olteanu.miha at gmail.com>
> Subject: Dictionary life cycle for the Large KB Gazetteer
> Date: 22 ??????? 2011 13:57:18 ????????+0200
> To: marin.nozhchev at ontotext.com, danko at ontotext.com, reneta.popova at ontotext.com
> 
> Hello,
>  
> I am using the Large KB Gazetteer with AllegroGraph. The ontology definition that I use can change while the pipeline is running.
> In this case I could simply reinitialize the processing resource. The question is: this is the only way of reloading the dictionary for large KB gazetteer? Can't this operation be done automatically, maybe setting the gazetteer as listener to my updated ontology event and on message reinitialize the component automatically?
> Is it possible to load only the newly added definitions, or the updates?
>  
> Thanks,
> Mihaela

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111122/45717efe/attachment.html>

From philip.alexiev at ontotext.com  Tue Nov 22 08:18:54 2011
From: philip.alexiev at ontotext.com (Philip Alexiev)
Date: Tue, 22 Nov 2011 15:18:54 +0200
Subject: [Kim-discussion] Dictionary life cycle for the Large KB
	Gazetteer
In-Reply-To: <9F97A94F-A977-464C-9EA6-7BBEFB6DCF86@ontotext.com>
References: <CA+zNo+EV0KXKAo4LyMDmFBVRPkkB2GqMMuPcAjqHb7-kSx5uYQ@mail.gmail.com>
	<9F97A94F-A977-464C-9EA6-7BBEFB6DCF86@ontotext.com>
Message-ID: <02106127-B08A-4498-9847-07DF89C8AB2C@ontotext.com>

Buna Mihaela,

  In the general case, changing the ontology on the fly introduces more complications than benefits. That is why it is not our practice and we haven't implemented the mechanisms for this process.  Currently, the LKB Gazetteer lacks this capability. Nevertheless, it is an interesting case, that we will take into consideration for the next releases.

  Usually we are working with two kinds of knowledge.  
  One consists of  well known facts from trusted sources. Those include objects like countries, cities, famous people etc. Those facts are carefully examined and can be relied upon. This quality data forms the gazetteer dictionary.  It does not change during the work of the system. It may improve between different deploys.
  The other type of data consists of facts, that have been recognized with the help of some rules and logic.  The quality here is not so good, so they don't have a place in the gazetteer. These facts are added to the RDF store when annotating the documents.

  So the ontology scheme and trusted data do not change often and there is no need to reload the gazetteer on the fly.  Also such changes may lead to serious inconsistencies, due to the complicated inference rules in the RDF databases.  That is why it is easy to add hard to modify and remove facts from them. This also answers the question about the differential update. It is easier to clear the dictionaries thus forcing the gazetteer to generate them anew. 

  It will be interesting to know more about your usecase and why you need such a flexible process. That way we might be able to help more.

Thank you for your interest in KIM

Philip Alexiev
Software Engineer, KIM team



On 22 Nov 2011, at 2:08 PM, Reneta Popova wrote:

> 
> 
> Begin forwarded message:
> 
>> From: Mihaela Olteanu <olteanu.miha at gmail.com>
>> Subject: Dictionary life cycle for the Large KB Gazetteer
>> Date: 22 ??????? 2011 13:57:18 ????????+0200
>> To: marin.nozhchev at ontotext.com, danko at ontotext.com, reneta.popova at ontotext.com
>> 
>> Hello,
>>  
>> I am using the Large KB Gazetteer with AllegroGraph. The ontology definition that I use can change while the pipeline is running.
>> In this case I could simply reinitialize the processing resource. The question is: this is the only way of reloading the dictionary for large KB gazetteer? Can't this operation be done automatically, maybe setting the gazetteer as listener to my updated ontology event and on message reinitialize the component automatically?
>> Is it possible to load only the newly added definitions, or the updates?
>>  
>> Thanks,
>> Mihaela
> 
> _______________________________________________
> Kim-discussion mailing list
> Kim-discussion at ontotext.com
> http://ontotext.com/mailman/listinfo/kim-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111122/66850caa/attachment.html>

From myfamily2488 at hotmail.com  Thu Nov 24 10:08:56 2011
From: myfamily2488 at hotmail.com (Jerry Gao)
Date: Thu, 24 Nov 2011 23:08:56 +0800
Subject: [Kim-discussion] Encountering with Nullpoint Exception when
 adding customised document feature into KIM feature map
In-Reply-To: <BAY151-W4946C6B3C3F092E59BF509CBCE0@phx.gbl>
References: <BAY151-W4946C6B3C3F092E59BF509CBCE0@phx.gbl>
Message-ID: <BAY151-W29078E559EEABD32D602BECBCE0@phx.gbl>


HI,
The email is to let you know just ignore the email below. 
We just solve the problem due to we put "java.net.URL" typed value into feature key 'URL' ,which seems only accept java.lang.String.
However, the exception is still strange and confusing for us.
Thanks, anyway.From: myfamily2488 at hotmail.com
To: kim-discussion at ontotext.com
Subject: Encountering with Nullpoint Exception when adding customised document feature into KIM feature map
Date: Thu, 24 Nov 2011 22:24:30 +0800








Hi,

 

I followed the instruction in : http://ontotext.com/pipermail/kim-discussion/2010-November/001192.html
to add our customised document feature.

 

1.       Add
new features in ../config/document.repository.properties:

 

com.ontotext.kim.KIMConstants.DOCUMENT_FEAT_LIST=KEYENTITIES,KEYPHRASES,TIMESTAMP,TITLE,URL,SUBJECT,DOCTYPE,LANGUAGE,SOURCE,AUTHORS,URLHASHCODE,SLICE

 

2.       Writing
java code using KIM API to annotate and store documents into the KIM server:

 

KIMJMIServiceImpl kimJMIService = new KIMJMIServiceImpl();

KIMDocument
testDocument = kimJMIService.prepareDocumentFromURL(new URL("http://searchconsumerization.techtarget.com/tip/Android-tablet-security-OS-features-and-Android-security-apps?asrc=EM_NLN_15536628&track=NL-544&ad=855854"), "UTF-8");

      
KIMDocument annotatedDoc =
kimJMIService.executeSemanticAnnotation(testDocument);

     
com.ontotext.kim.client.corpora.KIMFeatureMap docFeatureMap = new com.ontotext.kim.client.corpora.KIMFeatureMapImpl();

docFeatureMap.put("URL",testDocument.getSourceUrl());

      
docFeatureMap.put("TITLE", "Android tablet security: OS features
and Android security apps");

docFeatureMap.put("TIMESTAMP", Calendar.getInstance().getTime());

docFeatureMap.put("SOURCE",testDocument.getSourceUrl().toString());

             
docFeatureMap.put("URLHASHCODE", generateUrlHash(testDocument.getSourceUrl()));

             
docFeatureMap.put("SLICE", "http://searchconsumerization.techtarget.com/");

annotatedDoc.setFeatures(docFeatureMap);

kimJMIService.storeSyncKimDocument(annotatedDoc);

 

3.  Then, we run and
encountered with NullpointException:

 

Exception in thread
"main" java.lang.NullPointerException

      
at com.ontotext.kim.corpora.FeatureMapStore.storeFeature(FeatureMapStore.java:79)

      
at com.ontotext.kim.corpora.FeatureMapStore.store(FeatureMapStore.java:69)

      
at com.ontotext.kim.coredb.RdfCore.fillFeatures(RdfCore.java:217)

      
at com.ontotext.kim.coredb.RdfCore.addDocument(RdfCore.java:204)

      
at com.ontotext.kim.coredb.CachingDocumentRepository.addDocument(CachingDocumentRepository.java:40)

      
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

      
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

      
at java.lang.reflect.Method.invoke(Method.java:597)

      
at org.openrdf.util.rmirouting.ChannelIfaceImpl.invoke(ChannelIfaceImpl.java:513)

      
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

      
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

      
at java.lang.reflect.Method.invoke(Method.java:597)

      
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)

      
at sun.rmi.transport.Transport$1.run(Transport.java:159)

      
at java.security.AccessController.doPrivileged(Native Method)

      
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)

      
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)

      
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)

      
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)

      
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

      
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

      
at java.lang.Thread.run(Thread.java:662)

4.      
We also locate the code in
?com.ontotext.kim.corpora. FeatureMapStore? where the exception happens:

private void
storeFeature(EntityDescriptionImpl docEnt, Map.Entry ent, EntityDescriptionImpl
ftr)

  {

    String
key = ((String)ent.getKey()).toUpperCase();

    if
((ent.getValue() != null) && (this.fSchema.get().contains(key))) {

     
ftr.addAttribute(SAR_HAS_KEY, new LiteralImpl(key));

     
RDFDatatype typeByValue = this.typeMapper.getTypeByValue(ent.getValue());

     
URI typeUri = new URIImpl(typeByValue.getURI());

     
ftr.addAttribute(SAR_HAS_VALUE, new LiteralImpl(ent.getValue().toString(),
typeUri));

     
docEnt.addPrefetchedRelation(SAR_HAS_FEATURE, ftr);

    }

  }

5.      
Question:

 

Do we need to do any other
customization configuration in addition to the
?document.repository.properties??

Thanks & 
Regards

___________________

Jie Gao  		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111124/fba3eb72/attachment.html>

From myfamily2488 at hotmail.com  Thu Nov 24 11:35:16 2011
From: myfamily2488 at hotmail.com (Jerry Gao)
Date: Fri, 25 Nov 2011 00:35:16 +0800
Subject: [Kim-discussion] Strange KIMQuery problem
Message-ID: <BAY151-W38448B0AD818A7EB06535CBCE0@phx.gbl>


Strange 1. Encounter with KIMQueryException:
1. I put hashed web content into 'CONTENTHASHCODE'. The value is 64217a6726705e3e864e331074da011e.Java code:  DocumentQuery docQuery = new DocumentQuery();		 try{			 AtomExpr expr = new AtomExpr(map.get("CONTENTHASHCODE").toString(), "CONTENTHASHCODE");			 docQuery.setKeywordRestriction(expr);		     docQuery.setMaxResultLength(2);		     		     		     long size = irApi.getDocumentCount(docQuery); // DocumentRepositoryAPI irApi		     if (size > 0L){		    	 System.out.println("The document was stored");		         return true;		     }		} catch (Exception x) {	      KimLogs.logPOPULATER.debug("Error checking isStored() in DuplicateHunter.");	    }		return false;
Exception details:
com.ontotext.kim.client.query.KIMQueryException: Encountered " "<" "< "" at line 2, column 278.Was expecting one of:    "{" ...    "}" ...    <LANG_LITERAL> ...    <DATATYPED_LITERAL> ...    <STRING> ...    <URI> ...    <QNAME> ...    <BNODE> ...    <PREFIX_NAME> ...    <POS_INTEGER> ...    <NEG_INTEGER> ...    <DECIMAL> ...      while running this query: SELECT DISTINCT D FROM {D} <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 	{<http://proton.semanticweb.org/2006/05/protont#Document>} , {D} <http://ordi.ontotext.com/sar#hasFeature> {F0} <http://ordi.ontotext.com/sar#hasKey> {"CONTENTHASHCODE"}, {F0} <http://ordi.ontotext.com/sar#hasValue> {V0}, {<64217a6726705e3e864e331074da011e:64217a6726705e3e864e331074da011e>} <http://www.ontotext.com/prefixMatchIgnoreCase> {V0} LIMIT 1
However,...
when i change to set hashed URL value (e.g., e22c60e9d8cf41d897b2fae7da041f35), it works!!!!!! 
Java Code: DocumentQuery docQuery = new DocumentQuery();		 try{			 long docId = loadDocumentIdByContentHashCode(map.get("LINKURLHASHCODE").toString());			 			 AtomExpr expr = new AtomExpr(map.get("LINKURLHASHCODE").toString(), "LINKURLHASHCODE");			 docQuery.setKeywordRestriction(expr);		     docQuery.setMaxResultLength(2);		     		     		     long size = irApi.getDocumentCount(docQuery);		     if (size > 0L){		    	 System.out.println("The document was stored");		         return true;		     }		} catch (Exception x) {	      KimLogs.logPOPULATER.debug("Error checking isStored() in DuplicateHunter.");	    }		return false;
NO exception happens! 
Why???
Strange 2: We are not familiar with the difference between method ' irApi.getDocumentCount(DocumentQuery )' and method 'irApi.getDocumentIds(DocumentQuery )':
To our best knowledge of the two methods, they should response with consistent results.
However, the actual results are strange:
For example, we use the same query and value (e22c60e9d8cf41d897b2fae7da041f35) to query:1. first method 'isStored' (java code):	AtomExpr expr = new AtomExpr(map.get("LINKURLHASHCODE").toString(), "LINKURLHASHCODE");	docQuery.setKeywordRestriction(expr);	docQuery.setMaxResultLength(1);		     		     	long size = irApi.getDocumentCount(docQuery);       if (size > 0L){	   	 System.out.println("The document was stored");	         return true;	}2. second method 'loadDocumentIdByHashCode' (java code):
 	AtomExpr expr = new AtomExpr(contentHashCode, "LINKURLHASHCODE");		 	DocumentQuery docQuery = new DocumentQuery();	docQuery.setKeywordRestriction(expr);	docQuery.setMaxResultLength(2);		DocumentQueryResult docQR = irApi.getDocumentIds(docQuery);		Iterator<DocumentQueryResultRow> docIdIter = docQR.iterator();	while(docIdIter.hasNext()){		DocumentQueryResultRow result = (DocumentQueryResultRow)docIdIter.next();		return result.getDocumentId();        }
Result:   The first method return the value of 'size' -- 0, which means no document with the feature 'LINKURLHASHCODE' valued 'e22c60e9d8cf41d897b2fae7da041f35'   The second method, however, returned the document id, which is correct and expected.
Strange.

Thanks & 
Regards

___________________

Jie Gao  		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111125/44a92d61/attachment-0001.html>

From philip.alexiev at ontotext.com  Fri Nov 25 05:00:36 2011
From: philip.alexiev at ontotext.com (Philip Alexiev)
Date: Fri, 25 Nov 2011 12:00:36 +0200
Subject: [Kim-discussion] Strange KIMQuery problem
In-Reply-To: <BAY151-W38448B0AD818A7EB06535CBCE0@phx.gbl>
References: <BAY151-W38448B0AD818A7EB06535CBCE0@phx.gbl>
Message-ID: <DA9FCAAE-671A-47A5-A5FF-2AFFF8D1FFD6@ontotext.com>

Hello Jerry,

On the first question:

Serql uses the full URI syntax for the search literal. This means 
<64217a6726705e3e864e331074da011e:64217a6726705e3e864e331074da011e>
should conform to the URI specification. The specification can be found at  http://www.ietf.org/rfc/rfc2396.txt . You can see there that the schema (the first part of the URI - the part before the ':' ) can start only with a letter.  This explains the error at query validation and why it works with the hash code that starts with a letter.

A possible solution to this will be to generate your hash code and put a letter character at the beginning of it. So that if it is 'x' for example, all your codes will start with 'x'. You will then search for the code with an 'x' in front.


On the second question:

This behavior is strange.  Please make sure you are working with identical document queries.

If you send an example document and a test case that reproduces the error, it will be easy for us to track it down.

Thank you for your interest in KIM.

Philip Alexiev
Softrware Engineer , KIM team



On 24 Nov 2011, at 6:35 PM, Jerry Gao wrote:

> Strange 1. Encounter with KIMQueryException:
> 
> 1. I put hashed web content into 'CONTENTHASHCODE'. The value is 64217a6726705e3e864e331074da011e.
> Java code: 
>  DocumentQuery docQuery = new DocumentQuery();
> 		 try{
> 			 AtomExpr expr = new AtomExpr(map.get("CONTENTHASHCODE").toString(), "CONTENTHASHCODE");
> 			 docQuery.setKeywordRestriction(expr);
> 		     docQuery.setMaxResultLength(2);
> 		     		     
> 		     long size = irApi.getDocumentCount(docQuery); // DocumentRepositoryAPI irApi
> 		     if (size > 0L){
> 		    	 System.out.println("The document was stored");
> 		         return true;
> 		     }
> 		} catch (Exception x) {
> 	      KimLogs.logPOPULATER.debug("Error checking isStored() in DuplicateHunter.");
> 	    }
> 		return false;
> 
> Exception details:
> 
> com.ontotext.kim.client.query.KIMQueryException: Encountered " "<" "< "" at line 2, column 278.
> Was expecting one of:
>     "{" ...
>     "}" ...
>     <LANG_LITERAL> ...
>     <DATATYPED_LITERAL> ...
>     <STRING> ...
>     <URI> ...
>     <QNAME> ...
>     <BNODE> ...
>     <PREFIX_NAME> ...
>     <POS_INTEGER> ...
>     <NEG_INTEGER> ...
>     <! DECIMAL> ...
>      
>  while running this query: 
> SELECT DISTINCT D FROM 
> {D} <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> 
> 	{<http://proton.semanticweb.org/2006/05/protont#Document>} , 
> {D} <http://ordi.ontotext.com/sar#hasFeature> 
> {F0} <http://ordi.ontotext.com/sar#hasKey> {"CONTENTHASHCODE"}, 
> {F0} <http://ordi.ontotext.com/sar#hasValue> {V0}, 
> {<64217a6726705e3e864e331074da011e:64217a6726705e3e864e331074da011e>} 
> <http://www.ontotext.com/prefixMatchIgnoreCase> {V0} LIMIT 1
> 
> However,...
> 
> when i change to set hashed URL value (e.g., e22c60e9d8cf41d897b2fae7da041f35), it works!!!!!! 
> 
> Java Code:
>  DocumentQuery docQuery = new DocumentQuery();
> 		 try{
> 			 long docId = loadDocumentIdByContentHashCode(map.get("LINKURLHASHCODE").toString());
> 			 
> 			 AtomExpr expr = new AtomExpr(map.get("LINKURLHASHCODE").toString(), "LINKURLHASHCODE");
> 			 docQuery.setKeywordRestriction(expr);
> 		     docQuery.setMaxResultLength(2);
> 		 &nb! sp;   		     
> 		     long size = irApi.getDocumentCount(docQuery);
> 		     if (size > 0L){
> 		    	 System.out.println("The document was stored");
> 		         return true;
>     }
> 		} catch (Exception x) {
> 	      KimLogs.logPOPULATER.debug("Error checking isStored() in DuplicateHunter.");
> 	    }
> 		return false;
> 
> NO exception happens! 
> 
> Why???
> 
> Strange 2: We are not familiar with the difference between method ' irApi.getDocumentCount(DocumentQuery )' and method 'irApi.getDocumentIds(DocumentQuery )':
> 
> To our best knowledge of the two methods, they should response with consistent results.
> 
> However, the actual results are strange:
> 
> For example, we us! e the same query and value (e22c60e9d8cf41d897b2fae7da041f35) to query:
> 1. first method 'isStored' (java code):
> 	AtomExpr expr = new AtomExpr(map.get("LINKURLHASHCODE").toString(), "LINKURLHASHCODE");
> 	docQuery.setKeywordRestriction(expr);
> 	docQuery.setMaxResultLength(1);
> 		     		     
> 	long size = irApi.getDocumentCount(docQuery);
>        if (size > 0L){
> 	   	 System.out.println("The document was stored");
> 	         return true;
> 	}
> 2. second method 'loadDocumentIdByHashCode' (java code):
> 
>  	AtomExpr expr = new AtomExpr(contentHashCode, "LINKURLHASHCODE");
> 		 
> 	DocumentQuery docQuery = new DocumentQuery();
> 	docQuery.setKeywordRestriction(expr);
> 	docQuery.setMaxResultLen gth(2);
> 	
> 	DocumentQueryResult docQR = irApi.getDocumentIds(docQuery);
> 	
> 	Iterator<DocumentQueryResultRow> docIdIter = docQR.iterator();
> 	while(docIdIter.hasNext()){
> 		DocumentQueryResultRow result = (DocumentQueryResultRow)docIdIter.next();
> 		return result.getDocumentId();
>         }
> 
> Result:
>    The first method return the value of 'size' -- 0, which ! means no document with the feature 'LINKURLHASHCODE' valued 'e22c60e9d8cf41d897b2fae7da041f35'
>    The second method, however, returned the document id, which is correct and expected.
> 
> Strange.
> 
> Thanks & Regards
> ___________________
> Jie Gao
>  
> _______________________________________________
> Kim-discussion mailing list
> Kim-discussion at ontotext.com
> http://ontotext.com/mailman/listinfo/kim-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ontotext.com/pipermail/kim-discussion/attachments/20111125/8c7c20a4/attachment.html>