zorba-coders team mailing list archive

["Paul J. Lucas" <paul@xxxxxxxxxxxxx>]

Paul J. Lucas has proposed merging lp:~paul-lucas/zorba/bug-863320 into lp:zorba.

Requested reviews:
  Matthias Brantner (matthias-brantner)

For more details, see:
https://code.launchpad.net/~paul-lucas/zorba/bug-863320/+merge/77732

Added a test for the end-of-sentence to fix the bug.
-- 
https://code.launchpad.net/~paul-lucas/zorba/bug-863320/+merge/77732
Your team Zorba Coders is subscribed to branch lp:zorba.

=== modified file 'ChangeLog'
--- ChangeLog	2011-09-29 16:33:24 +0000
+++ ChangeLog	2011-09-30 16:11:31 +0000
@@ -37,7 +37,8 @@
   * Fixed bug #859968 (Using put on an attribute produces an invalid XDM instance and may cause segmentation faults)
   * Fixed bug #855314 (should not fold a constant expr that returns an error item)
   * Added ExternalFunctionParameter class to be registered in the DynamicContext
-	
+  * Fixed bug #863320 (Sentence is incorrectly incremented when token characters end without sentence terminator)
+
 version 2.0.1
 
   * Custom Full-text stemming.

=== modified file 'src/runtime/full_text/icu_tokenizer.cpp'
--- src/runtime/full_text/icu_tokenizer.cpp	2011-08-27 14:57:52 +0000
+++ src/runtime/full_text/icu_tokenizer.cpp	2011-09-30 16:11:31 +0000
@@ -365,7 +365,8 @@
     word_start = word_end, word_end = word_->next();
     if ( word_end >= sent_end && sent_end != BreakIterator::DONE ) {
       sent_start = sent_end, sent_end = sent_->next();
-      ++numbers().sent;
+      if ( sent_end != BreakIterator::DONE )
+        ++numbers().sent;
     }
   } // while
 

=== added file 'test/rbkt/ExpQueryResults/zorba/fulltext/ft-same-sentence-true-4.xml.res'
--- test/rbkt/ExpQueryResults/zorba/fulltext/ft-same-sentence-true-4.xml.res	1970-01-01 00:00:00 +0000
+++ test/rbkt/ExpQueryResults/zorba/fulltext/ft-same-sentence-true-4.xml.res	2011-09-30 16:11:31 +0000
@@ -0,0 +1,1 @@
+true

=== added file 'test/rbkt/Queries/zorba/fulltext/ft-same-sentence-true-4.xq'
--- test/rbkt/Queries/zorba/fulltext/ft-same-sentence-true-4.xq	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/fulltext/ft-same-sentence-true-4.xq	2011-09-30 16:11:31 +0000
@@ -0,0 +1,2 @@
+let $x := <msg>hello world</msg>
+return $x contains text "hello" ftand "world" same sentence