zorba-coders team mailing list archive

["Paul J. Lucas" <paul@xxxxxxxxxxxxx>]

Paul J. Lucas has proposed merging lp:~paul-lucas/zorba/bug-900552 into lp:zorba.

Requested reviews:
  Paul J. Lucas (paul-lucas)
  Matthias Brantner (matthias-brantner)
Related bugs:
  Bug #900552 in Zorba: "Full-text regressions, part 2"
  https://bugs.launchpad.net/zorba/+bug/900552

For more details, see:
https://code.launchpad.net/~paul-lucas/zorba/bug-900552/+merge/84555

Things should be reverted now.
-- 
https://code.launchpad.net/~paul-lucas/zorba/bug-900552/+merge/84555
Your team Zorba Coders is subscribed to branch lp:zorba.

=== modified file 'src/runtime/full_text/icu_tokenizer.cpp'
--- src/runtime/full_text/icu_tokenizer.cpp	2011-12-06 00:48:10 +0000
+++ src/runtime/full_text/icu_tokenizer.cpp	2011-12-06 03:30:30 +0000
@@ -329,10 +329,10 @@
       }
     }
 
-    else {
+    else /* if ( IS_WORD_BREAK( OTHER, rule_status ) ) */ {
       //
-      // "OTHER" tokens are for non-whitespace, non-digits, and non-numbers,
-      // i.e., word tokens.
+      // "OTHER" tokens are for non-whitespace, non-punctuation, and
+      // non-numbers, i.e., word tokens.
       //
 #     if DEBUG_TOKENIZER
       cout << "(OTHER)" << endl;
@@ -375,7 +375,9 @@
       sent_end = sent_it_->next();
       // The addition of the "if" fixes:
       // https://bugs.launchpad.net/bugs/863320
+#if 0
       if ( sent_end != BreakIterator::DONE )
+#endif
         ++numbers().sent;
     }
   } // while
@@ -387,7 +389,12 @@
   t.send( payload, callback );
   // Incrementing "sent" here fixes:
   // https://bugs.launchpad.net/bugs/897800
+#if 0
   ++numbers().sent;
+#endif
+#if DEBUG_TOKENIZER
+  cout << "--------------------\n";
+#endif /* DEBUG_TOKENIZER */
 }
 
 ///////////////////////////////////////////////////////////////////////////////

=== modified file 'test/rbkt/Queries/CMakeLists.txt'
--- test/rbkt/Queries/CMakeLists.txt	2011-12-06 00:06:46 +0000
+++ test/rbkt/Queries/CMakeLists.txt	2011-12-06 03:30:30 +0000
@@ -294,4 +294,4 @@
 
 EXPECTED_FAILURE(test/rbkt/zorba/reference/reference_5 868640)
 
-EXPECTED_FAILURE(test/rbkt/zorba/fulltext/ft-same-sentence-false-2  897800)
+EXPECTED_FAILURE(test/rbkt/zorba/fulltext/ft-same-sentence-true-4 900552)