team4alfanous team mailing list archive
-
team4alfanous team
-
Mailing list archive
-
Message #00072
[Branch ~team4alfanous/alfanous/alfanous-git] Rev 266: Create a new Text Analyser : APermissibleAnalyser
------------------------------------------------------------
revno: 266
git commit: 61b0e56a009f7f4bbbe9a13b0bb788c7d4170078
committer: Assem Chelli <assem.ch@xxxxxxxxx>
timestamp: Wed 2012-06-13 20:59:17 +0100
message:
Create a new Text Analyser : APermissibleAnalyser
modified:
src/alfanous/TextProcessing.py
--
lp:alfanous
https://code.launchpad.net/~team4alfanous/alfanous/alfanous-git
Your team Alfanous team is subscribed to branch lp:alfanous.
To unsubscribe from this branch go to https://code.launchpad.net/~team4alfanous/alfanous/alfanous-git/+edit-subscription
=== modified file 'src/alfanous/TextProcessing.py'
--- src/alfanous/TextProcessing.py 2012-06-13 12:49:23 +0000
+++ src/alfanous/TextProcessing.py 2012-06-13 19:59:17 +0000
@@ -234,6 +234,7 @@
#analyzers
QStandardAnalyzer = QSpaceTokenizer() | QArabicSymbolsFilter() #| QStopFilter()
+APermissibleAnalyzer = QSpaceTokenizer() | QArabicSymbolsFilter( shaping = True, tashkil = True, spellerrors = True, hamza = True )
QDiacAnalyzer = QSpaceTokenizer() | QArabicSymbolsFilter( tashkil = False )
QHighLightAnalyzer = QSpaceTokenizer() | QArabicSymbolsFilter()
QUthmaniAnalyzer = QSpaceTokenizer() | QArabicSymbolsFilter()