zorba-coders team mailing list archive

[Sorin Marian Nasoi <sorin.marian.nasoi@xxxxxxxxx>]

First of all let me mention that this type of communication in Launchpad messages is IMHO very hard to follow.

Anyway: this is how you chose to report your issues so here are my remarks below.

> Where did the file fots.xq go? Is it replaced by cli.xq? If so, the README.txt
> and the help
> in cli.xq should be adapted.
They were updated: are you using the latest version of the driver?


> I would be nice to get better error messages if one of the command line
> parameters is missing.
> For example,
> 
> ./bin/zorba -f -q ../sandbox/test/fots_driver/cli.xq -e
> fotsPath:=/home/mbrantner/w3c/checkout/2011/QT3-test-suite/  -e mode:=run-
> test-sets -e testSetPrefixes:=prod-WindowClause
> </home/mbrantner/zorba/build/URI_PATH/org/expath/ns/file.xq>:359,3: type error
> [err:XPTY0004]: empty path; raised at
> /home/mbrantner/zorba/sandbox/src/util/fs_util.cpp:196
> =================================================
> file:read-text#1 <http://expath.org/ns/file>
> /home/mbrantner/zorba/sandbox/test/fots_driver/zorba-fots-driver.xqy at line
> 175 column 40
> =================================================
> driver:run-fots#9 <http://www.zorba-xquery.com/zorba-fots-driver>
> /home/mbrantner/zorba/build/../sandbox/test/fots_driver/cli.xq at line 127
> column 10
Please open a Launchpad issue with the "fots-driver" TAG and assign it to me.

 
> Also, I think it became to hard to use. It would be nice if at least the path
> to the manifest
> file doesn't need to be given on the command line. Maybe a CMake-configured
> bash/bat
> script in the build directory would help to make it easier to run.
My experience with what you are proposing is this:
- it makes it easier to use, it is true
- everyone will forget what params were set in the file or that a file even exists. This makes it very hard to debug the reported issues because you also have to send the file with the configured params.

Also one needs to set different options to see what happens: for instance what happens when you enable the HOF feature? You can easily create a new ZorbaManifest_HOF_ENABLED.xml and invoke teh CLI with that file.
You see the difference in a second, you to not need to change any configuration file.

I would vote against your proposal.

 
> >> - Rename fots_driver to fots
> >> - Rename zorba-fots-driver.xqy to fots-driver.xq
> > People must get used to the terminology:
> > - FOTS means the testsuite from W3C: hence when a test is wrong => file a
> bug to W3C
> > - FOTS driver is the thing that runs the testsuite: hence when an error is
> detected in the FOTS driver = > file a bug in Launchpad with a "fots-driver"
> TAG
> > - test case is a single test
> > - test set is unit (a bunch) of tests grouped together because they test a
> certain feature/function/functionality
> >
> > The renaming you suggest would confuse people.
> At least the zorba in zorba-fots-driver is redundant.
No it is not: if we decide to put it in FOTS/drivers together with the Saxon, BaseX it will make sense right?

> >> - We generally use xq instead of xqy for xquery files. Maybe we should do
> this
> >> here to.
> > If you really insist then I can rename the files, sure...
> It's not a strong reject but I love consistency in the repository.
Please open a Launchpad issue with the "fots-driver" TAG and assign it to me.


> >> - Does the dependency output have to appear in every run?
> > Do not know what you mean by "dependency output": can you please be more
> specific?
> The output file is very verbose. At the beginning, it contains a huge list
> starting with the dependencies-satisfied
> file. Does it need to be there.
> 
> The output of a failing test is very long. You can't immediately see why
> something fails.
> For example,
> 
>       <test-case name="WindowingUseCase04" result="fail">
>         <environment ref="WindowingUseCases04"/>
>         <dependency type="spec" value="XQ30+"/>
>         <test>
> let $MAX_DIFF := 2
> 
> for sliding window $w in ./stream/event
>   start  $s_curr at $s_pos previous $s_prev
>     when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
>   only end next $e_next
>     when $e_next/@time - $s_curr/@time gt $MAX_DIFF
> return
>   avg( $w/@temp )
>       </test>
>         <result>
>           <assert-xml>8 9 17 18</assert-xml>
>         </result>
>         <info>
>           <environment name="WindowingUseCases04">
>             <source role="." file="WindowClause/temp_events.xml">
>               <description> temp_events.xml from the XQuery 3.0 Use Cases
> </description>
>               <created by="Matthias Brantner" on="2011-07-28"/>
>             </source>
>           </environment>
>           <query>import module namespace xqxq = 'http://www.zorba-
> xquery.com/modules/xqxq';
> variable $queryID := xqxq:prepare-main-module('
> xquery version ''3.0'';
> 
> 
> 
> let $MAX_DIFF := 2
> 
> for sliding window $w in ./stream/event
>   start  $s_curr at $s_pos previous $s_prev
>     when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
>   only end next $e_next
>     when $e_next/@time - $s_curr/@time gt $MAX_DIFF
> return
>   avg( $w/@temp )
>       '
> );
> variable $contextItem := doc("/home/mbrantner/w3c/checkout/2011/QT3-test-
> suite//prod/WindowClause/temp_events.xml");
> xqxq:bind-context-item($queryID, $contextItem);
> 
> 
> xqxq:evaluate($queryID)</query>
>           <out xmlns="">
>             <expected-result>
>               <assert-xml>8 9 17 18</assert-xml>
>             </expected-result>
>             <result>8 9 17 18</result>
>             <errors>Result
> '891718'
>  is different from the expected result
> '8 9 17 18'.</errors>
>           </out>
>         </info>
>       </test-case>
> 
> 
> This output contains a lot of stuff that might be helpful in some cases but it
> would be really nice
> to just have a summary of the failures and the reason why something failed.
Please add a -e verbose:=false to your command line.


> Also, this test seems to give the correct result but the comparison fails.
If you discovered an bug in a reported failed test that is great:
Please read http://my.zorba-xquery.com/dokuwiki/doku.php?id=zorba_fots_driver#debug_investigate_a_failed_test_case
And take the appropiate action.


-- 
https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980
Your team Zorba Coders is subscribed to branch lp:zorba.