zorba-coders team mailing list archive

[Matthias Brantner <matthias.brantner@xxxxxxxxxx>]

Where did the file fots.xq go? Is it replaced by cli.xq? If so, the README.txt and the help
in cli.xq should be adapted.

I would be nice to get better error messages if one of the command line parameters is missing.
For example,

./bin/zorba -f -q ../sandbox/test/fots_driver/cli.xq -e fotsPath:=/home/mbrantner/w3c/checkout/2011/QT3-test-suite/  -e mode:=run-test-sets -e testSetPrefixes:=prod-WindowClause
</home/mbrantner/zorba/build/URI_PATH/org/expath/ns/file.xq>:359,3: type error [err:XPTY0004]: empty path; raised at /home/mbrantner/zorba/sandbox/src/util/fs_util.cpp:196 
=================================================
file:read-text#1 <http://expath.org/ns/file> 
/home/mbrantner/zorba/sandbox/test/fots_driver/zorba-fots-driver.xqy at line 175 column 40
=================================================
driver:run-fots#9 <http://www.zorba-xquery.com/zorba-fots-driver> 
/home/mbrantner/zorba/build/../sandbox/test/fots_driver/cli.xq at line 127 column 10


Also, I think it became to hard to use. It would be nice if at least the path to the manifest
file doesn't need to be given on the command line. Maybe a CMake-configured bash/bat
script in the build directory would help to make it easier to run.

>> - Rename fots_driver to fots
>> - Rename zorba-fots-driver.xqy to fots-driver.xq
> People must get used to the terminology:
> - FOTS means the testsuite from W3C: hence when a test is wrong => file a bug to W3C
> - FOTS driver is the thing that runs the testsuite: hence when an error is detected in the FOTS driver = > file a bug in Launchpad with a "fots-driver" TAG
> - test case is a single test
> - test set is unit (a bunch) of tests grouped together because they test a certain feature/function/functionality
> 
> The renaming you suggest would confuse people.
At least the zorba in zorba-fots-driver is redundant.

>> - We generally use xq instead of xqy for xquery files. Maybe we should do this
>> here to.
> If you really insist then I can rename the files, sure...
It's not a strong reject but I love consistency in the repository.

> 
>> - Does the dependency output have to appear in every run?
> Do not know what you mean by "dependency output": can you please be more specific?
The output file is very verbose. At the beginning, it contains a huge list starting with the dependencies-satisfied
file. Does it need to be there.

The output of a failing test is very long. You can't immediately see why something fails.
For example,

      <test-case name="WindowingUseCase04" result="fail">
        <environment ref="WindowingUseCases04"/>
        <dependency type="spec" value="XQ30+"/>
        <test>
let $MAX_DIFF := 2

for sliding window $w in ./stream/event
  start  $s_curr at $s_pos previous $s_prev
    when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
  only end next $e_next
    when $e_next/@time - $s_curr/@time gt $MAX_DIFF
return
  avg( $w/@temp )
      </test>
        <result>
          <assert-xml>8 9 17 18</assert-xml>
        </result>
        <info>
          <environment name="WindowingUseCases04">
            <source role="." file="WindowClause/temp_events.xml">
              <description> temp_events.xml from the XQuery 3.0 Use Cases </description>
              <created by="Matthias Brantner" on="2011-07-28"/>
            </source>
          </environment>
          <query>import module namespace xqxq = 'http://www.zorba-xquery.com/modules/xqxq';
variable $queryID := xqxq:prepare-main-module('
xquery version ''3.0'';



let $MAX_DIFF := 2

for sliding window $w in ./stream/event
  start  $s_curr at $s_pos previous $s_prev
    when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
  only end next $e_next
    when $e_next/@time - $s_curr/@time gt $MAX_DIFF
return
  avg( $w/@temp )
      '
);
variable $contextItem := doc("/home/mbrantner/w3c/checkout/2011/QT3-test-suite//prod/WindowClause/temp_events.xml");
xqxq:bind-context-item($queryID, $contextItem);


xqxq:evaluate($queryID)</query>
          <out xmlns="">
            <expected-result>
              <assert-xml>8 9 17 18</assert-xml>
            </expected-result>
            <result>8 9 17 18</result>
            <errors>Result 
'891718'
 is different from the expected result 
'8 9 17 18'.</errors>
          </out>
        </info>
      </test-case>


This output contains a lot of stuff that might be helpful in some cases but it would be really nice
to just have a summary of the failures and the reason why something failed. 

Also, this test seems to give the correct result but the comparison fails.


Matthias
-- 
https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980
Your team Zorba Coders is subscribed to branch lp:zorba.