zorba-coders team mailing list archive

[Chris Hillery <chillery+launchpad@xxxxxxxxx>]

Chris Hillery has proposed merging lp:~zorba-coders/zorba/bug-1103432 into lp:zorba.

Commit message:
A number of fixes related to HTTP URI resolution.

Requested reviews:
  Zorba Coders (zorba-coders)

For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/bug-1103432/+merge/144674
-- 
https://code.launchpad.net/~zorba-coders/zorba/bug-1103432/+merge/144674
Your team Zorba Coders is requested to review the proposed merge of lp:~zorba-coders/zorba/bug-1103432 into lp:zorba.

=== modified file 'doc/zorba/options.dox'
--- doc/zorba/options.dox	2012-11-08 05:38:30 +0000
+++ doc/zorba/options.dox	2013-01-24 10:52:29 +0000
@@ -99,6 +99,8 @@
 it is specified inside query text. It must be specified using the C++ API
 or a Zorba command-line option. An error will be raised if this option is
 specified inside query text to avoid confusion.
+\note Also note that, unlike other features, this feature is always set or
+unset Zorba-wide.
 </ul>
 
 

=== modified file 'schemas/xhtml/xhtml1-frameset.dtd'
--- schemas/xhtml/xhtml1-frameset.dtd	2012-01-26 01:21:43 +0000
+++ schemas/xhtml/xhtml1-frameset.dtd	2013-01-24 10:52:29 +0000
@@ -23,19 +23,28 @@
 
 <!--================ Character mnemonic entities =========================-->
 
+<!-- PLEASE NOTE: This file is modified from the W3C original to
+  specify absolute URLs for the entities below. This is necessary due
+  to Zorba's interaction with Xerces: When Xerces processes this DTD
+  with the original relative entity URLs, it does not inform Zorba of
+  the base URI (ie, the URI of this DTD). Therefore Zorba cannot find
+  its local copy of the .ent file. This causes Xerces to fall back to
+  querying w3.org via HTTP for the .ent file, and w3.org punitively
+  puts a 30-second delay on such requests. -->
+
 <!ENTITY % HTMLlat1 PUBLIC
    "-//W3C//ENTITIES Latin 1 for XHTML//EN"
-   "xhtml-lat1.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent";>
 %HTMLlat1;
 
 <!ENTITY % HTMLsymbol PUBLIC
    "-//W3C//ENTITIES Symbols for XHTML//EN"
-   "xhtml-symbol.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent";>
 %HTMLsymbol;
 
 <!ENTITY % HTMLspecial PUBLIC
    "-//W3C//ENTITIES Special for XHTML//EN"
-   "xhtml-special.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent";>
 %HTMLspecial;
 
 <!--================== Imported Names ====================================-->

=== modified file 'schemas/xhtml/xhtml1-strict.dtd'
--- schemas/xhtml/xhtml1-strict.dtd	2012-01-26 01:21:43 +0000
+++ schemas/xhtml/xhtml1-strict.dtd	2013-01-24 10:52:29 +0000
@@ -23,19 +23,28 @@
 
 <!--================ Character mnemonic entities =========================-->
 
+<!-- PLEASE NOTE: This file is modified from the W3C original to
+  specify absolute URLs for the entities below. This is necessary due
+  to Zorba's interaction with Xerces: When Xerces processes this DTD
+  with the original relative entity URLs, it does not inform Zorba of
+  the base URI (ie, the URI of this DTD). Therefore Zorba cannot find
+  its local copy of the .ent file. This causes Xerces to fall back to
+  querying w3.org via HTTP for the .ent file, and w3.org punitively
+  puts a 30-second delay on such requests. -->
+
 <!ENTITY % HTMLlat1 PUBLIC
    "-//W3C//ENTITIES Latin 1 for XHTML//EN"
-   "xhtml-lat1.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent";>
 %HTMLlat1;
 
 <!ENTITY % HTMLsymbol PUBLIC
    "-//W3C//ENTITIES Symbols for XHTML//EN"
-   "xhtml-symbol.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent";>
 %HTMLsymbol;
 
 <!ENTITY % HTMLspecial PUBLIC
    "-//W3C//ENTITIES Special for XHTML//EN"
-   "xhtml-special.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent";>
 %HTMLspecial;
 
 <!--================== Imported Names ====================================-->

=== modified file 'schemas/xhtml/xhtml1-transitional.dtd'
--- schemas/xhtml/xhtml1-transitional.dtd	2012-01-26 01:21:43 +0000
+++ schemas/xhtml/xhtml1-transitional.dtd	2013-01-24 10:52:29 +0000
@@ -23,19 +23,28 @@
 
 <!--================ Character mnemonic entities =========================-->
 
+<!-- PLEASE NOTE: This file is modified from the W3C original to
+  specify absolute URLs for the entities below. This is necessary due
+  to Zorba's interaction with Xerces: When Xerces processes this DTD
+  with the original relative entity URLs, it does not inform Zorba of
+  the base URI (ie, the URI of this DTD). Therefore Zorba cannot find
+  its local copy of the .ent file. This causes Xerces to fall back to
+  querying w3.org via HTTP for the .ent file, and w3.org punitively
+  puts a 30-second delay on such requests. -->
+
 <!ENTITY % HTMLlat1 PUBLIC
    "-//W3C//ENTITIES Latin 1 for XHTML//EN"
-   "xhtml-lat1.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent";>
 %HTMLlat1;
 
 <!ENTITY % HTMLsymbol PUBLIC
    "-//W3C//ENTITIES Symbols for XHTML//EN"
-   "xhtml-symbol.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent";>
 %HTMLsymbol;
 
 <!ENTITY % HTMLspecial PUBLIC
    "-//W3C//ENTITIES Special for XHTML//EN"
-   "xhtml-special.ent">
+   "http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent";>
 %HTMLspecial;
 
 <!--================== Imported Names ====================================-->

=== modified file 'src/context/default_uri_mappers.cpp'
--- src/context/default_uri_mappers.cpp	2012-11-06 10:11:37 +0000
+++ src/context/default_uri_mappers.cpp	2013-01-24 10:52:29 +0000
@@ -193,14 +193,7 @@
 
   // Finally, append the original URI, so that it will be resolved
   // as-is if there's nothing appropriate on the local filesystem.
-  // Note: For module or schema imports, don't do this if it's a
-  // network (HTTP) URI and the "http_resolution" feature is disabled
-  // on this context.
-  if ( (lKind != EntityData::MODULE && lKind != EntityData::SCHEMA) ||
-       aSctx.is_feature_set(feature::http_resolution) ||
-       (HTTPURLResolver::isHTTPScheme(aUri) == false) ) {
-    oUris.push_back(aUri);
-  }
+  oUris.push_back(aUri);
 }
 
 ZorbaCollectionURIMapper::~ZorbaCollectionURIMapper()

=== modified file 'src/context/default_url_resolvers.cpp'
--- src/context/default_url_resolvers.cpp	2012-11-06 10:11:37 +0000
+++ src/context/default_url_resolvers.cpp	2013-01-24 10:52:29 +0000
@@ -26,6 +26,7 @@
 #include "store/api/collection.h"
 #include "store/api/item.h"
 #include "system/globalenv.h"
+#include <context/static_context.h>
 #include <fstream>
 #include <iostream>
 #include <sstream>
@@ -60,6 +61,12 @@
 HTTPURLResolver::resolveURL
 (zstring const& aUrl, EntityData const* aEntityData)
 {
+  // First, don't even try if http-uri-resolution is disabled.
+  if (!GENV.getRootStaticContext().is_feature_set(feature::http_resolution)) {
+    return nullptr;
+  }
+
+  // HTTP resolution doesn't make sense for collections or thesauri.
   switch ( aEntityData->getKind() ) {
     case EntityData::COLLECTION:
 #ifndef ZORBA_NO_FULL_TEXT

=== modified file 'src/context/static_context.cpp'
--- src/context/static_context.cpp	2013-01-11 17:37:08 +0000
+++ src/context/static_context.cpp	2013-01-24 10:52:29 +0000
@@ -65,6 +65,8 @@
 #include "types/typeops.h"
 #include "types/schema/validate.h"
 
+#include <util/uri_util.h>
+
 #include "functions/function.h"
 #include "functions/library.h"
 #include "functions/signature.h"
@@ -1694,6 +1696,23 @@
       oUris = lResultUris;
     }
   }
+
+  // We're all done, but for efficiency, we want to attempt all possible file:
+  // URIs first. So, post-sort the list.
+  size_t const lNumUris = oUris.size();
+  std::vector<zstring> lFileUris, lNonFileUris;
+  for (size_t i = 0; i < lNumUris; i++) {
+    zstring lUri = oUris.at(i);
+    if (uri::get_scheme(lUri) == uri::file) {
+      lFileUris.push_back(lUri);
+    }
+    else {
+      lNonFileUris.push_back(lUri);
+    }
+  }
+
+  oUris = lFileUris;
+  oUris.insert(oUris.end(), lNonFileUris.begin(), lNonFileUris.end());
 }
 
 
@@ -3595,10 +3614,14 @@
   feature::kind k;
   if (feature::kind_for(featureName->getLocalName().c_str(), k))
   {
+    // Special case: the http-uri-resolution feature is ALWAYS set on the
+    // root static context, to ensure it is applied system-wide.
+    static_context& target = (k != feature::http_resolution) ?
+          *this : GENV.getRootStaticContext();
     if (enable)
-      set_feature(k);
+      target.set_feature(k);
     else
-      unset_feature(k);
+      target.unset_feature(k);
   }
   else
   {

=== modified file 'src/types/schema/schema.cpp'
--- src/types/schema/schema.cpp	2013-01-17 04:43:05 +0000
+++ src/types/schema/schema.cpp	2013-01-24 10:52:29 +0000
@@ -259,12 +259,17 @@
             
           return lRetval;
         }
-        else
-          return NULL;          
+        else {
+          // We didn't find it. If we return NULL here, Xerces will try to
+          // resolve it its own way, which we don't want to happen.
+          throw XQUERY_EXCEPTION( err::XQST0059,
+              ERROR_PARAMS( lResolved ));
+        }
       }
       catch (ZorbaException const& e) {
         TRACE("!!! ZorbaException: " << e );
-        if ( e.diagnostic() == zerr::ZXQP0029_URI_ACCESS_DENIED )
+        if ( e.diagnostic() == zerr::ZXQP0029_URI_ACCESS_DENIED ||
+             e.diagnostic() == err::XQST0059 )
         {
           throw;
         }

=== modified file 'test/rbkt/Queries/CMakeLists.txt'
--- test/rbkt/Queries/CMakeLists.txt	2013-01-22 20:27:24 +0000
+++ test/rbkt/Queries/CMakeLists.txt	2013-01-24 10:52:29 +0000
@@ -485,13 +485,13 @@
 # Bug 921624. If this test takes more than a couple seconds, it must be
 # hitting w3.org for the DTD, which is bad.
 SET_TESTS_PROPERTIES(test/rbkt/zorba/schemas/local-xhtml
-  PROPERTIES TIMEOUT 5)
+  PROPERTIES TIMEOUT 10)
 # Bug 867363. These should fail or pass quickly because HTTP download
 # is disabled.
 SET_TESTS_PROPERTIES(test/rbkt/zorba/uris/disable-http
-  PROPERTIES TIMEOUT 5)
+  PROPERTIES TIMEOUT 10)
 SET_TESTS_PROPERTIES(test/rbkt/zorba/uris/disable-http-2
-  PROPERTIES TIMEOUT 5)
+  PROPERTIES TIMEOUT 10)
 
 
 # --------------------------------------------------------------------------

=== added file 'test/rbkt/Queries/zorba/schemas/bogus.xsd'
--- test/rbkt/Queries/zorba/schemas/bogus.xsd	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/schemas/bogus.xsd	2013-01-24 10:52:29 +0000
@@ -0,0 +1,2 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
+<html><body>Doesn't matter what's here, so long as it's not really a schema and it references the XHTML DTD</body></html>

=== modified file 'test/rbkt/Queries/zorba/schemas/local-xhtml.xq'
--- test/rbkt/Queries/zorba/schemas/local-xhtml.xq	2012-01-26 01:21:43 +0000
+++ test/rbkt/Queries/zorba/schemas/local-xhtml.xq	2013-01-24 10:52:29 +0000
@@ -2,5 +2,5 @@
 (: not available on the local filesystem, so it should be run with a    :)
 (: short timeout (~5 seconds).                                          :)
 
-import schema namespace bogus = "http://example.com/BOGUS";;
+import schema namespace bogus = "http://example.com/BOGUS"; at "bogus.xsd";
 1

=== added file 'test/rbkt/Queries/zorba/uris/bogus2.xsd'
--- test/rbkt/Queries/zorba/uris/bogus2.xsd	1970-01-01 00:00:00 +0000
+++ test/rbkt/Queries/zorba/uris/bogus2.xsd	2013-01-24 10:52:29 +0000
@@ -0,0 +1,6 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml2-transitional.dtd";>
+<html><body>Doesn't matter what's here, so long as it's not really a
+    schema and it references the XHTML DTD - in this case a
+    non-existent variation of the XHTML DTD, so even Zorba's local
+    copy of the XHTML1 can't save it
+</body></html>

=== modified file 'test/rbkt/Queries/zorba/uris/disable-http.xq'
--- test/rbkt/Queries/zorba/uris/disable-http.xq	2012-11-06 10:11:37 +0000
+++ test/rbkt/Queries/zorba/uris/disable-http.xq	2013-01-24 10:52:29 +0000
@@ -1,5 +1,5 @@
 (: For various reasons, importing a bad schema via HTTP can take :)
 (: minutes to complete. This test should time out unless the :)
 (: http-uri-resolution feature is disabled. :)
-import schema "http://www.w3.org/";;
+import schema "http://www.w3.org/"; at "bogus2.xsd";
 1

=== modified file 'test/rbkt/Queries/zorba/uris/slow.xqlib'
--- test/rbkt/Queries/zorba/uris/slow.xqlib	2012-11-06 10:11:37 +0000
+++ test/rbkt/Queries/zorba/uris/slow.xqlib	2013-01-24 10:52:29 +0000
@@ -3,7 +3,7 @@
 (: unless the http-uri-resolution feature is disabled. :)
 module namespace slow = "http://www.zorba-xquery.com/test/uris/slow";;
 
-import schema "http://www.w3.org/";;
+import schema "http://www.w3.org/"; at "bogus2.xsd";
 
 declare function slow:ok() as xs:string
 {