Debian Bug report logs - #567210
libsaxonb-java: doc-available always returns false without network

version graph

Package: libsaxonb-java; Maintainer for libsaxonb-java is Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>; Source for libsaxonb-java is src:saxonb.

Reported by: "brian m. carlson" <sandals@crustytoothpaste.net>

Date: Wed, 27 Jan 2010 23:24:08 UTC

Severity: grave

Tags: help, patch

Found in version saxonb/9.0.0.4+svn20080322-3

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 27 Jan 2010 23:24:11 GMT) Full text and rfc822 format available.

Acknowledgement sent to "brian m. carlson" <sandals@crustytoothpaste.ath.cx>:
New Bug report received and forwarded. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 27 Jan 2010 23:24:11 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: "brian m. carlson" <sandals@crustytoothpaste.ath.cx>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: libsaxonb-java: doc-available always returns false without network
Date: Wed, 27 Jan 2010 23:23:08 +0000
[Message part 1 (text/plain, inline)]
Package: libsaxonb-java
Version: 9.0.0.4+svn20080322-3
Severity: normal

In certain cases, the doc-available function wrongly returns false when
the document does, in fact, exist.  The attached testcase contains three
files.  Running "saxonb-xslt -s:foo.xml -xsl:foo.xsl" produces the
output:

Requested document is file:/tmp/saxonb-testcase/index.xhtml
Missing‽
Requested document is index.xhtml
Missing‽
Requested document is file:/tmp/saxonb-testcase/index.xhtml
Missing‽
Requested document is index.xhtml
Missing‽

Note that if Java networking works, it works correctly:

Requested document is file:/tmp/saxonb-testcase/index.xhtml
It exists!
Requested document is index.xhtml
It exists!
Requested document is file:/tmp/saxonb-testcase/index.xhtml
It exists!
Requested document is index.xhtml
It exists!

Nevertheless, there is no reason to require networking for file: URIs.
Saxon B should work correctly for file: URIs regardless of the state of
networking, period.  There is no legitimate reason to make a network
request when all documents are local.

Upon further examination, it appears that Saxon is attempting to do a
lookup of the DTD on www.w3.org, even when -expand:off and -dtd:off are
specified.  It does not use catalogs to do so, and therefore fails.

-- System Information:
Debian Release: squeeze/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-trunk-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages libsaxonb-java depends on:
ii  default-jre-headless [jav 1.6-34         Standard Java or Java compatible R
ii  gcj-4.4-jre-headless [jav 4.4.3-1        Java runtime environment using GIJ
ii  gcj-jre-headless [java2-r 4:4.4.2-3      Java runtime environment using GIJ
ii  gij-4.3 [java2-runtime-he 4.3.4-4        The GNU Java bytecode interpreter
ii  libdom4j-java             1.6.1+dfsg.2-2 flexible XML framework for Java
ii  libjdom1-java             1.1.1+dfsg-1   lightweight and fast library using
ii  libxom-java               1.2.1-1        A new XML object model for Java
ii  openjdk-6-jre-headless [j 6b17~pre3-1    OpenJDK Java runtime, using Hotspo

libsaxonb-java recommends no packages.

Versions of packages libsaxonb-java suggests:
pn  libsaxonb-java-doc            <none>     (no description available)

-- no debconf information

-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 713 440 7475 | http://crustytoothpaste.ath.cx/~bmc | My opinion only
OpenPGP: RSA v4 4096b 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Thu, 28 Jan 2010 01:18:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to "brian m. carlson" <sandals@crustytoothpaste.ath.cx>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Thu, 28 Jan 2010 01:18:08 GMT) Full text and rfc822 format available.

Message #10 received at 567210@bugs.debian.org (full text, mbox):

From: "brian m. carlson" <sandals@crustytoothpaste.ath.cx>
To: 567210@bugs.debian.org
Subject: Testcase
Date: Thu, 28 Jan 2010 01:15:17 +0000
[Message part 1 (text/plain, inline)]
I forgot to include the testcase, so it is now attached.  Also, since
I'm really only using Saxon-B via Ant, if a new feature is added to
resolve this problem, I really need it to be accessible via the Ant
task.

-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 713 440 7475 | http://crustytoothpaste.ath.cx/~bmc | My opinion only
OpenPGP: RSA v4 4096b 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187
[saxonb-testcase.tar.gz (application/octet-stream, attachment)]
[signature.asc (application/pgp-signature, inline)]

Changed Bug submitter to '"brian m. carlson" <sandals@crustytoothpaste.net>' from '"brian m. carlson" <sandals@crustytoothpaste.ath.cx>' Request was from "brian m. carlson" <sandals@crustytoothpaste.net> to control@bugs.debian.org. (Thu, 03 Feb 2011 20:52:00 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Mon, 29 Aug 2011 17:36:14 GMT) Full text and rfc822 format available.

Acknowledgement sent to Torsten Werner <twerner@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Mon, 29 Aug 2011 17:36:14 GMT) Full text and rfc822 format available.

Message #17 received at 567210@bugs.debian.org (full text, mbox):

From: Torsten Werner <twerner@debian.org>
To: "brian m. carlson" <sandals@crustytoothpaste.ath.cx>, 567210@bugs.debian.org
Subject: Re: Bug#567210: Testcase
Date: Mon, 29 Aug 2011 19:35:50 +0200
Hi Brian,

On Thu, Jan 28, 2010 at 2:15 AM, brian m. carlson
<sandals@crustytoothpaste.ath.cx> wrote:
> I forgot to include the testcase, so it is now attached.  Also, since
> I'm really only using Saxon-B via Ant, if a new feature is added to
> resolve this problem, I really need it to be accessible via the Ant
> task.

thanks for your report. I am afraid that we need someone to implement
the feature. Please note that I have just uploaded a newer upstream
version 9.1.0.8 of saxonb.

Sorry,
Torsten




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Mon, 29 Aug 2011 18:48:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to "brian m. carlson" <sandals@crustytoothpaste.net>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Mon, 29 Aug 2011 18:48:03 GMT) Full text and rfc822 format available.

Message #22 received at 567210@bugs.debian.org (full text, mbox):

From: "brian m. carlson" <sandals@crustytoothpaste.net>
To: Torsten Werner <twerner@debian.org>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: Testcase
Date: Mon, 29 Aug 2011 18:45:02 +0000
[Message part 1 (text/plain, inline)]
On Mon, Aug 29, 2011 at 07:35:50PM +0200, Torsten Werner wrote:
> On Thu, Jan 28, 2010 at 2:15 AM, brian m. carlson
> <sandals@crustytoothpaste.ath.cx> wrote:
> > I forgot to include the testcase, so it is now attached.  Also, since
> > I'm really only using Saxon-B via Ant, if a new feature is added to
> > resolve this problem, I really need it to be accessible via the Ant
> > task.
> 
> thanks for your report. I am afraid that we need someone to implement
> the feature. Please note that I have just uploaded a newer upstream
> version 9.1.0.8 of saxonb.

I'm not using saxonb via ant right now, so the ant task is not critical
for me anymore.  Do you still need someone to fix the underlying bug
(doc-available returns false for file URIs) or just the ant task?  I'll
try to look into the former sometime this wekk if that's the case to see
if I can fix it.

-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Mon, 29 Aug 2011 20:03:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Torsten Werner <twerner@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Mon, 29 Aug 2011 20:03:03 GMT) Full text and rfc822 format available.

Message #27 received at 567210@bugs.debian.org (full text, mbox):

From: Torsten Werner <twerner@debian.org>
To: "brian m. carlson" <sandals@crustytoothpaste.net>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: Testcase
Date: Mon, 29 Aug 2011 22:00:25 +0200
On Mon, Aug 29, 2011 at 8:45 PM, brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> I'm not using saxonb via ant right now, so the ant task is not critical
> for me anymore.  Do you still need someone to fix the underlying bug
> (doc-available returns false for file URIs) or just the ant task?  I'll
> try to look into the former sometime this wekk if that's the case to see
> if I can fix it.

That would be nice.

Thanks,
Torsten




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Fri, 17 May 2013 06:09:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mathieu Malaterre <malat@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Fri, 17 May 2013 06:09:04 GMT) Full text and rfc822 format available.

Message #32 received at 567210@bugs.debian.org (full text, mbox):

From: Mathieu Malaterre <malat@debian.org>
To: Control bugs server <control@bugs.debian.org>, 567210@bugs.debian.org
Date: Fri, 17 May 2013 08:04:58 +0200
tags 567210 important
thanks

Technically this render the package unusable if you do not have
internet connection, right ? Should'nt this be marked a 'grave' then ?



Severity set to 'important' from 'normal' Request was from Mathieu Malaterre <malat@debian.org> to control@bugs.debian.org. (Fri, 17 May 2013 06:36:04 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Thu, 17 Oct 2013 08:00:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mathieu Malaterre <malat@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Thu, 17 Oct 2013 08:00:05 GMT) Full text and rfc822 format available.

Message #39 received at 567210@bugs.debian.org (full text, mbox):

From: Mathieu Malaterre <malat@debian.org>
To: 567210@bugs.debian.org, Control bugs server <control@bugs.debian.org>
Date: Thu, 17 Oct 2013 09:55:54 +0200
tags 567210 patch
severity 567210 grave
thanks

Instead of:

$ cat /usr/bin/saxon-xslt
#!/bin/sh

exec java -classpath /usr/share/java/saxon.jar com.icl.saxon.StyleSheet "$@"

it should read:

#!/bin/sh

java -cp /etc/xml/resolver:/usr/share/java/xslthl.jar:/usr/share/java/xml-resolver.jar:/usr/share/java/saxonb.jar
net.sf.saxon.Transform -x
org.apache.xml.resolver.tools.ResolvingXMLReader -y
org.apache.xml.resolver.tools.ResolvingXMLReader -r
org.apache.xml.resolver.tools.CatalogResolver "$@"



Added tag(s) patch. Request was from Mathieu Malaterre <malat@debian.org> to control@bugs.debian.org. (Thu, 17 Oct 2013 08:00:07 GMT) Full text and rfc822 format available.

Severity set to 'grave' from 'important' Request was from Mathieu Malaterre <malat@debian.org> to control@bugs.debian.org. (Thu, 17 Oct 2013 08:00:08 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Tue, 31 Dec 2013 19:42:10 GMT) Full text and rfc822 format available.

Acknowledgement sent to tony mancill <tmancill@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Tue, 31 Dec 2013 19:42:10 GMT) Full text and rfc822 format available.

Message #48 received at 567210@bugs.debian.org (full text, mbox):

From: tony mancill <tmancill@debian.org>
To: 567210@bugs.debian.org
Cc: Mathieu Malaterre <malat@debian.org>
Subject: re: libsaxonb-java: doc-available always returns false without network
Date: Tue, 31 Dec 2013 11:40:53 -0800
[Message part 1 (text/plain, inline)]
Control: -1 tag +help

Hello,

The patch/different arguments for invoking saxonb-xslt attached to this
bug report doesn't appear to work with the test case provided by Brian
Carlson and current version of libsaxonb-java in the archive in
jessie/sid.  Or at least it's not working for me; it hangs after the
first "Requested document is file:/path/to/index.xhtml" and I can
confirm via wireshark that it is phoning home to www.w3.org.

I also tried specifying the Apache resolver (as in [0], but with the
paths updated to match Debian's libxml-commons-resolver1.1-java JAR,
etc.), and it hangs in the same way.

Can someone demonstrate test cases that show that this bug is addressed?

Thank you,
tony

[0] https://bugs.launchpad.net/ubuntu/+source/saxonb/+bug/400277

[signature.asc (application/pgp-signature, attachment)]

Added tag(s) help. Request was from tony mancill <tmancill@debian.org> to control@bugs.debian.org. (Tue, 31 Dec 2013 19:48:04 GMT) Full text and rfc822 format available.

Message sent on to "brian m. carlson" <sandals@crustytoothpaste.net>:
Bug#567210. (Tue, 07 Jan 2014 13:42:07 GMT) Full text and rfc822 format available.

Message #53 received at 567210-submitter@bugs.debian.org (full text, mbox):

From: Mathieu Malaterre <malat@debian.org>
To: 567210-submitter@bugs.debian.org
Subject: doc-available always returns false without network
Date: Tue, 7 Jan 2014 14:39:26 +0100
Dear submitter,

  Could you please confirm that patch proposed at:

  http://bugs.debian.org/567210#39

  Does solve the issue for you ?

Thanks much



Information stored :
Bug#567210; Package libsaxonb-java. (Wed, 08 Jan 2014 00:39:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to "brian m. carlson" <sandals@crustytoothpaste.net>:
Extra info received and filed, but not forwarded. (Wed, 08 Jan 2014 00:39:08 GMT) Full text and rfc822 format available.

Message #58 received at 567210-quiet@bugs.debian.org (full text, mbox):

From: "brian m. carlson" <sandals@crustytoothpaste.net>
To: Mathieu Malaterre <malat@debian.org>, 567210-quiet@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 8 Jan 2014 00:35:37 +0000
[Message part 1 (text/plain, inline)]
On Tue, Jan 07, 2014 at 02:39:26PM +0100, Mathieu Malaterre wrote:
> Dear submitter,
>
>   Could you please confirm that patch proposed at:
>
>   http://bugs.debian.org/567210#39
>
>   Does solve the issue for you ?

It does not appear to solve the problem.  It still does not work when
networking is disabled, and it takes a long time and hangs even when
networking is enabled (probably because of rate-limiting on the W3C's
server).  With networking disabled:

  vauxhall ok % cat saxonb-xslt
  #!/bin/sh
  
  java -cp /etc/xml/resolver:/usr/share/java/xslthl.jar:/usr/share/java/xml-resolver.jar:/usr/share/java/saxonb.jar net.sf.saxon.Transform -x org.apache.xml.resolver.tools.ResolvingXMLReader -y org.apache.xml.resolver.tools.ResolvingXMLReader -r org.apache.xml.resolver.tools.CatalogResolver "$@"
  vauxhall no % ./saxonb-xslt -s:foo.xml -xsl:foo.xsl
  Requested document is file:/tmp/saxonb-testcase/index.xhtml
  Missing‽
  Requested document is index.xhtml
  Missing‽
  Requested document is file:/tmp/saxonb-testcase/index.xhtml
  Missing‽
  Requested document is index.xhtml
  Missing‽

-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187
[signature.asc (application/pgp-signature, inline)]

Information stored :
Bug#567210; Package libsaxonb-java. (Thu, 09 Jan 2014 08:18:24 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mathieu Malaterre <malat@debian.org>:
Extra info received and filed, but not forwarded. (Thu, 09 Jan 2014 08:18:24 GMT) Full text and rfc822 format available.

Message #63 received at 567210-quiet@bugs.debian.org (full text, mbox):

From: Mathieu Malaterre <malat@debian.org>
To: Michael Kay <mike@saxonica.com>
Cc: 567210-quiet@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Thu, 9 Jan 2014 09:17:10 +0100
Dear Mike,

  I am trying to solve issue reported against debian: saxonb 9.1.0.8
package. The full report is at:

http://bugs.debian.org/567210

  In summary the documentation from:

http://www.saxonica.com/documentation/sourcedocs/xml-catalogs.html

  Does not seems to apply when used within debian installation. Indeed
if one download the testcase from:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=567210#10

  and run:

$ java -cp /etc/xml/resolver:/usr/share/java/xml-resolver.jar:/usr/share/java/saxonb.jar
 -Dxml.catalog.files=/etc/xml/catalog -Dxml.catalog.verbosity=1
net.sf.saxon.Transform -x
org.apache.xml.resolver.tools.ResolvingXMLReader -y
org.apache.xml.resolver.tools.ResolvingXMLReader -r
org.apache.xml.resolver.tools.CatalogResolver -s:foo.xml -xsl:foo.xsl

I always get an error without network, however it seems to be doing
something else when network is up. I have not been able to track down
what is actually missing during this XSLT transform.

The file /etc/xml/resolver/CatalogManager.properties comes from
libxml-commons-resolver1.1-java package.

Could you please let me know if there is a way to debug (log) what is
actually downloaded from the net that makes this test script fails.

Thanks much in advance,



Information stored :
Bug#567210; Package libsaxonb-java. (Thu, 09 Jan 2014 09:33:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Kay <mike@saxonica.com>:
Extra info received and filed, but not forwarded. (Thu, 09 Jan 2014 09:33:08 GMT) Full text and rfc822 format available.

Message #68 received at 567210-quiet@bugs.debian.org (full text, mbox):

From: Michael Kay <mike@saxonica.com>
To: Mathieu Malaterre <malat@debian.org>
Cc: 567210-quiet@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Thu, 9 Jan 2014 09:05:50 +0000
These can be tricky to debug. Are you seeing a trace of URI requests from the catalog resolver? If not, have you tried using a monitoring tool such as Wireshark?

The normal cause of problems is that a source file contains a DTD or schema reference to files hosted on www.w3.org. These requests will fail immediately if there is no network, and they will usually fail after a 30 second delay if there is a network, because W3C (since a couple of years ago) has been throttling requests to serve these files. Recent versions of Saxon deal with this by automatically redirecting requests to a local copy held within Saxon itself, but with earlier releases (including 9.1.0.8) the redirection has to be done at user level.

This is all several levels removed from Saxon itself: Saxon calls the XML parser (usually Xerces) to do the parsing, and requests for DTDs etc emanate from the parser, not from Saxon. So Saxon doesn't actually know what files are being requested.

Regards,

Michael Kay
Saxonica

On 9 Jan 2014, at 08:17, Mathieu Malaterre <malat@debian.org> wrote:

> Dear Mike,
> 
>  I am trying to solve issue reported against debian: saxonb 9.1.0.8
> package. The full report is at:
> 
> http://bugs.debian.org/567210
> 
>  In summary the documentation from:
> 
> http://www.saxonica.com/documentation/sourcedocs/xml-catalogs.html
> 
>  Does not seems to apply when used within debian installation. Indeed
> if one download the testcase from:
> 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=567210#10
> 
>  and run:
> 
> $ java -cp /etc/xml/resolver:/usr/share/java/xml-resolver.jar:/usr/share/java/saxonb.jar
> -Dxml.catalog.files=/etc/xml/catalog -Dxml.catalog.verbosity=1
> net.sf.saxon.Transform -x
> org.apache.xml.resolver.tools.ResolvingXMLReader -y
> org.apache.xml.resolver.tools.ResolvingXMLReader -r
> org.apache.xml.resolver.tools.CatalogResolver -s:foo.xml -xsl:foo.xsl
> 
> I always get an error without network, however it seems to be doing
> something else when network is up. I have not been able to track down
> what is actually missing during this XSLT transform.
> 
> The file /etc/xml/resolver/CatalogManager.properties comes from
> libxml-commons-resolver1.1-java package.
> 
> Could you please let me know if there is a way to debug (log) what is
> actually downloaded from the net that makes this test script fails.
> 
> Thanks much in advance,




Information stored :
Bug#567210; Package libsaxonb-java. (Tue, 28 Jan 2014 13:33:08 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and filed, but not forwarded. (Tue, 28 Jan 2014 13:33:08 GMT) Full text and rfc822 format available.

Message #73 received at 567210-quiet@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Michael Kay <mike@saxonica.com>
Cc: 567210-quiet@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Tue, 28 Jan 2014 15:28:03 +0200
Hi Michael,

Thanks for your previous replay.
I checked with tcpdump and Wireshark that requests are made to
www.w3.org [0]. Hence the failure without network.
Could you please elaborate how to patch Saxonb 9.1.0.8 to redirect
those requests to a local copy held within Saxon itself?

You can find previous discussion here:
http://bugs.debian.org/567210

[0] http://paste.debian.net/78819/

Thanks & Regards,
Eugene



Information stored :
Bug#567210; Package libsaxonb-java. (Tue, 28 Jan 2014 14:27:09 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Kay <mike@saxonica.com>:
Extra info received and filed, but not forwarded. (Tue, 28 Jan 2014 14:27:09 GMT) Full text and rfc822 format available.

Message #78 received at 567210-quiet@bugs.debian.org (full text, mbox):

From: Michael Kay <mike@saxonica.com>
To: Eugene Zhukov <jevgeni.zh@gmail.com>
Cc: 567210-quiet@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Tue, 28 Jan 2014 14:25:16 +0000
Saxon-B 9.1 does not include copies of these resources. 

You can always write a URIResolver and direct the request to copies held at application level, but it can't be done "behind the scenes".

My recommendation would be to move forward to a later Saxon release that fixes the problem. The current release is 9.5. We have no plans to issue further maintenance releases for 9.1, although we do appreciate that some users have been sticking with that release because of the discontinuities introduced between 9.1 and 9.2.

Michael Kay
Saxonica



On 28 Jan 2014, at 13:28, Eugene Zhukov <jevgeni.zh@gmail.com> wrote:

> Hi Michael,
> 
> Thanks for your previous replay.
> I checked with tcpdump and Wireshark that requests are made to
> www.w3.org [0]. Hence the failure without network.
> Could you please elaborate how to patch Saxonb 9.1.0.8 to redirect
> those requests to a local copy held within Saxon itself?
> 
> You can find previous discussion here:
> http://bugs.debian.org/567210
> 
> [0] http://paste.debian.net/78819/
> 
> Thanks & Regards,
> Eugene




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 29 Jan 2014 08:33:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 29 Jan 2014 08:33:04 GMT) Full text and rfc822 format available.

Message #83 received at 567210@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Michael Kay <mike@saxonica.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 29 Jan 2014 10:28:42 +0200
On Tue, Jan 28, 2014 at 4:25 PM, Michael Kay <mike@saxonica.com> wrote:
> Saxon-B 9.1 does not include copies of these resources.
>
> You can always write a URIResolver and direct the request to copies held at application level, but it can't be done "behind the scenes".
>
> My recommendation would be to move forward to a later Saxon release that fixes the problem. The current release is 9.5. We have no plans to issue further maintenance releases for 9.1, although we do appreciate that some users have been sticking with that release because of the discontinuities introduced between 9.1 and 9.2.
>

We have Saxon-HE 9.4.0.7 in Debian archive. So I tried the above
test-case with it:
$ java -cp /etc/xml/resolver:/usr/share/java/xml-resolver.jar:/usr/share/java/Saxon-HE.jar
-Dxml.catalog.files=/etc/xml/catalog -Dxml.catalog.verbosity=1
net.sf.saxon.Transform -s:foo.xml -xsl:foo.xsl

The result is it still fails without network. With network it works.
Also, when I look into the source code of Saxon-HE 9.4.0.7 at [0], I
cannot find the local copies of those resources. So I don't understand
how it would work without the network. What did I miss?

[0] https://dev.saxonica.com/repos/archive/opensource/tags/9.4.0.7/

Eugene



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 29 Jan 2014 09:33:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Kay <mike@saxonica.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 29 Jan 2014 09:33:07 GMT) Full text and rfc822 format available.

Message #88 received at 567210@bugs.debian.org (full text, mbox):

From: Michael Kay <mike@saxonica.com>
To: Eugene Zhukov <jevgeni.zh@gmail.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 29 Jan 2014 09:00:29 +0000
If you use the -t option on the command line, then attempts to use local copies of W3C DTDs will be traced on System.err. Hopefully this will shed more light on why the mechanism isn't working for you.

The EntityResolver that Saxon uses in 9.4 can be found here:

https://dev.saxonica.com/repos/archive/opensource/tags/9.4.0.7/hej/net/sf/saxon/lib/StandardEntityResolver.java

I'm not sure why the data files aren't included under the 9.4.0.7 Subversion tag, but the files are here:

https://dev.saxonica.com/repos/archive/opensource/latest9.4/data/w3c/

I note that your JAR file has been renamed, so it's possible it has also been rebuilt. Look inside it with a ZIP utility and check for the directory named "w3c".

A list of the W3C documents bundled with Saxon for 9.5 can also be found here:

http://www.saxonica.com/documentation/index.html#!sourcedocs/w3c-dtds

and the corresponding list for 9.4 is at:

http://www.saxonica.com/documentation9.4-demo/index.html#!sourcedocs/w3c-dtds

Michael Kay
Saxonica


On 29 Jan 2014, at 08:28, Eugene Zhukov <jevgeni.zh@gmail.com> wrote:

> On Tue, Jan 28, 2014 at 4:25 PM, Michael Kay <mike@saxonica.com> wrote:
>> Saxon-B 9.1 does not include copies of these resources.
>> 
>> You can always write a URIResolver and direct the request to copies held at application level, but it can't be done "behind the scenes".
>> 
>> My recommendation would be to move forward to a later Saxon release that fixes the problem. The current release is 9.5. We have no plans to issue further maintenance releases for 9.1, although we do appreciate that some users have been sticking with that release because of the discontinuities introduced between 9.1 and 9.2.
>> 
> 
> We have Saxon-HE 9.4.0.7 in Debian archive. So I tried the above
> test-case with it:
> $ java -cp /etc/xml/resolver:/usr/share/java/xml-resolver.jar:/usr/share/java/Saxon-HE.jar
> -Dxml.catalog.files=/etc/xml/catalog -Dxml.catalog.verbosity=1
> net.sf.saxon.Transform -s:foo.xml -xsl:foo.xsl
> 
> The result is it still fails without network. With network it works.
> Also, when I look into the source code of Saxon-HE 9.4.0.7 at [0], I
> cannot find the local copies of those resources. So I don't understand
> how it would work without the network. What did I miss?
> 
> [0] https://dev.saxonica.com/repos/archive/opensource/tags/9.4.0.7/
> 
> Eugene




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 29 Jan 2014 12:45:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 29 Jan 2014 12:45:04 GMT) Full text and rfc822 format available.

Message #93 received at 567210@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Michael Kay <mike@saxonica.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 29 Jan 2014 14:41:13 +0200
On Wed, Jan 29, 2014 at 11:00 AM, Michael Kay <mike@saxonica.com> wrote:
> If you use the -t option on the command line, then attempts to use local copies of W3C DTDs will be traced on System.err. Hopefully this will shed more light on why the mechanism isn't working for you.
>
> The EntityResolver that Saxon uses in 9.4 can be found here:
>
> https://dev.saxonica.com/repos/archive/opensource/tags/9.4.0.7/hej/net/sf/saxon/lib/StandardEntityResolver.java
>
> I'm not sure why the data files aren't included under the 9.4.0.7 Subversion tag, but the files are here:
>
> https://dev.saxonica.com/repos/archive/opensource/latest9.4/data/w3c/
>
> I note that your JAR file has been renamed, so it's possible it has also been rebuilt. Look inside it with a ZIP utility and check for the directory named "w3c".
>
> A list of the W3C documents bundled with Saxon for 9.5 can also be found here:
>
> http://www.saxonica.com/documentation/index.html#!sourcedocs/w3c-dtds
>
> and the corresponding list for 9.4 is at:
>
> http://www.saxonica.com/documentation9.4-demo/index.html#!sourcedocs/w3c-dtds
>
Thanks for the links!
I downloaded the official release from [0] and tried the test with it.
Here is the result with network: http://paste.debian.net/78983/
And here is the result without network: http://paste.debian.net/78984/
As you can see the test without network still fails. With this -t
option, when the test succeeds, you can see Saxon fetching a local
copy, but that doesn't seem to be the case without network.

[0] http://sourceforge.net/projects/saxon/files/Saxon-HE/9.4/SaxonHE9-4-0-7J.zip/download

Eugene



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 29 Jan 2014 13:03:13 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Kay <mike@saxonica.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 29 Jan 2014 13:03:13 GMT) Full text and rfc822 format available.

Message #98 received at 567210@bugs.debian.org (full text, mbox):

From: Michael Kay <mike@saxonica.com>
To: Eugene Zhukov <jevgeni.zh@gmail.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 29 Jan 2014 13:01:28 +0000
OK, so the problem seems to be here:

Cannot read xhtml11/xhtml-inlpres-1.mod file

and the reason would appear to be the absence of the w3c/ prefix on the file name.

This takes us to here:

https://saxonica.plan.io/boards/3/topics/5625

and that in turn leads me to

https://saxonica.plan.io/issues/1813

which I think is fixed in the 9.5 branch but not in 9.4.

The underlying cause is inconsistent use of system IDs and public IDs in the W3C-published DTDs.

Michael Kay
Saxonica


On 29 Jan 2014, at 12:41, Eugene Zhukov <jevgeni.zh@gmail.com> wrote:

> On Wed, Jan 29, 2014 at 11:00 AM, Michael Kay <mike@saxonica.com> wrote:
>> If you use the -t option on the command line, then attempts to use local copies of W3C DTDs will be traced on System.err. Hopefully this will shed more light on why the mechanism isn't working for you.
>> 
>> The EntityResolver that Saxon uses in 9.4 can be found here:
>> 
>> https://dev.saxonica.com/repos/archive/opensource/tags/9.4.0.7/hej/net/sf/saxon/lib/StandardEntityResolver.java
>> 
>> I'm not sure why the data files aren't included under the 9.4.0.7 Subversion tag, but the files are here:
>> 
>> https://dev.saxonica.com/repos/archive/opensource/latest9.4/data/w3c/
>> 
>> I note that your JAR file has been renamed, so it's possible it has also been rebuilt. Look inside it with a ZIP utility and check for the directory named "w3c".
>> 
>> A list of the W3C documents bundled with Saxon for 9.5 can also be found here:
>> 
>> http://www.saxonica.com/documentation/index.html#!sourcedocs/w3c-dtds
>> 
>> and the corresponding list for 9.4 is at:
>> 
>> http://www.saxonica.com/documentation9.4-demo/index.html#!sourcedocs/w3c-dtds
>> 
> Thanks for the links!
> I downloaded the official release from [0] and tried the test with it.
> Here is the result with network: http://paste.debian.net/78983/
> And here is the result without network: http://paste.debian.net/78984/
> As you can see the test without network still fails. With this -t
> option, when the test succeeds, you can see Saxon fetching a local
> copy, but that doesn't seem to be the case without network.
> 
> [0] http://sourceforge.net/projects/saxon/files/Saxon-HE/9.4/SaxonHE9-4-0-7J.zip/download
> 
> Eugene




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 29 Jan 2014 13:27:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 29 Jan 2014 13:27:04 GMT) Full text and rfc822 format available.

Message #103 received at 567210@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Michael Kay <mike@saxonica.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 29 Jan 2014 15:25:08 +0200
On Wed, Jan 29, 2014 at 3:01 PM, Michael Kay <mike@saxonica.com> wrote:
> OK, so the problem seems to be here:
>
> Cannot read xhtml11/xhtml-inlpres-1.mod file
>
I tried with latest 9.4 and it wasn't the case anymore, but main
problem persisted.

> and the reason would appear to be the absence of the w3c/ prefix on the file name.
>
> This takes us to here:
>
> https://saxonica.plan.io/boards/3/topics/5625
>
> and that in turn leads me to
>
> https://saxonica.plan.io/issues/1813
>
> which I think is fixed in the 9.5 branch but not in 9.4.
>
> The underlying cause is inconsistent use of system IDs and public IDs in the W3C-published DTDs.
>

I tried with the latest Saxon from [0] and it still fails without
network, but now it gives some valuable output:
[...]
Warning: SXXP0005: The source document is in namespace
http://www.w3.org/2005/Atom, but none of the
  template rules match elements in this namespace
[...]
Saxon does not have a local copy of PUBLIC -//W3C//DTD XHTML+RDFa
1.0//EN SYSTEM http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd
[...]

[0] http://sourceforge.net/projects/saxon/files/Saxon-HE/9.5/SaxonHE9-5-1-4J.zip/download



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Wed, 29 Jan 2014 14:27:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Michael Kay <mike@saxonica.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Wed, 29 Jan 2014 14:27:05 GMT) Full text and rfc822 format available.

Message #108 received at 567210@bugs.debian.org (full text, mbox):

From: Michael Kay <mike@saxonica.com>
To: Eugene Zhukov <jevgeni.zh@gmail.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Wed, 29 Jan 2014 14:25:00 +0000
> Warning: SXXP0005: The source document is in namespace
> http://www.w3.org/2005/Atom, but none of the
>  template rules match elements in this namespace

You can ignore that warning for present purposes.
> [...]
> Saxon does not have a local copy of PUBLIC -//W3C//DTD XHTML+RDFa
> 1.0//EN SYSTEM http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd

Unfortunately there is no complete list of DTDs on the W3C site that might potentially needed, and even if there were, I probably wouldn't want to ship them all with Saxon. So you might have to go back to using catalogs. On the other hand, if you can identify where this was referenced from, I can take a look and see if it ought to be included. It looks as if it comes from one of the XHTML variants, but there seem to be many of these in use.

Michael Kay
Saxonica




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Thu, 30 Jan 2014 11:51:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Thu, 30 Jan 2014 11:51:05 GMT) Full text and rfc822 format available.

Message #113 received at 567210@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Michael Kay <mike@saxonica.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Thu, 30 Jan 2014 13:49:57 +0200
On Wed, Jan 29, 2014 at 4:25 PM, Michael Kay <mike@saxonica.com> wrote:
>
>> Warning: SXXP0005: The source document is in namespace
>> http://www.w3.org/2005/Atom, but none of the
>>  template rules match elements in this namespace
>
> You can ignore that warning for present purposes.
>> [...]
>> Saxon does not have a local copy of PUBLIC -//W3C//DTD XHTML+RDFa
>> 1.0//EN SYSTEM http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd
>
> Unfortunately there is no complete list of DTDs on the W3C site that might potentially needed, and even if there were, I probably wouldn't want to ship them all with Saxon. So you might have to go back to using catalogs. On the other hand, if you can identify where this was referenced from, I can take a look and see if it ought to be included. It looks as if it comes from one of the XHTML variants, but there seem to be many of these in use.
>

Sorry about the confusion with catalogs. Back to initial test-case
with the latest Saxon-HE from [0]:
$ java -cp /etc/xml/resolver:/usr/share/java/xml-resolver.jar:../saxon9he.jar
-Dxml.catalog.files=/etc/xml/catalog -Dxml.catalog.verbosity=1
net.sf.saxon.Transform -t
-x:org.apache.xml.resolver.tools.ResolvingXMLReader
-y:org.apache.xml.resolver.tools.ResolvingXMLReader
-r:org.apache.xml.resolver.tools.CatalogResolver -s:foo.xml
-xsl:foo.xsl

It fails immediately without network, but hangs and succeeds with
network. "Saxon does not have a local copy of PUBLIC -//W3C//DTD
XHTML+RDFa" is not shown anymore! With tcpdump and wireshark I see a
request:
GET /MarkUp/DTD/xhtml-rdfa-1.dtd HTTP/1.1
User-Agent: Java/1.6.0_27
Host: www.w3.org
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive

As you say in the previous letter there is no complete list of DTDs on
the W3C site that might potentially be needed, and even if there were,
Saxon would not ship them all.
Should we conclude then this is a corner test-case and is not
supported by Saxon offline?

[0] http://sourceforge.net/projects/saxon/files/Saxon-HE/9.5/SaxonHE9-5-1-4J.zip/download

Eugene



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Fri, 31 Jan 2014 08:09:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Fri, 31 Jan 2014 08:09:05 GMT) Full text and rfc822 format available.

Message #118 received at 567210@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Michael Kay <mike@saxonica.com>
Cc: 567210@bugs.debian.org
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Fri, 31 Jan 2014 10:07:34 +0200
On Thu, Jan 30, 2014 at 1:49 PM, Eugene Zhukov <jevgeni.zh@gmail.com> wrote:
> It fails immediately without network, but hangs and succeeds with
> network. "Saxon does not have a local copy of PUBLIC -//W3C//DTD
> XHTML+RDFa" is not shown anymore! With tcpdump and wireshark I see a
> request:
> GET /MarkUp/DTD/xhtml-rdfa-1.dtd HTTP/1.1
> User-Agent: Java/1.6.0_27
> Host: www.w3.org
> Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
> Connection: keep-alive
I just wanted to amend this test-case with more-or-less full HTTP
request/response chain from tcpdump: http://paste.debian.net/79423/



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>:
Bug#567210; Package libsaxonb-java. (Thu, 06 Feb 2014 13:27:11 GMT) Full text and rfc822 format available.

Acknowledgement sent to Eugene Zhukov <jevgeni.zh@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Java Maintainers <pkg-java-maintainers@lists.alioth.debian.org>. (Thu, 06 Feb 2014 13:27:11 GMT) Full text and rfc822 format available.

Message #123 received at 567210@bugs.debian.org (full text, mbox):

From: Eugene Zhukov <jevgeni.zh@gmail.com>
To: Debian Java <debian-java@lists.debian.org>
Cc: 567210@bugs.debian.org, Michael Kay <mike@saxonica.com>
Subject: Re: Bug#567210: doc-available always returns false without network
Date: Thu, 6 Feb 2014 15:25:07 +0200
Hello,

A fresh look/opinion is welcome! Could someone please take a fresh
look at the bug #567210.
After closer look at my last test results (described in my two last
mails to the bug), I tend to think the problem lies in Saxon-{B|HE}.

Any comments are very much appreciated,
Eugene



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 17 15:37:45 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.