openjdk team mailing list archive

[tony mancill <tmancill@xxxxxxxxxx>]

On Tue, Sep 22, 2020 at 10:07:12AM +0100, Julian Gilbey wrote:
> On Mon, Sep 21, 2020 at 10:49:24PM -0700, tony mancill wrote:
> > [...]
> > 
> > One other thing I wanted to mention is what I'm using for a test case:
> > 
> >   jlink --add-modules java.desktop --output test
> > 
> > This fails, complaining about the hash of java.xml recorded in
> > java.base.  But other invocations of jlink are successful with the
> 
> That is super-weird; this command works fine for me.
> 
> Things to check (again, just trying to think of obvious things that
> I might well do wrong myself ;-): what is the output of
> 
> jlink describe debian/openjdk-11-jdk-headless/usr/lib/jvm/java-14-openjdk-amd64/jmods/java.base.jmod | grep java.xml
> sha256sum debian/openjdk-14-jdk-headless/usr/lib/jvm/java-14-openjdk-amd64/jmods/java.xml.jmod
> 
> (run in the build directory)?  Do these hashes match those in the
> error message that you're seeing with the failing jlink command?

No, the hashes don't match, although FWIW, they are consistent across
builds.  Looking only at java.xml.jmod...


>From the build logs:

Creating java.xml.jmod
Executing: [/bin/rm -f /<<PKGBUILDDIR>>/build/images/jmods/java.xml.jmod /<<PKGBUILDDIR>>/build/support/images/jmods/java.xml.jmod]
Executing: [/<<PKGBUILDDIR>>/build/jdk/bin/jmod -J-XX:+UseSerialGC -J-Xms32M -J-Xmx512M -J-XX:TieredStopAtLevel=1 create --module-version 14.0.2 --target-platform 'linux-amd64' --module-path /<<PKGBUILDDIR>>/build/images/jmods --class-path /<<PKGBUILDDIR>>/build/jdk/modules/java.xml --legal-notices "/<<PKGBUILDDIR>>/build/support/modules_legal/common:/<<PKGBUILDDIR>>/src/java.xml/share/legal" --exclude '**{_the.*,_*.marker,*.diz,*.debuginfo,*.dSYM/**,*.dSYM}' /<<PKGBUILDDIR>>/build/support/images/jmods/java.xml.jmod]
Executing: [strip-nondeterminism /<<PKGBUILDDIR>>/build/support/images/jmods/java.xml.jmod && sha256sum /<<PKGBUILDDIR>>/build/support/images/jmods/java.xml.jmod && /bin/mv /<<PKGBUILDDIR>>/build/support/images/jmods/java.xml.jmod /<<PKGBUILDDIR>>/build/images/jmods/java.xml.jmod]
4a15f120c6eedb70aee4bca50f90e62e9f48dd7517eb511087a0c05e26efa039  /<<PKGBUILDDIR>>/build/support/images/jmods/java.xml.jmod

 
On the filesystem after installing the binary packages:

$ sha256sum /usr/lib/jvm/java-14-openjdk-amd64/jmods/java.xml.jmod
22fa9b1ebb789375c8fb8e1644bc2b49620a63a13682dc4d5a4497d445d9ef89  /usr/lib/jvm/java-14-openjdk-amd64/jmods/java.xml.jmod

>From the error message:

Error: Hash of java.xml (22fa9b1ebb789375c8fb8e1644bc2b49620a63a13682dc4d5a4497d445d9ef89) differs to expected hash (4a15f120c6eedb70aee4bca50f90e62e9f48dd7517eb511087a0c05e26efa039) recorded in java.base


This leads me to question whether strip-nondeterminism is idempotent,
although based on some simple empirical testing, it appears that it is.
In any event, something is happening the jmod later during the build
such that it differs from what was recorded.  I'm still trying to grok
the logic around the "interim" builds of jmods; perhaps that's where the
difference lies.

 
> Which jlink are you using when you run your test command?
> (/usr/bin/jlink is a symlink to /etc/alternatives/jlink - where does
> that point?)

Everything looks sane with my openjdk-14

$ ls -al /usr/bin/jlink                          
lrwxrwxrwx 1 root root 23 Jul 16  2018 /usr/bin/jlink -> /etc/alternatives/jlink
$ ls -al /etc/alternatives/jlink
lrwxrwxrwx 1 root root 44 Sep 21 21:21 /etc/alternatives/jlink -> /usr/lib/jvm/java-14-openjdk-amd64/bin/jlink


> > patched build.  For example, the following is fine:
> > 
> >   jlink --add-modules java.base --output test
> > 
> > As does java.compiler and 32 others.  So 34 are okay and 38 of them are
> > failing.  The "good" vs. "fail" modules are attached in case it sparks
> > inspiration for someone.  I will take a look at the output of the
> > non-parallelized build in the (GMT-7) morning.
> > 
> > So there are approximately 50/50 odds that your test case is working
> > (hence the confusion), because the strip-determinism patch is only be
> > causing problems for a subset of modules.
> 
> There's nothing obvious to me in this list.  It's just a mystery...
> In my build, the SHA-256 hashes in java.base.jmod exactly match the
> SHA-256 hashes of the jmod files themselves.

Perhaps you'd like to prepare a source package and/or could make your
binaries available so I can try to reproduce with your build.

If this due to my local setup, and given that I was planning on a
delayed NMU anyway, another option would be for you to NMU your working
build (maybe to experimental we want to poke at it more).

> > Thank you for the ideas.  I will keep working on it.
> 
> Could you perhaps make your openjdk-*_*.debian.tar.xz available
> somewhere so that I can try it on my machine, and see if there's a
> difference between machines?  (I can get the .orig.tar.gz from the
> mirrors.)

Sure thing.  My debian.tar.xz and the build log from the .NONPARALLEL:
build, etc. can be found here:
https://people.debian.org/~tmancill/openjdk-14/944738/

FWIW, I ran the same "test all modules" with this most current build and
the same set of jmods fail with jlink.

Here's the test:

```
for x in $(ls -1 /usr/lib/jvm/java-14-openjdk-amd64/jmods/* | xargs -L1 basename | sed 's/\.jmod//')
do
   /usr/lib/jvm/java-14-openjdk-amd64/bin/jlink --add-modules ${x} --output test-${x}
done
```

$dayjob intrudes, but I will pick this up again this evening.

Thanks!
tony