dolfin team mailing list archive

[Martin Sandve Alnæs <martinal@xxxxxxxxx>]

On Tue, Jun 2, 2009 at 10:11 PM, Johan Hake <hake@xxxxxxxxx> wrote:
> On Tuesday 02 June 2009 17:05:28 Martin Sandve Alnæs wrote:
>> On Tue, Jun 2, 2009 at 3:17 PM, Johan Hake <hake@xxxxxxxxx> wrote:
>> > On Tuesday 02 June 2009 14:56:45 Martin Sandve Alnæs wrote:
>> >> On Tue, Jun 2, 2009 at 1:43 PM, Johan Hake <hake@xxxxxxxxx> wrote:
>> >> > On Tuesday 02 June 2009 12:37:51 Martin Sandve Alnæs wrote:
>> >> >> On Tue, Jun 2, 2009 at 12:34 PM, Johannes Ring <johannr@xxxxxxxxx>
> wrote:
>> >> >> > On Tue, June 2, 2009 12:24, Martin Sandve Alnæs wrote:
>> >> >> >> On Tue, Jun 2, 2009 at 10:47 AM, Johan Hake <hake@xxxxxxxxx> wrote:
>> >> >> >>> On Tuesday 02 June 2009 10:28:11 Martin Sandve Alnæs wrote:
>> >> >> >>>> I've tried to use dorsal on my home laptop which I just upgraded
>> >> >> >>>> to Jaunty.
>> >> >> >>>> I got some problems with trilinos (Harish: same as on my work
>> >> >> >>>> laptop), and will recompile now after removing all trilinos.pc
>> >> >> >>>> files around the system.
>> >> >> >>>>
>> >> >> >>>> I see from scons --help in dolfin that the path has been set
>> >> >> >>>> correctly there:
>> >> >> >>>>
>> >> >> >>>> withTrilinosDir: Specify path to Trilinos (
>> >> >> >>>> /path/to/withTrilinosDir ) default: None
>> >> >> >>>>     actual: /opt/fenics904
>> >> >> >>>>
>> >> >> >>>> I've found two issues I will call bugs:
>> >> >> >>>>
>> >> >> >>>> 1) dolfin scons didn't copy the trilinos.pc it generated to the
>> >> >> >>>> installation directory. Should be an easy fix in dolfin scons.
>> >> >> >>>
>> >> >> >>> This sounds strange. This line in the SConstruct file should do
>> >> >> >>> that:
>> >> >> >>>
>> >> >> >>>  # We also like to install all generated pkg-config files.
>> >> >> >>>  buildDataHash["pkgconfig"] +=
>> >> >> >>>    scons.globFiles(Dir("#/scons/pkgconfig/").srcnode().abspath,
>> >> >> >>> "*.pc")
>> >> >> >>
>> >> >> >> I can find no such line:
>> >> >> >>
>> >> >> >> martinal@martinal-znote:~/dev/fenics904/dolfin-0.9.2$ find -name
>> >> >> >> SConscript
>> >> >> >> ./data/SConscript
>> >> >> >> ./doc/SConscript
>> >> >> >> ./dolfin/SConscript
>> >> >> >> ./demo/SConscript
>> >> >> >> ./test/SConscript
>> >> >> >> ./sandbox/ilmarw/assembler_bench/Stokes_2D_TH/SConscript
>> >> >> >> ./sandbox/ilmarw/assembler_bench/Laplace_2D/SConscript
>> >> >> >> ./sandbox/ilmarw/assembler_bench/ICNS_3D_Momentum/SConscript
>> >> >> >> ./sandbox/ilmarw/assembler_bench/Elasticity_3D/SConscript
>> >> >> >> ./sandbox/ilmarw/assembler_bench/Stokes_2D_Stab/SConscript
>> >> >> >> martinal@martinal-znote:~/dev/fenics904/dolfin-0.9.2$ find -name
>> >> >> >> SConscript | xargs egrep buildData
>> >> >> >
>> >> >> > In DOLFIN 0.9.2 buildDataHash is called ret and the line you are
>> >> >> > looking for is 381 in dolfin/SConscript.
>> >> >> >
>> >> >> >>> Where is your trilinos.pc situated?
>> >> >> >>
>> >> >> >> Which one? I had several before, one in the path.
>> >> >> >> scons made a new one placed in the repository,
>> >> >> >> now I've copied that manually to the installation directory.
>> >> >> >> The libgaleri problem is now gone after the rebuild.
>> >> >> >>
>> >> >> >>
>> >> >> >> Compiling C++ demos works fine now, but "import dolfin" gives:
>> >> >> >>
>> >> >> >> ImportError: /opt/fenics904/lib/libdolfin.so.0: undefined symbol:
>> >> >> >> _Z26KSPMonitorTrueResidualNormP6_p_KSPidPv
>> >> >> >>
>> >> >> >> In [2]:
>> >> >> >> Do you really want to exit ([y]/n)?
>> >> >> >> martinal@martinal-znote:~$ c++filt
>> >> >> >> _Z26KSPMonitorTrueResidualNormP6_p_KSPidPv
>> >> >> >> KSPMonitorTrueResidualNorm(_p_KSP*, int, double, void*)
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> Turns out that petsc isn't handled correctly either. It's built
>> >> >> >> and installed by dorsal,
>> >> >> >> and dolfin scons gets the right directory for it,
>> >> >> >>
>> >> >> >> withPetscDir: Specify path to PETSc ( /path/to/withPetscDir )
>> >> >> >>     default: None
>> >> >> >>     actual: /opt/fenics904
>> >> >> >>
>> >> >> >> But no pkg-config file has even been generated in this case:
>> >> >> >> martinal@martinal-znote:~/dev/fenics904/dolfin-0.9.2$ ls
>> >> >> >> scons/pkgconfig/ boost.pc  gmp.pc  mtl4.pc  parmetis.pc  scotch.pc
>> >> >> >>  trilinos.pc
>> >> >> >
>> >> >> > No, because you already have one that is found in $PKG_CONFIG_PATH.
>> >> >>
>> >> >> I consider this behaviour a significant bug. A new one should be
>> >> >> generated if the $PKG_CONFIG_PATH version doesn't match the manually
>> >> >> specified version.
>> >> >
>> >> > This issue is returning, again and again. An old installed pkg-config
>> >> > file screws the construction and installation of new ones. Been there
>> >> > me too...
>> >> >
>> >> > The present "solution" has been to either manually delete it or run
>> >> >
>> >> >  scons install -c
>> >> >
>> >> > The later does only remove any pkg-config files in the install path
>> >> > and might miss out on others.
>> >>
>> >> That is not at all an acceptable solution, it requires that I modify
>> >> other parts of the system, parts that I may not even have access too.
>> >>
>> >> > So you suggest that we should check for the version of any installed
>> >> > pkg-config files and compare that to any manually specified libraries,
>> >> > let it be using FOO_DIR or withFOODir?
>> >>
>> >> Not sure about FOO_DIR, but withFOODir is a direct command
>> >> to the build system that should not be replaced by the default
>> >> system-wide option.
>> >>
>> >> > This is probably a not so intrusive check, however we should remember
>> >> > that caching is a feature. If we add too many checks the whole point
>> >> > of having cached pkg-config files goes away.
>> >>
>> >> This has nothing to do with cache (conceptually, don't know about
>> >> the implementation), this was the _first_ installation using dorsal.
>> >
>> > If you have a pkg-config file installed, scons has no way of knowing that
>> > it should not use that. If it is an erronious system wide file you cannot
>> > delete, we have a problem.
>>
>> Yes it has, I specify which directory it should use on the command line.
>> The information is there, but the scons system _choose_ to not use it.
>
> As it is now dolfin scons first check if there are a package config file
> installed in the pkg-config path. Is it there then we assume this is the
> right one. No further checks are done, so any withFooDir informations are not
> used.
>
> This goes into the whole design of the dolfin (simula) scons which is centered
> around pkg-config. The point with the withFooDir and any FOO_DIR environment
> variables are to help the pkg-config generators to generate a suitable
> pkg-config file, and then install this.


The consequence of this is that:

1) dolfin scons cannot generate pkg-config files for my
    local libraries on a system with a global installation.

2) dorsal doesn't work as intended because library paths
    are not respected by dolfin scons.

3) if somebody installs a global petsc.pc at our cluster,
    every user must provide pkg-config files manually
    and cannot use dolfin scons or dorsal properly.

and probably many more, as witnessed by many many user problems...


> This causes unexpected behaviours, as reported by you and others, when you are
> used with an other system. Personally I think dolfin scons reliance on
> pkg-config is nice.

There is no conceptual conflict between relying on pkg-config as the
fallback system
and adhering to the command line arguments the user provides to
override defaults.
If this is impossible to program in dolfin scons, then something is
seriously wrong
in the design of either scons itself or dolfin scons.


> However I see two ways of solving the issue.
>
>  1) add an un cachable option to scons to re build all pkg-config files
>     this will use any withFooDir variables. The generated pkg-config file
>     will be installed and used in subsequence builds.

Something like
  scons configure withTrilinosDir=/opt/fenics904
?

Basically an explicit configure step, but only if you ask for it directly.

Could we also have
  scons build
which skips the entire configuration phase? This takes some time today.

Then we can have just
  scons install # does everything
or separate steps:
  scons configure myoptions # no build
  scons build # no configure, just build using cached configuration
  scons install # full configuration check, build, install

Some action for just installing whatever is currently built would
also be nice in some rare cases, but not so important.


>  2) Change the order of presedence for the configuration. If withFooDir
>     is set we try to generate a pkg-config file using this directory. If not
>     try to find any installed pkg-config files, and last run the pkg-config
>     script using what ever information available to generate pkg-config file.
>
> The first suggestion is more in line with the design of dolfin scons, and I
> prefer this. The second one will try to generate a pkg-config file whenever a
> withFooDir is set, and keep doing this as the withFooDir is cached.

Clearly not desirable, I agree that (1) is better.


>> One very common reason for having to specify your own library
>> directories is when compiling in a multiuser environment.
>> For example at our cluster, I can't control what's installed or not
>> globally. Any other multiuser environments is likely to have the same
>> problem.
>
> I know this, and you can therefore alter the pkg-config path in for example
> the .bashrc file to change the precedence on which pkg-config file will be
> used.

Which renders the pkg-config generators in dolfin scons
useless when a global installation of some library exists.


>> >> Anyway, the cache system would still be broken if it ignored
>> >> my explicit "withPetscDir" argument. I ask for a specific version,
>> >> and scons gives me an arbitrary version it found on the system.
>> >> That's not a feature...
>> >
>> > Well, the withPetscDir option will be cached, and the second time you
>> > issues the command you write just scons. Is this an explicit argument
>> > then?
>>
>> Since arguments are cached, the behaviour should be equivalent to
>> specifying them directly. I don't know how scons works, and I don't
>> really care how that's implemented. Caching is an optimization,
>> and shouldn't change expected behaviour.
>
> Well I think you should care.

I'm a _user_ of dolfin scons. A user should never have to care
about the implementation, only about the interface and behaviour.


> As I have tried to explain, dolfin scons is
> built around the pkg-config files specifying the installed libraries. This
> give for example nice features of caching and modularization, a la pycc,
> which dolfin has not (yet?) utilized. However it also have side effects which
> we need to pay attention to.

Doesn't matter. Dolfin scons ignores my command line argument. Bug. Period.


>> > Would it be possible to handle the withFOODir options differently if they
>> > are called explicitly and not from cache? Maybe by setting some sort of
>> > flag to the internal option handling in dolfin scons? If it was set
>> > explicitly we regenerate the pkg-config file?
>>
>> I leave that to scons people. Just keep in mind that
>> a "cache system" that changes the behaviour is either:
>> A) NOT a cache system
>> B) Broken
>
> I do not think it is so simple, dolfin scons is not just a caching system.

Cache is a word with a meaning. When used, people expect certain things.

Wikipedia: "In computer science, a cache (pronounced /kæʃ/) is a
collection of data duplicating original values stored elsewhere or
computed earlier,..."

I stand by my words, but lets keep the continuation of this off the list ;)

Martin