yade-dev team mailing list archive

["Janek Kozicki (yade)" <jkozicki-yade@xxxxxxxxx>]

OK, I will experiment a bit and see what it takes to switch to pybind11.

Janek


Janek Kozicki (yade) said:     (by the date of Fri, 21 Feb 2020 22:30:04 +0100)

> Hi Anton,
> 
> Yes, the number of defines in Serialization.hpp is crazy.
> Changing/removing them to use pybind11 would take a lot of work.
> I'm sure it is possible, but not quickly.
> 
> We can open an issue about pybind11 switch. Discuss pros and cons,
> find a good strategy to deal with crazy defines and complete it later.
> 
> Doing this switch right now is unrealistic. Working on another moving
> target (pybind11) on top of an already moving target (minieigen HP
> not fully integrated) is irresponsible. If possible I would prefer:
> 
>   * minieigen-src package (slower compilation only for non-double)
>   * or integrate minieigen with yade sources for now (rather undesirable
>     due to compilation time).
> 
> Whatever we do I would prefer to integrate HP, because doubles will
> appear very quickly in the code.
> 
> Ah, if we talked about pybind11 switch three months ago it would be
> perfect timing for me :( In fact I was thinking about it that time,
> but I didn't want to push into something that wasn't guaranteed to be
> working [*]. Figuring out the correct way to convert high precision
> to/from python with boost was difficult, for a while I prefer to not
> repeat this task with pybind11. But surely we can get back to this
> later.
> 
> The extra time for compilation happens only when non-double type is used.
> If minieigen-src package gets to 20.04 it will be possible later for
> people to compile high precision yade from gitlab sources without big hassle.
> 
> The build_focal_simple [1] is 5 minutes because it's not parallelized.
> The problem is that pybuild is not accepting -j argument.
> With make -j 6 it is faster [2][3].
> 
> Also a fast quad_double (62 decimal places, package libqd-dev)
> integration is in the works with boost::multiprecision developers [4].
> 
> To summarize:
> 
>  * I would rather avoid pybind11 switch in a rush, better give it a few months
>  * I would prefer minieigen-src package (no extra compilation time for people who use `double`)
>  * minieigen-src in ubuntu 20.04 will let more people to test & use high precision.
> 
> I understand that this solution is temporary until we figure a
> proper way to deal with pybind11. The 4 year LTS is more than enough
> to solve pybind11 + Serialization.hpp problem.
> 
> It is better to finish HP integration to not lose what is already
> working.
> 
> best regards
> Janek
> 
> 
> [*] pybind11 is a new dependency and a new package, boost is well
>     integrated with itself, it was guaranteed to work.
> 
> [1] https://gitlab.com/yade-dev/minieigen/-/jobs/444927466
> 
> [2] https://gitlab.com/cosurgi/minieigen-real/pipelines/118211208 - I used this to test HP
> 
> [3] In the pipeline with ccache it's usually not a problem. On
>     debian build servers: it won't be a problem until we decide to
>     make yade-float128 package. But if we made an accompanying
>     minieigen-float128 this problem would disappear. But pybind11
>     alternative is good also, just unrealistic right now.
> 
> [4] https://github.com/boostorg/multiprecision/issues/184
> 
> 
> Anton Gladky said:     (by the date of Fri, 21 Feb 2020 20:09:07 +0100)
> 
> > Hi Janek,
> > 
> > we have misunderstanding here. python3-minieigen is the __binary__
> > package and it is a bad idea to ship the source code with the package.
> > 
> > Adding minieigen-src binary package is possible, but it looks like very
> > undesired way.
> > 
> > As I see, only Yade is using minieigen in the Debian. So, theoretically we
> > can merge it again with Yade. Not sure, whether it is a good way though....
> > Yade compiles too slow. Minieigen adds definitely 4-5 compilation minutes
> > and Gigabytes of used RAM.
> > 
> > We should really have a look at pybind11 alternative, if it accelerates the
> > compilation. But when I see Serialization.hpp, I am getting crazy from the
> > number of "defines" there.
> > 
> > Anton
> > 
> > Am Fr., 21. Feb. 2020 um 17:04 Uhr schrieb Janek Kozicki (yade)
> > <jkozicki-yade@xxxxxxxxx>:
> > >
> > > Uh Anton, I'm sorry to be so boring:
> > >
> > > I have checked with sid in /etc/apt/sources.list:
> > >
> > >   deb-src http://ftp.pl.debian.org/debian/ sid  main non-free contrib
> > >
> > > and built the python3-minieigen_0.50.3+dfsg1-12_amd64.deb package.
> > >
> > > There is no /usr/include/minieigen/*pp files inside :(
> > >
> > > For high precision to work, they are necessary. Maybe the proper way
> > > to do this is to introduce python3-minieigen-dev package? I'm not
> > > sure. These sources are needed because of these include [1] statements.
> > >
> > > I am attaching once again the file python3-minieigen.install which
> > > installs *pp files. Even the *.cpp files are used. If you feel
> > > that *.cpp files are too much, we could duplicate the .cpp files in
> > > yade (they are rather short) and only include the *.hpp files (these
> > > are quite long). But all *pp files in /usr/include/minieigen/ would
> > > be perfect.
> > >
> > > Best regards
> > > Janek
> 
> 
> -- 
> --
> Janek Kozicki, PhD. DSc. Arch. Assoc. Prof.
> Gdańsk University of Technology
> Faculty of Applied Physics and Mathematics
> Department of Theoretical Physics and Quantum Information
> --
> http://yade-dem.org/
> http://pg.edu.pl/jkozicki (click English flag on top right)
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~yade-dev
> Post to     : yade-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~yade-dev
> More help   : https://help.launchpad.net/ListHelp


-- 
--
Janek Kozicki, PhD. DSc. Arch. Assoc. Prof.
Gdańsk University of Technology
Faculty of Applied Physics and Mathematics
Department of Theoretical Physics and Quantum Information
--
http://yade-dem.org/
http://pg.edu.pl/jkozicki (click English flag on top right)