kicad-developers team mailing list archive

[Dick Hollenbeck <dick@xxxxxxxxxxx>]

On 08/20/2014 11:45 AM, Wayne Stambaugh wrote:
> On 8/19/2014 5:17 PM, Dick Hollenbeck wrote:
>> On 08/19/2014 03:37 PM, Wayne Stambaugh wrote:
>>> On 8/19/2014 4:17 PM, Dick Hollenbeck wrote:
>>>> On 08/18/2014 06:47 PM, Wayne Stambaugh wrote:
>>>>> On 8/18/2014 6:45 PM, Brian Sidebotham wrote:
>>>>>> On 16 August 2014 17:44, Wayne Stambaugh <stambaughw@xxxxxxxxxxx> wrote:
>>>>>>> One of the tasks that I have committed to working on in the KiCad road
>>>>>>> map is to clean up the current mess we have created by allowing
>>>>>>> dependency libraries to be built as part of the KiCad source build.  The
>>>>>>> only exception I see for the time being is Boost.  Although I am have my
>>>>>>> reservations on that as well.  Why you ask?  I've spent several days
>>>>>>> trying to get KiCad to build on Windows using MSYS2 as my build
>>>>>>> environment and mingw64 as my target environment.  Every single library
>>>>>>> dependency with the exception of our custom Boost and avhttp (which
>>>>>>> could easily be build and installed using CMake) are already packaged
>>>>>>> for me.  However, the current KiCad build insists on downloading and
>>>>>>> building some libraries from source that are already installed.  This is
>>>>>>> silly.  I can resolve the issues by passing all of the
>>>>>>> PACKAGE_ROOT_PATHs when I run CMake but that is silly as well since my
>>>>>>> build environment already points the correct path.
>>>>>>>
>>>>>>> Originally I intended to create a separate project to build the KiCad
>>>>>>> library dependencies but I have since changed my mind.  I do *not* think
>>>>>>> it is asking too much of developers to learn how to build and/or install
>>>>>>> libraries on their preferred platform.  If as a developer you must have
>>>>>>> this done for you automatically, I am going to please ask that *you*
>>>>>>> create a separate platform specific build tool such as the excellent
>>>>>>> kicad-winbuilder that Brian has created.  This will significantly
>>>>>>> simplify the KiCad CMake files and eliminate the situation I described
>>>>>>> above.  My preference and goal is that the KiCad CMake files be used to
>>>>>>> build KiCad, not library dependencies.
>>>>>>>
>>>>>>> Initially, I plan to remove the dependencies that do not require any
>>>>>>> patching to build which currently are avhttp, swig, cairo, libpng,
>>>>>>> pixman, pcre, pkgconfig, and glew.  Then I will remove the dependencies
>>>>>>> with platform specific patches which are openssl, wxwidgets and
>>>>>>> wxpython.  Then I will try to figure out what to do with the problem
>>>>>>> child that is Boost.  I would also suggest that all platform specific
>>>>>>> library dependency build patches be remove as well leaving only the
>>>>>>> Boost patches that are required for all platforms (except the context
>>>>>>> switching patches).
>>>>>>>
>>>>>>> My goal here is not to step on anyone's toes it is to get our build
>>>>>>> system under control so that I can build *KiCad* rather than figure out
>>>>>>> how to get the dependencies to build or not as the platform dictates.  I
>>>>>>> expect our code to be well designed and I don't think expecting the same
>>>>>>> from our build system design is out of line.  If any one has major
>>>>>>> objections then we will have to figure something out because I am not
>>>>>>> going to continue to spend valuable time fighting our build tools to get
>>>>>>> them to properly use the dependencies already installed on my platform.
>>>>>>>
>>>>>>> Wayne
>>>>>>>
>>>>>>
>>>>>> Hi Wayne,
>>>>>>
>>>>>> I think that sounds like a sane decision, although of course KiCad
>>>>>> will be subject to the bugs in other libraries, but it's up to
>>>>>> distribution maintainers to sort those out in my opinion.
>>>>>>
>>>>>> I hope that we can use some sort of deprecation system, please mark a
>>>>>> lib as being deprecated so that I can sort out Winbuilder before the
>>>>>> CMake system is broke. It's much easier to work that way round as
>>>>>> opposed to a reactionary approach where we break everything and then
>>>>>> everyone has to fix their build before being able to do anything. I
>>>>>> will do the leg work to keep the Winbuilder people happy and do any
>>>>>> projects necessary to package dependencies required by it.
>>>>>
>>>>> I will be making changes very slowly because I expect there to be some
>>>>> breakage with some of the FindFoo.cmake changes I'm going to make.  This
>>>>> will give everyone time to respond should I break anything.  I will be
>>>>> pushing hard against adding personal install paths to every
>>>>> FindFoo.cmake file.  CMake knows how to find the correct default paths
>>>>> on most platforms and I will always give the developer the chance to
>>>>> override those using either and environment variable or a definition on
>>>>> the CMake command line for custom library testing.  I will start out
>>>>> with the dependencies that don't require any patches to build and then
>>>>> address the more complex ones like wxWidgets and finish up with the
>>>>> nightmare that is Boost.  If I change more than one FindFoo.cmake every
>>>>> two weeks, that will be a blistering pace.  The goal is to get each
>>>>> dependency test working and properly designed before moving to the next one.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Wayne
>>>>
>>>>
>>>> The current thinking among those using CMake for several large projects is that it can
>>>> either find or it can build in the first pass.  It struggles with finding a prerequisite
>>>> and building the prerequisite as a fallback, then continuing to build that which does the
>>>> depending.  This is why when you dig into it, quite a number of larger projects have split
>>>> their CMake builders into two.  Sometimes this is done with or without a 3rd umbrella
>>>> builder on top of the two.
>>>>
>>>> a) prerequisite builder
>>>> b) core project builder
>>>
>>> I would be happy if b) would work well on every platform where the
>>> developer had the appropriate prerequisites already built and installed.
>>>  As far as a) goes, I would prefer that this be completely separate code
>>> and not a requirement to build KiCad.  
>>
>>
>> They are called prerequisites because they are.  I don't begin to understand this statement.
> 
> Put simply, my preference is that a) is optional and not the default
> setting. 


a) is like cosco.  You go in with a shopping cart and a shopping list, and when you come
out it can be empty.  There's more in the store than you need.  It is a repository of
intellectual property called build recipes.  Linux users would typically come out with
empty shopping carts because they have all that same stuff at home.  Maybe they don't even
go into the store.  CMake is the world's best build system, so having the intellectual
property (a.k.a package recipes) that Marco, Brian, and I developed is valuable, to
potentially the whole world.


> The current situation forces me to build dependencies in a
> manner that is not how I prefer to work. 

This was agreed to.  It is bad.

> I am more than capable of
> applying patches, configuring, building, and install library
> dependencies from source on all of the platforms that I use for
> development.  This allows me to control my builds the way I want so when
> things do go wrong, I know exactly where to look.  It also allows me to
> build dependencies and kicad from a single source.  Maybe I'm wrong here
> but my question is who is a) for, developers or users trying to build
> kicad on their own? 


If you want a precise answer, simply delete all the build recipes and see who complains.


> If it's for developers, my next question would be
> if you do not have enough experience to build and install libraries from
> source on your platform, given the complexity of kicad are you a
> competent enough developer to contribute in a meaningful way to kicad?

Right.  An architect fresh out of school shows up on the job working for Donald Trump.  Is
trump going to ask him to mix concrete in a wheel barrow just to prove that he knows how
to make a building's foundation?  What about consistency, what about productivity?  Cement
needs to be mixed the same way, and Donald has his own Cement factory in some cases.  He
wants the architect architecting.


> 
> Another solution I could live with is would be to wrap all of the
> external dependency build code inside an if condition such as:
> 
> option( KICAD_BUILD_DEPS "Build dependency libraries." OFF )
> 
> if( KICAD_BUILD_DEPS )
>     # Put all of external project build stuff inside here *only*.
> else()
>     # Use find_package() here.
> endif()


No, you either disbelieve what I said about a) and b) co-existing in the same build
script, or you don't believe I know what I'm talking about.  I am not sure which.




> 
> This way it's available if you need or want to use it.  On a properly
> configured system, KICAD_BUILD_DEPS=OFF should work without any special
> cmake command line incantations or environment variables.


http://stackoverflow.com/questions/17446981/cmake-externalproject-add-and-findpackage/23570741#23570741

Read that, then google for "CMake super-build" and see how many projects went that way.

There are about 10 links to the problem all over the internet.  I conscientious developer
that was going to spend their own time on this would pay for some phone time with kitware.

Again, CMake is the world's premier build system.  I know this because I have used many
many of them.  I maintain a linux distro in open-embedded, NOW.  I have used, cygwin,
msys, configure, buildroot, make, scons, distutils, blah blah blah.  Nothing compares to
CMake.

What we have are valuable CMake package recipes that Kitware could be interested in
helping support and fine tune.   The recipes are screaming for their own cookbook (to
switch metaphors).    The cookbook has many pages, each page can be consulted as needed.

The CMakeCache.txt can be preloaded as needed to assist in suggesting recipes, on a
platform specific basis.  You, Wayne, don't want to read the cookbook, you just want to
make KiCad and build your own ingredient.   Then simply separate the recipes.  This should
not be so difficult to understand.  Take a half day and read about bitbake some time.

And don't underestimate the value of these recipes.  Marco worked hard on them, as did
Brian, and did I.  They do not need to be mandatory cooking exercises however.  CMake is
actually that good.

Dick







> 
>>
>> It's like saying I wish my car never ran out of gas, so therefore I won't buy any.
>>
>> The prerequisites have to come from somewhere, the gas has to come from somewhere.
>>
>> Your statement confuses me to the point where I think its time for others to take up the
>> mantle here.
>>
>> You folks have a difficult struggle ahead.  Again, somebody needs to get on the phone with
>> kitware and get their advice and help.  You have the pieces, you need the expertise and help.
>>
>> I am out on this topic,  I tried.  I don't feel like I succeeded.
>>
>>
>> Dick
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>