← Back to team overview

kicad-developers team mailing list archive

Re: GitHub Plugin (my nemesis)

 

As a user, I always move to the edge of my seat when this conversation comes up. I sit there, waiting for awesome new features like in-program diff and pull requesting for kicad libraries, and then I inevitably sit back down because (IMHO) kicad wanted to use the github api and start wget vs. curl w/openssl license issues or whatever it was that happened on the mailing list years ago when the feature got developed, instead of trying to use an actual git client to be... well... a git client, and because of all the work that went into it, it seems no one wants to scrap it.

And I wholeheartedly agree that the libraries should be in one repo, i.e. why were they separate to begin with... I've heard that git submodules is not awesome, but surely it would apply in this situation? You'd get one encompassing repo, and then the submodules that you specifically want, no more.

I'm sure everyone thoroughly thought through the design decisions involved, it just seems really weird and kicad-style quirky.

Sorry if this email offended anyone, it was only my excitement of getting the thought out there, I didn't mean to be rude.
JS

On 9/21/2017 9:26 PM, Jon Evans wrote:
I think it would make the most sense to do option (a), possibly also leaving open the possibility of changing the updates source location (for example, it's conceivable to have a mirror server at kicad-pcb.org <http://kicad-pcb.org> that contains snapshots of the libraries that are compiled from the Git repo whenever it's updated using a Jenkins job).

-Jon

On Thu, Sep 21, 2017 at 9:21 PM, Oliver Walters <oliver.henry.walters@xxxxxxxxx <mailto:oliver.henry.walters@xxxxxxxxx>> wrote:

    Hi all,

    Ok, now that the website integration with the libraries is (pretty
    much) done, and the licensing issue seems to be sorted, there is
    one final puzzle piece to solve before I'm happy with the state of
    the libraries for a v5 release.

    *Goal: *Merge all footprint library repositories into a single
    repo to solve the ongoing dramas of maintaining 100+ repos.

    *Problem*: The *only* thing standing in the way of just doing this
    is that some users like the GitHub plugin and previous instruction
    is that this functionality cannot be removed.

    The GitHub plugin functions by downloading a .zip file of a
    .pretty repo. If we merge all footprint libs into a single repo
    with multiple subdirectories, this will not work anymore (as
    GitHub dosen't allow you to download a .zip of a single subdirectory).

    Merging the repos is the *right thing to do*. But how to proceed?

    *Options:*
    *
    *
    /a) Drop github plugin feature, replace with library-download tool/

    I don't think it is a good idea to live-load library data from
    GitHub (a lot of other users agree too). It's slow, and a waste of
    bandwidth to re-download the libs all the time.

    We drop support for loading libraries direct from GitHub. However,
    we add a tool for downloading libraries from GitHub and storing to
    disk. Users can update as they like. This can be integrated in
    KiCad and new users can run this tool when they first install
    KICad. This means that no libs need to distributed with the
    installer and users can update to latest libs whenever they want.

    /b) Improve github plugin to allow subdirectory traversal/

    This is difficult and will only result in the plugin being slower.
    There are two ways I can see to do this:

    i. Use Git API - tools exist that use this functionality -
    https://github.com/KinoLien/gitzip
    <https://github.com/KinoLien/gitzip>
    ii. Use subversion - GitHub actually provides subversion API -
    https://www.seanw.org/blog/download-git-repo-subdirectory/
    <https://www.seanw.org/blog/download-git-repo-subdirectory/>


    *Subversion*

    In either case, I think that using the subversion tool to
    partially download the libraries would be a good approach (I
    assume quicker than using wget and the GitHub API).

    1. Does anyone have any experience using the C API for SVN?
    2. The Python bindings are pretty good -
    https://pypi.python.org/pypi/svn
    <https://pypi.python.org/pypi/svn> - and much easier to use.
    However, can we make the library download tools dependent on
    enabling the python plugin?
    3. Is there a way to checkout a subversion remote to memory (to
    replicate the functionality of the current GitHub plugin)? If not,
    I'm not sure how to approach option b) above.


    Feedback appreciated. I think that it is very important especially
    for new users that this is improved. The GitHub plugin is
    constantly causing headaches!

    Cheers,
    Oliver

    _______________________________________________
    Mailing list: https://launchpad.net/~kicad-developers
    <https://launchpad.net/~kicad-developers>
    Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
    <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
    Unsubscribe : https://launchpad.net/~kicad-developers
    <https://launchpad.net/~kicad-developers>
    More help   : https://help.launchpad.net/ListHelp
    <https://help.launchpad.net/ListHelp>



_______________________________________________
Mailing list: https://launchpad.net/~kicad-developers
Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

References