kicad-developers team mailing list archive

[Maciej Sumiński <maciej.suminski@xxxxxxx>]

Hi Oliver,

IMHO, the library-download tool is the best choice here. I agree the
live download is not a good idea in many cases, but I imagine there are
people taking advantage of that (e.g. a centralized footprint repository
in a small company).

I think the main problem now, is we are relying too much on the Github
API, instead of using e.g. libgit2 [1]. Such library is not only more
flexible but also frees us from the Github. Even though it might
impossible to download a .zip file of a whole repository, there should
be an option to select single files to be downloaded or quickly list
directories. Apart from that, updates will be much faster, as you do not
need to download everything, but only deltas.

I am sure the downloader could be also implemented in Python and
integrated with pcbnew. IIRC all nightly builds are build with Python
scripting emabled, and I am not aware of any problems related to
compiling such version manually.

Regards,
Orson

1. https://github.com/libgit2/libgit2

On 09/22/2017 03:21 AM, Oliver Walters wrote:
> Hi all,
> 
> Ok, now that the website integration with the libraries is (pretty much)
> done, and the licensing issue seems to be sorted, there is one final puzzle
> piece to solve before I'm happy with the state of the libraries for a v5
> release.
> 
> *Goal: *Merge all footprint library repositories into a single repo to
> solve the ongoing dramas of maintaining 100+ repos.
> 
> *Problem*: The *only* thing standing in the way of just doing this is that
> some users like the GitHub plugin and previous instruction is that this
> functionality cannot be removed.
> 
> The GitHub plugin functions by downloading a .zip file of a .pretty repo.
> If we merge all footprint libs into a single repo with multiple
> subdirectories, this will not work anymore (as GitHub dosen't allow you to
> download a .zip of a single subdirectory).
> 
> Merging the repos is the *right thing to do*. But how to proceed?
> 
> *Options:*
> 
> *a) Drop github plugin feature, replace with library-download tool*
> 
> I don't think it is a good idea to live-load library data from GitHub (a
> lot of other users agree too). It's slow, and a waste of bandwidth to
> re-download the libs all the time.
> 
> We drop support for loading libraries direct from GitHub. However, we add a
> tool for downloading libraries from GitHub and storing to disk. Users can
> update as they like. This can be integrated in KiCad and new users can run
> this tool when they first install KICad. This means that no libs need to
> distributed with the installer and users can update to latest libs whenever
> they want.
> 
> *b) Improve github plugin to allow subdirectory traversal*
> 
> This is difficult and will only result in the plugin being slower. There
> are two ways I can see to do this:
> 
> i. Use Git API - tools exist that use this functionality -
> https://github.com/KinoLien/gitzip
> ii. Use subversion - GitHub actually provides subversion API -
> https://www.seanw.org/blog/download-git-repo-subdirectory/
> 
> 
> *Subversion*
> 
> In either case, I think that using the subversion tool to partially
> download the libraries would be a good approach (I assume quicker than
> using wget and the GitHub API).
> 
> 1. Does anyone have any experience using the C API for SVN?
> 2. The Python bindings are pretty good - https://pypi.python.org/pypi/svn -
> and much easier to use. However, can we make the library download tools
> dependent on enabling the python plugin?
> 3. Is there a way to checkout a subversion remote to memory (to replicate
> the functionality of the current GitHub plugin)? If not, I'm not sure how
> to approach option b) above.
> 
> 
> Feedback appreciated. I think that it is very important especially for new
> users that this is improved. The GitHub plugin is constantly causing
> headaches!
> 
> Cheers,
> Oliver
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>

Attachment: signature.asc
Description: OpenPGP digital signature