kicad-developers team mailing list archive

[Simon Küppers <simon.kueppers@xxxxxx>]

Your message got me thinking.

Maybe it would be a good way to implement two git repos, one for
footprints and one for 3D models (and a third one for schematic
symbols..). That is 3 repos to maintain instead of more than some dozens
as it is right now.
Then I would agree with you, I would personally be ok with downloading
either all footprints or none and not be able to select the libraries to
download. And then the user can choose if 3D models should be pulled
additionally. However that is my opinion only.
(What actually about stream compression for KiCad Libraries? Of course
this would be a big change but could reduce just the disk and download
size in general)

However, once the repos are pulled they need to be "tracked" somehow.
Because I strongly disagree with everyone saying "Just do it as you do
in software development, use the command line periodically to check for
KiCad footprint repo changes". Just No.
A simple first idea I was having in mind for a long time is just a check
on KiCad startup (by default, can be changed) what the status of the git
remote is and _notify_ the user if something has changed and if they
want to pull in the changes _to disk_. Not altering any of the designs
of course!
This could be improved later to provide a more sophisticated GIT
integration in KiCad (I get the warm fuzzy feeling when I write this :-))

> So why not use git as it was supposed to be used?
Strongly agree.

Best Regards


Am 23.09.2017 um 20:51 schrieb Thomas Kindler:
> Hi all!
> 
> After reading all the other messages, here are my two cents:
> 
> The KiCAD installer should just offer to do a full git clone of kicad-footprints
> and kicad-libray (which will be split into 3D, templates, etc. for the 5.0
> release as I understand).
> 
> A full clone takes 3.8 GB. A shallow clone takes 3.6 GB. The .git-direcotry size
> is 650 MB vs 486 MB.
> 
> So it's not worth the hassle IMHO as shallow clones don't allow to generate
> pull-requests without unshallowing them first.
> 
> BTW: 2 GB of that checkout is spent on the packages3d subdirectory. We could
> just support loading of .wrl.gz/.step.gz to save space.
> 
> In 2017, a few gig of harddisk space, and a < 1 gig download should be OK for
> everyone. All the other PCB tools, FPGA toolchains, Netflix, etc. are far bigger.
> 
> The other suggestions were quite surprising to me -- Github API, downloading
> individual subdirectories as .zip, abusing subversion (gasp), gitslave (last
> updated 2012), etc. would be a big hurdle and WTF for new KiCAD users.
> 
> So why not use git as it was supposed to be used?
> 
> best regards,
> thomas
> 
> 
> On 22.09.2017 03:21, Oliver Walters wrote:
>> Hi all,
>>
>> Ok, now that the website integration with the libraries is (pretty much) done,
>> and the licensing issue seems to be sorted, there is one final puzzle piece to
>> solve before I'm happy with the state of the libraries for a v5 release.
>>
>> *Goal: *Merge all footprint library repositories into a single repo to solve the
>> ongoing dramas of maintaining 100+ repos.
>>
>> *Problem*: The *only* thing standing in the way of just doing this is that some
>> users like the GitHub plugin and previous instruction is that this functionality
>> cannot be removed.
>>
>> The GitHub plugin functions by downloading a .zip file of a .pretty repo. If we
>> merge all footprint libs into a single repo with multiple subdirectories, this
>> will not work anymore (as GitHub dosen't allow you to download a .zip of a
>> single subdirectory).
>>
>> Merging the repos is the *right thing to do*. But how to proceed?
>>
>> *Options:*
>> *
>> *
>> /a) Drop github plugin feature, replace with library-download tool/
>>
>> I don't think it is a good idea to live-load library data from GitHub (a lot of
>> other users agree too). It's slow, and a waste of bandwidth to re-download the
>> libs all the time. 
>>
>> We drop support for loading libraries direct from GitHub. However, we add a tool
>> for downloading libraries from GitHub and storing to disk. Users can update as
>> they like. This can be integrated in KiCad and new users can run this tool when
>> they first install KICad. This means that no libs need to distributed with the
>> installer and users can update to latest libs whenever they want.
>>
>> /b) Improve github plugin to allow subdirectory traversal/
>>
>> This is difficult and will only result in the plugin being slower. There are two
>> ways I can see to do this:
>>
>> i. Use Git API - tools exist that use this functionality
>> - https://github.com/KinoLien/gitzip
>> ii. Use subversion - GitHub actually provides subversion API
>> - https://www.seanw.org/blog/download-git-repo-subdirectory/
>>
>>
>> *Subversion*
>>
>> In either case, I think that using the subversion tool to partially download the
>> libraries would be a good approach (I assume quicker than using wget and the
>> GitHub API).
>>
>> 1. Does anyone have any experience using the C API for SVN? 
>> 2. The Python bindings are pretty good - https://pypi.python.org/pypi/svn - and
>> much easier to use. However, can we make the library download tools dependent on
>> enabling the python plugin?
>> 3. Is there a way to checkout a subversion remote to memory (to replicate the
>> functionality of the current GitHub plugin)? If not, I'm not sure how to
>> approach option b) above.
>>
>>
>> Feedback appreciated. I think that it is very important especially for new users
>> that this is improved. The GitHub plugin is constantly causing headaches!
>>
>> Cheers,
>> Oliver
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>>
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>