kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #31016
Re: GitHub Plugin (my nemesis)
Thanks for your detailed description. I think it is a nice way to go. However two remarks:
Does it simplify the maintenance of the kicad library by the librarians? Is it github compatible? If not, we would need to find another platform to host the libraries.
If I look for git submodule, there is a lot of different opinions on why you should or should not use it. (e.g. https://www.google.de/amp/s/codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/amp/). Does it apply in our case? What about git subtree?
Best regards
Simon
Am 28. September 2017 02:37:09 MESZ schrieb David Godfrey <info@xxxxxxxxxxx>:
>Hi All
>
>I have to agree with some of the other posters, why not use git as it
>was intended?
>
>Specifically I think the following makes sense.....
>
>- Keep the multiple repositories, this helps when a company only wants
>to make certain sets of libs available to it's staff
>
>- Have a master repository that includes all other repositories as
>submodules.
>
>- Have branches that are matched to kicad versions. This allows
>footprint changes in a version safe way.
>
>- Use standard "git clone" (initial download) and "git pull" (update)
>on
>the main repo which provides the entire set of available libs without
>actually downloading the content for all libs.
>
>- When a specific lib is needed, do similar to what we do now, and use
>"git submodule update --init --recursive $submoduleName" to just pull
>that specific submodule
>
>- Allow the "main" library repo URI to be altered. This enables a
>companies fork of the repo to be used.
>
>- Allow the "main" library repo URI to be ANY valid git URI. That means
>
>a repo on a local fileserver rather than a http server can be used.
>Along with various security options etc.
>
>- Add a fairly simple scripted tool that's run "on release" to retrieve
>
>a README.md (an possibly a descriptor/index file) from each actual
>library repo and update those within the "main" repo. This removes the
>need to have the current case where manual edits are required to keep
>the "main" repo in sync with what's available in the individual lib's.
>
>To add a new lib, it's then as simple as (from the main repo) doing
>something like....
>- "git submodule add $URI"
>- "./scripts/update-library-indexes.sh"
>- "git commit -a -m $'added new module "modulename"\nupdated all module
>
>descriptions and index'"
>- "git push"
>
>
>All of the above allows the Main repo to be forked by a company (or
>individual) and have their own custom repo's added very easily.
>It even allows libraries to be easily excluded or replaced, all using
>extremely well developed management tools.
>
>On a side note, git submodules are stored in the main repository
>basically as URI that includes a commit reference.
>That means it's easy to specify a specific library version to include
>in
>a specific branch/tag of the main repo.
>
>
>As for the person that said "git isn't available for windows", sorry,
>that's bunkum.
>There are many many development environments out there that ONLY run on
>
>windows that have git integration.
>A quick google search for "git for windows" will show what you need to
>know there.
>
>Finally, wrapping git to duplicate the effective API that's currently
>in
>use should be relatively trivial, resulting in virtually no code
>changes
>required to KICAD it's self.
>Assuming the existing functionality is cleanly wrapped by an API, the
>only real change to KICAD would be the swapout of the module, and
>addition of an easy way to change the URI
>
>
>I know I haven't covered everything in this email, but it should be a
>good outline for further discussion
>
>
>On 22/09/17 09:21, Oliver Walters wrote:
>> Hi all,
>>
>> Ok, now that the website integration with the libraries is (pretty
>> much) done, and the licensing issue seems to be sorted, there is one
>> final puzzle piece to solve before I'm happy with the state of the
>> libraries for a v5 release.
>>
>> *Goal: *Merge all footprint library repositories into a single repo
>to
>> solve the ongoing dramas of maintaining 100+ repos.
>>
>> *Problem*: The *only* thing standing in the way of just doing this is
>
>> that some users like the GitHub plugin and previous instruction is
>> that this functionality cannot be removed.
>>
>> The GitHub plugin functions by downloading a .zip file of a .pretty
>> repo. If we merge all footprint libs into a single repo with multiple
>
>> subdirectories, this will not work anymore (as GitHub dosen't allow
>> you to download a .zip of a single subdirectory).
>>
>> Merging the repos is the *right thing to do*. But how to proceed?
>>
>> *Options:*
>> *
>> *
>> /a) Drop github plugin feature, replace with library-download tool/
>>
>> I don't think it is a good idea to live-load library data from GitHub
>
>> (a lot of other users agree too). It's slow, and a waste of bandwidth
>
>> to re-download the libs all the time.
>>
>> We drop support for loading libraries direct from GitHub. However, we
>
>> add a tool for downloading libraries from GitHub and storing to disk.
>
>> Users can update as they like. This can be integrated in KiCad and
>new
>> users can run this tool when they first install KICad. This means
>that
>> no libs need to distributed with the installer and users can update
>to
>> latest libs whenever they want.
>>
>> /b) Improve github plugin to allow subdirectory traversal/
>>
>> This is difficult and will only result in the plugin being slower.
>> There are two ways I can see to do this:
>>
>> i. Use Git API - tools exist that use this functionality -
>> https://github.com/KinoLien/gitzip
>> ii. Use subversion - GitHub actually provides subversion API -
>> https://www.seanw.org/blog/download-git-repo-subdirectory/
>>
>>
>> *Subversion*
>>
>> In either case, I think that using the subversion tool to partially
>> download the libraries would be a good approach (I assume quicker
>than
>> using wget and the GitHub API).
>>
>> 1. Does anyone have any experience using the C API for SVN?
>> 2. The Python bindings are pretty good -
>> https://pypi.python.org/pypi/svn - and much easier to use. However,
>> can we make the library download tools dependent on enabling the
>> python plugin?
>> 3. Is there a way to checkout a subversion remote to memory (to
>> replicate the functionality of the current GitHub plugin)? If not,
>I'm
>> not sure how to approach option b) above.
>>
>>
>> Feedback appreciated. I think that it is very important especially
>for
>> new users that this is improved. The GitHub plugin is constantly
>> causing headaches!
>>
>> Cheers,
>> Oliver
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help : https://help.launchpad.net/ListHelp
>Regards
>David Godfrey
>SB Tech Services
>mb: +61 437 286 200 <tel:+61437286200>
>
>_______________________________________________
>Mailing list: https://launchpad.net/~kicad-developers
>Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
>Unsubscribe : https://launchpad.net/~kicad-developers
>More help : https://help.launchpad.net/ListHelp
Follow ups
References