kicad-developers team mailing list archive

[Simon Küppers <simon.kueppers@xxxxxx>]

Thanks for your detailed description. I think it is a nice way to go. However two remarks:
Does it simplify the maintenance of the kicad library by the librarians? Is it github compatible? If not, we would need to find another platform to host the libraries. 

If I look for git submodule, there is a lot of different opinions on why you should or should not use it. (e.g. https://www.google.de/amp/s/codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/amp/). Does it apply in our case? What about git subtree? 

Best regards 
Simon 

Am 28. September 2017 02:37:09 MESZ schrieb David Godfrey <info@xxxxxxxxxxx>:
>Hi All
>
>I have to agree with some of the other posters, why not use git as it 
>was intended?
>
>Specifically I think the following makes sense.....
>
>- Keep the multiple repositories, this helps when a company only wants 
>to make certain sets of libs available to it's staff
>
>- Have a master repository that includes all other repositories as 
>submodules.
>
>- Have branches that are matched to kicad versions. This allows 
>footprint changes in a version safe way.
>
>- Use standard "git clone" (initial download) and "git pull" (update)
>on 
>the main repo which provides the entire set of available libs without 
>actually downloading the content for all libs.
>
>- When a specific lib is needed, do similar to what we do now, and use 
>"git submodule update --init --recursive $submoduleName" to just pull 
>that specific submodule
>
>- Allow the "main" library repo URI to be altered. This enables a 
>companies fork of the repo to be used.
>
>- Allow the "main" library repo URI to be ANY valid git URI. That means
>
>a repo on a local fileserver rather than a http server can be used. 
>Along with various security options etc.
>
>- Add a fairly simple scripted tool that's run "on release" to retrieve
>
>a README.md (an possibly a descriptor/index file) from each actual 
>library repo and update those within the "main" repo. This removes the 
>need to have the current case where manual edits are required to keep 
>the "main" repo in sync with what's available in the individual lib's.
>
>To add a new lib, it's then as simple as (from the main repo) doing 
>something like....
>- "git submodule add $URI"
>- "./scripts/update-library-indexes.sh"
>- "git commit -a -m $'added new module "modulename"\nupdated all module
>
>descriptions and index'"
>- "git push"
>
>
>All of the above allows the Main repo to be forked by a company (or 
>individual) and have their own custom repo's added very easily.
>It even allows libraries to be easily excluded or replaced, all using 
>extremely well developed management tools.
>
>On a side note, git submodules are stored in the main repository 
>basically as URI that includes a commit reference.
>That means it's easy to specify a specific library version to include
>in 
>a specific branch/tag of the main repo.
>
>
>As for the person that said "git isn't available for windows", sorry, 
>that's bunkum.
>There are many many development environments out there that ONLY run on
>
>windows that have git integration.
>A quick google search for "git for windows" will show what you need to 
>know there.
>
>Finally, wrapping git to duplicate the effective API that's currently
>in 
>use should be relatively trivial, resulting in virtually no code
>changes 
>required to KICAD it's self.
>Assuming the existing functionality is cleanly wrapped by an API, the 
>only real change to KICAD would be the swapout of the module, and 
>addition of an easy way to change the URI
>
>
>I know I haven't covered everything in this email, but it should be a 
>good outline for further discussion
>
>
>On 22/09/17 09:21, Oliver Walters wrote:
>> Hi all,
>>
>> Ok, now that the website integration with the libraries is (pretty 
>> much) done, and the licensing issue seems to be sorted, there is one 
>> final puzzle piece to solve before I'm happy with the state of the 
>> libraries for a v5 release.
>>
>> *Goal: *Merge all footprint library repositories into a single repo
>to 
>> solve the ongoing dramas of maintaining 100+ repos.
>>
>> *Problem*: The *only* thing standing in the way of just doing this is
>
>> that some users like the GitHub plugin and previous instruction is 
>> that this functionality cannot be removed.
>>
>> The GitHub plugin functions by downloading a .zip file of a .pretty 
>> repo. If we merge all footprint libs into a single repo with multiple
>
>> subdirectories, this will not work anymore (as GitHub dosen't allow 
>> you to download a .zip of a single subdirectory).
>>
>> Merging the repos is the *right thing to do*. But how to proceed?
>>
>> *Options:*
>> *
>> *
>> /a) Drop github plugin feature, replace with library-download tool/
>>
>> I don't think it is a good idea to live-load library data from GitHub
>
>> (a lot of other users agree too). It's slow, and a waste of bandwidth
>
>> to re-download the libs all the time.
>>
>> We drop support for loading libraries direct from GitHub. However, we
>
>> add a tool for downloading libraries from GitHub and storing to disk.
>
>> Users can update as they like. This can be integrated in KiCad and
>new 
>> users can run this tool when they first install KICad. This means
>that 
>> no libs need to distributed with the installer and users can update
>to 
>> latest libs whenever they want.
>>
>> /b) Improve github plugin to allow subdirectory traversal/
>>
>> This is difficult and will only result in the plugin being slower. 
>> There are two ways I can see to do this:
>>
>> i. Use Git API - tools exist that use this functionality - 
>> https://github.com/KinoLien/gitzip
>> ii. Use subversion - GitHub actually provides subversion API - 
>> https://www.seanw.org/blog/download-git-repo-subdirectory/
>>
>>
>> *Subversion*
>>
>> In either case, I think that using the subversion tool to partially 
>> download the libraries would be a good approach (I assume quicker
>than 
>> using wget and the GitHub API).
>>
>> 1. Does anyone have any experience using the C API for SVN?
>> 2. The Python bindings are pretty good - 
>> https://pypi.python.org/pypi/svn - and much easier to use. However, 
>> can we make the library download tools dependent on enabling the 
>> python plugin?
>> 3. Is there a way to checkout a subversion remote to memory (to 
>> replicate the functionality of the current GitHub plugin)? If not,
>I'm 
>> not sure how to approach option b) above.
>>
>>
>> Feedback appreciated. I think that it is very important especially
>for 
>> new users that this is improved. The GitHub plugin is constantly 
>> causing headaches!
>>
>> Cheers,
>> Oliver
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>Regards
>David Godfrey
>SB Tech Services
>mb: +61 437 286 200 <tel:+61437286200>
>
>_______________________________________________
>Mailing list: https://launchpad.net/~kicad-developers
>Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>Unsubscribe : https://launchpad.net/~kicad-developers
>More help   : https://help.launchpad.net/ListHelp