kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #31012
Re: GitHub Plugin (my nemesis)
Hi All
I have to agree with some of the other posters, why not use git as it
was intended?
Specifically I think the following makes sense.....
- Keep the multiple repositories, this helps when a company only wants
to make certain sets of libs available to it's staff
- Have a master repository that includes all other repositories as
submodules.
- Have branches that are matched to kicad versions. This allows
footprint changes in a version safe way.
- Use standard "git clone" (initial download) and "git pull" (update) on
the main repo which provides the entire set of available libs without
actually downloading the content for all libs.
- When a specific lib is needed, do similar to what we do now, and use
"git submodule update --init --recursive $submoduleName" to just pull
that specific submodule
- Allow the "main" library repo URI to be altered. This enables a
companies fork of the repo to be used.
- Allow the "main" library repo URI to be ANY valid git URI. That means
a repo on a local fileserver rather than a http server can be used.
Along with various security options etc.
- Add a fairly simple scripted tool that's run "on release" to retrieve
a README.md (an possibly a descriptor/index file) from each actual
library repo and update those within the "main" repo. This removes the
need to have the current case where manual edits are required to keep
the "main" repo in sync with what's available in the individual lib's.
To add a new lib, it's then as simple as (from the main repo) doing
something like....
- "git submodule add $URI"
- "./scripts/update-library-indexes.sh"
- "git commit -a -m $'added new module "modulename"\nupdated all module
descriptions and index'"
- "git push"
All of the above allows the Main repo to be forked by a company (or
individual) and have their own custom repo's added very easily.
It even allows libraries to be easily excluded or replaced, all using
extremely well developed management tools.
On a side note, git submodules are stored in the main repository
basically as URI that includes a commit reference.
That means it's easy to specify a specific library version to include in
a specific branch/tag of the main repo.
As for the person that said "git isn't available for windows", sorry,
that's bunkum.
There are many many development environments out there that ONLY run on
windows that have git integration.
A quick google search for "git for windows" will show what you need to
know there.
Finally, wrapping git to duplicate the effective API that's currently in
use should be relatively trivial, resulting in virtually no code changes
required to KICAD it's self.
Assuming the existing functionality is cleanly wrapped by an API, the
only real change to KICAD would be the swapout of the module, and
addition of an easy way to change the URI
I know I haven't covered everything in this email, but it should be a
good outline for further discussion
On 22/09/17 09:21, Oliver Walters wrote:
Hi all,
Ok, now that the website integration with the libraries is (pretty
much) done, and the licensing issue seems to be sorted, there is one
final puzzle piece to solve before I'm happy with the state of the
libraries for a v5 release.
*Goal: *Merge all footprint library repositories into a single repo to
solve the ongoing dramas of maintaining 100+ repos.
*Problem*: The *only* thing standing in the way of just doing this is
that some users like the GitHub plugin and previous instruction is
that this functionality cannot be removed.
The GitHub plugin functions by downloading a .zip file of a .pretty
repo. If we merge all footprint libs into a single repo with multiple
subdirectories, this will not work anymore (as GitHub dosen't allow
you to download a .zip of a single subdirectory).
Merging the repos is the *right thing to do*. But how to proceed?
*Options:*
*
*
/a) Drop github plugin feature, replace with library-download tool/
I don't think it is a good idea to live-load library data from GitHub
(a lot of other users agree too). It's slow, and a waste of bandwidth
to re-download the libs all the time.
We drop support for loading libraries direct from GitHub. However, we
add a tool for downloading libraries from GitHub and storing to disk.
Users can update as they like. This can be integrated in KiCad and new
users can run this tool when they first install KICad. This means that
no libs need to distributed with the installer and users can update to
latest libs whenever they want.
/b) Improve github plugin to allow subdirectory traversal/
This is difficult and will only result in the plugin being slower.
There are two ways I can see to do this:
i. Use Git API - tools exist that use this functionality -
https://github.com/KinoLien/gitzip
ii. Use subversion - GitHub actually provides subversion API -
https://www.seanw.org/blog/download-git-repo-subdirectory/
*Subversion*
In either case, I think that using the subversion tool to partially
download the libraries would be a good approach (I assume quicker than
using wget and the GitHub API).
1. Does anyone have any experience using the C API for SVN?
2. The Python bindings are pretty good -
https://pypi.python.org/pypi/svn - and much easier to use. However,
can we make the library download tools dependent on enabling the
python plugin?
3. Is there a way to checkout a subversion remote to memory (to
replicate the functionality of the current GitHub plugin)? If not, I'm
not sure how to approach option b) above.
Feedback appreciated. I think that it is very important especially for
new users that this is improved. The GitHub plugin is constantly
causing headaches!
Cheers,
Oliver
_______________________________________________
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~kicad-developers
More help : https://help.launchpad.net/ListHelp
Regards
David Godfrey
SB Tech Services
mb: +61 437 286 200 <tel:+61437286200>
Follow ups
References
