kicad-developers team mailing list archive

[Miron Iancu <miancu@xxxxxxxx>]

Hello.

Maybe you might be interested in the opinion of a newbie user and contributor.

My experience:

- worked mostly with Altium and Eagle for my hobby projects; 
- I earn my living as a project manager for a large multinational business software vendor, with +20 years PM experience.

I have to agree with Oliver and Bernardo. There are several aspects that I would like to kindly address, to your attention

1. When we design tools intended for maintenance of any library one should make these tools easily understandable not by experience programmers but by hobbyists and electronics enthusiasts. Or, better said, by the user base that one intends to attract as contributors and/or users. Many of us have very low programming skills or experience in collaborative platforms like git, subversion, whatever. Ideally, the versioning and control of the sources should be opaque (abstracted) for the average user. Notions like forking, pulling, pushing, committing etc are, maybe, second nature for you, but require a certain learning curve for someone like me that is mainly interested in 1/ using the software and 2/ quickly adding up to a library a missing component or footprint and using it immediately in a project. Or to have different versions in an online repository with clear descriptions about deltas between versions. If this objective is achieved, one will also increase KiCad's contributor base because more people that are usually put back by complicated use cases will be enrolled as contributors. Highlight: abstract the git use case; as contributor and user I should not be concerned where is located and how should I access a collection/ library in order to do my job (use it or maintain it).

2. Even if it is relatively easy to add a new component schematic to a component library, the overall workflow from creating the shape in library editor to adding attributes, footprints etc is heterogenous. I found it challenging to explore the footprint libraries in order to add a FPFilter or to see which are the available options based on the requirements in the component Datasheet. I struggled with easily understanding and finding available footprints in order to add a FP filter and I had to go back and forth between applications. Maybe it is an easier way that I don’t know in order to achieve this, but if so it is not intuitive. The way I see the component design workflow is to have the footprint viewer handy (as link, button etc) inside the library editor or from the component attributes window, from the component properties -> edit footprint filter; in this way one can immediately add a filter by looking into the fp browser/ navigator. 

3. When a user is working in the library editor, i.e. woring in, let’s say, „contributor mode“, KLC-base rules should be, by default, enforced at application level. One concern less, that saves time. The default values are different than the ones required by KLC to be compliant with and I had to refactor several components because I forgot about this. Moreover, even so, if I save the component editor options these should remain as saved. At least on mac, they revert to some defaults at each application start. 

4. All these modal windows should have contextual help; there should be some entries to explain the structure of a FP filter, or what an alias is etc. I had to open several components and look for examples or look into documentation which is time consuming.

5. For maintenance, it should be easier to „separate the concerns“ and have one repo for the component schematic, one repo for footprints (all) and one repo for 3D models. By „one repo“ I want to say one single clonable git address. I noticed that, for each .pretty entry there is a separate git repo that I have to clone locally and maintain separately. Really ? Why ? Somebody suggested to use another, different tool from Google to manage these sources. An external tool will really complicate things and will - sooner or later - chase away all newbies and porgramming-agnostic people that want to contribute. 

6. Who is doing what ? There is a risk of overlapping tasks on same objectives, which seem to be taken lightly :) but can be quite frustrating for somebody that spent some considerable amount of time creating a library just to find out that somebody else’s pull request, for the same, was taken first. ;) There should be some kind of a resource assignment to tasks or even larger deliverables. That would also help planning the releases.

Again, all these are from a fresh perspective: from somebody that is new in this. An outsider. 

Thank you all for reading and for all your wonderful achievements !

Regards,

Miron






> Pe 17 feb. 2017, la 09:15, Nick Østergaard <oe.nick@xxxxxxxxx> a scris:
> 
> 2017-02-17 1:18 GMT+01:00 Cirilo Bernardo <cirilo.bernardo@xxxxxxxxx>:
>> Hi Oliver,
>> 
>> I agree with pretty much everything, especially removing the 3D models
>> from the tree. I have argued for that a number of times in the past but I
>> think as library dictator you may simply have to enforce it at some stage
>> even if angry hordes complain later.
>> 
>> On Fri, Feb 17, 2017 at 10:19 AM, Oliver Walters
>> <oliver.henry.walters@xxxxxxxxx> wrote:
>>> Hi everyone,
>>> 
>>> I am looking for some input on some thoughts I have regarding the future of
>>> the Kicad "standard libraries" on GitHub. I think there are some serious
>>> maintainability issues with the way the libraries are currently handled, and
>>> this will only get worse once the new eeschema / .sweet changes are rolled
>>> in.
>>> 
>>> 1. Number of repositories
>>> 
>>> There are currently 96 .pretty repos at https://github.com/KiCad . This is a
>>> huge number of repos to maintain especially considering that for each pull
>>> request, the user has to fork / branch / PR yet another repository (and then
>>> the librarians have to do the same to check it ). And then, you must pull
>>> the latest changes from (up to) 96 repos to keep all the libs up to date.
>>> 
>>> There are currently 90 .lib files (symbol libraries) at
>>> https://github.com/KiCad/kicad-library . If the .sweet format follows the
>>> .pretty format we will be asking users (at least the ones to want to make
>>> use of the GitHub library feature) to manage over 180 separate GitHub
>>> repositories.
>>> 
>>> That's a whole lotta repos.
>>> 
>> 
>> I would suggest putting a commit freeze on all current repos and merging
>> them into one footprint repo. Contributors can pull the new repo and
>> continue their work from there with little hassle. Individual (frozen) repos
>> should remain until we're happy that everything is working and any helper
>> scripts have been updated to work with the new structure.
>> 
>> Changes to the schematic repos should be made when we get to the point
>> of converting existing symbols - at that point a new schematic repo should
>> be created and the 3D models moved.
>> 
>> Some thought needs to go into how we associate data between the 3
>> disjoint repos (symbols, footprints, 3D). One simple solution is to require
>> the repos to be cloned to specific directories at the same directory level,
>> for example:
>> 
>> kicad_libs
>> +--symbols
>> +--footprints
>> +--3D
>> 
>> With such an arrangement we can always define a ${KICAD_DATA_ROOT}
>> and make modifications to the filename search algorithms; stock symbols
>> and footprints can then reference a 3D model via ${KICAD_DATA_ROOT}/3D.
>> 
>> Some thought needs to go into the github plugin and how it can continue
>> to be useful to people who do use it. I don't believe there will be any
>> problems encountered in merging the footprint libs which would not be
>> encountered with the libs separated as they currently are. Pegging specific
>> files to specific commits will always be a nightmare to manage and simply
>> isn't a problem for the kicad team to solve. One thing which may displease
>> users is that they will not see a "Group_A" vs "Group_B" of footprints -
>> it will appear that all footprints are stuffed into one single group, but this
>> sort of thing can all be managed.
>> 
>>> 2. Library Management Tools
>>> 
>>> Currently we are investigating using some CI tools to help manage pull
>>> requests. If we go ahead with this we need to roll out these changes to 96
>>> repositories. And then maintain those CI tools.
>>> 
> 
> I don't see how having many repos dictate that we need so many "CI
> tools" individually. We could potentially have the CI to generate
> previews or somthing similar for each pull request.
> 
>> 
>> Yup, that's just crazy. Aim for the 3, but put on your flameproof suit.
>> 
>>> 3. GitHub "plugin" drawbacks
>>> 
>>> AFAIK (from discussing this issue on IRC) the Kicad git plugin does not
>>> actually make use of git, but rather makes use of the download-as-zip
>>> functionality that github provides. This, to me, presents the following
>>> issues:
>>> 
>>> i) GitHub may break this feature at any time - even by changing the format
>>> of the download URL!
>>> 
>>> ii) Incompatible with non-github repositories - No facility to link in with
>>> company repositories, for e.g., that are not served on GitHub
>>> 
>>> iii) Excessive bandwidth - Each time a .pretty lib is loaded, it downloads a
>>> zip of the entire library? If we were making use of git functionality it
>>> would be simple to just update the library (and store a local cache) and
>>> thus only download the diff rather than re-download the entire thing.
> 
> IIRC there is a session cache, so it will only re-download it when you
> restart kicad or pcbnew.
> 
>>> 
>>> iv) Increased github load - As a follow-on from c) if Kicad attracts
>>> significant user base then constant download of github libs will impact
>>> significantly on github bandwidth. Other repositories have been shut or
>>> throttled for high bandwidth use.
>>> 
>> 
>> Definitely a problem. Personally I'd prefer a custom tool running over git
>> to help users manage selection of footprints and their revisions etc. - the
>> question is who has time to do that. Personally I'd never used the github
>> plugin as it just doesn't sit well with the way I do things. Other people
>> love it though. Some serious thought needs to go into the concept of
>> remote access of footprints etc. Actually using git rather than pulling
>> zip files with no history is much more inviting - we can then use commit
>> hashes of the files to help manage revision control of those files within
>> a project. (Do I update this footprint, can I roll it back, is there a new
>> revision available, etc.)
>> 
>>> 4. User Ease-Of-Use
>>> 
>>> Many users (newbies, especially) are confused by the library management.
>>> Other EDA packages provide links to packaged library downloads, whereas the
>>> initial library setup for Kicad is a lot more obtuse.
>>> 
>>> Suggestions:
>>> 
>>> I suggest that this problem be tackled as follows:
>>> 
>>> A) Combine .pretty libs. The entire size of the pretty libraries is
>>> currently ~50MB which is not huge. These should be combined into a single
>>> repository on github which would make library management infinitely easier.
>>> 
>> 
>> Definitely.
>> 
>>> B) Single repo for .sweet libs (future) - As per A)
>>> 
>> 
>> Definitely - as I said above, this should be done when we've converted and
>> tested the .lib files to the new format.
> 
> I don't think it is best to combine the pretty libs. I think it is
> nice to store the data that way. We could start using a tool like
> "repo" (from google, used to manage android sources) which is a tool
> to manage many git repos. Then all that is needed when a new lib is
> added is to add one line in a manifest file, and it is possible to
> point to speficic versions in each repo making it easy to fetch a
> specific release.
> 
>> 
>>> C) Single repo for .3dshapes libs - Currently there is no real reason that
>>> the 3D shapes have to be with the symbols. In fact, they are really bloating
>>> the symbols library!
>>> 
>> 
>> Definitely - I've always argued that they have no business being in there and
>> simply waste bandwidth for people who have no interest whatsoever in VRML
>> models. The ability to use STEP models in the dev branch will make matters
>> much worse.
>> 
>>> D) Versioned downloads on the website - The website should offer a libraries
>>> page under ./Downloads which has versioned releases of the libraries. Users
>>> can choose to download the entire set of footprints/symbols/3dmodels, or
>>> download subdirectories individually. This presents an easy introduction to
>>> the libraries and also means that users can be assured that unless they
>>> specifically download a newer version, their libraries are not going to
>>> change underneath them.
>>> 
>> 
>> I prefer to tag a git repository and provide instructions for retrieval. This is
>> easily done with a single repository for each of symbols, footprints, and 3D;
>> it is an absolute nuisance with the current huge number of footprint repos
>> (that tagging script had better work just right ...). Github in particular has a
>> "Releases" scheme in its API. Maybe that could be of some help.
>> 
>>> E) As an extension of D) users who wish to contribute or be at the
>>> bleeding-edge of the libraries can fork the repos (3, not 186) and keep
>>> these up to date very easily. The 3D model lib will be large but a one-time
>>> download is fine as git will keep the changes up to date with very little
>>> bandwidth.
>>> 
>>> F) Improve the github plugin - This should be turned into a proper GIT
>>> plugin (not a github plugin). This will have the following advantages
>>> i) Not reliant on github (users can manage their own git repos for e.g.)
>>> ii) Potential to partially check out repos (e.g. only pull a single
>>> subdirectory)
>>> 
>>> OR
>>> 
>>> G) Remove the github plugin - Library versioning might be better performed
>>> by the user external to Kicad. I do not rely on Kicad to manage the
>>> libraries for me, I do that in git externally. I know that many other users
>>> do this too. From an architectural point of view, I do not think that Kicad
>>> should "force" the user to manage their libraries in this way. Should we add
>>> hard-coded support for other online services too? Personally I think the
>>> github plugin makes life harder than any problems it solves.
>>> 
> 
> We are not really forcing users to manage their libraries in this way.
> It is just the default.
> 
> 
>> 
>> I'm all for (G), but see my notes above. I don't find the github plugin useful
>> for me, but we have to be mindful of people who do use it.
>> 
>> - Cirilo
>> 
>>> TL;DR - If / when the .sweet format change goes ahead, if we add another
>>> ~100 github repositories, the librarians are going to go postal.
>>> 
>>> Cheers,
>>> Oliver
>>> 
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp