kicad-developers team mailing list archive

[Oliver Walters <oliver.henry.walters@xxxxxxxxx>]

>
> I noticed that your symbol library table uses absolute paths rather than
> using the SYMBOL_LIB_DIR path expansion.  This should not make any
> difference as long as the path you used is the same path as old library
> list look up path.  If it's not, that is your problem.  The remapping
> will not look up by symbol name in the symbol library table if the
> symbol is not found in the original library in the old library list that
> it was loaded from.  I will attempt to explain this again so everyone
> who is having this issue can understand what is happening.


The sym-lib-table was created by the project-update tool, and I had not
defined SYMBOL_LIB_DIR. However the generated sym-lib-table did point to
the libraries I was using previously in this project (v4)

SYMBOL_LIB_DIR was defined as "C:\msys64\mingw64\share\kicad\library". I
don't have a build setup on this machine and the signed windows were
unavailable. This is a jenkins nightly.

The process makes sense (I think) but the failures are quite transparent.

Currently the "rescue orphans" procedure in the other dialog works very
well. Perhaps we could:

a) Run this operation after "rescue" if the rescue did not work?
b) Explicitly point users to make use of this dialog.

On Fri, Dec 15, 2017 at 10:42 AM, Wayne Stambaugh <stambaughw@xxxxxxxxx>
wrote:

> I noticed that your symbol library table uses absolute paths rather than
> using the SYMBOL_LIB_DIR path expansion.  This should not make any
> difference as long as the path you used is the same path as old library
> list look up path.  If it's not, that is your problem.  The remapping
> will not look up by symbol name in the symbol library table if the
> symbol is not found in the original library in the old library list that
> it was loaded from.  I will attempt to explain this again so everyone
> who is having this issue can understand what is happening.
>
> Here are the steps that occur for remapping each symbol (not including
> the final rescue since that is just another library in the old project
> library list):
>
> Find the full path and file name of the old library list lookup method
> used obtain the symbol link.  This is found in LIB_PART::m_library
> member.  This will point to the same library as if you were using stable
> version 4.
>
> Look for the library by path and full file name in the global symbol
> library table.  If the library cannot be found in the global symbol
> library table and the file exists, add this library to the project
> symbol library table and give it a unique name if necessary.
>
> Set the symbol library nickname to the entry in the symbol library table.
>
> Once all symbols are remapped, write the project symbol library table
> and refresh if necessary and remove all of the libraries from the old
> project library list.
>
> I suspect that you are running a development version of kicad that
> cannot find the original symbol libraries in the project library list
> because they are not in a path that it expects them to be in.  You can
> fix this by manually editing the project file to include this path so
> the remapping will work correctly.
>
> Once the remapping is complete,
> On 12/14/2017 06:05 PM, Oliver Walters wrote:
> > Wayne,
> >
> > Here is the issue I am facing. Maybe I have misunderstood the process,
> > or maybe I have found a bug.
> >
> > 1. Load simple legacy project and attempt to remap. Failures across the
> > board.
> > Inline image 1
> >
> > 2. Check the sym-lib-table
> >
> > Inline image 2
> >
> >
> > 3. Confirm that the symbols do actually exist in the loaded libs
> >
> >
> > Inline image 3
> >
> > 4. No dice
> >
> > Inline image 4
> >
> > 5. Try again?
> >
> > Inline image 5
> >
> > 6. Weep, etc
> >
> > What am I doing wrong?
> >
> > Oliver
> >
> > On Thu, Dec 14, 2017 at 10:41 AM, Wayne Stambaugh <stambaughw@xxxxxxxxx
> > <mailto:stambaughw@xxxxxxxxx>> wrote:
> >
> >     On 12/13/2017 01:56 AM, Oliver Walters wrote:
> >     > I finally had a chance to try the symbol remapping on a real
> project,
> >     > and I have a few thoughts on how I think it could be improved from
> a
> >     > users perspective (with code to back up my thoughts!)
> >     >
> >     > 1. "Remap" should replace "rescue"
> >     >
> >     > The remap tool, with some work, should replace the "symbol rescue"
> tool.
> >     > There are two major cases which it should cover:
> >     >
> >     > a) Handling "legacy" (v4 and prior) symbols which do not specify a
> >     > library nickname
> >     > b) Handling missing libraries or symbols
> >     >  i) A library has been moved or deleted
> >     > ii) An item has been renamed in a library
> >
> >
> >     These are two distinctly different processes and need to stay
> separate.
> >     Please do not merge them.
> >
> >     >
> >     > The rescue tool does not seem to work at all now, with the new
> >     > sym-lib-table approach. I think I have come up with a way to
> combine the
> >     > two tools that provides a better approach anyhow.
> >
> >     Please explain.  The last testing I did, both the legacy and fully
> >     defined LIB_ID rescue works as required.  Are you saying if you
> delete
> >     or modify a library symbol used in a schematic with a fully defined
> >     LIB_ID that is does not get rescued properly from the cache
> library?  If
> >     so, please file a bug report so I can fix it.
> >
> >     >
> >     > 2. Let users "remap" multiple times.
> >     >
> >     > The current implementation only allows remap if *all* the symbols
> are
> >     > un-mapped. If a partial remap occurs then the users are left high
> and
> >     > dry. Partial remaps should be allowed, this gives the users chance
> to
> >     > fix their library tables or whatever.
> >
> >     I'm not opposed to this but only for symbols that do not already
> have a
> >     library nickname assigned to the schematic symbol LIB_ID.  Once a
> symbol
> >     is remapped and mapping is correct, the only other valid operation
> is to
> >     choose a new symbol.  I would reject any change that fell back to
> >     looking up symbols by name alone (no library).  This was the entire
> >     point of the remapping and would reintroduce the bug that has been
> >     around since the beginning of the project.
> >
> >     >
> >     > 3. Separate project-remap from symbol-remap
> >     >
> >     > The regeneration of library tables should be separate from symbol
> >     > remapping. Rescue the project once, (with a visible warning to
> user at
> >     > first project load, as is currently the case). But, allow symbol
> remap
> >     > from the menu at any stage (even if there are no symbols to remap!)
> >
> >     This will likely not work because more often than not, successful
> >     remapping depends on the project specific symbol libraries not just
> the
> >     global symbol libraries.  This is why the remapping utility first
> >     creates a project specific symbol library table based upon the
> libraries
> >     it needs (contains symbols used in the schematic) from the old
> library
> >     list.  Keep in mind, the remapping does not just look for the first
> >     occurrence of a symbol with a given name in the symbol library table
> >     (that would be broken), it actually matches the full file path and
> file
> >     name from the symbol library table to do the mapping.  There is a bug
> >     where symlinks will get mapped as a different library.  I have a fix
> for
> >     that but I have had time to test it.
> >
> >     >
> >     > 4. Group similar symbols together when remapping
> >     >
> >     > If a schematic has 100+ 'R' symbols, they should be grouped
> together so
> >     > that they can *all* be remapped at once. Collect similar symbols
> >     > together. (See demonstrator image below).
> >
> >     Nice but not crucial to the remapping.  Sorting could be added to the
> >     existing remapping feature as well.
> >
> >     >
> >     > 5. Allow manual remap selection
> >     >
> >     > If auto-remapping fails for a particular symbol, allow the user to
> >     > select a new symbol manually
> >
> >     This isn't really remapping.  This is reassigning a library symbol.
> If
> >     a symbol cannot be located in any library in the old library list,
> there
> >     is no way to recover it.  This was true before the switch to fully
> >     defined LIB_IDs.
> >
> >     >
> >     > 6. Allow skipping of remapped symbols
> >     >
> >     > Some may be not able to be remaped at the current time. Allow user
> to
> >     > skip and come back later.
> >
> >     Do you really want listen to users scream about broken symbol links?
> >     Keep in mind, once the remapping is complete, all of the old
> libraries
> >     from the symbol library table are removed from the project files so
> they
> >     are no longer available for future remapping which will almost
> certainly
> >     lead to issues.  I will reject keeping the old library list in tact
> as
> >     it was part of the original issue.
> >
> >     >
> >     > 7. Provide a dry-run first and let the user see what is happening.
> >
> >     I'm fine with a dry run feature.  I had it on my todo list but if you
> >     get to it before I do.  My feelings wont be hurt.
> >
> >     >
> >     >
> >     > I have actually implemented all of the points above. See
> demonstrator
> >     > images here:
> >     >
> >     > A) Dry run with explicit output messages, and information on what
> will
> >     > be changed
> >     >
> >     > Inline image 1
> >     >
> >     > B) Success!
> >     >
> >     > Inline image 2
> >     >
> >     > If this is a good approach I will spend some more time on this,
> there
> >     > are still some issues to work out. However, It is (to me) a much
> cleaner
> >     > approach.
> >     >
> >     > Thoughts?
> >     >
> >     > Oliver
> >     >
> >     >
> >     > _______________________________________________
> >     > Mailing list: https://launchpad.net/~kicad-developers
> >     <https://launchpad.net/~kicad-developers>
> >     > Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> >     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
> >     > Unsubscribe : https://launchpad.net/~kicad-developers
> >     <https://launchpad.net/~kicad-developers>
> >     > More help   : https://help.launchpad.net/ListHelp
> >     <https://help.launchpad.net/ListHelp>
> >     >
> >
> >     _______________________________________________
> >     Mailing list: https://launchpad.net/~kicad-developers
> >     <https://launchpad.net/~kicad-developers>
> >     Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> >     <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
> >     Unsubscribe : https://launchpad.net/~kicad-developers
> >     <https://launchpad.net/~kicad-developers>
> >     More help   : https://help.launchpad.net/ListHelp
> >     <https://help.launchpad.net/ListHelp>
> >
> >
>