kicad-developers team mailing list archive

[Lorenzo Marcantonio <l.marcantonio@xxxxxxxxxxxx>]

On Wed, Aug 21, 2013 at 01:58:33PM -0400, Wayne Stambaugh wrote:
> I should have qualified that with the utf8 character set because
> internally all of our file parsers work on utf8 encoded files.

There's a *slight* imprecision in that. With UTF8 *encoding* (character
set is Unicode, ISO 10646-1). Given the properties of UTF8 this mean
standard ASCII (i.e. ISO646 7 bit stuff) plus any byte in the 80h-FFh
range. This is guaranteed by UTF8 design (for example you will not
encounter a '/' whatever codepoint you use different from '/').

In practice, if the filesystem handles 8 bit clean, it will handle UTF-8
without problem. BTW Windows works in UCS-2, so there are bigger
restriction on file length when using extended characters...

> While I agree with you on principal, anyone could create a kicad_mod
> file by hand and name it anything the platform accepts.  There is no way
> to enforce that.  As long as the file name can be parsed in from the
> appropriate KiCad file, we should be able to open it.  At least that
> should be goal.

Uhmmm... pondering... let's see if I got the situation...

1) we want to know the footprint name just by the file name (using
   readdir) instead of having to read the file. This seems reasonable
   for performance reasons;

2) the name of the footprint is both the file name and a field inside
   the footprint file itself.

Point 2) has the following issues:

2.1) user renames the file, internal name mismatches with the external
     one

2.2) user uses different case with internal and external name. Works
     on Windows (and AFAIK even HFSplus should handle it), another un*x
     user can't refresh the footprints since it mismatches.

Point 2.1 is clearly a user error, and at least a warning should be
given. It should be decided if reject the file, accept it with the
external name or accept it with the internal name. IMHO priority should
be given to the internal name, since it's the name given by the part
designer, hopefully... a simple library check/fix function could be
implemented to check (and maybe rename) the footprints in a directory,
as an utility.

Point 2.2 means that we should be 'case preserving'; a straight open
wouldn't do, a readdir and a collated name match have to be done to find
the right file. This is not necessarily trivial with UTF8, it depends on
the locale support from the OS. Alternatively there are libraries around
that does that (like the giant ICU, but even smaller things). Code for
this kind of match should be readily available both from Samba and
probably Apache httpd.

That would also mean that file Module and MODULE on a un*x system are
in conflict and should be handled by the same validation utility as
before (it would be same case of file foo containing module foo and file
bar containing module foo!)

Remember however that at the moment the module name in pcbnew is case
sensitive. This could give problems (or maybe not); further study is
required. In particular the presentation of the list extracted from the
directory will not always match in case the actual modules in the file!

Another alternative would be using a cache/index file like fontconfig
does: the file name has no significance in itself and the index would
have to be updated following the timestamp. The conflict possibilies are
exactly the same, I think that the routines for maintaining such kind of
index would be on the same level of complexity of the validation
routine. The index itself don't need to be particularly sophisticated,
something like fonts.dir would suffice (unless you want to scale to
1000s of modules for library, but then probably the problem would be
NTFS...)

In database terms: the primary key has to match with another candidate
key which could vary unexpectedly. That would be blasphemy for a DBMS:D
So we switch the primary key to a synthetic key (the actual, 'random'
filename) and put an index to find the synthetic key from the real
primary key. The filename then doesn't actually matter anymore (by
default it should be the same to easily find the file, however)

-- 
Lorenzo Marcantonio
Logos Srl