kicad-developers team mailing list archive

[Wayne Stambaugh <stambaughw@xxxxxxxxx>]

On 2/26/2017 4:04 PM, Cirilo Bernardo wrote:
> There is one other way which I found after much digging
> and it involves a GCC extension. Since we use GCC on
> Windows this might be acceptable:
> 
> a. create a derived class of std::ifstream/ofstream. On
> Windows the derived class will be used while on other
> OS it will simply be typedef to std::ifstream/ofstream

This seems reasonable to me.

> 
> b. overload open() to use the gcc extension like this:
> __gnu_cxx::stdio_filebuf buf( _wopen ( utf8_filename, _O_RDONLY ) );
> std::istream mystream ( &buf );

If this is portable, than I'm file with this as well but on the surface
it looks gcc specific.  If that is the case, then I would rather got
with option a.

> 
> The destructor must contain code to delete buf since
> the istream will not delete it on destruction.
> 
> If that would be acceptable I'll make some test
> programs and work on a patch set. This solution
> would also work on OCE and I can talk to the OCE
> patch team to see what they think of it.
> 
> - Cirilo
> 
> 
> On Mon, Feb 27, 2017 at 4:54 AM, Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>> On 2/26/2017 1:50 AM, Cirilo Bernardo wrote:
>>> Hi folks,
>>>
>>>  This whole thing with UTF-8 filenames in Windows is a disaster.
>>> What I've found so far:
>>>
>>> 1. Regarding OCE: Since OCE 0.17 (OpenCascade 6.8) UTF8
>>> filenames have been supported when built with MSVC but
>>> obviously not with MinGW.
>>>
>>> 2. MinGW does not provide any means of transparently using
>>> UTF-8 filenames. All filenames within the STL *must* be
>>> char* and MinGW *will* simply pass these on to OpenFileA()
>>> on Windows resulting in UTF-8 being interpreted as ASCII-8
>>> (and who uses ASCII-8 filenames anyway).
>>>
>>> So everything hinges on (2). If OCE uses std::stream then
>>> fixing all issues under Windows is a lost cause. If OCE
>>> simply plays with FILE* then it can be patched to work
>>> in MinGW by invoking _wfopen() rather than fopen().
>>>
>>> As for kicad itself, std::stream is used in:
>>> (a) VRML export
>>> (b) IDF static library
>>> (c) Scenegraph dynamic library for 3D plugins
>>>
>>> 2 paths forward come to mind and both will involve some work:
>>>
>>> (1) Move to the MSVC build system on Windows: this makes
>>> it possible for us to use Microsoft extensions to STL to deal
>>> with non-ASCII filename issues. There is no need to dig into
>>> the OCE code since we know it will work correctly when built
>>> with MSVC.
>>
>> This is not an acceptable solution.  It's not portable and would limit
>> windows builds to using msvc.
>>
>>>
>>> (2) Rework kicad code to play with FILE* (or wxFileStream)
>>> rather than std::ifstream/ofstream. Although this will fix the
>>> issues which are confined to kicad's source, it does nothing
>>> to address the OCE issue. Whether or not OCE in MinGW
>>> is a lost cause remains to be seen.
>>
>> FILE* is how we pretty much do it everywhere else in KiCad with good
>> results so I don't see any reason not to do it this way with the model
>> parser code.  At least it's portable across all build platforms.
>> Doesn't oce have a reader function that takes a FILE *?
>>
>>>
>>> One other possibility (but one which I hadn't looked into)
>>> is to see if the STL implementation within MinGW uses the
>>> MinGW-CRT. If it does then it may be possible to fix
>>> everything by ensuring that the MinGW-CRT converts all
>>> filenames to UTF16 and opens a file using FileOpenW().
>>> In all cases this is not a pleasant task.
>>>
>>> Any comments/suggestions?
>>>
>>> - Cirilo
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp