kicad-developers team mailing list archive

[Wayne Stambaugh <stambaughw@xxxxxxxxx>]

On 5/3/2019 10:49 AM, Jeff Young wrote:
> @Wayne, are you sure about those settings?  (That’s the flag I proposed
> we flip in the previous incantation of this thread.)  I’m fairly sure
> (although not 100% certain) that the bug only exists in
> wxUSE_UNICODE_UTF8 mode.

If that is the case, could it be that simple to just rebuild wxWidgets
with the appropriate settings?  That would be something wouldn't it?  I
double checked my system, ubuntu on my laptop, both 32and 64 bit builds
on windows and they are all configured the same way.  Keep in mind that
I am just looking at the packaged versions of wxWidgets on these
platforms.  If folks are build their own custom wxWidgets libraries, I
cannot vouch for them.  I wonder if Tom's new crash reporter can grab
the wx/setup.h file so we can have another data point in our bug
reports, particularly for those unexplained random crashes that we seem
to have.

> 
> @Jon & @Tom, I know you guys have also fixed some of these crashes.
>  What platform did you find them on?  The ones I’ve fixed have been
> encountered on OSX (which is the only platform I have).
> 
>> On 3 May 2019, at 15:41, Wayne Stambaugh <stambaughw@xxxxxxxxx
>> <mailto:stambaughw@xxxxxxxxx>> wrote:
>>
>> On 5/3/19 5:22 AM, Jeff Young wrote:
>>> Hi Dick,
>>>
>>>>> h) What is the list of deficiencies with current string usage?
>>>
>>> I only have one issue with the current use of wxString, but it’s a big
>>> one: it crashes (unpredictably) when used multi-threaded in UTF8 mode.
>>
>> I thought it was wxString itself that was not thread safe not
>> necessarily the utf-8 build but thread safety is the primary goal now
>> that we are using threads in multiple places within KiCad.
>>
>> On my Debian system wx/setup.h shows
>>
>> #define wxUSE_UNICODE 1
>>
>> and
>>
>> #define wxUSE_UNICODE_UTF8 0
>>
>> so it would appear that wxString is built for unicode not utf8 mode on
>> linux.  I'm also pretty sure windows builds are unicode as well.
>>
>> There is a secondary goal of removing wxWidgets from our low level
>> objects.  Maybe some day we can build the low level KiCad non-ui
>> libraries sans wxWdigets.  My thinking is that wxString should only come
>> into play at the UI level when dealing with wxWidgets UI code.  Being
>> able to use a standard C++ string implementation would (may?) go a long
>> way in helping with that goal.
>>
>>>
>>> This design document makes for fascinating
>>> reading: https://wiki.wxwidgets.org/Development:_UTF-8_Support.  It
>>> appears that the current wxString is at least in part modelled on
>>> QtString.
>>>
>>> There’s also a bunch of interesting info
>>> here: https://docs.wxwidgets.org/trunk/overview_string.html, which I
>>> believe is more up-to-date than the previous link.  In particular,
>>> there’s the mention that wxString handles extra-BMP characters
>>> transparently when compiled in UTF8 mode (currently used by Kicad), but
>>> does NOT when compiled in default mode (in which case the app must
>>> handle surrogate pairs).  This of course directly leads to your point
>>> (d):
>>>
>>>>>> d) What does the set of characters that don't fall into UCS2
>>>>>> actually look like?  How big
>>>>>> is this set, really?  (UTF16 is bigger than UCS2 and picks up the
>>>>>> difference.)
>>>
>>> Do we really need to handle extra-BMP characters?
>>>
>>> An even more recent version of the second document
>>> (https://docs.wxwidgets.org/trunk/classwx_string.html) finally makes an
>>> oblique reference to the multi-threading issue by starting with this
>>> (rather unhelpful) suggestion:
>>>
>>> Note
>>>    While the use of wxString
>>>    <https://docs.wxwidgets.org/trunk/classwx_string.html> is
>>>    unavoidable in wxWidgets program, you are encouraged to use the
>>>    standard string classes |std::string| or |std::wstring| in your
>>>    applications and convert them to and from wxString
>>>    <https://docs.wxwidgets.org/trunk/classwx_string.html> only when
>>>    interacting with wxWidgets.
>>>
>>>
>>> Cheers,
>>> Jeff.
>>>
>>>
>>>> On 3 May 2019, at 02:03, Dick Hollenbeck <dick@xxxxxxxxxxx
>>>> <mailto:dick@xxxxxxxxxxx>
>>>> <mailto:dick@xxxxxxxxxxx>> wrote:
>>>>
>>>> On 5/2/19 5:32 PM, Dick Hollenbeck wrote:
>>>>> On 4/30/19 4:36 AM, Jeff Young wrote:
>>>>>> We had talked earlier about throwing the wxWidgets UTF8 compile
>>>>>> switch to get rid of our wxString re-entrancy problems.  However, I
>>>>>> noticed that the 6.0 work packages doc includes an item for
>>>>>> std::string-ization of the BOARD.  (While a lot more work, this is a
>>>>>> better solution because it also increases our gui-toolkit-choice
>>>>>> flexibility.)
>>>>>>
>>>>>> I’d like to propose that we use std::wstring for that.  UTF8 should
>>>>>> *only* be an encoding format (similar to s-expr).  It should never
>>>>>> be used internally. That’s what unicode wchar_t’s are for.
>>>>>>
>>>>>> And I’d like to propose that we extend std::wstring-ization to
>>>>>> SCH_ITEM and LIB_ITEM.  (Then we can get rid of a bunch of our ugly
>>>>>> mutex hacks.)
>>>>>
>>>>>
>>>>> I've been looking at this for a few months now.  I think it is so
>>>>> important, that a
>>>>> sub-committee should be formed, and if that committee takes as long
>>>>> as 4 months to come to
>>>>> a recommendation, this would not be too long.  This issue is simply
>>>>> too critical.
>>>>>
>>>>> I would like to volunteer to be on that committee.  For the entire
>>>>> list to participate in
>>>>> this simply does not make sense to me.  I would welcome the
>>>>> opportunity to study this with
>>>>> a team of 5-6 players.  More than that probably leads to anxiety.
>>>>>  Then, given the
>>>>> recommendations, the list would of course have an opportunity to
>>>>> raise questions and take
>>>>> shots, before a strategy is formulated, and before anything is
>>>>> implemented.
>>>>>
>>>>> Again, approximately:
>>>>>
>>>>>  committee recommendations -> list approval -> strategy formulation
>>>>> -> implementation
>>>>>
>>>>>
>>>>> Up to now I have looked at many libraries and have [way *too* much]
>>>>> experience in multiple
>>>>> languages on multiple platforms, so I think I can be valuable
>>>>> contributor.
>>>>>
>>>>> The final work product initially would simply be a list of
>>>>> recommendations, that quickly
>>>>> transforms to a strategy thereafter.  This is an enormous
>>>>> undertaking, so I suggest
>>>>> against racing to a solution.  It could look a lot easier than it
>>>>> will ultimately be, as
>>>>> is typical in software development.  But the return on investment
>>>>> needs to be near optimal
>>>>> in the end.
>>>>>
>>>>> Some questions to answer are:
>>>>>
>>>>> a) How did wxString get to its current state?  Is is merely a
>>>>> conglomeration of after
>>>>> thought, or is is anywhere near optimal.
>>>>>
>>>>> b) Why so many forms of it?  Can one form be chosen for all platforms?
>>>>>
>>>>> c) How does wxString it compare to QtString?
>>>>>
>>>>> d) What does the set of characters that don't fall into UCS2 actually
>>>>> look like?  How big
>>>>> is this set, really?  (UTF16 is bigger than UCS2 and picks up the
>>>>> difference.)
>>>>>
>>>>> e) For data files, I think UTF8 is fine.  So the change is for RAM
>>>>> manipulation of
>>>>> strings.  Aren't we talking about a RAM resident string that bridges
>>>>> into the GUI seamlessly?
>>>>>
>>>>> f) What does new C++ language support offer?
>>>>>
>>>>> g) What do C++ language designers suggest?
>>>>
>>>> h) What is the list of deficiencies with current string usage?
>>>>
>>>>
>>>>>
>>>>>
>>>>> etc.
>>>>>
>>>>> But this is best continued in a smaller group, as said.
>>>>>
>>>>>
>>>>> The other thing that I bring to this is vast familiarity with KiCad's
>>>>> internal workings,
>>>>> string use cases, and goals.
>>>>>
>>>>> Let me know if I can help.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Dick
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~kicad-developers
>>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~kicad-developers
>>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>