← Back to team overview

kicad-developers team mailing list archive

Re: [PATCH] Fuzzable PCB parsing test harness

 

> … Let's just hope the macos build doesn't choke.

He he.  And it did. :)

I think I fixed it but someone might want to check that I didn’t do something stupid.

https://git.launchpad.net/kicad/commit/?id=4061f1cce3b177ceca894fd2f01e0eedd928273b <https://git.launchpad.net/kicad/commit/?id=4061f1cce3b177ceca894fd2f01e0eedd928273b>

Cheers,
Jeff.


> On 22 Oct 2018, at 18:32, Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
> 
> No problem.  I have gone through this exercise a few times myself.
> Let's just hope the macos build doesn't choke.
> 
> On 10/22/2018 1:30 PM, John Beard wrote:
>> Thanks, Wayne! Sorry for all the drama with it. Hopefully I can learn
>> from it for the libcommon tests.
>> 
>> Cheers,
>> 
>> John
>> On Mon, Oct 22, 2018 at 6:14 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>> 
>>> It's merged.  Thanks John.
>>> 
>>> On 10/22/2018 1:09 PM, John Beard wrote:
>>>> Hi Wayne,
>>>> 
>>>> Hurray!
>>>> 
>>>> No extras needed, this should be ready to merge. There are additional
>>>> things we can do to enable testing of PLUGINs as opposed to the PCB
>>>> parser, but this at least provides the base to build on. This commit
>>>> is self-contained and includes the fuzzing documentation.
>>>> 
>>>> Cheers,
>>>> 
>>>> John
>>>> On Mon, Oct 22, 2018 at 5:45 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>>>> 
>>>>> Hey John,
>>>>> 
>>>>> Finally!  That was easy.  I'm guessing this is ready to merge or is
>>>>> there anything else that needs to be done?
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Wayne
>>>>> 
>>>>> On 10/22/2018 12:19 PM, John Beard wrote:
>>>>>> Hi Wayne,
>>>>>> 
>>>>>> Hmm, didn't expect that to be an issue! Try a ToStdString() for size?
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> John
>>>>>> On Mon, Oct 22, 2018 at 2:39 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>>>>>> 
>>>>>>> Hey John,
>>>>>>> 
>>>>>>> You not giving up on this are you ;).  I builds fine for mingw32 and
>>>>>>> passes all of the tests.  However, the mingw64 build fails with:
>>>>>>> 
>>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/pcb_parse_input/main.cpp:
>>>>>>> In function 'int main(int, char**)':
>>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/pcb_parse_input/main.cpp:148:32:
>>>>>>> error: call of overloaded 'open(const wxString&)' is ambiguous
>>>>>>>             fin.open( filename );
>>>>>>>                                ^
>>>>>>> In file included from
>>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/qa_utils/stdstream_line_reader.h:32,
>>>>>>>                 from
>>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/pcb_parse_input/main.cpp:32:
>>>>>>> C:/msys64/mingw64/include/c++/8.2.0/fstream:653:7: note: candidate:
>>>>>>> 'void std::basic_ifstream<_CharT, _Traits>::open(const char*,
>>>>>>> std::ios_base::openmode) [with _CharT = char; _Traits =
>>>>>>> std::char_traits<char>; std::ios_base::openmode = std::_Ios_Openmode]'
>>>>>>>       open(const char* __s, ios_base::openmode __mode = ios_base::in)
>>>>>>>       ^~~~
>>>>>>> C:/msys64/mingw64/include/c++/8.2.0/fstream:673:7: note: candidate:
>>>>>>> 'void std::basic_ifstream<_CharT, _Traits>::open(const wchar_t*,
>>>>>>> std::ios_base::openmode) [with _CharT = char; _Traits =
>>>>>>> std::char_traits<char>; std::ios_base::openmode = std::_Ios_Openmode]'
>>>>>>>       open(const wchar_t* __s, ios_base::openmode __mode = ios_base::in)
>>>>>>>       ^~~~
>>>>>>> 
>>>>>>> On 10/22/2018 4:57 AM, John Beard wrote:
>>>>>>>> Hi Wayne,
>>>>>>>> 
>>>>>>>> Since I'm out of better ideas and can't replicate the link failure on
>>>>>>>> Jenkins, here's a patch that uses the long list from pcb_test_window.
>>>>>>>> Presumably it's there for some reason, though there is no comment or
>>>>>>>> Git log clue that I can see.
>>>>>>>> 
>>>>>>>> Could you see if that works for you?
>>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> 
>>>>>>>> John
>>>>>>>> On Fri, Oct 19, 2018 at 12:09 PM John Beard <john.j.beard@xxxxxxxxx> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Wayne,
>>>>>>>>> 
>>>>>>>>> That's pretty odd, the incantations are practically the same.
>>>>>>>>> 
>>>>>>>>> The only other difference between this and the other pcb-based
>>>>>>>>> programs is this line:
>>>>>>>>> 
>>>>>>>>>    add_dependencies( pnsrouter pcbcommon pcad2kicadpcb
>>>>>>>>> ${GITHUB_PLUGIN_LIBRARIES} )
>>>>>>>>> 
>>>>>>>>> Which I can't really see making the difference (I think it's there for
>>>>>>>>> a header dependency).
>>>>>>>>> 
>>>>>>>>> And the very long target_link_libraries list that lists everything 4
>>>>>>>>> times and then a bit more. I thing replicating that long list might
>>>>>>>>> fix it, but it doesn't seem very tidy. The list from test_window:
>>>>>>>>> 
>>>>>>>>>     polygon pnsrouter common pcbcommon bitmaps polygon pnsrouter
>>>>>>>>> common pcbcommon bitmaps polygon pnsrouter common pcbcommon bitmaps
>>>>>>>>> polygon pnsrouter common pcbcommon 3d-viewer bitmaps gal pcad2kicadpcb
>>>>>>>>> common
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> 
>>>>>>>>> John
>>>>>>>>> On Thu, Oct 18, 2018 at 2:22 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hey John,
>>>>>>>>>> 
>>>>>>>>>> Below is the verbose output.  I hope it helps.
>>>>>>>>>> 
>>>>>>>>>> [ 98%] Linking CXX executable qa_pcb_parse_input.exe
>>>>>>>>>> cd
>>>>>>>>>> /C/msys64/home/wstambaugh/build32/kicad/trunk-release/qa/pcb_parse_input
>>>>>>>>>> && /C/msys64/mingw32/bin/cmake.exe -E remove -f
>>>>>>>>>> CMakeFiles/qa_pcb_parse_input.dir/objects.a
>>>>>>>>>> cd
>>>>>>>>>> /C/msys64/home/wstambaugh/build32/kicad/trunk-release/qa/pcb_parse_input
>>>>>>>>>> && /C/msys64/mingw32/bin/ar.exe cr
>>>>>>>>>> CMakeFiles/qa_pcb_parse_input.dir/objects.a
>>>>>>>>>> @CMakeFiles/qa_pcb_parse_input.dir/objects1.rsp
>>>>>>>>>> cd
>>>>>>>>>> /C/msys64/home/wstambaugh/build32/kicad/trunk-release/qa/pcb_parse_input
>>>>>>>>>> && /C/msys64/mingw32/bin/g++.exe -Wall  -Wsuggest-override
>>>>>>>>>> -Wno-unused-local-typedefs -Wno-strict-aliasing -mthreads -fpermissive
>>>>>>>>>> -O3 -DNDEBUG  -s -Wl,--whole-archive
>>>>>>>>>> CMakeFiles/qa_pcb_parse_input.dir/objects.a -Wl,--no-whole-archive  -o
>>>>>>>>>> qa_pcb_parse_input.exe
>>>>>>>>>> -Wl,--major-image-version,0,--minor-image-version,0
>>>>>>>>>> ../../common/libpcbcommon.a ../../common/liblegacy_wx.a
>>>>>>>>>> ../../common/libcommon.a ../../bitmaps_png/libbitmaps.a
>>>>>>>>>> ../../polygon/libpolygon.a ../../common/libgal.a
>>>>>>>>>> ../../pcbnew/pcad2kicadpcb_plugin/libpcad2kicadpcb.a
>>>>>>>>>> ../../pcbnew/github/libgithub_plugin.a ../qa_utils/libqa_utils.a
>>>>>>>>>> -LC:/msys64/mingw32/lib -pipe -Wl,--subsystem,windows -mwindows
>>>>>>>>>> -lwx_mswu_gl-3.0 -lwx_mswu_aui-3.0 -lwx_mswu_adv-3.0 -lwx_mswu_html-3.0
>>>>>>>>>> -lwx_mswu_core-3.0 -lwx_baseu_net-3.0 -lwx_baseu-3.0 -lwx_baseu_xml-3.0
>>>>>>>>>> -lwx_mswu_stc-3.0 -lws2_32 ../../common/libcommon.a
>>>>>>>>>> ../../common/libgal.a -lglew32 -lcairo -lpixman-1 -lopengl32 -lglu32
>>>>>>>>>> -lcurl -lssl -lcrypto -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32
>>>>>>>>>> -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32
>>>>>>>>>> C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/7.3.0/../../../../i686-w64-mingw32/bin/ld.exe:
>>>>>>>>>> ../../common/libcommon.a(marker_base.cpp.obj):marker_base.cpp:(.text+0xee9):
>>>>>>>>>> undefined reference to `DRC_ITEM::ShowHtml(EDA_UNITS_T) const'
>>>>>>>>>> collect2.exe: error: ld returned 1 exit status
>>>>>>>>>> make[2]: ***
>>>>>>>>>> [qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/build.make:163:
>>>>>>>>>> qa/pcb_parse_input/qa_pcb_parse_input.exe] Error 1
>>>>>>>>>> make[2]: Leaving directory '/home/wstambaugh/build32/kicad/trunk-release'
>>>>>>>>>> make[1]: *** [CMakeFiles/Makefile2:3431:
>>>>>>>>>> qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/all] Error 2
>>>>>>>>>> make[1]: Leaving directory '/home/wstambaugh/build32/kicad/trunk-release'
>>>>>>>>>> make: *** [Makefile:141: all] Error 2
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 10/17/2018 6:42 PM, John Beard wrote:
>>>>>>>>>>> Hi Wayne,
>>>>>>>>>>> 
>>>>>>>>>>> Could you try to build with VERBOSE=1 so I can see what the failed link
>>>>>>>>>>> command line looks like and compare to Jenkins?
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> 
>>>>>>>>>>> John
>>>>>>>>>>> 
>>>>>>>>>>> On 17 October 2018 13:17:34 BST, Wayne Stambaugh <stambaughw@xxxxxxxxx>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>    Hey John,
>>>>>>>>>>> 
>>>>>>>>>>>    Close but no cigar.  I'm still getting a single link error.
>>>>>>>>>>> 
>>>>>>>>>>>    C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/7.3.0/../../../../i686-w64-mingw32/bin/ld.exe:
>>>>>>>>>>>    ../../common/libcommon.a(marker_base.cpp.obj):marker_base.cpp:(.text+0xee9):
>>>>>>>>>>>    undefined reference to `DRC_ITEM::ShowHtml(EDA_UNITS_T) const'
>>>>>>>>>>>    collect2.exe: error: ld returned 1 exit status
>>>>>>>>>>>    [100%] Built target eeschema
>>>>>>>>>>>    make[2]: ***
>>>>>>>>>>>    [qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/build.make:163:
>>>>>>>>>>>    qa/pcb_parse_input/qa_pcb_parse_input.exe] Error 1
>>>>>>>>>>>    make[1]: *** [CMakeFiles/Makefile2:3431:
>>>>>>>>>>>    qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/all] Error 2
>>>>>>>>>>>    make[1]: *** Waiting for unfinished jobs....
>>>>>>>>>>>    [100%] Built target pcbnew_kiface
>>>>>>>>>>>    make: *** [Makefile:141: all] Error 2
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>    On 10/16/2018 9:28 AM, John Beard wrote:
>>>>>>>>>>> 
>>>>>>>>>>>        Hi Wayne,
>>>>>>>>>>> 
>>>>>>>>>>>        I think I might have fixed the ordering in the link libraries (at
>>>>>>>>>>>        least, it now builds on Jenkins).
>>>>>>>>>>> 
>>>>>>>>>>>        This is just the first patch to focus on the build error, I'll
>>>>>>>>>>>        rebase
>>>>>>>>>>>        the other docs stuff later if/when it works.
>>>>>>>>>>> 
>>>>>>>>>>>        Cheers,
>>>>>>>>>>> 
>>>>>>>>>>>        John
>>>>>>>>>>>        On Fri, Oct 12, 2018 at 7:58 PM Wayne Stambaugh
>>>>>>>>>>>        <stambaughw@xxxxxxxxx> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>            John,
>>>>>>>>>>> 
>>>>>>>>>>>            This patch fails to link on windows. I've attached the build
>>>>>>>>>>>            error.
>>>>>>>>>>> 
>>>>>>>>>>>            Wayne
>>>>>>>>>>> 
>>>>>>>>>>>            On 10/9/2018 9:53 AM, John Beard wrote:
>>>>>>>>>>> 
>>>>>>>>>>>                Hi,
>>>>>>>>>>> 
>>>>>>>>>>>                Here is an update patch that rebases over the commenting
>>>>>>>>>>>                out of
>>>>>>>>>>>                pcb_test_window, polygon_triangulation and
>>>>>>>>>>>                polygon_generator and fixes
>>>>>>>>>>>                a link error to do with base_screen.cpp.
>>>>>>>>>>> 
>>>>>>>>>>>                Cheers,
>>>>>>>>>>> 
>>>>>>>>>>>                John
>>>>>>>>>>>                On Mon, Oct 8, 2018 at 5:27 PM John Beard
>>>>>>>>>>>                <john.j.beard@xxxxxxxxx> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                    Sorry,
>>>>>>>>>>> 
>>>>>>>>>>>                    I wrote "ms", I meant "us" - the times are in the
>>>>>>>>>>>                    handful-of-millsecond range.
>>>>>>>>>>> 
>>>>>>>>>>>                    Cheers,
>>>>>>>>>>> 
>>>>>>>>>>>                    John
>>>>>>>>>>>                    On Mon, Oct 8, 2018 at 5:24 PM John Beard
>>>>>>>>>>>                    <john.j.beard@xxxxxxxxx> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>                        Hi,
>>>>>>>>>>> 
>>>>>>>>>>>                        This is a patch to add a test program that
>>>>>>>>>>>                        allows to parse a Pcbnew
>>>>>>>>>>>                        file from command line params or stdin. This
>>>>>>>>>>>                        means you can use it for
>>>>>>>>>>>                        fuzz testing.
>>>>>>>>>>> 
>>>>>>>>>>>                        I have done a little bit of fuzz testing so far
>>>>>>>>>>>                        (8 million execs,
>>>>>>>>>>>                        about 70% of a cycle), and have not found any
>>>>>>>>>>>                        crashes, but I can make
>>>>>>>>>>>                        it hang in a few ways. These all seem to be in
>>>>>>>>>>>                        streams which contain
>>>>>>>>>>>                        nul's. This is actually not reachable from the
>>>>>>>>>>>                        UI due to reading files
>>>>>>>>>>>                        into wxStrings first (nut quite sure why),
>>>>>>>>>>>                        whereas this program uses
>>>>>>>>>>>                        the parser directly. Thus, the bug is probably
>>>>>>>>>>>                        not very critical.
>>>>>>>>>>>                        Example hanging input attached (note there's a
>>>>>>>>>>>                        nul in it, so your
>>>>>>>>>>>                        editor may or may not like that).
>>>>>>>>>>> 
>>>>>>>>>>>                        This program can also be fed a number of files,
>>>>>>>>>>>                        which means it could
>>>>>>>>>>>                        be used for automated testing that all files in
>>>>>>>>>>>                        a batch can be parsed
>>>>>>>>>>>                        successfully, and also provides a handy way to
>>>>>>>>>>>                        put GDB on a program
>>>>>>>>>>>                        when debugging the parser against specific input.
>>>>>>>>>>> 
>>>>>>>>>>>                        There is timing on the parsing too, mostly for
>>>>>>>>>>>                        interest (use the -v
>>>>>>>>>>>                        flag). It takes about 150-3000ms per FP on my
>>>>>>>>>>>                        machine for the FPs in
>>>>>>>>>>>                        Connector_PinSocket_2.54mm.pretty.
>>>>>>>>>>> 
>>>>>>>>>>>                        There's also some centralisation of some
>>>>>>>>>>>                        QA-related utils into a
>>>>>>>>>>>                        qa_utils library.
>>>>>>>>>>> 
>>>>>>>>>>>                        Cheers,
>>>>>>>>>>> 
>>>>>>>>>>>                        John
>>>>>>>>>>>                        ------------------------------------------------------------------------
>>>>>>>>>>>                        Mailing list:
>>>>>>>>>>>                        https://launchpad.net/~kicad-developers
>>>>>>>>>>>                        Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>                        Unsubscribe :
>>>>>>>>>>>                        https://launchpad.net/~kicad-developers
>>>>>>>>>>>                        More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>> 
>>>>>>>>>>>            ------------------------------------------------------------------------
>>>>>>>>>>>            Mailing list: https://launchpad.net/~kicad-developers
>>>>>>>>>>>            Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>            Unsubscribe : https://launchpad.net/~kicad-developers
>>>>>>>>>>>            More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp


Follow ups

References