← Back to team overview

kicad-developers team mailing list archive

Re: [PATCH] Fuzzable PCB parsing test harness

 

No problem.  I have gone through this exercise a few times myself.
Let's just hope the macos build doesn't choke.

On 10/22/2018 1:30 PM, John Beard wrote:
> Thanks, Wayne! Sorry for all the drama with it. Hopefully I can learn
> from it for the libcommon tests.
> 
> Cheers,
> 
> John
> On Mon, Oct 22, 2018 at 6:14 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>
>> It's merged.  Thanks John.
>>
>> On 10/22/2018 1:09 PM, John Beard wrote:
>>> Hi Wayne,
>>>
>>> Hurray!
>>>
>>> No extras needed, this should be ready to merge. There are additional
>>> things we can do to enable testing of PLUGINs as opposed to the PCB
>>> parser, but this at least provides the base to build on. This commit
>>> is self-contained and includes the fuzzing documentation.
>>>
>>> Cheers,
>>>
>>> John
>>> On Mon, Oct 22, 2018 at 5:45 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>>>
>>>> Hey John,
>>>>
>>>> Finally!  That was easy.  I'm guessing this is ready to merge or is
>>>> there anything else that needs to be done?
>>>>
>>>> Cheers,
>>>>
>>>> Wayne
>>>>
>>>> On 10/22/2018 12:19 PM, John Beard wrote:
>>>>> Hi Wayne,
>>>>>
>>>>> Hmm, didn't expect that to be an issue! Try a ToStdString() for size?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> John
>>>>> On Mon, Oct 22, 2018 at 2:39 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>>>>>
>>>>>> Hey John,
>>>>>>
>>>>>> You not giving up on this are you ;).  I builds fine for mingw32 and
>>>>>> passes all of the tests.  However, the mingw64 build fails with:
>>>>>>
>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/pcb_parse_input/main.cpp:
>>>>>> In function 'int main(int, char**)':
>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/pcb_parse_input/main.cpp:148:32:
>>>>>> error: call of overloaded 'open(const wxString&)' is ambiguous
>>>>>>              fin.open( filename );
>>>>>>                                 ^
>>>>>> In file included from
>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/qa_utils/stdstream_line_reader.h:32,
>>>>>>                  from
>>>>>> C:/msys64/home/wstambaugh/src/kicad-trunk/qa/pcb_parse_input/main.cpp:32:
>>>>>> C:/msys64/mingw64/include/c++/8.2.0/fstream:653:7: note: candidate:
>>>>>> 'void std::basic_ifstream<_CharT, _Traits>::open(const char*,
>>>>>> std::ios_base::openmode) [with _CharT = char; _Traits =
>>>>>> std::char_traits<char>; std::ios_base::openmode = std::_Ios_Openmode]'
>>>>>>        open(const char* __s, ios_base::openmode __mode = ios_base::in)
>>>>>>        ^~~~
>>>>>> C:/msys64/mingw64/include/c++/8.2.0/fstream:673:7: note: candidate:
>>>>>> 'void std::basic_ifstream<_CharT, _Traits>::open(const wchar_t*,
>>>>>> std::ios_base::openmode) [with _CharT = char; _Traits =
>>>>>> std::char_traits<char>; std::ios_base::openmode = std::_Ios_Openmode]'
>>>>>>        open(const wchar_t* __s, ios_base::openmode __mode = ios_base::in)
>>>>>>        ^~~~
>>>>>>
>>>>>> On 10/22/2018 4:57 AM, John Beard wrote:
>>>>>>> Hi Wayne,
>>>>>>>
>>>>>>> Since I'm out of better ideas and can't replicate the link failure on
>>>>>>> Jenkins, here's a patch that uses the long list from pcb_test_window.
>>>>>>> Presumably it's there for some reason, though there is no comment or
>>>>>>> Git log clue that I can see.
>>>>>>>
>>>>>>> Could you see if that works for you?
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> John
>>>>>>> On Fri, Oct 19, 2018 at 12:09 PM John Beard <john.j.beard@xxxxxxxxx> wrote:
>>>>>>>>
>>>>>>>> Hi Wayne,
>>>>>>>>
>>>>>>>> That's pretty odd, the incantations are practically the same.
>>>>>>>>
>>>>>>>> The only other difference between this and the other pcb-based
>>>>>>>> programs is this line:
>>>>>>>>
>>>>>>>>     add_dependencies( pnsrouter pcbcommon pcad2kicadpcb
>>>>>>>> ${GITHUB_PLUGIN_LIBRARIES} )
>>>>>>>>
>>>>>>>> Which I can't really see making the difference (I think it's there for
>>>>>>>> a header dependency).
>>>>>>>>
>>>>>>>> And the very long target_link_libraries list that lists everything 4
>>>>>>>> times and then a bit more. I thing replicating that long list might
>>>>>>>> fix it, but it doesn't seem very tidy. The list from test_window:
>>>>>>>>
>>>>>>>>      polygon pnsrouter common pcbcommon bitmaps polygon pnsrouter
>>>>>>>> common pcbcommon bitmaps polygon pnsrouter common pcbcommon bitmaps
>>>>>>>> polygon pnsrouter common pcbcommon 3d-viewer bitmaps gal pcad2kicadpcb
>>>>>>>> common
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> John
>>>>>>>> On Thu, Oct 18, 2018 at 2:22 PM Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>>>>>>>>>
>>>>>>>>> Hey John,
>>>>>>>>>
>>>>>>>>> Below is the verbose output.  I hope it helps.
>>>>>>>>>
>>>>>>>>> [ 98%] Linking CXX executable qa_pcb_parse_input.exe
>>>>>>>>> cd
>>>>>>>>> /C/msys64/home/wstambaugh/build32/kicad/trunk-release/qa/pcb_parse_input
>>>>>>>>> && /C/msys64/mingw32/bin/cmake.exe -E remove -f
>>>>>>>>> CMakeFiles/qa_pcb_parse_input.dir/objects.a
>>>>>>>>> cd
>>>>>>>>> /C/msys64/home/wstambaugh/build32/kicad/trunk-release/qa/pcb_parse_input
>>>>>>>>> && /C/msys64/mingw32/bin/ar.exe cr
>>>>>>>>> CMakeFiles/qa_pcb_parse_input.dir/objects.a
>>>>>>>>> @CMakeFiles/qa_pcb_parse_input.dir/objects1.rsp
>>>>>>>>> cd
>>>>>>>>> /C/msys64/home/wstambaugh/build32/kicad/trunk-release/qa/pcb_parse_input
>>>>>>>>> && /C/msys64/mingw32/bin/g++.exe -Wall  -Wsuggest-override
>>>>>>>>> -Wno-unused-local-typedefs -Wno-strict-aliasing -mthreads -fpermissive
>>>>>>>>> -O3 -DNDEBUG  -s -Wl,--whole-archive
>>>>>>>>> CMakeFiles/qa_pcb_parse_input.dir/objects.a -Wl,--no-whole-archive  -o
>>>>>>>>> qa_pcb_parse_input.exe
>>>>>>>>> -Wl,--major-image-version,0,--minor-image-version,0
>>>>>>>>> ../../common/libpcbcommon.a ../../common/liblegacy_wx.a
>>>>>>>>> ../../common/libcommon.a ../../bitmaps_png/libbitmaps.a
>>>>>>>>> ../../polygon/libpolygon.a ../../common/libgal.a
>>>>>>>>> ../../pcbnew/pcad2kicadpcb_plugin/libpcad2kicadpcb.a
>>>>>>>>> ../../pcbnew/github/libgithub_plugin.a ../qa_utils/libqa_utils.a
>>>>>>>>> -LC:/msys64/mingw32/lib -pipe -Wl,--subsystem,windows -mwindows
>>>>>>>>> -lwx_mswu_gl-3.0 -lwx_mswu_aui-3.0 -lwx_mswu_adv-3.0 -lwx_mswu_html-3.0
>>>>>>>>> -lwx_mswu_core-3.0 -lwx_baseu_net-3.0 -lwx_baseu-3.0 -lwx_baseu_xml-3.0
>>>>>>>>> -lwx_mswu_stc-3.0 -lws2_32 ../../common/libcommon.a
>>>>>>>>> ../../common/libgal.a -lglew32 -lcairo -lpixman-1 -lopengl32 -lglu32
>>>>>>>>> -lcurl -lssl -lcrypto -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32
>>>>>>>>> -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32
>>>>>>>>> C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/7.3.0/../../../../i686-w64-mingw32/bin/ld.exe:
>>>>>>>>> ../../common/libcommon.a(marker_base.cpp.obj):marker_base.cpp:(.text+0xee9):
>>>>>>>>> undefined reference to `DRC_ITEM::ShowHtml(EDA_UNITS_T) const'
>>>>>>>>> collect2.exe: error: ld returned 1 exit status
>>>>>>>>> make[2]: ***
>>>>>>>>> [qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/build.make:163:
>>>>>>>>> qa/pcb_parse_input/qa_pcb_parse_input.exe] Error 1
>>>>>>>>> make[2]: Leaving directory '/home/wstambaugh/build32/kicad/trunk-release'
>>>>>>>>> make[1]: *** [CMakeFiles/Makefile2:3431:
>>>>>>>>> qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/all] Error 2
>>>>>>>>> make[1]: Leaving directory '/home/wstambaugh/build32/kicad/trunk-release'
>>>>>>>>> make: *** [Makefile:141: all] Error 2
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 10/17/2018 6:42 PM, John Beard wrote:
>>>>>>>>>> Hi Wayne,
>>>>>>>>>>
>>>>>>>>>> Could you try to build with VERBOSE=1 so I can see what the failed link
>>>>>>>>>> command line looks like and compare to Jenkins?
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> John
>>>>>>>>>>
>>>>>>>>>> On 17 October 2018 13:17:34 BST, Wayne Stambaugh <stambaughw@xxxxxxxxx>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>     Hey John,
>>>>>>>>>>
>>>>>>>>>>     Close but no cigar.  I'm still getting a single link error.
>>>>>>>>>>
>>>>>>>>>>     C:/msys64/mingw32/bin/../lib/gcc/i686-w64-mingw32/7.3.0/../../../../i686-w64-mingw32/bin/ld.exe:
>>>>>>>>>>     ../../common/libcommon.a(marker_base.cpp.obj):marker_base.cpp:(.text+0xee9):
>>>>>>>>>>     undefined reference to `DRC_ITEM::ShowHtml(EDA_UNITS_T) const'
>>>>>>>>>>     collect2.exe: error: ld returned 1 exit status
>>>>>>>>>>     [100%] Built target eeschema
>>>>>>>>>>     make[2]: ***
>>>>>>>>>>     [qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/build.make:163:
>>>>>>>>>>     qa/pcb_parse_input/qa_pcb_parse_input.exe] Error 1
>>>>>>>>>>     make[1]: *** [CMakeFiles/Makefile2:3431:
>>>>>>>>>>     qa/pcb_parse_input/CMakeFiles/qa_pcb_parse_input.dir/all] Error 2
>>>>>>>>>>     make[1]: *** Waiting for unfinished jobs....
>>>>>>>>>>     [100%] Built target pcbnew_kiface
>>>>>>>>>>     make: *** [Makefile:141: all] Error 2
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>     On 10/16/2018 9:28 AM, John Beard wrote:
>>>>>>>>>>
>>>>>>>>>>         Hi Wayne,
>>>>>>>>>>
>>>>>>>>>>         I think I might have fixed the ordering in the link libraries (at
>>>>>>>>>>         least, it now builds on Jenkins).
>>>>>>>>>>
>>>>>>>>>>         This is just the first patch to focus on the build error, I'll
>>>>>>>>>>         rebase
>>>>>>>>>>         the other docs stuff later if/when it works.
>>>>>>>>>>
>>>>>>>>>>         Cheers,
>>>>>>>>>>
>>>>>>>>>>         John
>>>>>>>>>>         On Fri, Oct 12, 2018 at 7:58 PM Wayne Stambaugh
>>>>>>>>>>         <stambaughw@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>             John,
>>>>>>>>>>
>>>>>>>>>>             This patch fails to link on windows. I've attached the build
>>>>>>>>>>             error.
>>>>>>>>>>
>>>>>>>>>>             Wayne
>>>>>>>>>>
>>>>>>>>>>             On 10/9/2018 9:53 AM, John Beard wrote:
>>>>>>>>>>
>>>>>>>>>>                 Hi,
>>>>>>>>>>
>>>>>>>>>>                 Here is an update patch that rebases over the commenting
>>>>>>>>>>                 out of
>>>>>>>>>>                 pcb_test_window, polygon_triangulation and
>>>>>>>>>>                 polygon_generator and fixes
>>>>>>>>>>                 a link error to do with base_screen.cpp.
>>>>>>>>>>
>>>>>>>>>>                 Cheers,
>>>>>>>>>>
>>>>>>>>>>                 John
>>>>>>>>>>                 On Mon, Oct 8, 2018 at 5:27 PM John Beard
>>>>>>>>>>                 <john.j.beard@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     Sorry,
>>>>>>>>>>
>>>>>>>>>>                     I wrote "ms", I meant "us" - the times are in the
>>>>>>>>>>                     handful-of-millsecond range.
>>>>>>>>>>
>>>>>>>>>>                     Cheers,
>>>>>>>>>>
>>>>>>>>>>                     John
>>>>>>>>>>                     On Mon, Oct 8, 2018 at 5:24 PM John Beard
>>>>>>>>>>                     <john.j.beard@xxxxxxxxx> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                         Hi,
>>>>>>>>>>
>>>>>>>>>>                         This is a patch to add a test program that
>>>>>>>>>>                         allows to parse a Pcbnew
>>>>>>>>>>                         file from command line params or stdin. This
>>>>>>>>>>                         means you can use it for
>>>>>>>>>>                         fuzz testing.
>>>>>>>>>>
>>>>>>>>>>                         I have done a little bit of fuzz testing so far
>>>>>>>>>>                         (8 million execs,
>>>>>>>>>>                         about 70% of a cycle), and have not found any
>>>>>>>>>>                         crashes, but I can make
>>>>>>>>>>                         it hang in a few ways. These all seem to be in
>>>>>>>>>>                         streams which contain
>>>>>>>>>>                         nul's. This is actually not reachable from the
>>>>>>>>>>                         UI due to reading files
>>>>>>>>>>                         into wxStrings first (nut quite sure why),
>>>>>>>>>>                         whereas this program uses
>>>>>>>>>>                         the parser directly. Thus, the bug is probably
>>>>>>>>>>                         not very critical.
>>>>>>>>>>                         Example hanging input attached (note there's a
>>>>>>>>>>                         nul in it, so your
>>>>>>>>>>                         editor may or may not like that).
>>>>>>>>>>
>>>>>>>>>>                         This program can also be fed a number of files,
>>>>>>>>>>                         which means it could
>>>>>>>>>>                         be used for automated testing that all files in
>>>>>>>>>>                         a batch can be parsed
>>>>>>>>>>                         successfully, and also provides a handy way to
>>>>>>>>>>                         put GDB on a program
>>>>>>>>>>                         when debugging the parser against specific input.
>>>>>>>>>>
>>>>>>>>>>                         There is timing on the parsing too, mostly for
>>>>>>>>>>                         interest (use the -v
>>>>>>>>>>                         flag). It takes about 150-3000ms per FP on my
>>>>>>>>>>                         machine for the FPs in
>>>>>>>>>>                         Connector_PinSocket_2.54mm.pretty.
>>>>>>>>>>
>>>>>>>>>>                         There's also some centralisation of some
>>>>>>>>>>                         QA-related utils into a
>>>>>>>>>>                         qa_utils library.
>>>>>>>>>>
>>>>>>>>>>                         Cheers,
>>>>>>>>>>
>>>>>>>>>>                         John
>>>>>>>>>>                         ------------------------------------------------------------------------
>>>>>>>>>>                         Mailing list:
>>>>>>>>>>                         https://launchpad.net/~kicad-developers
>>>>>>>>>>                         Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>                         Unsubscribe :
>>>>>>>>>>                         https://launchpad.net/~kicad-developers
>>>>>>>>>>                         More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>
>>>>>>>>>>             ------------------------------------------------------------------------
>>>>>>>>>>             Mailing list: https://launchpad.net/~kicad-developers
>>>>>>>>>>             Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>             Unsubscribe : https://launchpad.net/~kicad-developers
>>>>>>>>>>             More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>


Follow ups

References