kicad-developers team mailing list archive

[John Beard <john.j.beard@xxxxxxxxx>]

On Thu, Mar 2, 2017 at 10:30 PM, Wayne Stambaugh <stambaughw@xxxxxxxxx> wrote:
>
> One thing I overlooked is wxWidgets provides wxBufferedInputStream[1]
> which takes a reference to a wxInputStream object as an argument.  The
> reason wxFileInputStream and wxFFileInputStream may be so slow is that
> they are not buffered at all internally and need to be wrapped by
> wxBufferedInputStream to perform the buffering.  It might be something
> worth taking a look at.

I tried this out - there's a factor of just over 10 improvment over a
plain wxFileInputStream and a 30% ish speedup over wxFFileStream, but
it's still 20 times slower than std::ifstream. I can't help but think
I'm just doing all the wx stuff wrong - it cannot be that bad for
reals!


I have also noticed, doing this in a quieter room, that the 4000ms
wxInputStream benchmarks (File, FFile and Buffered) cause the fans on
my computer to kick up to full, so something is being computed
furiously!

std::fstream                   317858 lines, acc: 25103384 in 16 ms
std::fstream, reused           317858 lines, acc: 25103384 in 16 ms
RICHIO                         317858 lines, acc: 25103384 in 95 ms
RICHIO, reused                 317858 lines, acc: 25103384 in 93 ms
std::ifstream L_R              317858 lines, acc: 25103384 in 21 ms
std::ifstream L_R, reused      317858 lines, acc: 25103384 in 21 ms
wxFileIStream                  317858 lines, acc: 25103384 in 4218 ms
wxFileIStream, reused          317858 lines, acc: 25103384 in 4225 ms
wxFFileIStream                 317858 lines, acc: 25103384 in 568 ms
wxFFileIStream, reused         317858 lines, acc: 25103384 in 567 ms
wxFileIStream. buf'd           317858 lines, acc: 25103384 in 380 ms
wxFileIStream, buf'd, reused   317858 lines, acc: 25103384 in 383 ms
wxFFileIStream. buf'd          317858 lines, acc: 25103384 in 376 ms
wxFFileIStream, buf'd, reused  317858 lines, acc: 25103384 in 376 ms

For the wx streams, if you do more reps, there's a huge increase in
releative slowness
 (above is 2, below is 10):

std::fstream                   1589290 lines, acc: 125516920 in 81 ms
wxFileIStream                  1589290 lines, acc: 125516920 in 21279 ms
wxFileIStream. buf'd           1589290 lines, acc: 125516920 in 14394 ms

No idea what's behind all this, any I'll be happy to be told im
Holding It Wrong (TM)!

Patch for adding buffered wx streams to the benchmark tool attached.

Cheers,

John

From d1d066d65a4e5525f3e47f731d7d132715aa6b88 Mon Sep 17 00:00:00 2001
From: John Beard <john.j.beard@xxxxxxxxx>
Date: Fri, 3 Mar 2017 00:26:39 +0800
Subject: [PATCH] io_benchmark: Add wxBufferedInputStream benchmarks

When run, these show a 10x speed up over wxFileInputStream alone and
about 30% over wxFFInputStream.
---
 tools/io_benchmark/io_benchmark.cpp | 62 +++++++++++++++++++++++++++++++++++--
 1 file changed, 59 insertions(+), 3 deletions(-)

diff --git a/tools/io_benchmark/io_benchmark.cpp b/tools/io_benchmark/io_benchmark.cpp
index 3eea0c095..1d7071199 100644
--- a/tools/io_benchmark/io_benchmark.cpp
+++ b/tools/io_benchmark/io_benchmark.cpp
@@ -157,7 +157,7 @@ static void bench_line_reader_reuse( const wxString& aFile, int aReps, BENCH_REP
 /**
  * Benchmark using an INPUTSTREAM_LINE_READER with a given
  * wxInputStream implementation.
- * The INPUTSTREAM_LINE_READER is reset for each cycle.
+ * The wxInputStream is recreated for each cycle.
  */
 template<typename S>
 static void bench_wxis( const wxString& aFile, int aReps, BENCH_REPORT& report )
@@ -182,7 +182,7 @@ static void bench_wxis( const wxString& aFile, int aReps, BENCH_REPORT& report )
 /**
  * Benchmark using an INPUTSTREAM_LINE_READER with a given
  * wxInputStream implementation.
- * The INPUTSTREAM_LINE_READER is recreated for each cycle.
+ * The wxInputStream is reset for each cycle.
  */
 template<typename S>
 static void bench_wxis_reuse( const wxString& aFile, int aReps, BENCH_REPORT& report )
@@ -202,6 +202,58 @@ static void bench_wxis_reuse( const wxString& aFile, int aReps, BENCH_REPORT& re
     }
 }
 
+
+/**
+ * Benchmark using a INPUTSTREAM_LINE_READER with a given
+ * wxInputStream implementation, buffered with wxBufferedInputStream.
+ * The wxInputStream is recreated for each cycle.
+ */
+template<typename WXIS>
+static void bench_wxbis( const wxString& aFile, int aReps, BENCH_REPORT& report )
+{
+    WXIS fileStream( aFile );
+    wxBufferedInputStream bufferedStream( fileStream );
+
+    for( int i = 0; i < aReps; ++i)
+    {
+        INPUTSTREAM_LINE_READER istr( &bufferedStream, aFile );
+
+        while( istr.ReadLine() )
+        {
+            report.linesRead++;
+            report.charAcc += (unsigned char) istr.Line()[0];
+        }
+
+        fileStream.SeekI( 0 );
+    }
+}
+
+
+/**
+ * Benchmark using a INPUTSTREAM_LINE_READER with a given
+ * wxInputStream implementation, buffered with wxBufferedInputStream.
+ * The wxInputStream is reset for each cycle.
+ */
+template<typename WXIS>
+static void bench_wxbis_reuse( const wxString& aFile, int aReps, BENCH_REPORT& report )
+{
+    WXIS fileStream( aFile );
+    wxBufferedInputStream bufferedStream( fileStream );
+
+    INPUTSTREAM_LINE_READER istr( &bufferedStream, aFile );
+
+    for( int i = 0; i < aReps; ++i)
+    {
+        while( istr.ReadLine() )
+        {
+            report.linesRead++;
+            report.charAcc += (unsigned char) istr.Line()[0];
+        }
+
+        fileStream.SeekI( 0 );
+    }
+}
+
 /**
  * List of available benchmarks
  */
@@ -217,6 +269,10 @@ static std::vector<BENCHMARK> benchmarkList =
     { 'W', bench_wxis<wxFileInputStream>, "wxFileIStream, reused" },
     { 'g', bench_wxis<wxFFileInputStream>, "wxFFileIStream" },
     { 'G', bench_wxis_reuse<wxFFileInputStream>, "wxFFileIStream, reused" },
+    { 'b', bench_wxbis<wxFileInputStream>, "wxFileIStream. buf'd" },
+    { 'B', bench_wxbis_reuse<wxFileInputStream>, "wxFileIStream, buf'd, reused" },
+    { 'c', bench_wxbis<wxFFileInputStream>, "wxFFileIStream. buf'd" },
+    { 'C', bench_wxbis_reuse<wxFFileInputStream>, "wxFFileIStream, buf'd, reused" },
 };
 
 
@@ -312,7 +368,7 @@ int main( int argc, char* argv[] )
 
         BENCH_REPORT report = executeBenchMark( bmark, reps, inFile );
 
-        os << wxString::Format( "%-25s %u lines, acc: %u in %u ms",
+        os << wxString::Format( "%-30s %u lines, acc: %u in %u ms",
                 bmark.name, report.linesRead, report.charAcc, (int) report.benchDurMs.count() )
             << std::endl;;
     }
-- 
2.12.0