maria-developers team mailing list archive

[Anshu Avinash <anshu.avinash35@xxxxxxxxx>]

Hi all,

Here is a blog post describing my progress so far:
http://igniting.in/gsoc2014/2014/08/04/progress-so-far/

Comments and suggestions are welcome.

Regards
Anshu Avinash


On Wed, Jul 9, 2014 at 1:14 AM, Roberto Spadim <roberto@xxxxxxxxxxxxx>
wrote:

> please include the output (solutions.txt) link too, to check what happened
>
> 2014-07-08 16:24 GMT-03:00 Roberto Spadim <roberto@xxxxxxxxxxxxx>:
> > http://www.netlib.org/lapack/lapacke.html
> > a C api to lapack
> >
> > 2014-07-08 16:23 GMT-03:00 Roberto Spadim <roberto@xxxxxxxxxxxxx>:
> >> check lapack lib, i used it some years ago, and it solve linear
> >> equations, at least you don't waste time with 'how to solve linear
> >> equations', if you want to study :) lapack was a nice lib, at least i
> >> used without problems
> >>
> >> 2014-07-08 16:04 GMT-03:00 Roberto Spadim <roberto@xxxxxxxxxxxxx>:
> >>> a1,a2 is what your solve equation function is saving?
> >>> and you want know t1,t2,t3..t130, to understand how much time each
> >>> 'read function' take, that's it?
> >>>
> >>> doing this, what's the next step? this is a start point to select
> >>> what's better? index vs table scan?
> >>>
> >>>
> >>>
> >>> 2014-07-08 15:54 GMT-03:00 Anshu Avinash <anshu.avinash35@xxxxxxxxx>:
> >>>> Hi,
> >>>>
> >>>> a1, a2, ..., a130 are coefficients. t1, t2, .., t130 are unknowns. We
> need
> >>>> 130 linearly independent equations to solve for these variables. We
> can
> >>>> never get 130 linearly independent equations as some of the
> coefficients
> >>>> would be 0 every time. Hence, we get an approximate solution by
> forming an
> >>>> overdetermined system (
> http://en.wikipedia.org/wiki/Overdetermined_system).
> >>>> Let me know if you have any further doubts.
> >>>>
> >>>> Regards
> >>>> Anshu Avinash
> >>>>
> >>>>
> >>>> On Wed, Jul 9, 2014 at 12:18 AM, Roberto Spadim <
> roberto@xxxxxxxxxxxxx>
> >>>> wrote:
> >>>>>
> >>>>> ops, linear equation
> >>>>>
> >>>>> 2014-07-08 15:47 GMT-03:00 Roberto Spadim <roberto@xxxxxxxxxxxxx>:
> >>>>> > going back...
> >>>>> > a1t1 + a2t2 + … + a130t130= ttotal
> >>>>> >
> >>>>> > a1, t1...
> >>>>> >
> >>>>> > a1 is something you don't know
> >>>>> > t1 is the coefficients[i]?
> >>>>> >
> >>>>> > it's a first order equation, right?
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > 2014-07-08 15:20 GMT-03:00 Anshu Avinash <
> anshu.avinash35@xxxxxxxxx>:
> >>>>> >> Hi,
> >>>>> >>
> >>>>> >> The idea is we know the total time the query took, and how many
> times
> >>>>> >> an
> >>>>> >> operation was performed. For example, consider the case of
> 'read_time'.
> >>>>> >> We
> >>>>> >> know how many times an index read took place, but don't know how
> much
> >>>>> >> time
> >>>>> >> does it take to do an index read. By solving these equations, we
> are
> >>>>> >> trying
> >>>>> >> to find out time for individual operations. coefficients[i].value
> is
> >>>>> >> `how
> >>>>> >> many time the operation i took place in a single query.`
> >>>>> >>
> >>>>> >> Hope this clears things up.
> >>>>> >>
> >>>>> >> Regards
> >>>>> >> Anshu Avinash
> >>>>> >>
> >>>>> >>
> >>>>> >> On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim <
> roberto@xxxxxxxxxxxxx>
> >>>>> >> wrote:
> >>>>> >>>
> >>>>> >>> just to understand...
> >>>>> >>> --- the solve_equation part, today only used to save information:
> >>>>> >>>   std::ofstream datafile;
> >>>>> >>>   char file_name[100];
> >>>>> >>>   my_snprintf(file_name, 100,
> >>>>> >>> "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id);
> >>>>> >>>   datafile.open(file_name, std::ios::app);
> >>>>> >>>   for(int i=0; i < MAX_CONSTANTS; i++)
> >>>>> >>>     datafile << coefficients[i].value << " ";
> >>>>> >>>   datafile << total_time << "\n";
> >>>>> >>>   datafile.close();
> >>>>> >>> ----
> >>>>> >>>
> >>>>> >>> the idea is: given a query and some coefficients[i].value, you
> got
> >>>>> >>> total_time need to execute the query
> >>>>> >>> you want to "train" something to tell you how many time the same
> query
> >>>>> >>> should execute?
> >>>>> >>> or, what's the "x[i]" variables from your system (hardware/hard
> >>>>> >>> disk/etc), and extend this to others queries?
> >>>>> >>>
> >>>>> >>>
> >>>>> >>> 2014-07-08 14:20 GMT-03:00 Roberto Spadim <roberto@xxxxxxxxxxxxx
> >:
> >>>>> >>> > =] nice
> >>>>> >>> >
> >>>>> >>> > 2014-07-08 14:18 GMT-03:00 Anshu Avinash
> >>>>> >>> > <anshu.avinash35@xxxxxxxxx>:
> >>>>> >>> >> Hi all,
> >>>>> >>> >>
> >>>>> >>> >> You can download it here
> >>>>> >>> >>
> >>>>> >>> >>
> >>>>> >>> >> (
> https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharing
> ).
> >>>>> >>> >> It is around 26M. I have added the link on blog too.
> >>>>> >>> >>
> >>>>> >>> >> Regards
> >>>>> >>> >> Anshu
> >>>>> >>> >>
> >>>>> >>> >>
> >>>>> >>> >> On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim
> >>>>> >>> >> <roberto@xxxxxxxxxxxxx>
> >>>>> >>> >> wrote:
> >>>>> >>> >>>
> >>>>> >>> >>> could you 'display' the dataset you used with octave?
> >>>>> >>> >>>
> >>>>> >>> >>> 2014-07-08 13:55 GMT-03:00 Anshu Avinash
> >>>>> >>> >>> <anshu.avinash35@xxxxxxxxx>:
> >>>>> >>> >>> > Hi all,
> >>>>> >>> >>> >
> >>>>> >>> >>> > This week's blog post is at:
> >>>>> >>> >>> >
> http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/
> >>>>> >>> >>> > .
> >>>>> >>> >>> > Sorry
> >>>>> >>> >>> > for
> >>>>> >>> >>> > the delay.
> >>>>> >>> >>> > Suggestions for an approach to solve the system of linear
> >>>>> >>> >>> > equations
> >>>>> >>> >>> > are
> >>>>> >>> >>> > welcome.
> >>>>> >>> >>> >
> >>>>> >>> >>> > Regards
> >>>>> >>> >>> > Anshu Avinash
> >>>>> >>> >>> >
> >>>>> >>> >>> >
> >>>>> >>> >>> > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim
> >>>>> >>> >>> > <roberto@xxxxxxxxxxxxx>
> >>>>> >>> >>> > wrote:
> >>>>> >>> >>> >>
> >>>>> >>> >>> >> " MDEV. "
> >>>>> >>> >>> >> it's nice to put full name (MDEV-350), since google and
> others
> >>>>> >>> >>> >> search
> >>>>> >>> >>> >> engines help when someone try to find information about
> mdev
> >>>>> >>> >>> >> 350
> >>>>> >>> >>> >>
> >>>>> >>> >>> >> text is ok :)
> >>>>> >>> >>> >>
> >>>>> >>> >>> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash
> >>>>> >>> >>> >> <anshu.avinash35@xxxxxxxxx>:
> >>>>> >>> >>> >> > Hi,
> >>>>> >>> >>> >> >
> >>>>> >>> >>> >> > Sorry for the confusion, this is the new link:
> >>>>> >>> >>> >> >
> http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/
> >>>>> >>> >>> >> > Thanks for pointing out.
> >>>>> >>> >>> >> >
> >>>>> >>> >>> >> > Regards
> >>>>> >>> >>> >> > Anshu
> >>>>> >>> >>> >> >
> >>>>> >>> >>> >> >
> >>>>> >>> >>> >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim
> >>>>> >>> >>> >> > <roberto@xxxxxxxxxxxxx>
> >>>>> >>> >>> >> > wrote:
> >>>>> >>> >>> >> >>
> >>>>> >>> >>> >> >> "Sorry this page does not exist =("
> >>>>> >>> >>> >> >>
> >>>>> >>> >>> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash
> >>>>> >>> >>> >> >> <anshu.avinash35@xxxxxxxxx>:
> >>>>> >>> >>> >> >> > Hi all,
> >>>>> >>> >>> >> >> >
> >>>>> >>> >>> >> >> > You can find this week's blog entry at:
> >>>>> >>> >>> >> >> > http://igniting.in/2014/06/23/work-before-mid-term/
> >>>>> >>> >>> >> >> > Suggestions/reviews are welcome.
> >>>>> >>> >>> >> >> >
> >>>>> >>> >>> >> >> > Regards
> >>>>> >>> >>> >> >> > Anshu Avinash
> >>>>> >>> >>> >> >> >
> >>>>> >>> >>> >> >> >
> >>>>> >>> >>> >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim
> >>>>> >>> >>> >> >> > <roberto@xxxxxxxxxxxxx>
> >>>>> >>> >>> >> >> > wrote:
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >> Well i wws reading your posts
> >>>>> >>> >>> >> >> >> Do you need big data to test read and scan times?
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash
> >>>>> >>> >>> >> >> >> <anshu.avinash35@xxxxxxxxx> escreveu:
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >>> Hi all,
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> You can find this week's blog entry at
> >>>>> >>> >>> >> >> >>>
> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm
> >>>>> >>> >>> >> >> >>> now
> >>>>> >>> >>> >> >> >>> maintaining the
> >>>>> >>> >>> >> >> >>> code only on github:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> https://github.com/igniting/server/tree/selfTuningOptimizer.
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Regards
> >>>>> >>> >>> >> >> >>> Anshu Avinash
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash
> >>>>> >>> >>> >> >> >>> <anshu.avinash35@xxxxxxxxx> wrote:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Hi all,
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> You can find my this week's blog entry at
> >>>>> >>> >>> >> >> >>>
> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/
> >>>>> >>> >>> >> >> >>> . I
> >>>>> >>> >>> >> >> >>> have
> >>>>> >>> >>> >> >> >>> created a
> >>>>> >>> >>> >> >> >>> branch on launchpad for my work:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211
> >>>>> >>> >>> >> >> >>> .
> >>>>> >>> >>> >> >> >>> You
> >>>>> >>> >>> >> >> >>> can
> >>>>> >>> >>> >> >> >>> give your suggestions/reviews either on this
> thread or
> >>>>> >>> >>> >> >> >>> as a
> >>>>> >>> >>> >> >> >>> comment
> >>>>> >>> >>> >> >> >>> on
> >>>>> >>> >>> >> >> >>> the
> >>>>> >>> >>> >> >> >>> blog itself.
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Regards
> >>>>> >>> >>> >> >> >>> Anshu Avinash
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim
> >>>>> >>> >>> >> >> >>> <roberto@xxxxxxxxxxxxx>
> >>>>> >>> >>> >> >> >>> wrote:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> wow a big work, congratulation guy, i will read
> part by
> >>>>> >>> >>> >> >> >>> part
> >>>>> >>> >>> >> >> >>> to
> >>>>> >>> >>> >> >> >>> better
> >>>>> >>> >>> >> >> >>> understand mariadb code
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash
> >>>>> >>> >>> >> >> >>> <anshu.avinash35@xxxxxxxxx>:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Hi all,
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> This week's blog entry would get delayed by couple
> of
> >>>>> >>> >>> >> >> >>> days.
> >>>>> >>> >>> >> >> >>> I
> >>>>> >>> >>> >> >> >>> have
> >>>>> >>> >>> >> >> >>> started coding though and would like to give heads
> up on
> >>>>> >>> >>> >> >> >>> what
> >>>>> >>> >>> >> >> >>> I'm
> >>>>> >>> >>> >> >> >>> doing.
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> I've looked at the diffs for "Cost model project"
> of
> >>>>> >>> >>> >> >> >>> mysql:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596
> >>>>> >>> >>> >> >> >>> and
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 .
> >>>>> >>> >>> >> >> >>> These
> >>>>> >>> >>> >> >> >>> give a pretty good idea about what are the
> hard-coded
> >>>>> >>> >>> >> >> >>> constants
> >>>>> >>> >>> >> >> >>> and
> >>>>> >>> >>> >> >> >>> where
> >>>>> >>> >>> >> >> >>> are they being used.
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and
> >>>>> >>> >>> >> >> >>> "SCAN_TIME_FACTOR"
> >>>>> >>> >>> >> >> >>> to
> >>>>> >>> >>> >> >> >>> the
> >>>>> >>> >>> >> >> >>> values returned by read_time() and scan_time() in
> >>>>> >>> >>> >> >> >>> handler.h,
> >>>>> >>> >>> >> >> >>> while
> >>>>> >>> >>> >> >> >>> returning. These values would be read from a table
> in
> >>>>> >>> >>> >> >> >>> mysql
> >>>>> >>> >>> >> >> >>> db.
> >>>>> >>> >>> >> >> >>> For
> >>>>> >>> >>> >> >> >>> that
> >>>>> >>> >>> >> >> >>> I've looked at sql_statistics.cc. After completing
> this,
> >>>>> >>> >>> >> >> >>> I'll
> >>>>> >>> >>> >> >> >>> first
> >>>>> >>> >>> >> >> >>> change
> >>>>> >>> >>> >> >> >>> the values of these constants manually and check
> if the
> >>>>> >>> >>> >> >> >>> better
> >>>>> >>> >>> >> >> >>> or
> >>>>> >>> >>> >> >> >>> worse
> >>>>> >>> >>> >> >> >>> query plans are being selected. I'll first do the
> last
> >>>>> >>> >>> >> >> >>> step
> >>>>> >>> >>> >> >> >>> manually,
> >>>>> >>> >>> >> >> >>> to
> >>>>> >>> >>> >> >> >>> check if everything is working as expected and
> later
> >>>>> >>> >>> >> >> >>> automate
> >>>>> >>> >>> >> >> >>> it.
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Regards
> >>>>> >>> >>> >> >> >>> Anshu
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash
> >>>>> >>> >>> >> >> >>> <anshu.avinash35@xxxxxxxxx> wrote:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Hi all,
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> You can find my blog entry for this week at
> >>>>> >>> >>> >> >> >>>
> http://igniting.in/gsoc2014/2014/05/11/first-steps/ .
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Regards
> >>>>> >>> >>> >> >> >>> Anshu Avinash
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash
> >>>>> >>> >>> >> >> >>> <anshu.avinash35@xxxxxxxxx> wrote:
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Hi all,
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> Sorry for the irregular updates. I had been busy
> for
> >>>>> >>> >>> >> >> >>> last
> >>>>> >>> >>> >> >> >>> couple
> >>>>> >>> >>> >> >> >>> of
> >>>>> >>> >>> >> >> >>> days
> >>>>> >>> >>> >> >> >>> and might still be busy for 1-2 days more. I would
> be
> >>>>> >>> >>> >> >> >>> completely
> >>>>> >>> >>> >> >> >>> free
> >>>>> >>> >>> >> >> >>> starting next week, and would be updating my blog
> weekly
> >>>>> >>> >>> >> >> >>> on
> >>>>> >>> >>> >> >> >>> every
> >>>>> >>> >>> >> >> >>> Monday (so
> >>>>> >>> >>> >> >> >>> 1st update would be on May 12). I would also send
> the
> >>>>> >>> >>> >> >> >>> link
> >>>>> >>> >>> >> >> >>> of my
> >>>>> >>> >>> >> >> >>> post
> >>>>> >>> >>> >> >> >>> weekly
> >>>>> >>> >>> >> >> >>> on the mailing list.
> >>>>> >>> >>> >> >> >>>
> >>>>> >>> >>> >> >> >>> As discussed on irc, I started to explore the pair
> of
> >>>>> >>> >>> >> >> >>> constants:
> >>>>> >>> >>> >> >> >>> handler::scan_time() and handler::read_time().
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >> --
> >>>>> >>> >>> >> >> >> Roberto Spadim
> >>>>> >>> >>> >> >> >> SPAEmpresarial
> >>>>> >>> >>> >> >> >> Eng. Automação e Controle
> >>>>> >>> >>> >> >> >>
> >>>>> >>> >>> >> >> >
> >>>>> >>> >>> >> >>
> >>>>> >>> >>> >> >>
> >>>>> >>> >>> >> >>
> >>>>> >>> >>> >> >> --
> >>>>> >>> >>> >> >> Roberto Spadim
> >>>>> >>> >>> >> >> SPAEmpresarial
> >>>>> >>> >>> >> >> Eng. Automação e Controle
> >>>>> >>> >>> >> >
> >>>>> >>> >>> >> >
> >>>>> >>> >>> >>
> >>>>> >>> >>> >>
> >>>>> >>> >>> >>
> >>>>> >>> >>> >> --
> >>>>> >>> >>> >> Roberto Spadim
> >>>>> >>> >>> >> SPAEmpresarial
> >>>>> >>> >>> >> Eng. Automação e Controle
> >>>>> >>> >>> >
> >>>>> >>> >>> >
> >>>>> >>> >>>
> >>>>> >>> >>>
> >>>>> >>> >>>
> >>>>> >>> >>> --
> >>>>> >>> >>> Roberto Spadim
> >>>>> >>> >>> SPAEmpresarial
> >>>>> >>> >>> Eng. Automação e Controle
> >>>>> >>> >>
> >>>>> >>> >>
> >>>>> >>> >
> >>>>> >>> >
> >>>>> >>> >
> >>>>> >>> > --
> >>>>> >>> > Roberto Spadim
> >>>>> >>> > SPAEmpresarial
> >>>>> >>> > Eng. Automação e Controle
> >>>>> >>>
> >>>>> >>>
> >>>>> >>>
> >>>>> >>> --
> >>>>> >>> Roberto Spadim
> >>>>> >>> SPAEmpresarial
> >>>>> >>> Eng. Automação e Controle
> >>>>> >>
> >>>>> >>
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > --
> >>>>> > Roberto Spadim
> >>>>> > SPAEmpresarial
> >>>>> > Eng. Automação e Controle
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Roberto Spadim
> >>>>> SPAEmpresarial
> >>>>> Eng. Automação e Controle
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Roberto Spadim
> >>> SPAEmpresarial
> >>> Eng. Automação e Controle
> >>
> >>
> >>
> >> --
> >> Roberto Spadim
> >> SPAEmpresarial
> >> Eng. Automação e Controle
> >
> >
> >
> > --
> > Roberto Spadim
> > SPAEmpresarial
> > Eng. Automação e Controle
>
>
>
> --
> Roberto Spadim
> SPAEmpresarial
> Eng. Automação e Controle
>