maria-discuss team mailing list archive

[Xiaofei Du <xiaofei.du008@xxxxxxxxx>]

Laurynas,

We cannot recover from a torn page only using redo log. But wouldn't undo
log record enough information for recovery in the case of a torn page? Undo
log should have old values of affected rows. So shouldn't it be enough to
recover a torn page using information from undo log?

Xiaofei

On Sat, May 9, 2015 at 12:07 AM, Laurynas Biveinis <
laurynas.biveinis@xxxxxxxxx> wrote:

> Xiaofei -
>
> We can indeed detect the torn page write without the doublewrite
> buffer (and WebScaleSQL has a patch utilising this observation). But
> we need not only to detect, but to recover the page as well. And
> without the doublewrite, if we discard the page, we have nothing: a
> half-old half-new page on the disk and the redo log records for that
> page are not enough to recover it.
>
> 2015-05-09 8:44 GMT+03:00 Xiaofei Du <xiaofei.du008@xxxxxxxxx>:
> > Justin,
> >
> > I think the fsync I was concerning and the torn page problem are two
> > different things. But now I have a question about double write buffer.
> If we
> > can detect a torn page by checking the top and bottom of a page, why
> would
> > we still need double write buffer? If the page is consistent, then we use
> > it, otherwise, we just discard it. Maybe this is a naive question. But
> > please let me know. Thanks.
> >
> > Xiaofei
> >
> > On Fri, May 8, 2015 at 9:24 PM, Justin Swanhart <greenlion@xxxxxxxxx>
> wrote:
> >>
> >> Hi,
> >>
> >> The log does not have whole pages.  Pages must not be torn for the
> >> recovery process to work.  A fsync is required when a page is written to
> >> disk.  During recovery all changes since the last checkpoint are
> replayed,
> >> then transactions that do not have a commit marker are rolled back.
> This is
> >> called roll forward/roll back recovery.
> >>
> >> --Justin
> >>
> >> On Fri, May 8, 2015 at 6:09 PM, Xiaofei Du <xiaofei.du008@xxxxxxxxx>
> >> wrote:
> >>>
> >>> Justin,
> >>>
> >>> I was thinking of if fsync is needed each time after a write. The
> >>> operations are already in the log. So recovery can always be done from
> the
> >>> log. The difference is that during recovery, we need to go back
> further in
> >>> the log and it will take longer. But in that way, I guess it would be
> hard
> >>> to coordinate with the kernel flush thread.
> >>>
> >>> Xiaofei
> >>>
> >>> On Fri, May 8, 2015 at 2:06 PM, Justin Swanhart <greenlion@xxxxxxxxx>
> >>> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> InnoDB recovery can not handle torn pages.  An fsync is required to
> >>>> ensure that the page is fully written to disk.  This is also why the
> >>>> doublewrite buffer is used.  Before pages are written down to disk,
> they are
> >>>> first written sequentially into the doublewrite buffer.  This buffer
> is
> >>>> synced, then async page writing can proceed.  If the database
> crashes, the
> >>>> pages in flight will be rewritten by the doublewrite buffer.  The
> detection
> >>>> mechanism for torn pages comes from an LSN, which is written into the
> top
> >>>> and the bottom of the page.  If the LSN at the top and bottom do not
> match
> >>>> the page is torn.
> >>>>
> >>>> Regards,
> >>>>
> >>>> --Justin
> >>>>
> >>>> On Fri, May 8, 2015 at 12:43 PM, Xiaofei Du <xiaofei.du008@xxxxxxxxx>
> >>>> wrote:
> >>>>>
> >>>>> Laurynas,
> >>>>>
> >>>>> This is exactly what I was looking for. I went through these
> functions
> >>>>> before. I disabled double write buffer, so I didn't pay attention to
> code
> >>>>> under buf_dblwr... The reason I asked this question is because I
> didn't know
> >>>>> how the recovery process works, so I was wondering if it's necessary
> to
> >>>>> fsync after each write. It's a performance concern. Anyway, thank
> you very
> >>>>> much!
> >>>>>
> >>>>> Jan -- Thank you for your answer too!
> >>>>>
> >>>>> Xiaofei
> >>>>>
> >>>>> On Thu, May 7, 2015 at 9:59 PM, Laurynas Biveinis
> >>>>> <laurynas.biveinis@xxxxxxxxx> wrote:
> >>>>>>
> >>>>>> Xiaofei -
> >>>>>>
> >>>>>> fsync is performed for all the flush types (LRU, flush, single page)
> >>>>>> if it is asked for (innodb_flush_method != O_DIRECT_NO_FSYNC). The
> >>>>>> apparent difference in sync and async is not because of the sync
> >>>>>> difference itself, but because of the flush type difference. The
> >>>>>> single page flush flushes one page, and requests a fsync for its
> file.
> >>>>>> Other flushes flush in batches, don't have to fsync for each written
> >>>>>> page individually but rather sync once at the end. Then doublewrite
> >>>>>> complicates this further. If it is disabled, fsync will happen in
> >>>>>> buf_dblwr_sync_datafiles called from buf_dblwr_flush_buffered_writes
> >>>>>> called from buf_flush_common called at the end of either LRU or
> flush
> >>>>>> list flush. If doublewrite is enabled, fsync will happen in
> >>>>>> buf_dblwr_update called from buf_flush_write_complete.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> 2015-05-07 9:01 GMT+03:00 Xiaofei Du <xiaofei.du008@xxxxxxxxx>:
> >>>>>> > Hi Laurynas,
> >>>>>> >
> >>>>>> > On Wed, May 6, 2015 at 9:14 PM, Laurynas Biveinis
> >>>>>> > <laurynas.biveinis@xxxxxxxxx> wrote:
> >>>>>> >>
> >>>>>> >> Xiaofei -
> >>>>>> >>
> >>>>>> >> > Does InnoDB maintain a dirty
> >>>>>> >> > page table?
> >>>>>> >>
> >>>>>> >> You must be referring to the buffer pool flush_list.
> >>>>>> >
> >>>>>> >
> >>>>>> > You are right. The flush_list is can be used for recovery and
> >>>>>> > checkpoint.
> >>>>>> >
> >>>>>> >>
> >>>>>> >>
> >>>>>> >> > Is fsync called to guarantee the page to be on persistent
> >>>>>> >> > storage so that the dirty page table can be updated? If this is
> >>>>>> >> > the
> >>>>>> >> > case,
> >>>>>> >> > when is the dirty page table updated for asynchronous IOs?
> >>>>>> >>
> >>>>>> >> Check buf_flush_write_complete in buf0flu.cc. For async IO it is
> >>>>>> >> called from buf_page_io_complete in buf0buf.cc.
> >>>>>> >
> >>>>>> >
> >>>>>> > You are right that this is the place it updates the dirty page
> >>>>>> > information.
> >>>>>> > But I still don't understand why the fsync is needed for
> synchronous
> >>>>>> > IOs,
> >>>>>> > but not for the AIOs. Jan Lindstrom said fsync is also called for
> >>>>>> > other AIO
> >>>>>> > operations. But I could only it true in one of many AIO
> operations.
> >>>>>> > Or maybe
> >>>>>> > I am missing something still?
> >>>>>> >
> >>>>>> >>
> >>>>>> >>
> >>>>>> >> --
> >>>>>> >> Laurynas
> >>>>>> >
> >>>>>> >
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Laurynas
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Mailing list: https://launchpad.net/~maria-discuss
> >>>>> Post to     : maria-discuss@xxxxxxxxxxxxxxxxxxx
> >>>>> Unsubscribe : https://launchpad.net/~maria-discuss
> >>>>> More help   : https://help.launchpad.net/ListHelp
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
>
> --
> Laurynas
>