← Back to team overview

holland-discuss team mailing list archive

Re: MongoDB

 

Andy,
Thanks! FYI I'm still on this, just thinking about the best way to set it up, because mongodump has rather different modes of operation. I suppose they all need to be supported.

Normally it creates a directory "dump/" and underneath that creates a directory for each DB, and in that creates a BSON (binary JSON) file for each collection. So it does not normally dump things to stdout where we can pass it to the compression method of choice.

You *can* get that behavior, but only if you dump each collection (table) individually with a separate call to mongodump. I suppose that would be the default as it would avoid creating the dumps uncompressed and then compressing them afterwards.

However if you use the --oplog operation, which has guaranteed point-in-time consistency by using the oplog (binlog) then you *must* dump everything -- no DB exclusions at all. That will require the creation of the directory tree. That sucks for space reasons.

LVM is looking better and better here ...


On 01/12/2012 09:23 AM, Andrew Garner wrote:
That sounds good.

As far as mongo-lvm this can probably be handled by the generic
lvmsnapshot plugin (currently still sitting in my abg branch) in most
sane deployments where journalling is enabled - this works for
Postgres/MySQL+InnoDB (where myisam consistency isn't required) and
other transactionally stable software as well.  Otherwise it's a
pretty simple extension of the lvmsnapshot as is done with the
mysql-lvm plugin to support any other specific mongo actions
(fsync+lock,etc).

~Andy

On Thu, Jan 12, 2012 at 9:15 AM, Micah Yoder<micah.yoder@xxxxxxxxxxxxx>  wrote:
Just popping in here to note that I'm starting work on a MongoDB mongodump
driver.  We'll see how it goes.  Thought I'd check to see if anyone else is
working on one or has even thought about it before I get too far.

After that, a mongo-lvm plugin would make a good amount of sense.

I checked out the Amsterdam branch to get started.


_______________________________________________
Mailing list: https://launchpad.net/~holland-discuss
Post to     : holland-discuss@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~holland-discuss
More help   : https://help.launchpad.net/ListHelp




References