← Back to team overview

drizzle-discuss team mailing list archive

Re: about replacing loadfile with filesystem storage engine


On Thu, 29 Jul 2010 14:24:30 -0700, Clint Byrum <clint@xxxxxxxxxx> wrote:
> I don't think many people are using LOAD DATA INFILE with direct file
> access on the servers. Thats just a really bad paradigm, especially
> since the code that does that in MySQL is really flaky (for instance,
> does not check for errors during read, so if your NFS mount where your
> source file hiccups, you just go "Oh, EOF, return" instead of "Hey
> user your data wasn't fully loaded!" LOAD DATA LOCAL INFILE is far
> more common, and has its own set of weirdness.

I think people aren't as well for all sorts of reasons too...

of course, our code should not be flaky.

One benefit of code reuse here is that we could do wacky things like
only have one place where we have to make sure error handling works :)

> I'd much rather see the LOAD DATA commands dropped in favor of the way
> postgresql does this, with the COPY command. With COPY, you can
> basically just say "COPY INTO table" and it uses the socket as a data
> stream of text to fill the table.

We should probably have something like that too.

> It would be awesome if this command can leverage the code that the
> filesystem_engine uses, but I don't know if it makes sense to have the
> engine itself be at the core.

It probably does, we're wanting things in the row format eventually
anyway, so having the same bit of code doing the conversion, not so bad.

Stewart Smith