pbxt-discuss team mailing list archive

[Paul McCullagh <paul.mccullagh@xxxxxxxxxxxxx>]

Hi Michael,

Thanks for the info. Strange is, I would have expected the memory map  
to be able to grow closer to the size of the virtual memory space  
(4GB, I am guessing in your case).

We did improve the error handling, but, because other users have  
experienced similar problems with large tables, we have turned off  
memory mapping by default in the latest 1.0.08 revision.

Some work would be required before we can enable it again, but this is  
undecided at the moment.

On Jul 7, 2009, at 11:15 AM, Michael Han wrote:

> I was importing a dump of some tables with a couple of million rows  
> (5.9 million to be exact, got rid of the indices before loading). I  
> could basically watch the file grow to 2.5 gigs (which is the main  
> memory on the test server) and stall.
>
> In the logs I got error messages with PBXT (1.0.0.08) trying to  
> reallocate memory, failing and ending up in an endless loop.
>
> Feeling-wise it was because it tried to map the file, couldn't map  
> it into half main / half swap, and failed.
>
> I got rid of the logs, but I  can try it again, as i still have the  
> dumps and binaries flying around.

1.0.08 (RC2) should handle this without a problem, but I am doing a  
lot of work on disk I/O at the moment (in lp:~paul-mccullagh/pbxt/ 
staging).

So as soon as this is pushed to the trunk, it would be great if you  
could try the import again.

I will post a message to the list.

Thanks,

Paul

--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com