← Back to team overview

dhis2-users team mailing list archive

Re: Major speed issue with pg_restore

 

Agree, the system has two independent disks, I hadn’t set up any mirroring or striping – I had basically just setup the OS in both cases and performed OS updates.  Pretty much a vanilla config on both.  My experience is that if the SATA channel drivers aren’t properly loaded, the disks will work in legacy mode which slows things down considerably since no interrupts are used.  In the old days with the IDE channel, it was interrupt 14 and 15 for channel 1 and 2 but that’s no longer the case with SATA.  To be honest I haven’t come across this before in Linux since most good distros have all the software required to run all but the newest hardware and given that these machines are a few years old, I would assume they are supported (but I imagine this may be something worth troubleshooting).  Ultimately I do think online hosting for this project will probably be the best given the difficulty supporting any hardware in this location so we are exploring those options.

Some things I hadn’t tried yet:
https://serverfault.com/questions/790723/postgres-9-5-restoring-dumps-is-very-very-slow
“




It does not seem to be a PG problem. More likely it's a language or locale problem with PG on Ubuntu 16.04, because installing huge indexed tables with foreign languages and therefore with many utf-8 chars cause the problem. Do we miss a language package? On another server with Ubuntu 14.04 everything works fine.

“

It’s not this, since I had dumped using the default copy method:
http://manpages.ubuntu.com/manpages/xenial/man1/pg_dump.1.html

“--column-inserts
       --attribute-inserts
           Dump data as INSERT commands with explicit column names (INSERT
           INTO table (column, ...) VALUES ...). This will make restoration
           very slow; it is mainly useful for making dumps that can be loaded
           into non-PostgreSQL databases. However, since this option generates
           a separate command for each row, an error in reloading a row causes
           only that row to be lost rather than the entire table contents.

--inserts

           Dump data as INSERT commands (rather than COPY). This will make

           restoration very slow; it is mainly useful for making dumps that

           can be loaded into non-PostgreSQL databases. However, since this

           option generates a separate command for each row, an error in

           reloading a row causes only that row to be lost rather than the

           entire table contents. Note that the restore might fail altogether

           if you have rearranged column order. The --column-inserts option is

           safe against column order changes, though even slower.
“


References