So, how it started, DB started throwing errors about "database with OID 1"
2014-02-07 22:54:46.561 GET >ERROR: database is not accepting commands to avoid wraparound data loss in database with OID 1
2014-02-07 22:54:46.561 GET >HINT: Stop the postmaster and use a standalone backend to vacuum that database.
You might also need to commit or roll back old prepared transactions.
OK, let's find DB with OID = 1:
backend> select oid,datname from pg_database where oid=1
1: oid (typeid = 26, len = 4, typmod = -1, byval = t)
2: datname (typeid = 19, len = 64, typmod = -1, byval = f)
----
backend>
result = nothing. I've checked all DB's I've got and all of them have unique OID, none with value of "1".
Configuration
Software:
CentOS 6 x86_64 (always fresh updates), postgresql 9.3.2 / RPM's from postgresql.org. fsync=on, autovacuum=on, rest is default (commented).
Block layers:
SAS Disks (write cache off) -> Dell Perc RAID (RAID 10, battery backed write cache) -> LVM LV for pgsql -> ext4 (defaults). No errors reported on block level by OS or HW RAID consistency check, no power failures (never occured).
DB with OID=1
By default, when running initdb, postgres creates database with template0 and template1. DB OID for template1 = 1, but in my case for some reason - it's not.
postgresql.conf
- in particular the autovacuum parameters and thefsync
parameter. Look in the log files for prior messages about wraparound and show any you find. Checkdmesg
for I/O errors, and the kernel logs. Describe any recent crashes, power losses or unexpected restarts (pg doesn't mind these IF your server's storage system works OK). – Craig Ringer Feb 9 at 23:47md
, filesystem type, filesystem mount options, etc). – Craig Ringer Feb 9 at 23:51-P -F i
? (see postgresql.org/docs/current/static/app-postgres.html) . Oh, and what's the output ofpg_controldata /path/to/database/directory
and ofselect oid, datname, datfrozenxid, datminmxid FROM pg_catalog.pg_database
? – Craig Ringer Feb 10 at 1:03