As you all know, we have to start the server with "-c" option in order to generate core dump during server crash. However if you want to know whether your server is already started to generate core dump, then here you go:
Start the PostgreSQL cluster with core dump option
.pg_ctl -c -D <Data directory location>
Here are the steps to find whether the server is started with -c option.
1. Check the procpid of the postmaster. Go to the location /proc/<procpid of postmaster>.
2. At this location, you can verify limits file to check whether server has started with "-c" option. If you do not have any limitations for enterprisedb user in "limits.conf" file at "/etc/security" location, then the "Soft Limit" of "Max core file size" in "limits" file will be "unlimited". The entry in limits file should be like below:
Limit Soft Limit Hard Limit Units
Max core file size unlimited unlimited bytes
If "Soft Limit" of "Max core file size" in "limits" file is "0", it indicates that your postmaster was not started with "-c" option.
Limit Soft Limit Hard Limit Units
Max core file size 0 unlimited bytes
If you have any limitations for all users or enterprisedb user in "limits.conf" file under "/etc/sercurity", then the above value for Soft Limit or Hard Limit will depends on value in "limits.conf" file.
In simple way, if your postmaster has started with "-c" option, then the "Soft Limit" of "Max core file size" in "limits" file should be some value or unlimited otherthan "0"(which depends on limits.conf file).
If we have started our postmaster with "-c" option, a core dump file will be generated under $PG_DATA directory during a backend crash. A core dump will be generated by the operating system, and we will be able to attach gdb to it to collect a stack trace or other information.
Here is one simple test case which shows us how core dump will be generated:
1. We have started the postmaster with "-c" option.
-bash-4.1$ /opt/PostgreSQL/9.3/bin/pg_ctl -D /opt/PostgreSQL/9.3/data -c start
server starting
2. Go to the location /proc/<procpid of postmaster> and check the limits file to know whether postmaster was started with "-c" option.
-bash-4.1$ ps -ef|grep PostgreSQL|grep 9.3
postgres 48892 1 0 14:28 pts/15 00:00:00 /opt/PostgreSQL/9.3/bin/postgres -D /opt/PostgreSQL/9.3/data
postgres 48980 46943 0 14:30 pts/15 00:00:00 grep PostgreSQL
-bash-4.1$
-bash-4.1$ cd /proc/48892/
-bash-4.1$ ls -ltrh
enterprisedb@ubuntu:/proc/6711$ ls -ltr limits
-r-------- 1 enterprisedb enterprisedb 0 2012-08-28 05:15 limits
-bash-4.1$ grep 'core' limits
Max core file size unlimited unlimited bytes
-bash-4.1$ cat limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size unlimited unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 1024 15777 processes
Max open files 1024 1024 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 15777 15777 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
3. For example, aborting any backend process will generate a core dump file under $PG_DATA directory.
-bash-4.1$ ps -ef|grep 48892
postgres 48892 1 0 14:28 pts/15 00:00:00 /opt/PostgreSQL/9.3/bin/postgres -D /opt/PostgreSQL/9.3/data
postgres 48893 48892 0 14:28 ? 00:00:00 postgres: logger process postgres 49090 48892 0 14:32 ? 00:00:00 postgres: checkpointer process postgres 49091 48892 0 14:32 ? 00:00:00 postgres: writer process postgres 49092 48892 0 14:32 ? 00:00:00 postgres: wal writer process
postgres 49093 48892 0 14:32 ? 00:00:00 postgres: autovacuum launcher process
postgres 49094 48892 0 14:32 ? 00:00:00 postgres: stats collector process4. We can attach gdb to core dump to collect a stack trace by using below command.
postgres 49114 46943 0 14:32 pts/15 00:00:00 grep 48892
-bash-4.1$
-bash-4.1$ kill -ABRT 49091
-bash-4.1$ ls -ltrh /opt/PostgreSQL/9.3/data/core*
-rw-------. 1 postgres postgres 143M Mar 8 14:33 /opt/PostgreSQL/9.3/data/core.49091
-bash-4.1$ gdb -q -c /opt/PostgreSQL/9.3/data/core.49091 /opt/PostgreSQL/9.3/bin/postgres
we should have debug symbols in our installer which was used for installation to get proper trace?, maybe, I'm poor at that, so better stay here.. ;-)