I was working on qmail (patching it heavily ) and at one point I was getting a segmentation fault. Trying to trace it in a busy environment is hard, especially in qmail that is very modular and processes come and go away all the time.
So I thought a core dump could be the answer.The only problem was my system wouldn't dump the core of the processes where the 'segmentation fault' occurred. I have set the core size file high enough ( ulimit -c 10000000 ) in the qmail start script but still not core dump so I decided to do some research.
From core man page it seems like a core could not be dumped for a number of reasons:
- the process owner doesn't have write permission to the directory where the core file should be written.
- the core file size limits ( RLIMIT_CORE , RLIMIT_FSIZE ) are less then the size of the core that would be dumped
- the process was setuid and in this case dumping the core would depend on the setting in /proc/sys/fs/suid_dumpable ( proc(5) )
- the file being executed doesn't have read permission
In my case #2 and #4 were not an issue so it must have been #1 and/or #3 .
To make sure #1 was not an issue I created a directory /tmp/cores, made it 777 to make sure any process can write to it ( not safe in a multiuser environment but works ) and then set /proc/sys/kernel/core_pattern to /tmp/cores/core so now all core files would be in /tmp/cores instead of the process working directory as it is by default. It's also a good idea to make sure /proc/sys/kernel/core_uses_pid is set to 1 so the pid of the process will be appended to the core file name.
To fix #3 I set /proc/sys/fs/suid_dumpable to 2 . I restarted qmail and there I had the core in /tmp/cores .