Debugging Session Creation

In this short blog post I’m going to explain how to attach gdb to a dedicated process during the early stage of the session creation on an Oracle database. The main challenge arises from the fact that the sqlplus or listener, depending on whether you’re connecting locally or remotely, respectively, forks a process which will eventually become the dedicated process for the session. Consequently, the available timeslot to attach to the process at a point of interest is rather short. So, we have to cease the forked process early enough in order to allow gdb to attach itself to the dedicated process. As a matter of fact, the suggested foot-in-the-door technique not only can be generally employed for inspecting all sorts of forked processes, but its usage could also be easily expanded to debugging short-livers of any kind.

As far as I know, there isn’t any possibility to follow a forked process with gdb. Therefore, I’m going to use DTrace to stop the forked process. For instance, the following script is going to pause the execution of a forked dedicated process for any bequeath connection done from sqlplus to the database whose name has been passed through the first argument.

#pragma D option quiet

proc:::exec
/ execname == "sqlplus" /
{
  self->parent = pid ;
}
  
proc:::exec-success
/ self->parent && 
  strtok(strtok(curpsinfo->pr_psargs," "),"oracle") == $1 /
{
  printf("DB: %s, pid: %d \n",
    strtok(strtok(curpsinfo->pr_psargs," "),"oracle"),
    pid
  );
  stop();
}

Obviously, the script is extremely intrusive, as it inherently prevents new sessions from logging into the database. Accordingly, it is not intended for production environments. You might also want to further limit its scope by putting additional conditions in proc:::exec provider, such as zonename and uid. In particular, this might be required when, for example, databases with the same name are running in different Solaris containers.

Finally, the script prints the database name and the pid of the forked dedicated process:

dtrace -w -s stop_forked.d '"DBX100"'
DB: DBX100, pid: 15771

Since the process has halted, you have all the time in the world now to attach to it, set some breakpoints and finally resume its execution for instance:

gdb
attach 15771
break kpoauth
continue