Debugging Session Creation

In this short blog post I’m going to explain how to attach gdb to a dedicated process during the early stage of the session creation on an Oracle database. The main challenge arises from the fact that the sqlplus or listener, depending on whether you’re connecting locally or remotely, respectively, forks a process which will eventually become the dedicated process for the session. Consequently, the available timeslot to attach to the process at a point of interest is rather short. So, we have to cease the forked process early enough in order to allow gdb to attach itself to the dedicated process. As a matter of fact, the suggested foot-in-the-door technique not only can be generally employed for inspecting all sorts of forked processes, but its usage could also be easily expanded to debugging short-livers of any kind.

As far as I know, there isn’t any possibility to follow a forked process with gdb. Therefore, I’m going to use DTrace to stop the forked process. For instance, the following script is going to pause the execution of a forked dedicated process for any bequeath connection done from sqlplus to the database whose name has been passed through the first argument.


#pragma D option quiet

proc:::exec
/ execname == "sqlplus" /
{
  self->parent = pid ;
}
  
proc:::exec-success
/ self->parent && 
  strtok(strtok(curpsinfo->pr_psargs," "),"oracle") == $1 /
{
  printf("DB: %s, pid: %d \n",
    strtok(strtok(curpsinfo->pr_psargs," "),"oracle"),
    pid
  );
  stop();
}

Obviously, the script is extremely intrusive, as it inherently prevents new sessions from logging into the database. Accordingly, it is not intended for production environments. You might also want to further limit its scope by putting additional conditions in proc:::exec provider, such as zonename and uid. In particular, this might be required when, for example, databases with the same name are running in different Solaris containers.

Finally, the script prints the database name and the pid of the forked dedicated process:

dtrace -w -s stop_forked.d '"DBX100"'
DB: DBX100, pid: 15771

Since the process has halted, you have all the time in the world now to attach to it, set some breakpoints and finally resume its execution for instance:

gdb
attach 15771
break kpoauth
continue
Thanks for sharing

Nenad Noveljic

4 Comments

  1. Hi ,
    Thanks for sharing 🙂
    In case of Linux we can use GDB with the following options for that :

    set follow-fork-mode child
    set detach-on-fork on
    symbol-file /oracle/home/bin/oracle

    And keep up with your greet blog !

    • Hi Hatem,

      Thank you for the kind words and the informaton. I’m glad to hear that you like the blog. Also, I’ve been learning a lot by studying your blog posts, especially the ones where you were using gdb for figuring out Oracle internals.

      With regard to following a child process with gdb, I couldn’t get it working on Solaris x86. As a matter of fact, it’s not the only gdb feature that can’t be used there. For example, reverse debugging doesn’t work too.

  2. Believe it or not, I actually got to this post because I was looking for kpoauth description. Can you elaborate on the Oracle functions involved in the authentication and connection establishment? (As far as you know, of course)..

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.