uprobe

uprobes don’t work correctly with Oracle database software, for example:

bpftrace -e 'uprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick
{
  printf("HERE\n") ;
}'

The uprobe on the function entry kstmgetsectick, one of the Oracle C functions, cannot be attached:

Attaching 1 probe...
cannot attach uprobe, Invalid argument
ERROR: Error attaching probe: uprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick

chris_skyflier explained that uprobes can’t deal with the byte sequence “0x66 0x90”, which Oracle inserted in each function’s prologue to facilitate hot patching. In other words, the Oracle patching feature broke uprobes.

Hatem Mahmoud explained the workaround – adding an offset to a uprobe to skip the problematic byte sequence:

bpftrace -e 'uprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick+2
{
  printf("HERE\n") ;
}'

Attaching 1 probe...
HERE

uretprobe

uretprobe, the probe that fires on the function exit, doesn’t work for the same reason:

bpftrace -e 'uretprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick
{
  printf("HERE\n") ;
}'

Attaching 1 probe...
cannot attach uprobe, Invalid argument
ERROR: Error attaching probe: uretprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick

Unluckily, the workaround with offset doesn’t function either because uretprobes don’t support offsets.

bpftrace -e 'uretprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick+2
{
  printf("HERE\n") ;
}'

stdin:1:1-84: ERROR: Offset not allowed
uretprobe:/u00/oracle/orabase/product/21.6.0.0.220419_a/bin/oracle:kstmgetsectick+2

What other workaround can we use?

Workaround

We can configure uprobe on the program address where the function returns. But, first, we have to identify the address of the ret instruction (or its cousin retq). Since we have to disassemble the function, and the licence agreement with Oracle prohibits that, I’m going to demonstrate this technique on the following simple C program:

int main( int argc, char *argv[] ){
    return 10 ;
}

cc ret.c -o ret

gdb ret <<< "disas main"

0x0000000000400536 <+0>:     push   %rbp
0x0000000000400537 <+1>:     mov    %rsp,%rbp
0x000000000040053a <+4>:     mov    %edi,-0x4(%rbp)
0x000000000040053d <+7>:     mov    %rsi,-0x10(%rbp)
0x0000000000400541 <+11>:    mov    $0xa,%eax
0x0000000000400546 <+16>:    pop    %rbp
0x0000000000400547 <+17>:    retq

The retq instruction is at the address 0x0000000000400547, which is at the offset +17. We can set uprobe with either the address or the offset:

bpftrace -e 'uprobe:./ret:main+17
{
  printf("%d\n", reg("ax")) ;
}'

bpftrace -e 'uprobe:./ret:0x0000000000400547
{
  printf("%d\n", reg("ax")) ;
}'

Both variants produce the same result.
The probe prints the function’s return value, which, according to the x86 calling convention, is stored in the eax CPU register.

Attaching 1 probe...
10

Summary

In summary, uprobe and uretprobe don’t work with Oracle database software. The workaround for uprobe is adding an offset to the function to skip the byte sequence that Oracle inserted into a prologue of each function. Unfortunately, this doesn’t work for uretprobes because they don’t allow offsets. Alternatively, you can emulate uretprobe with an uprobe having an offset on the function call return(s). The return value can be extracted from the eax CPU register. To determine the address/offset of the ret* calls, you’d need to disassemble the traced function. Keep in mind that Oracle Corp generally doesn’t allow disassembling.