{"id":4250,"date":"2022-06-22T15:50:24","date_gmt":"2022-06-22T15:50:24","guid":{"rendered":"https:\/\/nenadnoveljic.com\/blog\/?p=4250"},"modified":"2022-06-22T15:50:27","modified_gmt":"2022-06-22T15:50:27","slug":"workaround-for-non-functioning-uretprobes-with-oracle-database-software","status":"publish","type":"post","link":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/","title":{"rendered":"Workaround for Non-Functioning uretprobes with Oracle Database Software"},"content":{"rendered":"<h1>uprobe<\/h1>\n<p>uprobes don&#8217;t work correctly with Oracle database software, for example:<\/p>\n<pre><code>bpftrace -e '<span style=\"color:red\">uprobe<\/span>:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:<span style=\"color:red\">kstmgetsectick<\/span>\n{\n  printf(\"HERE\\n\") ;\n}'<\/code><\/pre>\n<p>The <span style=\"color:red\">uprobe<\/span> on the function entry <span style=\"color:red\">kstmgetsectick<\/span>, one of the Oracle C functions, cannot be attached:<\/p>\n<pre><code>Attaching 1 probe...\n<span style=\"color:red\">cannot attach uprobe, Invalid argument<\/span>\nERROR: Error attaching probe: uprobe:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:kstmgetsectick<\/code><\/pre>\n<p><a href=\"https:\/\/twitter.com\/chris_skyflier\/status\/1537766809700638721\">chris_skyflier explained <\/a> that uprobes can&#8217;t deal with the byte sequence &#8220;0x66 0x90&#8221;, which Oracle inserted in each function&#8217;s prologue to facilitate hot patching. In other words, the Oracle patching feature broke uprobes.<\/p>\n<p><a href=\"https:\/\/mahmoudhatem.wordpress.com\/2017\/03\/21\/uprobes-issue-with-oracle-12c\/\">Hatem Mahmoud explained the workaround <\/a> &#8211; adding an offset to a uprobe to skip the problematic byte sequence:<\/p>\n<pre><code>bpftrace -e 'uprobe:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:kstmgetsectick<span style=\"color:blue\">+2<\/span>\n{\n  printf(\"HERE\\n\") ;\n}'\n\nAttaching 1 probe...\nHERE<\/code><\/pre>\n<h1>uretprobe<\/h1>\n<p>uretprobe, the probe that fires on the function exit, doesn&#8217;t work for the same reason:<\/p>\n<pre><code>bpftrace -e '<span style=\"color:red\">uretprobe<\/span>:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:kstmgetsectick\n{\n  printf(\"HERE\\n\") ;\n}'<\/code><\/pre>\n<pre><code>Attaching 1 probe...\n<span style=\"color:red\">cannot attach uprobe, Invalid argument<\/span>\nERROR: Error attaching probe: uretprobe:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:kstmgetsectick<\/code><\/pre>\n<p>Unluckily, the workaround with offset doesn&#8217;t function either because uretprobes don&#8217;t support offsets.<\/p>\n<pre><code>bpftrace -e 'uretprobe:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:kstmgetsectick<span style=\"color:red\">+2<\/span>\n{\n  printf(\"HERE\\n\") ;\n}'\n\nstdin:1:1-84: <span style=\"color:red\">ERROR: Offset not allowed<\/span>\nuretprobe:\/u00\/oracle\/orabase\/product\/21.6.0.0.220419_a\/bin\/oracle:kstmgetsectick+2<\/code><\/pre>\n<p>What other workaround can we use?<\/p>\n<h1>Workaround<\/h1>\n<p>We can configure uprobe on the program address where the function returns. But, first, we have to identify the address of the ret instruction (or its cousin retq). Since we have to disassemble the function, and the licence agreement with Oracle prohibits that, I&#8217;m going to demonstrate this technique on the following simple C program:<\/p>\n<pre><code>int main( int argc, char *argv[] ){\n    return 10 ;\n}<\/code><\/pre>\n<pre><code>cc ret.c -o ret<\/code><\/pre>\n<pre><code>gdb ret &lt;&lt;&lt; \"disas main\"<\/code><\/pre>\n<pre><code>0x0000000000400536 &lt;+0&gt;:     push   %rbp\n0x0000000000400537 &lt;+1&gt;:     mov    %rsp,%rbp\n0x000000000040053a &lt;+4&gt;:     mov    %edi,-0x4(%rbp)\n0x000000000040053d &lt;+7&gt;:     mov    %rsi,-0x10(%rbp)\n0x0000000000400541 &lt;+11&gt;:    <span style=\"color:brown\">mov    $0xa,%eax<\/span>\n0x0000000000400546 &lt;+16&gt;:    pop    %rbp\n<span style=\"color:blue\">0x0000000000400547 &lt;+17&gt;:    retq<\/span><\/code><\/pre>\n<p>The <span style=\"color:blue\">retq<\/span> instruction is at the address <span style=\"color:blue\">0x0000000000400547<\/span>, which is at the offset <span style=\"color:blue\">+17<\/span>. We can set uprobe with either the address or the offset:<\/p>\n<pre><code>bpftrace -e 'uprobe:.\/ret:main+<span style=\"color:blue\">17<\/span>\n{\n  printf(\"%d\\n\", <span style=\"color:brown\">reg(\"ax\")<\/span>) ;\n}'<\/code><\/pre>\n<pre><code>bpftrace -e 'uprobe:.\/ret:<span style=\"color:blue\">0x0000000000400547<\/span>\n{\n  printf(\"%d\\n\", <span style=\"color:brown\">reg(\"ax\")<\/span>) ;\n}'<\/code><\/pre>\n<p>Both variants produce the same result.<br \/>\nThe probe prints the function&#8217;s return value, which, according to the <a href=\"https:\/\/en.wikipedia.org\/wiki\/X86_calling_conventions\">x86 calling convention<\/a>, is stored in the <span style=\"color:brown\">eax<\/span> CPU register.<\/p>\n<pre><code>Attaching 1 probe...\n<span style=\"color:brown\">10<\/span><\/code><\/pre>\n<h1>Summary<\/h1>\n<p>In summary, uprobe and uretprobe don&#8217;t work with Oracle database software. The workaround for uprobe is adding an offset to the function to skip the byte sequence that Oracle inserted into a prologue of each function. Unfortunately, this doesn&#8217;t work for uretprobes because they don&#8217;t allow offsets. Alternatively, you can emulate uretprobe with an uprobe having an offset on the function call return(s). The return value can be extracted from the eax CPU register. To determine the address\/offset of the ret* calls, you&#8217;d need to disassemble the traced function. Keep in mind that Oracle Corp generally doesn&#8217;t allow disassembling.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known &#8211; adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround. <a href=\"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/\" class=\"more-link\">Continue Reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[53,5],"tags":[],"class_list":["post-4250","post","type-post","status-publish","format-standard","hentry","category-ebpf","category-oracle"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Workaround for Non-Functioning uretprobes with Oracle Database Software - All-round Database Topics<\/title>\n<meta name=\"description\" content=\"Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known - adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Workaround for Non-Functioning uretprobes with Oracle Database Software - All-round Database Topics\" \/>\n<meta property=\"og:description\" content=\"Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known - adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/\" \/>\n<meta property=\"og:site_name\" content=\"All-round Database Topics\" \/>\n<meta property=\"article:published_time\" content=\"2022-06-22T15:50:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-06-22T15:50:27+00:00\" \/>\n<meta name=\"author\" content=\"Nenad Noveljic\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@NenadNoveljic\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nenad Noveljic\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/\"},\"author\":{\"name\":\"Nenad Noveljic\",\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/#\\\/schema\\\/person\\\/51458d9dd86dbbdd19f5add451d44efa\"},\"headline\":\"Workaround for Non-Functioning uretprobes with Oracle Database Software\",\"datePublished\":\"2022-06-22T15:50:24+00:00\",\"dateModified\":\"2022-06-22T15:50:27+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/\"},\"wordCount\":343,\"commentCount\":0,\"articleSection\":[\"eBPF\",\"Oracle\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/\",\"url\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/\",\"name\":\"Workaround for Non-Functioning uretprobes with Oracle Database Software - All-round Database Topics\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/#website\"},\"datePublished\":\"2022-06-22T15:50:24+00:00\",\"dateModified\":\"2022-06-22T15:50:27+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/#\\\/schema\\\/person\\\/51458d9dd86dbbdd19f5add451d44efa\"},\"description\":\"Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known - adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Workaround for Non-Functioning uretprobes with Oracle Database Software\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/\",\"name\":\"All-round Database Topics\",\"description\":\"Nenad Noveljic\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/#\\\/schema\\\/person\\\/51458d9dd86dbbdd19f5add451d44efa\",\"name\":\"Nenad Noveljic\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a97b796613ea48ec8a7b79c8ffe1c685dcffc920c68121f6238d5caab5070670?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a97b796613ea48ec8a7b79c8ffe1c685dcffc920c68121f6238d5caab5070670?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a97b796613ea48ec8a7b79c8ffe1c685dcffc920c68121f6238d5caab5070670?s=96&d=mm&r=g\",\"caption\":\"Nenad Noveljic\"},\"sameAs\":[\"nenad-noveljic-9b746a6\",\"https:\\\/\\\/x.com\\\/NenadNoveljic\"],\"url\":\"https:\\\/\\\/nenadnoveljic.com\\\/blog\\\/author\\\/nenad\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Workaround for Non-Functioning uretprobes with Oracle Database Software - All-round Database Topics","description":"Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known - adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/","og_locale":"en_US","og_type":"article","og_title":"Workaround for Non-Functioning uretprobes with Oracle Database Software - All-round Database Topics","og_description":"Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known - adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround.","og_url":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/","og_site_name":"All-round Database Topics","article_published_time":"2022-06-22T15:50:24+00:00","article_modified_time":"2022-06-22T15:50:27+00:00","author":"Nenad Noveljic","twitter_card":"summary_large_image","twitter_creator":"@NenadNoveljic","twitter_misc":{"Written by":"Nenad Noveljic","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/#article","isPartOf":{"@id":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/"},"author":{"name":"Nenad Noveljic","@id":"https:\/\/nenadnoveljic.com\/blog\/#\/schema\/person\/51458d9dd86dbbdd19f5add451d44efa"},"headline":"Workaround for Non-Functioning uretprobes with Oracle Database Software","datePublished":"2022-06-22T15:50:24+00:00","dateModified":"2022-06-22T15:50:27+00:00","mainEntityOfPage":{"@id":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/"},"wordCount":343,"commentCount":0,"articleSection":["eBPF","Oracle"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/","url":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/","name":"Workaround for Non-Functioning uretprobes with Oracle Database Software - All-round Database Topics","isPartOf":{"@id":"https:\/\/nenadnoveljic.com\/blog\/#website"},"datePublished":"2022-06-22T15:50:24+00:00","dateModified":"2022-06-22T15:50:27+00:00","author":{"@id":"https:\/\/nenadnoveljic.com\/blog\/#\/schema\/person\/51458d9dd86dbbdd19f5add451d44efa"},"description":"Oracle database software breaks uprobes and uretprobes. The workaround for uprobes is well known - adding an offset to the function entry. This article explains how to emulate uretprobes with uprobes as a workaround.","breadcrumb":{"@id":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/nenadnoveljic.com\/blog\/workaround-for-non-functioning-uretprobes-with-oracle-database-software\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/nenadnoveljic.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Workaround for Non-Functioning uretprobes with Oracle Database Software"}]},{"@type":"WebSite","@id":"https:\/\/nenadnoveljic.com\/blog\/#website","url":"https:\/\/nenadnoveljic.com\/blog\/","name":"All-round Database Topics","description":"Nenad Noveljic","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/nenadnoveljic.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/nenadnoveljic.com\/blog\/#\/schema\/person\/51458d9dd86dbbdd19f5add451d44efa","name":"Nenad Noveljic","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/a97b796613ea48ec8a7b79c8ffe1c685dcffc920c68121f6238d5caab5070670?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/a97b796613ea48ec8a7b79c8ffe1c685dcffc920c68121f6238d5caab5070670?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a97b796613ea48ec8a7b79c8ffe1c685dcffc920c68121f6238d5caab5070670?s=96&d=mm&r=g","caption":"Nenad Noveljic"},"sameAs":["nenad-noveljic-9b746a6","https:\/\/x.com\/NenadNoveljic"],"url":"https:\/\/nenadnoveljic.com\/blog\/author\/nenad\/"}]}},"_links":{"self":[{"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/posts\/4250","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/comments?post=4250"}],"version-history":[{"count":1,"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/posts\/4250\/revisions"}],"predecessor-version":[{"id":4268,"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/posts\/4250\/revisions\/4268"}],"wp:attachment":[{"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/media?parent=4250"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/categories?post=4250"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nenadnoveljic.com\/blog\/wp-json\/wp\/v2\/tags?post=4250"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}