The alert logs of some of our databases have been cluttered with the “TNS-12599: TNS:cryptographic checksum mismatch” error messages.

We use the following configuration:

SQLNET.CRYPTO_CHECKSUM_SERVER  = required
SQLNET.ENCRYPTION_SERVER       = required
SQLNET.ENCRYPTION_TYPES_SERVER = (AES256)

A plethora of articles on Internet suggests downgrading encryption to a less strict mode to get rid of the error messages. But that would severely compromise the security – “required” is the only value that really enforces encryption. After all, you wouldn’t remove an ugly lock from your door just to improve the esthethics, would you?

In this article I’ll explain the circumstances under which this error occur and how to get rid of it without compromising security.

The error message doesn’t contain much useful information despite its verbosity:

NI cryptographic checksum mismatch error: 12599.

  VERSION INFORMATION:
        TNS for Solaris: Version 19.0.0.0.0 - Production
        Oracle Bequeath NT Protocol Adapter for Solaris: Version 19.0.0.0.0 - Production
        TCP/IP NT Protocol Adapter for Solaris: Version 19.0.0.0.0 - Production
  Version 19.9.0.0.0
  Time: 01-MAY-2021 13:01:05
  Tracing not turned on.
  Tns error struct:
    ns main err code: 12599

TNS-12599: TNS:cryptographic checksum mismatch
    ns secondary err code: 12656
    nt main err code: 0
    nt secondary err code: 0
    nt OS err code: 0

The error could be traced down to JDBC connections by correlating the timestamp with listener log.

Since I couldn’t reproduce the error just by connecting to a database, I gathered more information about the process with the following configuration:

alter system set events '12599 trace name errorstack forever, level 3';

An additional trace file for the dedicated server process was generated:

Errors in file /u00/oracle/orabase/diag/rdbms/db_site1/DB/trace/DB_ora_20019.trc:

Besides the process ID, the trace file didn’t contain any other useful information. The process was short-lived and impossible to observe.

Once more, I used the technique for enriching the log file with call stacks to get an idea about what could have led to the problem:

syscall::write:entry
/ strstr( fds[arg0].fi_pathname, $1 ) !=  0 /
{
  trace(copyinstr(arg1));
  ustack(100);
}

dtrace -s alertlog_writes.d '"alert_DB"'

ORA-12599: TNS:cryptographic checksum mismatch

libc.so.1`__write+0xa
oracle`sdbgrfuwf_write_file+0x64
oracle`sdbgrfwf_write_file+0x3c
oracle`dbglWriteTextOnly+0x65c
oracle`dbglWriteLogCommon+0x2cf
oracle`dbgrlWriteAlertDetail+0x25a
oracle`dbgrlWriteAlertText+0x26
oracle`dbkrlPril+0x245
oracle`kgedes+0xb9
oracle`dbkedDefDump+0x6fd
oracle`ksedmp+0x2b8
oracle`dbkdaKsdActDriver+0xd03
oracle`dbgdaExecuteAction+0x178
oracle`dbgdaRunAction+0x29b
oracle`dbgdRunActions+0x90
oracle`dbgdProcessEventActions+0x124
oracle`dbgdChkEventKgErr+0x925
oracle`dbkdChkEventRdbmsErr+0x4a
oracle`ksfpec+0x6f
oracle`dbgePostErrorKGE+0x495
oracle`dbkePostKGE_kgsf+0x3b
oracle`kgeade+0x447
oracle`kgeselv+0x67
oracle`ksesecl0+0xb7
oracle`opiino+0xf83
oracle`opiodr+0x439
oracle`opidrv+0x33f
oracle`sou2o+0x97
oracle`opimai_real+0x121
oracle`ssthrdmain+0x26b
oracle`main+0xa9
oracle`0x77e2b14

The error was generated in the Oracle C function ksesecl0, whose description is “kernel service error security”, see Frits Hoogland’s orafun.

This function handles authentication – I already saw it when analyzing an RMAN authentication problem .

I could indeed reproduce the TNS error by supplying wrong credentials to the following Java program:

import java.sql.DriverManager;
import java.sql.Connection;
import java.sql.SQLException;

public class ConnectDB {
    public static void main(String[] argv) {
        Connection connection = null ;
        try {
            Class.forName("oracle.jdbc.driver.OracleDriver");
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
            return;
        }

        try {
            connection = DriverManager.getConnection(
                "jdbc:oracle:thin:@ " + argv[0], argv[1], argv[2]);
        } catch (SQLException e) {
            e.printStackTrace();
            return;
        }
    }
}

Compile it with:

javac ConnectDB.java

Run it as follows:

java -cp .:./ojdbc7.jar ConnectDB server:port:db user password

Since the error doesn’t appear after a successful connection, we can conclude that the code that handles exception after a login error doesn’t implement the checksumming correctly.

The error is generated with OJDBC versions 6 and 7, but not with the OJDBC version 8 and fat client.

Some people have identified that Oracle Enterprise Manager connections have been causing these error messages. That’s because OEM 13.4 sadly uses the version 7 even though the deployed java version is 8 :

ps -ef | grep agent | grep java
oracle    3224  2765   0   Mar 02 ?        4843:07 /oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/jdk/bin/java -Xmx128M -XX:MaxMetaspaceSize=224M -server -d64 -Djava.security.egd=file:///dev/./urandom -Dsun.lang.ClassLoader.allowArraySyntax=true -XX:-DoEscapeAnalysis -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+UseCompressedOops -XX:-UseLWPSynchronization -Dsun.security.pkcs11.enable-solaris=false -Dwatchdog.pid=2765 -cp /oracle/product/agent13c/agent_13.4.0.0.0/jdbc/lib/ojdbc7.jar:/oracle/product/agent13c/agent_13.4.0.0.0/ucp/lib/ucp.jar:/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/modules/jsch-0.1.54.jar:/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/modules/com.oracle.http_client.http_client.jar:/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/modules/oracle.xdk/xmlparserv2.jar:/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/modules/oracle.dms/dms.jar:/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/modules/oracle.odl/ojdl.jar:/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/modules/oracle.odl/ojdl2.jar:/oracle/product/agent13c/agent_13.4.0.0.0/lib/optic.jar:/oracle/product/agent13c/agent_13.4.0.0.0/sysman/jlib/log4j-core.jar:/oracle/product/agent13c/agent_13.4.0.0.0/jlib/gcagent_core.jar:/oracle/product/agent13c/agent_13.4.0.0.0/sysman/jlib/emagentSDK-intg.jar:/oracle/product/agent13c/agent_13.4.0.0.0/sysman/jlib/emagentSDK.jar oracle.sysman.gcagent.tmmain.TMMain

/oracle/product/agent13c/agent_13.4.0.0.0/oracle_common/jdk/bin/java -version
java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 1.8.0_231-b34)
Java HotSpot(TM) 64-Bit Server VM (build 25.231-b34, mixed mode)

If your applications can’t upgrade to OJDBC 8, you can redirect these error messages to the SQLNet server log with the following configuration:

DIAG_ADR_ENABLED=off
LOG_FILE_SERVER=server.log
LOG_DIRECTORY_SERVER=/oracle/network/log

Summary

In summary, OJDBC versions below 8 don’t handle encryption correctly after a log on error. As a consequence, the TNS-12599 error message gets written in the alert log file. This error message is normally of no interest to DBAs – the client received the error message for the failed log on and should respond to it. Massive failed logons from JDBC clients can clutter the alert log.

Therefore, instruct the applications to upgrade to OJDBC 8. If not feasible, you can redirect those message to tne SQLNet server log.