opatchauto failed with error code 2

Opatchauto:

In general, when we invoke opatchauto, opatch will patch both the GI stack and the database software stack. Since we do not have a database running, patch will skip the database software stack and only apply the PSU to the GI Home. Before we invoke the opatchauto command, let’s create the ocm.rsp response file by executing the OCM Installation Response Generator (emocmrsp).

Now we are going to patch the ORACLE_HOME,its automatically bounce the databases which are running under this ORACLE_HOME.We have to execute a command as a root user.
ERROR:



While applying 12c patch, it failed with below error

[root@test.doy.com] /oracle/app/oracle/product/12.1.0/dbrac_1/OPatch/opatchauto apply  /orashare1/OPatch/psu/gipsu/25901062 -oh /oracle/app/oracle/product/12.1.0/dbrac_1 -ocmrf /oracle/app/oracle/ocm.rsp

OPatch Automation Tool
Copyright (c) 2015, Oracle Corporation.  All rights reserved.

OPatchauto version : 12.1.0.1.5
OUI version        : 12.1.0.2.0
Running from       : /oracle/product/grid/12.1.0

opatchauto log file: /oracle/product/grid/12.1.0/cfgtoollogs/opatchauto/20132450/opatch_gi_2017-02-24_03-25-09_deploy.log

Parameter Validation: Successful

Grid Infrastructure home:
/oracle/product/grid/12.1.0
RAC home(s):
/oracle/product/db/12.1.0

Configuration Validation: Successful

Patch Location: /oracle/archivelog/patch/20132450
Grid Infrastructure Patch(es): 19769473 19769479 19769480 19872484
RAC Patch(es): 19877336 19769479 19769480

Patch Validation: Successful

Stopping RAC (/oracle/product/db/12.1.0) ... Successful

Applying patch(es) to "/oracle/product/db/12.1.0" ...
Command "/oracle/product/db/12.1.0/OPatch/opatch napply -phBaseFile /tmp/OraDB12Home1_patchList -local  -invPtrLoc /oracle/product/grid/12.1.0/oraInst.loc -oh /oracle/product/db/12.1.0 -silent -ocmrf /oracle/archivelog/patch/unconfig_GI.rsp" execution failed:
UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.

Log file Location for the failed command: /oracle/product/db/12.1.0/cfgtoollogs/opatch/opatch2017-02-24_03-26-10AM_1.log

For more details, please refer to the log file "/oracle/product/grid/12.1.0/cfgtoollogs/opatchauto/20132450/opatch_gi_2017-02-24_03-25-09_deploy.debug.log".

Apply Summary:

Following patch(es) failed to be installed:
GI Home: /oracle/product/grid/12.1.0: 19769473, 19769479, 19769480, 19872484
RAC Home: /oracle/product/db/12.1.0: 19877336, 19769479, 19769480

opatchauto failed with error code 2.


Checking the log it shows that

[Feb 24, 2017 3:26:17 AM]    Finish fuser command /sbin/fuser /oracle/product/db/12.1.0/lib/libclntsh.so.12.1 at Tue Feb 24 03:26:17 PST 2015
[Feb 24, 2017 3:26:17 AM]    Start fuser command /sbin/fuser /oracle/product/db/12.1.0/lib/libasmclntsh12.so at Tue Feb 24 03:26:17 PST 2015
[Feb 24, 2017 3:26:17 AM]    Finish fuser command /sbin/fuser /oracle/product/db/12.1.0/lib/libasmclntsh12.so at Tue Feb 24 03:26:17 PST 2015
[Feb 24, 2017 3:26:17 AM]    Following executables are active :
            7                /oracle/product/db/12.1.0/lib/libclntsh.so.12.1
[Feb 24, 2017 3:26:17 AM]    Prerequisite check "CheckActiveFilesAndExecutables" failed.
            7                The details are:
            7
            7
            7                Following executables are active :
            7                /oracle/product/db/12.1.0/lib/libclntsh.so.12.1
[Feb 24, 2017 3:26:17 AM]    OUI-67073:UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.
[Feb 24, 2017 3:26:17 AM]    Finishing UtilSession at Tue Feb 24 03:26:17 PST 2015
[Feb 24, 2017 3:26:17 AM]    Log file location: /oracle/product/db/12.1.0/cfgtoollogs/opatch/opatch2015-02-24_03-26-10AM_1.log
[Feb 24, 2017 3:26:17 AM]    Stack Description: java.lang.RuntimeException: Prerequisite check "CheckActiveFilesAndExecutables" failed.
[Feb 24, 2017 3:26:17 AM]    StackTrace: oracle.opatch.OPatchSessionHelper.runApplyPrereqs(OPatchSessionHelper.java:6516)
[Feb 24, 2017 3:26:17 AM]    StackTrace: oracle.opatch.opatchutil.NApply.process(NApply.java:1617)


Reason:

Someone is accessing Oracle library

Solution:

Using fuser, find who is accessing and kill it.

fuser  /oracle/app/oracle/product/12.1.0/dbrac_1/lib/libclntsh.so.12.1

kill -9 pid

[root@test.doy.com] fuser  /oracle/app/oracle/product/12.1.0/dbrac_1/lib/libclntsh.so.12.1
/oracle/app/oracle/product/12.1.0/dbrac_1/lib/libclntsh.so.12.1: 112053m

[root@test.doy.com] /oracle/home/oracle$ ps -ef|grep 112053
oracle   112053 111991  0 Feb23 pts/0    00:00:00 sqlplus

[root@test.doy.com] /oracle/home/oracle$ kill -9 112053

[root@test.doy.com] /oracle/app/oracle/product/12.1.0/dbrac_1/OPatch/opatchauto apply  /orashare1/OPatch/psu/gipsu/25901062 -oh /oracle/app/oracle/product/12.1.0/dbrac_1 -ocmrf /oracle/app/oracle/ocm.rsp