ISSUE:

While trying to apply a patch on a Fusion Middleware or WebLogic Server Oracle home, the following error occurs:

$ ./opatch lsinventory -jre /usr/java/jdk1.8.0_141/jre

OPatch Version : 13.9.0.0.0
OUI Version : 12.1.0.2.0
The OPatch version is not applicable for current OUI version.

Since OUI Version starts with 12.1, Please go to 'My Oracle Support' and get right OPatch 12.1.*.*.* based on patch 6880880, release version 12.1.0.0.0 and appropriate platform.

OPatch failed with error code 19


CAUSE:

As the error states, OUI and OPatch version do not match.

Upon further investigation, it is observed that the ORACLE_HOME is not set correctly and pulling a different OUI version from another environment.


SOLUTION:


1. Set ORACLE_HOME to the directory being patches. Also look at the PATH environment variable to ensure you don't have the OPatch directory from another location. The following should always be set before running opatch commands:

UNIX:
ORACLE_HOME=$MW_HOME/Oracle_FRHome1; export ORACLE_HOME
PATH=$ORACLE_HOME/OPatch:$PATH; export PATH

Windows:
set ORACLE_HOME=%MW_HOME%\OHS
set PATH=%ORACLE_HOME%\OPatch;%PATH%

2. Now you should be able to invoke the "opatch lsinventory" see the correct opatch version and intended Oracle home. Then run the "opatch apply" command to apply the patch.

Note: The error says, "get right OPatch 12.1.*.*.* based on patch 6880880", this is actually referring to an Oracle Database version. For Oracle Fusion Middleware, it is very rare to need to update OPatch or use Patch 6880880 to obtain a new one. Always use the one installed in the Oracle home or a backup if accidental overwritten with another version. Problems will occur if the incorrect Opatch is installed

Reference:

OPatch Failed with Error Code 19 - "The OPatch version is not applicable for current OUI version" (Doc ID 2364024.1)

ISSUE:

When using BNE Profile BNE: Enable OOXML Standard being set to Yes, Web ADI Performance for Create Document and Upload functionality improves as the performance does not depend on the Client PC desktop capacity, when being set. The performance for the Web ADI Upload and Create Document functionality is better when using the Middle Tier server capacity.

With this being said, if there are issues seen with the Web ADI Uploads related to Performance or Create Document taking time to build the Integrator, then you will need to look into Fine Tuning the Middle Tier's resources. If the Middle Tier is not sized correctly to handle the added functionality of the BNE Profile BNE: Enable OOXML Standard being set to Yes you can experience overall Web Server/Middle Tier Performance degradation when Server is under load.

This is mainly seen when uploading large amounts of Data in the Web ADI Integrator containing many Rows and even Columns and can occur during times of month end.

CHANGES:

Setting Profile BNE: Enable OOXML Standard = Yes

If the profile option, BNE: Enable OOXML Standard, would be set No, testing the same amount of rows and data would not be able to upload, the desktop Excel will probably hang. Then, the cause of the problem is not that BNE: Enable OOXML Standard is set to Yes. The cause would be related to the Middle Tiers needing to be fine tuned.

CAUSE:

Web ADI was never intended or designed to handle such a large upload. Web ADI was never envisioned to handle more then 20k, 30k or even 40k rows to be uploaded at a single time. However it can be done. In many cases Customer have to Benchmark test the Upload Functionality to see where there limitations are meet.

Test your Integrator with smaller amounts of rows. Try to keep this at or around 10k for testing purposes. Then slowing increase the number of rows added to the Integrator and test.
Increase the Rows until the thresh-hold is hit when the issues mentioned in the SR are hit.

There is no Benchmark testing for Uploading from Oracle as there are too many variables and each customer using this functionality in different methods to be able to provide accurate Benchmarks on the Upload Functionality.

If profile Option "BNE_OOXML_ENABLED" is Yes, larger memory is needed as explained below
https://docs.oracle.com/cd/E26401_01/doc.122/e22007/T443182T443186.htm
Line Region Properties

If your site is using the OOXML format, then the document creation and upload processing is performed on the server. In this case the processing is significantly faster than if you do not use OOXML, but requires a larger server heap memory. You should review the server heap memory setting to ensure it is appropriate for this configuration.

Beginning in Release 12.2.5 or ATG and Framework patching delivering bneprof.ldt 120.14.12020000.5 or higher, you can choose to create Web ADI templates according to the Office Open XML (OOXML) standard, an XML format developed by Microsoft to represent spreadsheets and other types of documents. When using the OOXML format, Oracle Web ADI generates the spreadsheet entirely on the Oracle E-Business Suite server, and the completed spreadsheet is then downloaded to the desktop.

SOLUTION:

Debug:

========



The Required Debug for Support in order to assist in determining if JVM Middle Tier Performance related or hit limitation of the number of Rows and Columns being updated for current sizing of Middle Tier.



1. Please provide the BNE Log File for this issue.

NOTE: Setting BNE to TRACE mode may help identifying the integrator and the operation (upload/download) BUT will degrade even more the performance. It is important quickly enable the BNE Trace reproduce the issue and immediately return BNE Log Level to Error.

1a. Clear existing BNE.log so that there is no old/unrelated data in it.

1b. From the EBS apps (System Administrator > Profile > System) Set the following System Profile Options to enable Debug:
Remember that the BNE Server log profiles only have effect at SITE LEVEL.

* BNE Server Log Filename : bne_perf_ooxml.log
* BNE Server Log Level : TRACE (This must be set at the Site level, it will not work at the other levels)
* BNE Server Log Path : free to choose, eg. in the $LOG_HOME/logs (use the full path, make sure that the applmgr has write permissions)

1c. Restart the Apache Server for changes to take effect.

1d. Have User follow the Steps to reproduce the problem.
1e. Upload the log file defined by profile BNE Server Log Filename for review .
1f. Afterwards, set the BNE Server Log Level to ERROR to avoid excessive growth of the log file. In production, this Profile should be set to ERROR unless debugging or diagnosing an issue.

2. At the same time of enabling BNE Logging, enable *JDBC logging* from the console for the WLS managed server.
2a. Provide the exact date and time the outage occurs with the external Application tier node.
Example:
- Login to the console
- Environment > Server > oacore_serverX (ie: oacore_server1)
- Click on Debug tab
- Expand weblogic
- Select jdbc
- Click Enable
- Activate Changes

2b. Enable *Servlet logging* from the console for managed server.

Example
- Login to the console
- Environment > Server > oacore_serverX (ie: oacore_server1)
- Click on Debug tab
- Expand weblogic
- Select servlet
- Click Enable
- Activate Changes

2c. Stop all application services.

2d. Backup/move the existing FMW/OHS log files for managed server X and
clean the logs (ie: move all current logging to a different backup
location so new logs will only exist after reproducing the issue).

Example for OACore managed server:
$EBS_DOMAIN_HOME/servers/oacore*/logs/*
$EBS_DOMAIN_HOME/servers/oacore_server*/adr/diag/ofm/EBS_domain_*/oac*/incident/*
$IAS_ORACLE_HOME/instances/*/diagnostics/logs/OHS/*/*
$IAS_ORACLE_HOME/instances/*/diagnostics/logs/OPMN/opmn/*

2e. Start the middle tier.

2f. Reproduce the issue while **noting the time stamp of the failure.

2g. Upload the following
$EBS_DOMAIN_HOME/servers/oacore*/logs/*
$EBS_DOMAIN_HOME/servers/oacore_server*/adr/diag/ofm/EBS_domain_*/oac*/incident/*
$IAS_ORACLE_HOME/instances/*/diagnostics/logs/OHS/*/*
$IAS_ORACLE_HOME/instances/*/diagnostics/logs/OPMN/opmn/*

3. Provide the value of the BNE Profiles by running the following SQL:
spool bne_profiles
set linesize 132
set pagesize 132
col NAME format A35
col LEVEL_SET format a10
col CONTEXT format a10
col VALUE format A20 wrap
col Server format a10
col resp format a8 wrap
col application format a10
break on NAME
select
p.profile_option_name NAME,
decode(v.level_id,
10001, 'Site',
10002, 'Application',
10003, 'Responsibility',
10004, 'User',
10005, 'Server',
'UnDef') LEVEL_SET,
decode(to_char(v.level_id),
'10001', '',
'10002', app.application_short_name,
'10003', rsp.responsibility_key,
'10005', svr.node_name,
'10006', org.name,
'10004', usr.user_name,
'UnDef') "CONTEXT",
v.profile_option_value VALUE
from
fnd_profile_options p,
fnd_profile_option_values v,
fnd_profile_options_tl n,
fnd_user usr,
fnd_application app,
fnd_responsibility rsp,
fnd_nodes svr,
hr_operating_units org
where
p.profile_option_id = v.profile_option_id (+) and
p.profile_option_name = n.profile_option_name and
n.user_profile_option_name like '%BNE%' and
usr.user_id (+) = v.level_value and
rsp.application_id (+) = v.level_value_application_id and
rsp.responsibility_id (+) = v.level_value and
app.application_id (+) = v.level_value and
svr.node_id (+) = v.level_value and
org.organization_id (+) = v.level_value
order by name, level_set
/

4. Please provide the out of the following SQL:
SELECT b.bug_number bug,
b.creation_date creation_date ,
b.last_update_date last_update_date ,
decode(bug_number,
17007206, 'R12.ATG_PF.C.delta.3',
17909318, 'R12.ATG_PF.C.delta.4',
19245366, 'R12.ATG_PF.C.delta.5',
21900895, 'R12.ATG_PF.C.delta.6',
24690680, 'R12.ATG_PF.C.delta.7',
23706874, 'BNE:Web Application Desktop Integrator Post 12.2.6 Consolidated Critical Fixes'
) Patch
FROM AD_BUGS b
WHERE b.BUG_NUMBER IN ('17007206','17909318','19245366','21900895','24690680','23706874')
ORDER BY patch;

5. Please provide the integrator having the issue. Is the Integrator having issues with the Create Document download or the Upload functionality having the issue?

6. Please compare the operation (download/upload) time BNE: Enable OOXML Standard = Yes vs. BNE: Enable OOXML Standard = No and provide the results.

7. Upload diagnostic Integrator Meatadata for the Integrator you are testing with. See How To Run Diagnostic Script Integrator Metadata Report For Custom Integrators? (Doc ID 1488669.1)

Run the Diagnostics for the Integrator Metadata while still under the Web Applications Desktop Integrator Diagnostics for the Integrator you are testing with.
7a. Expand the Web ADI Diagnostics region and choose the Test "Web ADI Integrator Metadata" and Click 'Execute'
7b. Add appropriate input parameters for Responsibility ID and the Integrator name and click on the Submit button.
7c. Check the results by refreshing until the diagnostics complete running.
Initially the report status page will show "In Progress".
Click on the Refresh button until the report(s) show up under the Success column.
7d. Once the report shows up in the Success column, click on the number 1 under the Success column which is a hyper link.
7e. Then click on the View Report green check mark on the right side of the page to open the report in your browser.
7f. Click on "File > Save as" to save the file to your desktop and then upload the report to the SR.

8.You can have issues with Temp Space used for Web ADI when creating the xls/xlsm file being passed to the Client PC.

NFS / NAS / SAN storage can cause performance issue when network storage is not mounted with nolock / llock option.
Below document explains the same.

Oracle E-Business suite do not directly support or certify any Network addressable Storage (NAS) (SAN and NFS fall under this terminology). There is a very good article on this...

Choosing a Shared File System for Oracle E-Business Suite

Frequently Asked Questions: Sharing the Application Tier File System in Oracle Applications 11i (Note 243880.1)
Sharing The Application Tier File System in Oracle E-Business Suite Release 12.0 and 12.1 (Doc ID 384248.1)

The problem comes around the physical differences between a local hard disk and using Network addressable Storage. For example, problems can arise with the increased amount of time that operating system file locks are held on certain files when they are held on a Shared File systems. These lock timings are a lot smaller for local disk so there is no issue. Also this issue might only ever arise on a system that is put under load, this may answer why other systems do not have the issue.

In the first instance I would contact the Shared File System vendor and ask them what settings they recommend for an E-Business suite web/forms tier. Most vendors (e.g. NetApp) have their own specific recommendations.

If the vendor has no preferences then a number of basic recommendations are made by Oracle to ensure that we do not hit any known issues. The recommendations for the Apps tier are

rw,intr,bg,hard,timeo=600,wsize=32768,rsize=32768,nfsvers=3,tcp,nolock,acregmin=0,acregmax=0

This means that you will need to change your 'Mount' options to match. If any of these mount options are not recognized then you will need to contact either the Shared File System vendor or the operating system vendor for more help.

The outstanding issues for using the E-Business suite on Shared File Systems are

- Include the 'nolock' option in the mount command.

On Linux, it should be using "nolock" option.

On Solaris, it should be using "llock" option.

- The Apache Lockfile MUST be stored on a local disk

You MUST update the $CONTEXT_FILE and change the variable "s_lock_pid_dir" to point to a local directory on the server. Once the update is done autoconfig must be ran on ALL web/forms tiers.

8a. Ensure that the following variables in the $CONTEXT_FILE are set to a local disk (*NON* NFS Mount Point) on EACH of your Application (web/forms) tiers.

- s_lock_pid_dir
- s_pids_dir
- s_web_pid_file

8b. Run autoconfig to make the changes

8c. Per Note 384248.1 verify that your NFS mounts have the "no lock" option set for all your NFS mounts.

For example:

NFS Parameter Recommendations
-----------------------------------------
For Linux Application Tier File System -
rw,intr,bg,hard,timeo=600,wsize=32768,rsize=32768,nfsvers=3,tcp,nolock,acregmin=0,acregmax=0

~~~~~~~~~~~~~~
Per Development:

If one is using any NAS implementation (NetApps, EMC, SGI, etc) then they MUST make sure to mount their NFS mounts with the
"nolock" option. This parameter is different for different OS's so have your customers refer to the "man pages" in regards to NFS.

For Linux:
mount -t nfs -o nolock nfs_server:/nfs_devices /nfs_mount

Typical Shared File System Logs are under the /var/log. For NFS, you should see a nfs.log. This will help in debugging .

9. Check the OACore_Server(x) JVM Memory settings and the number of OACores being used in the WLS Console for the JVM Start Options.

 Need the -XX:PermSize -XX:MaxPermSize -Xms and -Xmx settings.

The Defualt JVM Memory settings with EBS tend to not be sized correctly to handle Month End Loads with Web ADI.

An Xmx of 4G had been the original recommendation from Development, but they have entirely backed off from that now in favor of 2G. Garbage Collection starts when heap memory gets low and when there is 4G to clean that takes a lot of time and that JVM cannot do any EBS work when garbage collection is going on so therefore more JVMs with smaller heaps are better than fewer JVMs with large heaps.





Typical Solutions:

============



1. Increasing the number of OACores and Increasing the Memory Heap.

Use the following steps to customize the managed server configuration via the WebLogic Server Administration Console.

1a. Log on to the WebLogic Server Administration Console.
1b. Click on the 'Servers' link. This link takes you to a page containing a summary of the WebLogic Administration Server and all managed servers.
1c. Click on the managed server whose configuration needs to be updated. A page containing various tabs for the settings of the managed server appears. For eg. oacore_server1.
1d. Navigate to Configuration Tab > Server Start Tab > You can see the Arguments Section.
1e. Click on Lock and Edit button in the 'Change Center' panel.
1f. Update the Heap space to the desired value.

2. Using Profile Option BNE: Enable Upload Compression if you are not using BNE: Enable OOXML Standard as set to Yes. If you use the Profile Option BNE: Enable OOXML Standard = Yes, this uses Zip Functionality in the background to Zip and Upload the Data from the Integrator. You can see better Performance when using BNE: Enable OOXML Standard = Yes. However this comes at a cost of Performance issues on the Middle Tier.  This would be a situation were you set BNE: Enable OOXML Standard = No and use the BNE: Enable Upload Compression. This will put the Upload and Zip Functionality back on the Client PC's Resources rather then the Middle Tier. However Upload Times will be much Slower.

This were Benchmark Testing comes in to test what works best for your company needs.

With Release 12.2, you can use the Profile Option BNE: Enable Upload Compression for larger uploads.
When Upload Compression mode is enabled, Web ADI spreadsheet gives a new Menu Option: Oracle > Settings. Accessing this menu option, user defines the compressing utility and its executable.
Possible compressing utilities to use are: None, WinZip, and 7Zip.

3. Using smaller chunk of data for the Upload. Using the Benchmark Testing as mentioned above to Benchmark yourself.

Test with Seeded Integrator and Seeded Tables vs. Custom Integrator and Custom Tables being used.

Most Performance issues seen with BNE/Web ADI Upload is due to Custom Tables that the Interface Tables are trying to insert data to.

In this case Oracle Web ADI Support can only address the Performance with seeded Integrators and seeded Database Tables.

4. If BNE: Enable OOXML Standard = Yes is having Performance or Out of Memory issues related to Uploading/Downloading, test using Profile BNE: Enable OOXML Standard = No. This will return the functionality of the Integrator being created on the Client PC rather then the Server as described above. If the Client PC is still hanging with BNE: Enable OOXML Standard = No, ensure Patch 27953931:R12.BNE.C has been applied and retest with BNE: Enable OOXML Standard = Yes.

5. Check for BNE/Web ADI Connection Leaks. Currently there are known issues with BNE holding connections from the JDBC Connection Pool.

The following are the Known Patches to resolve the current Connection Leaks.

Patch 25232083:R12.BNE.C
Patch 22575948:R12.BNE.C
Patch 22575951:R12.BNE.C
Patch 20027711:R12.BNE.C

You can run the Connection Leak/Login Analyzer to get current BNE related Patching for Connection Leaks.

Please run the Login/Connection Leak Analyzer, E-Business Suite Applications Login Analyzer (Doc ID 2319360.1)
Upload the Results.

6. Check for Performance Patch 27953931:R12.BNE.C - 1OFF: LOW PERFORMANCE RENDERING INTEGRATOR LAYOUT WHEN BNE OOXML IS ON


REFERENCE:

R12:Web ADI Performance and Out of Memory issues seen when uploading large amounts of data (Doc ID 2506299.1)

ISSUE:

On Oracle Applications 12.1.3 version,
when attempt to register the target instance on a central instance,getting following error:

Message not found. Application: AZ, Message Name: oracle.apps.fnd.common.PoolException: Not able to create new database connection: FNDSECURITY_APPL_SERVER_ID.

The issue can be reproduced at will with the following steps:
1.Register the target instance on a central instance.

The issue has the following business impact:
Unable to register the target instance.

CAUSE:

The required correct DBC file was not in place / corrupted.

SOLUTION:

1. Please confirm whether the DBC (Database Connectivity) File created for instance that you want to map to.

2. Check the path $FND_SECURE. of central instance to check and confirm the DBC file of each remote instance.

3. Run Autoconfig to regenerate DBC file in the required instance and copy the file to the path $FND_SECURE.

4. Re-test.

REFERENCE:

Not Able To Create A New Database Connection: FNDSECURITY_APPL_SERVER_ID with Instance Mapping For iSetup (Doc ID 1627952.1)

There are times where we need to clear the space of the mount point when the database grows.
Sometimes, we run out of space and we may need to wait for storage team to allocate the space.

Below script will be life saving script, where you can shrink the datafiles and save space immediately.

Upon allocation of the storage, you may resize the datafiles again if needed.

Below query will give you the last added datafile in oracle.

This will be useful to check whenever you have datafile related issues.

DBA_HIST_DATAFILE is the view stores information of datafiles.

select a.FILE#,a.FILENAME,a.TSNAME,b.creation_time from DBA_HIST_DATAFILE a,V$datafile b where  a.FILE#=b.FILE# order by b.creation_time;

Sometime you wonder about the standard oracle views that where it gets information.  Also, you might be interested in getting base table information so that you can write your own sql queries.

This will help you to get the details.

set lines 300 pages 300
select VIEW_DEFINITION from v$fixed_view_definition where VIEW_NAME='&view_name';--V$log

Get the component id from below command

col component_name for a20;
col COMPONENT_STATUS for a20;
select COMPONENT_NAME, COMPONENT_STATUS,Component_Id from fnd_svc_components;

To start the services from back end - connect to apps user

declare
l_Component_Id number :=Component id; --Enter workflow Component_Id which needs to be started
l_errcode number;
l_errstr varchar2(4000);
begin
FND_SVC_COMPONENT.Start_Component(l_Component_Id, l_errcode, l_errstr);
commit;
end;
/

SET ECHO off
REM NAME:   TFSCLOCK.SQL
REM USAGE:"@path/tfsclock"
REM ------------------------------------------------------------------------
REM REQUIREMENTS:
REM    SELECT on V_$LOCK, V_$SESSION, SYS.USER$, SYS.OBJ$
REM ------------------------------------------------------------------------
REM PURPOSE:
REM    The following locking information script provides fully DECODED
REM    information regarding the locks currently held in the database.
REM    The report generated is fairly complex and difficult to read,
REM    but has considerable detail.
REM
REM    The TFTS series contains scripts to provide (less detailed) lock 
REM    information in a formats which are somewhat less difficult to read:
REM    TFSMLOCK.SQL and TFSLLOCK.SQL.
REM ------------------------------------------------------------------------
REM EXAMPLE:
REM    Too complex to show a representative sample here
REM 
REM ------------------------------------------------------------------------
REM DISCLAIMER:
REM    This script is provided for educational purposes only. It is NOT 
REM    supported by Oracle World Wide Technical Support.
REM    The script has been tested and appears to work as intended.
REM    You should always run new scripts on a test instance initially.
REM ------------------------------------------------------------------------
REM

set lines 200
set pagesize 66
break on Kill on sid on  username on terminal
column Kill heading 'Kill String' format a13
column res heading 'Resource Type' format 999
column id1 format 9999990
column id2 format 9999990
column locking heading 'Lock Held/Lock Requested' format a40
column lmode heading 'Lock Held' format a20
column request heading 'Lock Requested' format a20
column serial# format 99999
column username  format a10  heading "Username"
column terminal heading Term format a6
column tab format a30 heading "Table Name"
column owner format a9
column LAddr heading "ID1 - ID2" format a18
column Lockt heading "Lock Type" format a40
column command format a25
column sid format 990

select
nvl(S.USERNAME,'Internal') username,
        L.SID,
        nvl(S.TERMINAL,'None') terminal,
        decode(command,
0,'None',decode(l.id2,0,U1.NAME||'.'||substr(T1.NAME,1,20),'None')) tab,
decode(command,
0,'BACKGROUND',
1,'Create Table',
2,'INSERT',
3,'SELECT',
4,'CREATE CLUSTER',
5,'ALTER CLUSTER',
6,'UPDATE',
7,'DELETE',
8,'DROP',
9,'CREATE INDEX',
10,'DROP INDEX',
11,'ALTER INDEX',
12,'DROP TABLE',
13,'CREATE SEQUENCE',
14,'ALTER SEQUENCE',
15,'ALTER TABLE',
16,'DROP SEQUENCE',
17,'GRANT',
18,'REVOKE',
19,'CREATE SYNONYM',
20,'DROP SYNONYM',
21,'CREATE VIEW',
22,'DROP VIEW',
23,'VALIDATE INDEX',
24,'CREATE PROCEDURE',
25,'ALTER PROCEDURE',
26,'LOCK TABLE',
27,'NO OPERATION',
28,'RENAME',
29,'COMMENT',
30,'AUDIT',
31,'NOAUDIT',
32,'CREATE EXTERNAL DATABASE',
33,'DROP EXTERNAL DATABASE',
34,'CREATE DATABASE',
35,'ALTER DATABASE',
36,'CREATE ROLLBACK SEGMENT',
37,'ALTER ROLLBACK SEGMENT',
38,'DROP ROLLBACK SEGMENT',
39,'CREATE TABLESPACE',
40,'ALTER TABLESPACE',
41,'DROP TABLESPACE',
42,'ALTER SESSION',
43,'ALTER USER',
44,'COMMIT',
45,'ROLLBACK',
46,'SAVEPOINT',
47,'PL/SQL EXECUTE',
48,'SET TRANSACTION',
49,'ALTER SYSTEM SWITCH LOG',
50,'EXPLAIN',
51,'CREATE USER',
52,'CREATE ROLE',
53,'DROP USER',
54,'DROP ROLE',
55,'SET ROLE',
56,'CREATE SCHEMA',
57,'CREATE CONTROL FILE',
58,'ALTER TRACING',
59,'CREATE TRIGGER',
60,'ALTER TRIGGER',
61,'DROP TRIGGER',
62,'ANALYZE TABLE',
63,'ANALYZE INDEX',
64,'ANALYZE CLUSTER',
65,'CREATE PROFILE',
66,'DROP PROFILE',
67,'ALTER PROFILE',
68,'DROP PROCEDURE',
69,'DROP PROCEDURE',
70,'ALTER RESOURCE COST',
71,'CREATE SNAPSHOT LOG',
72,'ALTER SNAPSHOT LOG',
73,'DROP SNAPSHOT LOG',
74,'CREATE SNAPSHOT',
75,'ALTER SNAPSHOT',
76,'DROP SNAPSHOT',
79,'ALTER ROLE',
85,'TRUNCATE TABLE',
86,'TRUNCATE CLUSTER',
87,'-',
88,'ALTER VIEW',
89,'-',
90,'-',
91,'CREATE FUNCTION',
92,'ALTER FUNCTION',
93,'DROP FUNCTION',
94,'CREATE PACKAGE',
95,'ALTER PACKAGE',
96,'DROP PACKAGE',
97,'CREATE PACKAGE BODY',
98,'ALTER PACKAGE BODY',
99,'DROP PACKAGE BODY',
command||' - ???') COMMAND,
        decode(L.LMODE,1,'No Lock',
                2,'Row Share',
                3,'Row Exclusive',
                4,'Share',
                5,'Share Row Exclusive',
                6,'Exclusive','NONE') lmode,
        decode(L.REQUEST,1,'No Lock',
                2,'Row Share',
                3,'Row Exclusive',
                4,'Share',
                5,'Share Row Exclusive',
                6,'Exclusive','NONE') request,
l.id1||'-'||l.id2 Laddr,
l.type||' - '||
decode(l.type,
'BL','Buffer hash table instance lock',
'CF',' Control file schema global enqueue lock',
'CI','Cross-instance function invocation instance lock',
'CS','Control file schema global enqueue lock',
'CU','Cursor bind lock',
'DF','Data file instance lock',
'DL','Direct loader parallel index create',
'DM','Mount/startup db primary/secondary instance lock',
'DR','Distributed recovery process lock',
'DX','Distributed transaction entry lock',
'FI','SGA open-file information lock',
'FS','File set lock',
'HW','Space management operations on a specific segment lock',
'IN','Instance number lock',
'IR','Instance recovery serialization global enqueue lock',
'IS','Instance state lock',
'IV','Library cache invalidation instance lock',
'JQ','Job queue lock',
'KK','Thread kick lock',
'MB','Master buffer hash table instance lock',
'MM','Mount definition gloabal enqueue lock',
'MR','Media recovery lock',
'PF','Password file lock',
'PI','Parallel operation lock',
'PR','Process startup lock',
'PS','Parallel operation lock',
'RE','USE_ROW_ENQUEUE enforcement lock',
'RT','Redo thread global enqueue lock',
'RW','Row wait enqueue lock',
'SC','System commit number instance lock',
'SH','System commit number high water mark enqueue lock',
'SM','SMON lock',
'SN','Sequence number instance lock',
'SQ','Sequence number enqueue lock',
'SS','Sort segment lock',
'ST','Space transaction enqueue lock',
'SV','Sequence number value lock',
'TA','Generic enqueue lock',
'TD','DDL enqueue lock',
'TE','Extend-segment enqueue lock',
'TM','DML enqueue lock',
'TO','Temporary Table Object Enqueue',
'TT','Temporary table enqueue lock',
'TX','Transaction enqueue lock',
'UL','User supplied lock',
'UN','User name lock',
'US','Undo segment DDL lock',
'WL','Being-written redo log instance lock',
'WS','Write-atomic-log-switch global enqueue lock',
'TS',decode(l.id2,0,'Temporary segment enqueue lock (ID2=0)',
                    'New block allocation enqueue lock (ID2=1)'),
'LA','Library cache lock instance lock (A=namespace)',
'LB','Library cache lock instance lock (B=namespace)',
'LC','Library cache lock instance lock (C=namespace)',
'LD','Library cache lock instance lock (D=namespace)',
'LE','Library cache lock instance lock (E=namespace)',
'LF','Library cache lock instance lock (F=namespace)',
'LG','Library cache lock instance lock (G=namespace)',
'LH','Library cache lock instance lock (H=namespace)',
'LI','Library cache lock instance lock (I=namespace)',
'LJ','Library cache lock instance lock (J=namespace)',
'LK','Library cache lock instance lock (K=namespace)',
'LL','Library cache lock instance lock (L=namespace)',
'LM','Library cache lock instance lock (M=namespace)',
'LN','Library cache lock instance lock (N=namespace)',
'LO','Library cache lock instance lock (O=namespace)',
'LP','Library cache lock instance lock (P=namespace)',
'LS','Log start/log switch enqueue lock',
'PA','Library cache pin instance lock (A=namespace)',
'PB','Library cache pin instance lock (B=namespace)',
'PC','Library cache pin instance lock (C=namespace)',
'PD','Library cache pin instance lock (D=namespace)',
'PE','Library cache pin instance lock (E=namespace)',
'PF','Library cache pin instance lock (F=namespace)',
'PG','Library cache pin instance lock (G=namespace)',
'PH','Library cache pin instance lock (H=namespace)',
'PI','Library cache pin instance lock (I=namespace)',
'PJ','Library cache pin instance lock (J=namespace)',
'PL','Library cache pin instance lock (K=namespace)',
'PK','Library cache pin instance lock (L=namespace)',
'PM','Library cache pin instance lock (M=namespace)',
'PN','Library cache pin instance lock (N=namespace)',
'PO','Library cache pin instance lock (O=namespace)',
'PP','Library cache pin instance lock (P=namespace)',
'PQ','Library cache pin instance lock (Q=namespace)',
'PR','Library cache pin instance lock (R=namespace)',
'PS','Library cache pin instance lock (S=namespace)',
'PT','Library cache pin instance lock (T=namespace)',
'PU','Library cache pin instance lock (U=namespace)',
'PV','Library cache pin instance lock (V=namespace)',
'PW','Library cache pin instance lock (W=namespace)',
'PX','Library cache pin instance lock (X=namespace)',
'PY','Library cache pin instance lock (Y=namespace)',
'PZ','Library cache pin instance lock (Z=namespace)',
'QA','Row cache instance lock (A=cache)',
'QB','Row cache instance lock (B=cache)',
'QC','Row cache instance lock (C=cache)',
'QD','Row cache instance lock (D=cache)',
'QE','Row cache instance lock (E=cache)',
'QF','Row cache instance lock (F=cache)',
'QG','Row cache instance lock (G=cache)',
'QH','Row cache instance lock (H=cache)',
'QI','Row cache instance lock (I=cache)',
'QJ','Row cache instance lock (J=cache)',
'QL','Row cache instance lock (K=cache)',
'QK','Row cache instance lock (L=cache)',
'QM','Row cache instance lock (M=cache)',
'QN','Row cache instance lock (N=cache)',
'QO','Row cache instance lock (O=cache)',
'QP','Row cache instance lock (P=cache)',
'QQ','Row cache instance lock (Q=cache)',
'QR','Row cache instance lock (R=cache)',
'QS','Row cache instance lock (S=cache)',
'QT','Row cache instance lock (T=cache)',
'QU','Row cache instance lock (U=cache)',
'QV','Row cache instance lock (V=cache)',
'QW','Row cache instance lock (W=cache)',
'QX','Row cache instance lock (X=cache)',
'QY','Row cache instance lock (Y=cache)',
'QZ','Row cache instance lock (Z=cache)','????') Lockt
from    V$LOCK L, 
        V$SESSION S,
        SYS.USER$ U1,
        SYS.OBJ$ T1
where   L.SID = S.SID 
and     T1.OBJ#  = decode(L.ID2,0,L.ID1,1) 
and     U1.USER# = T1.OWNER#
and     S.TYPE != 'BACKGROUND'
order by 1,2,5
/

-- Check what the sessions in our instance are waiting for
select event,count(*) from v$session group by event order by count(*);

-- Flexible query to check what's currently running in the system
-- Where statement and column lists can be modified by the case
-- Written for RAC DBs
select
s.inst_id,
--      'alter system kill session '''|| s.SID||',' || s.serial# ||'''' ,
--'!kill -9 ' || p.spid,
      p.SPID UnixProcess ,s.SID,s.serial#,s.USERNAME,s.COMMAND,s.MACHINE,s.blocking_session
      ,s.program, status,state,event,s.sql_id,sql_text,COMMAND_TYPE
--    ,sbc.name,to_char(sbc.last_captured,'yyyy-mm-dd hh24:mi:ss'),sbc.value_string
    from gv$session s
left outer join gv$process p on p.ADDR = s.PADDR and s.inst_id=p.inst_id
left outer join gv$sqlarea sa on sa.ADDRESS = s.SQL_ADDRESS and s.inst_id=sa.inst_id
--left outer join gV$SQL_BIND_CAPTURE sbc on sbc.ADDRESS = s.SQL_ADDRESS and s.inst_id=p.inst_id
where 1=1 and sql_text like '...'

-- Check what a specific session is doing:
select
      p.SPID UnixProcess ,s.SID,s.serial#,s.USERNAME,s.COMMAND,s.MACHINE,s.SQL_ADDRESS,s.SQL_HASH_VALUE
      ,s.program, status,sql_text,COMMAND_TYPE
    from gv$session s,gv$process p, gv$sqlarea sa
where p.ADDR = s.PADDR and s.inst_id=p.inst_id
and sa.ADDRESS = s.SQL_ADDRESS and s.inst_id=sa.inst_id
and s.sid=1722;

-- Find all sessions that are blocked and which session is blocking them
-- Then find what the blocking session is doing
select sid,blocking_session,username,sql_id,event,machine,osuser,program from v$session where blocking_session > 0;
select sid,blocking_session,username,sql_id,event,machine,osuser,program from v$session where sid=491;


-- generate commands to kill all sessions from a specific user on specific instance
select 'alter system kill session '''|| SID||',' || serial# ||''' immediate;' from gv$session where username='BAD_USER' and inst_id=1;


-- Kill all sessions waiting for specific events by a specific user
select       'alter system kill session '''|| s.SID||',' || s.serial# ||''';'
from gv$session s
where 1=1
and (event='latch: shared pool' or event='library cache lock') and s.USERNAME='DBSNMP';

-- kill all sessions executing a bad SQL
select
     'alter system kill session '''|| s.SID||',' || s.serial# ||''';'
    from v$session s
where s.sql_id='0vj44a7drw1rj';


-- Sessions taking most PGA memory
-- Can be used to find leaks
select addr,SPID,username,program,pga_alloc_mem/1024 mem_alloc_Kb from v$process order by pga_alloc_mem;

-- Check what is the top SQL executed by parallel slaves
select sql_id,count(*) from v$session where program like '%P0%' group by sql_id;



-- Find inactive sessions
-- This can be used to decide which sessions to kill if the DB is running out of processes
select sid, blocking_session,username,program,machine,osuser,  sql_id, prev_sql_id, event,LAST_CALL_ET from v$session where status != 'ACTIVE' and last_call_et>3600;


-- How many sessions openned by each app server
select machine,count(*) from gv$session s group by machine;

-- Find sql_id for a specific sql snippet
select sql_id,sql_text from v$sql where dbms_lob.instr(sql_text, 'create INDEX',1,1) > 0

-- Find SQL with too many child cursors:
select version_count,sql_text from v$sqlarea order by version_count desc



-- Get the longops status for a specific session
select sid
,      message, start_time,time_remaining
from   v$session_longops
where  sid = 28
order by start_time;

-- Check status for long ops executing right now
  select s.sid,s.serial#,opname, target, program,sofar, totalwork,units, elapsed_seconds, message,start_time,time_remaining 
from v$session_longops l
join v$session s on l.sid=s.sid and s.serial#=l.serial#
where time_remaining>0
order by start_time desc




-- Find out how much memory each session is using
COLUMN sid                     FORMAT 999            HEADING 'SID'
COLUMN oracle_username         FORMAT a12            HEADING 'Oracle User'     JUSTIFY right
COLUMN os_username             FORMAT a9             HEADING 'O/S User'        JUSTIFY right
COLUMN session_program         FORMAT a18            HEADING 'Session Program' TRUNC
COLUMN session_module         FORMAT a18            HEADING 'Session module' TRUNC
COLUMN session_action         FORMAT a18            HEADING 'Session action' TRUNC
COLUMN session_machine         FORMAT a8             HEADING 'Machine'   JUSTIFY right TRUNC
COLUMN session_pga_memory      FORMAT 9,999,999,999  HEADING 'PGA Memory'
COLUMN session_pga_memory_max  FORMAT 9,999,999,999  HEADING 'PGA Memory Max'
COLUMN session_uga_memory      FORMAT 9,999,999,999  HEADING 'UGA Memory'
COLUMN session_uga_memory_max  FORMAT 9,999,999,999  HEADING 'UGA Memory MAX'
COLUMN session_total_memory  FORMAT 9,999,999,999  HEADING 'Total Memory'

select sid,oracle_username,os_username,session_program,session_module,session_action,
session_pga_memory,session_pga_memory_max,session_uga_memory,session_uga_memory_max,session_pga_memory+session_uga_memory session_total_memory from (
SELECT
    s.sid                sid
  , lpad(s.username,12)  oracle_username
  , lpad(s.osuser,9)     os_username
  , s.program            session_program
  , s.module            session_module
  , s.action            session_action
  , lpad(s.machine,8)    session_machine
  , (select ss.value from v$sesstat ss, v$statname sn
     where ss.sid = s.sid and
           sn.statistic# = ss.statistic# and
           sn.name = 'session pga memory')        session_pga_memory
  , (select ss.value from v$sesstat ss, v$statname sn
     where ss.sid = s.sid and
           sn.statistic# = ss.statistic# and
           sn.name = 'session pga memory max')    session_pga_memory_max
  , (select ss.value  from v$sesstat ss, v$statname sn
     where ss.sid = s.sid and
           sn.statistic# = ss.statistic# and
           sn.name = 'session uga memory')        session_uga_memory
  , (select ss.value from v$sesstat ss, v$statname sn
     where ss.sid = s.sid and
           sn.statistic# = ss.statistic# and
           sn.name = 'session uga memory max')   as session_uga_memory_max
FROM
    v$session  s )
ORDER BY session_total_memory DESC;

-- Find segments with most read operations, and hopefully relatively few writes
-- These are good candidates for placing on SSD storage
WITH segment_stats
          AS (SELECT ss.owner || '.' || ss.object_name
                    || DECODE (ss.subobject_name,NULL, '',
                    '(' || ss.subobject_name || ')')  segment_name,
                    ss.object_type,
                    SUM ( CASE WHEN statistic_name LIKE 'physical reads%'
                            THEN VALUE ELSE 0  END) reads,
                    SUM ( CASE WHEN statistic_name LIKE 'physical writes%'
                            THEN VALUE  ELSE 0 END) writes,
                    ROUND (SUM (bytes) / 1048576) mb
              FROM   v$segment_statistics ss
              JOIN   dba_segments s
               ON (s.owner = ss.owner AND s.segment_name = ss.object_name
                   AND NVL (ss.subobject_name, 'x') =NVL (s.partition_name, 'x'))
             WHERE statistic_name LIKE 'physical reads%'
                OR statistic_name LIKE 'physical writes%'
             GROUP BY ss.owner,ss.object_name,ss.subobject_name,ss.object_type)
 SELECT segment_name, object_type, reads,writes,
       ROUND (reads * 100 / SUM (reads) OVER (), 2) pct_reads,
       ROUND (writes * 100 / SUM (writes) OVER (), 2) pct_writes,
       mb FROM segment_stats
 ORDER BY reads DESC

 -- find segments with full scans
 -- These are not ideal for placing on SSD storage
col segment_name format a60
select
        owner || '.' || object_name || DECODE (subobject_name,NULL, '','(' || subobject_name || ')') segment_name,object_type,
        value scans
from
        V$segment_statistics
where
        statistic_name = 'segment scans'
and     value != 0
order by
value desc
;

During tuning a query, I found one table has stale statistics. While running gather stats for that table, got below error. Let me demonstrate with a demo table:

SQL> execute dbms_stats.gather_table_stats(ownname => 'HR', tabname => 'EMPLOYEE', estimate_percent => DBMS_STATS.AUTO_SAMPLE_SIZE, method_opt => 'FOR ALL COLUMNS SIZE AUTO');

BEGIN dbms_stats.gather_table_stats(ownname => 'HR', tabname => 'EMPLOYEE', estimate_percent => DBMS_STATS.AUTO_SAMPLE_SIZE, method_opt => 'FOR ALL COLUMNS SIZE AUTO'); END;

*
ERROR at line 1:
ORA-20005: object statistics are locked (stattype = ALL)
ORA-06512: at "SYS.DBMS_STATS", line 34634
ORA-06512: at line 1


SQL> 

Analysis:

Verify whether statistics is locked for that table or not.


SQL> col OWNER for a12
SQL> col table_name for a30
SQL> col STATTYPE_LOCKED for a15
SQL> select owner,table_name,STATTYPE_LOCKED from dba_tab_statistics where table_name='EMPLOYEE' and owner='HR';

OWNER        TABLE_NAME                     STATTYPE_LOCKED
------------ ------------------------------ ---------------
HR           EMPLOYEE                           ALL

SQL> 

So, the column "stattype_locked" value is showing "ALL", that means statistics is locked for that table.

Solution Description:

The table statistics can be locked from the further update by using dbms_stats package. This will help to restrict from automatic statistics updation. When you get this error you can check whether your table is locked on further statistics updation using following query. The stattype_locked should be null to update the statistics.

STATTYPE_LOCKED – ALL -> stats are locked for the table
STATTYPE_LOCKED – NULL – > stats are not locked

To Unlock stats:

SQL> EXEC DBMS_STATS.unlock_table_stats('HR','EMPLOYEE');

PL/SQL procedure successfully completed.

SQL> 


Now try to run stats again:


SQL> execute dbms_stats.gather_table_stats(ownname => 'HR', tabname => 'EMPLOYEE', estimate_percent => DBMS_STATS.AUTO_SAMPLE_SIZE, method_opt => 'FOR ALL COLUMNS SIZE AUTO');

PL/SQL procedure successfully completed.

SQL> 

Similarly we can unlock stats for a schema also.

SQL> EXEC DBMS_STATS.unlock_schema_stats('HR');

PL/SQL procedure successfully completed.

How To Allow ADOP To Continue As If It Were Successful After Hitting An Error While Applying Patch(es)



Applying multiple patches in a given adop cycle, where by one of the patches fails to compile a form.  Instead of having adop error out, and having to redo all of the patches, just have adop log the message and continue on with the patch application.

Typical message that a user sees when adop fails during a patching cycle.


Continue as if it were successful :
***

AutoPatch could not find a response to the above prompt
or found an incorrect response in the defaults file.

You must run AutoPatch in an interactive session and provide a correct value.


Solution :


Currently the only way to "Continue as if a patch were successful" is by making use of the following parameter "flags=autoskip" in conjunction with the "abandon=no" parameter at the command-line.

For example: 
           


adop phase=apply patches=17020683 abandon=no restart=yes flags=autoskip