Doyensys Allappsdba Blog..

TX (Row Locks)

A TX enqueue is created as soon as a transaction is started. It is uniquely identified by the rollback segment number, slot number (in rollback segment’s transaction table) and slot number’s sequence number.

The wait event for this lock is normally "enq: TX - row lock contention".

A session, whilst carrying out DML on a row (or SELECT …. FOR UPDATE) can be waiting on a TX for the following main reasons:

Another session is carrying out DML on the requested row. Or has used SELECT … FOR UPDATE.

Another session has inserted the same row (defined by unique key) into the same table.

There are no free ITL (interested transaction list) slots in the block header. i.e. too many concurrent DML transactions are modifying rows in the same block. Increase INITRANS (and/or MAXTRANS) for the segment. Alternatively the number of rows in a block can be reduced by re-organizing the table; either reducing the block size or increasing the PCTFREE. The wait event for this situation is "enq: TX - allocate ITL entry".

My Oracle Support document "Waits for 'Enq: TX - ...' Type Events - Transaction (TX) Lock Example Scenarios (Document 62354.1)" explains some scenarios for TX locks, including:

Insufficient ITL slots (INITRANS) in a block

Waits due to rows being covered by the same BITMAP index fragment.

TM (Table Locks)

This lock is acquired by a transaction when using DML (or SELECT …. FOR UPDATE) or by the LOCK TABLE statement.

DML operations require table locks to reserve DML access to the table on behalf of a transaction and to prevent DDL operations that would conflict with the transaction.

The wait for this lock is "enq: TM – contention".

UL (User Lock)

These are user specified locks.

They can be requested and released using the DBMS_LOCK package procedures REQUEST and RELEASE.

The wait for this lock is "enq: UL - contention".

Reference:

Oracle E-Business Suite Performance Guide (Doc ID 1672174.1)

A session simply sleeps for a pre-determined length of time specified in a call to the USER_LOCK.SLEEP or DBMS_LOCK.SLEEP procedures.

However it can appear that a session is locked or there is a performance issue.

The details of sessions with "PL/SQL lock timer" waits can be queried using the following SQL:

SELECT
s.sid,
s.seconds_in_wait,
s.wait_class,
s.event,
s.p1text,
s.p1,
s.p1raw,
s.serial#,
s.program,
s.client_identifier,
s.terminal,
s.command,
ct.command_name,
s.service_name,
s.module,
s.action,
s.username,
s.machine,
s.sql_id,
st.sql_text, -- just the first 64 characters of the SQL text
st.hash_value
FROM v$session s,
v$sqltext st,
v$sqlcommand ct
WHERE st.sql_id (+) = s.sql_id
AND st.piece (+) = 0
AND ct.command_type (+) = s.command
AND s.event = 'PL/SQL lock timer'
ORDER BY s.seconds_in_wait DESC;

Reference:

Oracle E-Business Suite Performance Guide (Doc ID 1672174.1)

Add Options to the Concurrent Program Definition

This method is available in both 11i and R12.

To produce a Reports Client Process Trace at Concurrent Program Level:

1. Log in to the environment that will be traced.

2. Choose the relevant responsibility for defining concurrent programs.

3. Navigate to the Concurrent > Program > Define window.

4. Query for the concurrent program to be traced.

5. Turn on Reports Trace for the concurrent program by entering TRACEOPTS=TRACE_ALL and TRACEFILE=<full path and filename> in the Options field. The two keywords are delimited by a space. TRACEOPTS can also be set to TRACE_PRF.

Options must contain TRACEFILE along with TRACEOPTS, otherwise the request will error with "REP-0715: Trace file must be specified when tracing."

6. Navigate to the Submit Request window (e.g. Request > Run).

7. Submit a concurrent request for the concurrent program.

8. Make a note of the request_id of the concurrent request.

9. Check the progress of the concurrent request in the requests window.

10. When the request has completed navigate to the Concurrent > Program > Define window and disable the Reports Trace by clearing the Options field.

11. Obtain the trace file from the location specified in TRACEFILE above.

Other values for TRACEOPTS are available. See here for a full list.

The TRACEMODE parameter can also be used to specify if trace information is appended to the file (TRACE_APPEND) or overwrites it (TRACE_REPLACE).

Specify Debug Options when submitting a Concurrent Request
This method is available in R12 and 11.5.10 (with latest ATG patch levels).

To produce a Reports Client Process Trace at Concurrent Request Level:

1. Log in to the environment that will be traced.

2. Set the profile option "Concurrent: Allow Debugging" (internal name FND_CONC_ALLOW_DEBUG) user-level value to "Yes".

3. Navigate to the Submit Request window (e.g. Request > Run).

4. Select the Concurrent Request to be run.

5. Click on the "Debug Options" button.

6. Enable Reports Trace by checking the "Reports Trace" Debug Option and then select Debug Option Value "Log all possible trace information". Debug Option Value "Log report performance statistics" can also be used.

7. Click the "OK" button to confirm the selection. A "Create Debug Rule : Debug Option" page is displayed. Click on the "OK" button again.

8. Submit the concurrent request.

9. Make a note of the request_id of the concurrent request.

10. Check the progress of the concurrent request in the requests window.

11. When the request has completed, the trace file name and location (full path) can be obtained from the concurrent program log (file) .

Other levels are available in the Debug Option Value. See here for a full list.

Reference:

Oracle E-Business Suite Performance Guide (Doc ID 1672174.1)

Error in R12.2.5:

oracle.apps.fnd.framework.OAException: Could not load application module 'oracle.apps.XXCUST.xxom.xxDOYOrders.server.XxOmDOYOrdersAM'.

at oracle.apps.fnd.framework.webui.OAJSPApplicationRegistry.registerApplicationModule(OAJSPApplicationRegistry.java:279)at oracle.apps.fnd.framework.webui.OAJSPApplicationRegistry.registerApplicationModule(OAJSPApplicationRegistry.java:78)at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(OAPageBean.java:1348)

at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(OAPageBean.java:582)

at oracle.apps.fnd.framework.webui.OAPageBean.preparePage(OAPageBean.java:470)

at _pages.__oa._jspService(__oa.java:233)

at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)

at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)

at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)

at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)

at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60) at oracle.apps.jtf.cabo.interceptor.JTFWrapperFilter.doFilter(JTFWrapperFilter.java:149) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60) at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:527 at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:253) at weblogic.servlet.jsp.PageContextImpl.forward(PageContextImpl.java:159) at _pages.__rf._jspService(__rf.java:239) at weblogic.servlet.jsp.JspBase.service(JspBase.java:34) at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227) at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)

Reason :

This was due to incorrect/corrupted/missing jar file generation.

Pre-Check

1. Cross verified the custom jar files which is inside $JAVA_TOP

a) Just copied the customXXCUST.jar to a different location ex:

cp $JAVA_TOP/customXXCUST.jar /home/appDOY/s/

b) Now extract the customXXCUST.jar

unzip customXXCUST.jar

c) Now I checked the files inside that

/home/appDOY/s/oracle/apps /XXCUST

Here This should list all the files which we have zipped . But I can find only the following file

XXCUST_bck

Solution:

If you are taking backup of any folders,and place that in a different Directory which is out of the zip which you are using below.

Steps to be followed to include all the custom files to a single Jar file.

Please check the note (Note: 1609939.1) for additional details.

Step 1. Backup existing $JAVA_TOP/customXXCUST.jar into a different location.

Step 2. Remove all duplicate/backup folders under

$JAVA_TOP/oracle/apps/XXCUST/

Step 3. Steps to recreate customXXCUST.jar including the files from $XXCUST_TOP/java

cd $JAVA_TOP

zip -r customXXCUST.zip oracle/apps/XXCUST

zip -r customXXCUST.zip Doyen

Note : Before doing the below copy check and remove the customXXCUST.zip if already presents in $XXCUST_TOP/java

cp $JAVA_TOP/customXXCUST.zip $XXCUST_TOP/java

cd $XXCUST_TOP/java

zip -r customXXCUST.zip Doyen

zip -r customXXCUST.zip oracle/apps/

Note : Before doing the below copy check and remove the customXXCUST.zip if already presents in $APPL_TOP/cns_cust/java

cp $XXCUST_TOP/java/customXXCUST.zip $APPL_TOP/cns_cust/java

cd $APPL_TOP/cns_cust/java

zip -r customXXCUST.zip oracle

zip -r customXXCUST.zip commod

Note : Before doing the below copy check and remove the customXXCUST.zip if already presents in $JAVA_TOP

cp $APPL_TOP/cns_cust/java/customXXCUST.zip $JAVA_TOP

Step 4: Create JAR file

adjava oracle.apps.ad.jri.adjmx -areas $JAVA_TOP/customXXCUST.zip -outputFile $JAVA_TOP/customXXCUST.jar -jar $CONTEXT_NAME 1 CUST jarsigner -storePass -keyPass

Step 5: Cleanup

rm $JAVA_TOP/customXXCUST.zip

rm $XXCUST_TOP/java/customXXCUST.zip

rm $APPL_TOP/cns_cust/java/customXXCUST.zip

chmod 775 $JAVA_TOP/customXXCUST.jar

Note :

After Doing the above steps, The above jar file is there in Java_TOP and this will be known to Weblogic only if the below steps is done.

Follow steps below to make the custom jar available for WebLogic

a) Take a back-up of existing <FND_TOP>/admin/template/ebsProductManifest_xml.tmp

b) Modify <FND_TOP>/admin/template/ebsProductManifest_xml.tmp to add below entry for customXXCUST.jar (after customall.jar)

<library>customXXCUST.jar</library>

c) Run AutoConfig

d) Bounce the Mid-tier services

MICR font is used for check printing. It needs to be purchased and configured into XML Publisher to use in BI Publisher reports.

In DOYEN (11i) instance, found the MICR font is configured. Below are the steps to review.

1. Log in as XML Publisher Administrator.

2. Navigate to Administration->Font Files

3. Search by name “MICR

4. Navigate to Administration -> Font Mappings

5. Search by Mapping Name “MICR”

6. Drill down to the MICR details

Below are the steps to configure new fonts and font mapping:

1. Log in as XML Publisher Administrator.

2. Navigate to Administration->Font Files->Create Font File.

3. Fields are Font Name and File.

For Font Name choose any descriptive name.

File will browse your PC to locate the font file.

4. Navigate to Font Mappings->Create Font Mapping Set.

5. Mapping name is the name you will give to a set of fonts.

6. Mapping code is the internal name you will give to this set.

7. Type: 'PDF Form' for PDF templates. 'FO to PDF' for all other template types.

8. Create Font Mapping (this allows you to add multiple fonts to a set).

9. Font Family is the exact same name you see in Word under Fonts.

If you don't use the same name the font will not be picked up at runtime.

10. Style and weight must also match how you use the font in windows.

Normal and Normal are good defaults.

Scenario:

Got the Forms error while adding responsibility to a user and so, I did the workflow sync validation and it does not fixed. So, I ran autoconfig in Apps tier and it end up with Error and ran in db node and that too endup with the same error.

1. Auto config in apps Node completed with the following Error:

WARNING: [AutoConfig Error Report]
The following report lists errors AutoConfig encountered during each
phase of its execution. Errors are grouped by directory and phase.
The report format is:
<filename> <phase> <return code where appropriate>

[PROFILE PHASE]
AutoConfig could not successfully execute the following scripts:
Directory: /Doyen5/app/PROJ06/fs2/FMW_Home/webtier/perl/bin/perl -I /Doyen5/app/PROJ06/fs2/FMW_Home/webtier/perl/lib/5.10.0 -I /Doyen5/app/PROJ06/fs2/FMW_Home/webtier/perl/lib/site_perl/5.10.0 -I /Doyen5/app/PROJ06/fs2/EBSapps/appl/au/12.0.0/perl -I /Doyen5/app/PROJ06/fs2/FMW_Home/webtier/ohs/mod_perl/lib/site_perl/5.10.0/x86_64-linux-thread-multi /Doyen5/app/PROJ06/fs2/inst/apps/PROJ06_tlvda2/admin/scripts/adexecsql.pl sqlfile=/Doyen5/app/PROJ06/fs2/inst/apps/PROJ06_tlvda2/admin/install
afwebprf.sql INSTE8_PRF 1

AutoConfig is exiting with status 1

2. Autoconfig in the DB node completed with the following error:

WARNING: [AutoConfig Error Report]
The following report lists errors AutoConfig encountered during each
phase of its execution. Errors are grouped by directory and phase.
The report format is:
<filename> <phase> <return code where appropriate>

[PROFILE PHASE]
AutoConfig could not successfully execute the following scripts:
Directory: /Doyen5/orahome/Doyen5db/11.2.0.4/appsutil/install/PROJ06_testdb2
afdbprf.sh INSTE8_PRF 1

AutoConfig is exiting with status 1

Now when i tried running the script (afwebprf.sql) manually which errored out in apps autoconfig and i faced the same error which we got in forms.

Now i tried running the script (afdbprf.sh) manually which errored out in db node and found the same error:

[oraDoyen5@testdb2:DOYEN5_testdb2]# sh afdbprf.sh

afdbprf.sh started at Thu Jun 22 19:54:29 IDT 2017

The environment settings are as follows ...

ORACLE_HOME : /Doyen5/orahome/Doyen5db/11.2.0.4
ORACLE_SID : PROJ06
PATH : /Doyen5/orahome/Doyen5db/11.2.0.4/perl/bin:/Doyen5/orahome/Doyen5db/11.2.0.4/perl/bin:/Doyen5/orahome/Doyen5db/11.2.0.4/bin:/usr/bin:/usr/sbin:/Doyen5/orahome/Doyen5db/11.2.0.4/appsutil/jre/bin:/bin:/usr/bin/X11:/usr/local/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:.:.
Library Path : /Doyen5/orahome/Doyen5db/11.2.0.4/lib:/usr/X11R6/lib:/usr/openwin/lib:/Doyen5/orahome/Doyen5db/11.2.0.4/lib:/usr/dt/lib:/Doyen5/orahome/Doyen5db/11.2.0.4/ctx/lib

Executable : /Doyen5/orahome/Doyen5db/11.2.0.4/bin/sqlplus

Enter the APPS username: apps
Enter the APPS password:
SQL*Plus: Release 11.2.0.4.0 Production on Thu Jun 22 19:54:33 2017
Copyright (c) 1982, 2013, Oracle. All rights reserved.

Enter value for 1: Enter value for 2: Enter value for 3: Connected.
[ FND_DB_WALLET_DIR ]
Application Id : 0
Profile Value : /Doyen5/orahome/Doyen5db/11.2.0.4/appsutil/wallet
Level Name: SITE
INFO : Updated/created profile option value.
.
[ ECX_UTL_XSLT_DIR ]
Application Id : 174
Profile Value : /usr/tmp
Level Name: SITE
INFO : Updated/created profile option value.
.
[ ECX_UTL_LOG_DIR ]
Application Id : 174
Profile Value : /usr/tmp
Level Name: SITE
INFO : Updated/created profile option value.
.
[ BIS_DEBUG_LOG_DIRECTORY ]
Application Id : 0
Profile Value : /usr/tmp
Level Name: SITE
INFO : Updated/created profile option value.
.
[ UTL_FILE_LOG ]
Application Id : 1
Profile Value : /usr/tmp, /Doyen5/logout/tmp, /tmp/, /Doyen5/oratmp/tempdata,
/Doyen5/logout/Agile, /Doyen5/logout/Agile/in, /Doyen5/logout/Agile/out,
/Doyen5/winshare/APAC/AP, /Doyen5/winshare/crm/scorecard,
/Doyen5/winshare/APAC/WESTPAC, /Doyen5/logout/cm,
/Doyen5/orahome/Doyen5db/11.2.0.4/appsutil/outbound/PROJ06_testdb2
Level Name : SITE
INFO : Error updating/creating profile option value.
.
begin
*
ERROR at line 1:
ORA-04063: rule set "APPLSYS.WF_JAVA_DEFERRED_R" has errors
ORA-06512: at "APPS.WF_EVENT", line 1194
ORA-06512: at "APPS.FND_WF_ENGINE", line 1025
ORA-06512: at "APPS.FND_PROFILE", line 1102
ORA-06512: at "APPS.FND_PROFILE", line 2353
ORA-06512: at "APPS.ADX_PRF_PKG", line 192
ORA-06512: at line 42

The above error seems to be an issue with the WF Ruleset.

Solution:

Checked the following Rule Set is valid and found to be invalid.

select object_name, object_type, status from dba_objects where object_name='WF_JAVA_DEFERRED_R';

WF_JAVA_DEFERRED_RRULE SETINVALID

This is because of the issues with the Workflow queues. So i did the clean up and enque processes as suggested in the below Note id.

After following the following noteid , It fixed the issue:

Fixing Invalid Workflow Rule Sets such as WF_DEFERRED_R and Related Errors on Workflow Queues:ORA-24033 (Doc ID 337294.1)

Once it is done, I have checked the status of the Ruleset and it is valid.

select object_name, object_type, status from dba_objects where object_name='WF_JAVA_DEFERRED_R';

WF_JAVA_DEFERRED_RRULE SETVALID

Now , Ran the Autoconfig and it has completed Normal.

Scenario :

The server has a capacity to handle more process but there is no oacore component to do so and we need to add a oacore .

Solution:

Add oacore :

Syntax :

1. $ perl <AD_TOP>/patch/115/bin/adProvisionEBS.pl \
ebs-create-managedserver -contextfile=<CONTEXT_FILE> \
-managedsrvname=oacore_server2 -servicetype=oacore \
-managedsrvport=7203 -logfile=<APPLRGF>/TXK/addMS_oacoreserver2.log

Example :

perl /devl/app/DEV/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/adProvisionEBS.pl \
ebs-create-managedserver -contextfile=/devl/app/DEV/fs1/inst/apps/DEV_tstsara24/appl/admin/DEV_tstsara24.xml \
-managedsrvname=oacore_server2 -servicetype=oacore \
-managedsrvport=7212 -logfile=/devl/app/DEV/fs1/inst/apps/DEV_tstsara24/logs/appl/rgf/TXK/addMS_oacoreserver2.log

2. Add this to the conf file do the following:

perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl -contextfile=$CONTEXT_FILE \
-configoption=addMS -oacore=tstsara24.corp.doyensystems.com:7212

perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl -contextfile=$CONTEXT_FILE \
-configoption=addMS -oacore=tstsara24.corp.doyensystems.com:7337

log file shows :

/devl/app/DEV/fs1/inst/apps/DEV_tstsara24/logs/appl/rgf/TXK/txkSetAppsConf_06301835.log

Reading /devl/app/DEV/fs1/FMW_Home/webtier/instances/EBS_web_DEV_OHS1/config/OHS/EBS_web_DEV/mod_wl_ohs.conf ...

Reading /devl/app/DEV/fs1/FMW_Home/webtier/instances/EBS_web_DEV_OHS1/config/OHS/EBS_web_DEV/apps.conf ...

In the config file it will add a entry like this.

WebLogicCluster tstsara24.corp.doyensystems.com:7221,tstsara24.corp.doyensystems.com:7212,tstsara24.corp.doyensystems.com:7337

Reference :

Managing Configuration of Oracle HTTP Server and Web Application Services in Oracle E-Business Suite Release 12.2 (Doc ID 1905593.1)

This will help you when AD to OID connection is broken and users are waiting to be loaded to SSO:

Get the User extract from AD :

ldapsearch -h <dummy.ad.srv.com> -p 389 -D "CN=ORASSO,CN=Users,DC=us,DC=dummy,DC=com" -w "passwd" -b "dc=us,dc=dummy,dc=com"
-s sub sAMAccountName="*astathom*" dn sAMAccountName cn displayName title mail description employeeid objectclass sn manager description title userprincipalname whenCreated whenChanged
CN=Andrew.Stathom,OU=Users,DC=us,DC=dummy,DC=com
objectClass=top
objectClass=person
objectClass=organizationalPerson
objectClass=user
cn=Andrew.Stathom
sn=Andrew
title=Collections Rep
description=Collections Rep
displayName=Stathom, Andrew
employeeID=2393661
sAMAccountName=astathom
userPrincipalName=astathom@ironmountain.com
mail=Andrew.Stathom@ironmountain.com
manager=CN=Kent_Bob,CN=Users,DC=us,DC=dummy,DC=com

Use the below as a Template for creating a ldif file for loading the user into SSO :

dn:
orclsourceobjectdn:
objectclass: inetorgperson
objectclass: organizationalperson
objectclass: orcluserv2
objectclass: orcladuser
objectclass: orcladobject
objectclass: person
objectclass: top
cn:
orclsamaccountname:
uid:
sn:
title:
description:
displayName:
facsimiletelephonenumber:
employeenumber:
mail:
krbprincipalname:
manager:

Prepare ldif file using the extract from AD & the above template :

vi user_Andrew.ldif
dn: cn=Andrew.Stathom,ou=users,cn=adus,cn=users,dc=dummy,dc=com (as per the tree strcuture in your OID)
orclsourceobjectdn: CN=Andrew.Stathom,OU=Users,DC=us,DC=dummy,DC=com # (dn from AD)
objectclass: inetorgperson # mandatory attribute
objectclass: organizationalperson # mandatory attribute
objectclass: orcluserv2 # mandatory attribute
objectclass: orcladuser # mandatory attribute
objectclass: orcladobject # mandatory attribute
objectclass: person # mandatory attribute
objectclass: top # mandatory attribute
cn: Andrew.Stathom  # cn from AD
orclsamaccountname: dummy\astathom # domain\<sAMAccountName from AD>
uid: dummy.astathom # domain.sAMAccountName
sn: Andrew # from AD
title: Collections Rep # from AD
description: Collections Rep # from AD
displayName: Stathom, Andrew  # from AD
facsimiletelephonenumber: 23646 # employeeID from AD
employeenumber: 23646 # employeeID from AD
mail: Andrew.Stathom@dummy.com # mail from AD
krbprincipalname: astathom@dummy.com # userPrincipalName from AD
manager: CN=Kent_Bob,CN=Users,DC=us,DC=dummy,DC=com  # from AD

Load the User to SSO :

ldapadd -h oidapp.dummy.com -p 3060 -D "cn=orcladmin" -w "passwd" -f user_Andrew.ldif

or

You can as well load from ODSM console using User Import option .

This is a workaround that is really helpful when Oracle's directory integration is down/broken and you need to quickly load a few users to EBS :

Extract the user details from OID (not from AD) to a .ldif file :

ldapsearch -h dummy.ad.srv -p 3060 -D "cn=orcladmin" -s sub -L -b "cn=users,dc=dummy,dc=com" -w "passwd""uid=dummy.jsmith"
uid mail facsimiletelephonenumber description employeenumber orclsamaccountname >> /tmp/jsmith.ldif

dn: cn=John_Smith,ou=users,cn=adus,cn=users,dc=dummy,dc=com
uid: dummy.jsmith
mail: John.Smith@dummy.com
orclsamaccountname: dummy\jsmith
facsimiletelephonenumber: 20777 ---> Employee ID for EBS
employeenumber: 20777
description: Manager, Accounting

Note: In the above example I took only those attributes that we need for FND User form.

Load the user to EBS from LDIF file generated above :

Copy the above ldif file to EBS Application server and run the below java command :

java oracle.apps.fnd.oid.LDAPUserImport -v -dbc $FND_SECURE/PROD.dbc -d -f jsmith.ldif -n uid -tcaRecord N -defresp N
LDAP user import from jsmith.ldif started..
LDAP user import completed sucessfully. For further details refer to log file at LDAPUserImport.log

Guys, I would like to share with you all a scenario which you may come across very seldom, but really important.

RMAN is not deleting ArchiveLogs on Primary Database when the State for the Archive Destination to the Standby Database has already been set to 'defer'. When you are trying to delete archive logs, the below error is raised:

RMAN-08137: WARNING: archived log not deleted, needed for standby or upstream capture process

As long as the Archive Destination (log_archive_dest_2) is still set, we consider the Standby Database as still alive and keep the ArchiveLogs on Primary Database to perform Gap Resolution when the Archive Destination is valid again.

There are Situations when this is not wanted, eg :

The Standby Database was activated or removed, but you still keep the Archive Destination because you want to rebuild the Standby Database later again. In this Case you can set the hidden Parameter "_deferred_log_dest_is_valid" to FALSE (the default is TRUE) which will consider the deferred Archive Destinations as completely unavailable and will not preserve ArchiveLogs for those Destinations any more. It is a dynamic Parameter and can be set as shown below :

SQL> alter system set "_deferred_log_dest_is_valid" = FALSE scope=both;

I prefer to do this than deleting the archive logs forcefully.

Just remember to enable this parameter once you re-build your Standby, to avoid unintentional deletions ... you know what I mean 😉

I have recently come across an issue with our Shareplex Target instance where a couple of Post Processes went down and there were no significant errors in the log.

After going through the Database alert log I understood the Archive dest is full and I have cleared the archive dest. But, the Post Processes were still not coming up.

So, I had to exclusively execute these commands to bring up those 2 Post Processes.

start post queue OTH4

start post queue IEX01

Remember If the backlogs are huge the sync up time will be longer, which is expected.

Error :

User reported the following error

”Unable to make any changes to invoice #589 as it is stuck as 'Could not reserve record.

SYMPTOMS :

When attempting to update an invoice, the following error occurs:

APP-SQLAP-10771: Could not reserve record.

CAUSE :

A database lock from another session has occurred on the table, AP_INVOICES_ALL, AP_INVOICE_LINES_ALL, AP_INVOICE_DISTRIBUTIONS_ALL.

SOLUTION :

To implement the solution on your instance follow the below steps:

1. Run the following to determine what tables are locked:

   SELECT a.object_id, a.session_id, substr(b.object_name, 1, 40)
                FROM v$locked_object a, dba_objects b
                WHERE a.object_id = b.object_id
                AND b.object_name like 'AP_%'
                ORDER BY b.object_name;

         2. Look at the results and insert whatever AP_% tables are returned from a) into the script below:

   SELECT l.*, o.owner object_owner, o.object_name
                FROM SYS.all_objects o, v$lock l
                WHERE l.TYPE = 'TM'
                AND o.object_id = l.id1
                AND o.object_name in ('AP_INVOICES_ALL', 'AP_INVOICE_LINES_ALL', 'AP_INVOICE_DISTRIBUTIONS_ALL');

3. Run the following:

    SELECT SID, SERIAL#
                FROM v$session
                WHERE SID = <SID from b)>;

4. Once the locking sessions have been identified, please use the below command to kill such sessions.

ALTER SYSTEM KILL SESSION 'sid,serial#' IMMEDIATE;

Reference:

R12: AP: Attempting to Update an Invoice Errors with APP-SQLAP-10771: could not reserve record (Doc ID 1200053.1)

Oracle WLS java settings.

Edit setdomainenv file under weblogic home \bin bin/setDomainEnv.sh set SET JAVA_OPTIONS="................"

Error:

APP-SQLAP-10771: Could not reserve record.

Unable to pay the AP invoice in full, Got above error while performing the action –> pay in full

Solution:

Database lock from another session is present on the table AP_INVOICES_ALL, AP_INVOICE_LINES_ALL, AP_INVOICE_DISTRIBUTIONS_ALL for the same record that user was trying to modify from application.

a) To determine what session has a lock on this record, please execute the following steps:

1) Run the following to determine what tables are locked:

--- --- ---

SELECT a.object_id, a.session_id, substr(b.object_name, 1, 40)

FROM v$locked_object a, dba_objects b

WHERE a.object_id = b.object_id

AND b.object_name like 'AP_%'

ORDER BY b.object_name;

--- --- ---

2) Look at the results and insert whatever AP_% tables are returned from a) into the script below:

--- --- ---

SELECT l.*, o.owner object_owner, o.object_name

FROM SYS.all_objects o, v$lock l

WHERE l.TYPE = 'TM'

AND o.object_id = l.id1

AND o.object_name in ('AP_INVOICES_ALL', 'AP_INVOICE_LINES_ALL', 'AP_INVOICE_DISTRIBUTIONS_ALL');

--- ---

SELECT SID, SERIAL#

FROM v$session

WHERE SID = <SID from b)>;

--- --- ---

b) Once the locking sessions have been identified, please use the below command to kill such sessions.

--- --- ---

ALTER SYSTEM KILL SESSION 'sid,serial#' IMMEDIATE;

---

Note: SID and serial# to be taken from the output from 1c).

--- --- ---

Another Solution

==============

If there are many items locked you may find it easier to follow this alternative solution:

a) Ensure no-one is working and no processes are running in the system.

b) Bounce the database.

c) Retest the issue

Some useful fnd scripts which available in $FND_TOP/sql directory

FNDATPRG.sql Purge audit trail tables before a given date

FNDCPDC2.sql Delete a concurrent program and SRS definition if needed, calls FNDCPDCP.sql

FNDCPDCP.sql Delete a concurrent program and SRS definition if needed

fndfbdpm.sql Deletes a flexbuilder parameter.

fndfbfxn.sql Generate a list of FlexBuilder functions and parameters for a given application. Some parameters have multiple definitions, but this report lists each parameter only once.

fndfbprm.sql Generate a list of FlexBuilder functions and parameters, including detailed information about how the parameter is defined for a given application. This report includes all definitions for each parameter, ordered by sequence number. The information in this report corresponds to information in the Define FlexBuilder Parameters form.

fndffbdd.sql Finds independant segment values which don't have the default dependant segment value defined for them.

FNDFFCVS.sql Copy flexfield value set from one database to another creates a sql script filename.sql which can be run on another database to copy value sets

FNDFFDDS.sql Delete a descriptive flexfield definition from AOL tables

FNDFMFXR.sql Forms Trigger Exception Report

fndgofpr.sql Creates Grants Only for a Full PRivileged Oracle ID. This assumes fndgsspr.sql was run first to create a limited set of SELECT only privileges which are not recreated here.

fndgsnpa.sql Revokes all AOL grants and drops all AOL synonyms from a user. Must also revoke synonym GL_CURRENCIES which uses FND_CURRENCIES.

fndgsspr.sql Creates Grants and Synonyms for a Select only Privileged Oracle ID.

FNDMDCMR.sql Creates a report of all messages for an application in the given language.The report is located in the application's message directory and has the name {language short name}.FDDTMFEXT. For example: /applications/fnd/5.0/mesg/usaeng.msg. The standalone FNDMDCMF (Create Message File) runs this report after creating the binary message file to produce a human-readable version.

fndmncpy.sql Copy application menus to another, custom application (Rel. 9 to 10 backup)

FNDNLADD.sql Add missing translation rows for FND _TL tables.

FNDNLCHK.sql Check _TL tables for inconsistent or missing translation data. FNDNLCHK does not fix anything, it only reports bad data. Use FNDNLADD to fix errors found by FNDNLCHK.

fndnlhlp.sql Updates fnd_loader_formats

FNDNLICR.sql NLS Insert non-ISO currency

FNDNLINS.sql Update _TL tables to populate rows for new language when a new language is installed.

FNDNLMVL.sql Install multi-language versions of _VL views.

fndnmts.sql Populate FND%_TL subtables with data from main FND% tables ( In 10.5 this script is superceded by FNDNLADD.sql and should not be used by customers. )

fndnstm.sql Populate FND_ main tables with data from subtables.

fndrspfm.sql Report on Users who Access a Given Form

FNDRSTST.sql SRS Test Program - expects three arguments and prints them

fndscats.sql Signon Audit Time Stamp

FNDSCETS.sql Set end time stamp (FND_LOGINS, FND_LOGIN_RESPONSIBILITIES, FND_LOGIN_RESP_FORMS)

FNDSCGRP.sql Reports on which applications, sets and programs have been assigned to which responsibility. Accepts application name and responsibility name.

FNDSCPRG.sql Purge signon audit tables from given date

FNDUDUAL.sql This script ensures only one record exists in FND_DUAL.

fndutcsq.sql Converts rows in FND_UNIQUE_IDENTIFIER_CONTROL to Sequences

wfbkg.sql WorkFlow BacKGround engine, starts the background engine, running for the indicated number of minutes.

wfbkgchk.sql WorkFlow Background Check. It displays a status report on background work waiting to be processed.

wfdirchk.sql WorkFlow Directoy Servoices Data Model Check

WFNLADD.sql Add missing translation rows for WF _TL tables.

wfnldat.sql Add default language data for standalone Workflow install.

wfnlena.sql Enable/disable an installed language (workflow)

wfntfsh.sql WorkFlow NoTiFication SHow status

wfprot.sql WorkFlow PROTection level reset. It resets the protection level for all objects in a specified item type to the supplied value. After resetting the protection level NOTHING in the item type will be customizable by a higher access level.

wfrefchk.sql WorkFlow Primary,Unique and Foreign Key constraint checker. It checks for all invalid workflow data that is missing primary key data for a foreign key

wfretry.sql WorkFlow Handle error'ed activity. It displays a list of errored activities for the indicated item. Type in the name of the activity, and command to skip, retry, or reset.

wfrmall.sql WorkFlow ReMove ALL. It DELETES all workflow information. ALL OF IT.

wfrmbref.sql Deletes all invalid workflow data that is missing primary key data for a foreign key

wfrmita.sql WorkFlow ReMove ITem Attribute. It deletes all workflow information for the specified item attribute.

wfrmitms.sql WorkFlow ReMove ITMeS. It removes item status information for items which match the supplied type and key patterns.

wfrmitt.sql WorkFlow ReMove ITemType. It deletes ALL workflow information for the specified item type.

wfrmtype.sql WorkFlow ReMove TYPE. It purges ALL runtime data associated with a given item type.

wfrun.sql WorkFlow RUN a process. It creates and starts the specified process.

wfstat.sql WorkFlow item STATUS report. It displays a status report on the indicated item (132 charachter output).

wfstatus.sql WorkFlow item STATUS report. It displays a status report on the indicated item (132 charachter output).

wfver.sql WorkFlow VERsion display. It displays version information for all WF source.

wfverchk.sql WorkFlow Version Check. It checks all workflow activities for potentially invalid version histories (more than one version of an activity active at any given time). Correct any errors found.

Cannot communicate with Cluster Ready Services

ISSUE:-

[oracle@server1:+ASM1] sudo crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4530: Communications failure contacting Cluster Synchronization Services daemon

[oracle@server1:+ASM1] sudo crsctl stat res -t -init

--------------------------------------------------------------------------------

Name Target State Server State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

1 ONLINE ONLINE server1 STABLE

ora.cluster_interconnect.haip

1 ONLINE ONLINE server1 STABLE

ora.crf

1 OFFLINE OFFLINE STABLE

ora.crsd

1 ONLINE ONLINE server1 STABLE

ora.cssd

1 ONLINE ONLINE server1 STABLE

ora.cssdmonitor

1 ONLINE ONLINE server1 STABLE

ora.ctssd

1 ONLINE ONLINE server1 OBSERVER,STABLE

ora.diskmon

1 OFFLINE OFFLINE STABLE

ora.drivers.acfs

1 ONLINE ONLINE server1 STABLE

ora.evmd

1 ONLINE ONLINE server1 STABLE

ora.gipcd

1 ONLINE ONLINE server1 STABLE

ora.gpnpd

1 ONLINE ONLINE server1 STABLE

ora.mdnsd

1 ONLINE ONLINE server1 STABLE

ora.storage

1 ONLINE ONLINE server1 STABLE

--------------------------------------------------------------------------------

[oracle@server1:+ASM1] sudo crsctl start res ora.diskmon -init

CRS-2672: Attempting to start 'ora.diskmon' on 'server1'

CRS-2676: Start of 'ora.diskmon' on 'server1' succeeded

[oracle@server1:+ASM1] sudo crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

[oracle@server1:+ASM1]

Action Plan for 18c Grid:-

sudo crsctl stop crs -f

cd /tmp

ls -al

rm -rf Ora*

ls -al

rm -rf .oracle

ls -al

rm -rf orphan_indfile.lst osw.hb

ls -al

rm -rf CVU*

ls -la

sudo crsctl start crs

[oracle@server1:+ASM1] sudo crsctl stat res -t -init

--------------------------------------------------------------------------------

Name Target State Server State details

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

1 ONLINE ONLINE server1 STABLE

ora.cluster_interconnect.haip

1 ONLINE ONLINE server1 STABLE

ora.crf

1 OFFLINE OFFLINE STABLE

ora.crsd

1 ONLINE ONLINE server1 STABLE

ora.cssd

1 ONLINE ONLINE server1 STABLE

ora.cssdmonitor

1 ONLINE ONLINE server1 STABLE

ora.ctssd

1 ONLINE ONLINE server1 OBSERVER,STABLE

ora.diskmon

1 OFFLINE OFFLINE STABLE

ora.drivers.acfs

1 ONLINE ONLINE server1 STABLE

ora.evmd

1 ONLINE ONLINE server1 STABLE

ora.gipcd

1 ONLINE ONLINE server1 STABLE

ora.gpnpd

1 ONLINE ONLINE server1 STABLE

ora.mdnsd

1 ONLINE ONLINE server1 STABLE

ora.storage

1 ONLINE ONLINE server1 STABLE

--------------------------------------------------------------------------------

Action plan:-

[oracle@server1:+ASM1] sudo crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

[oracle@server1:+ASM1]

DBCA fails with errors: [FATAL] [DBT-50000] Unable to check for available memory

Recently I have installed Oracle 19c Database binary to windows machine. And after that I started DBCA to create database.

In "Configuration options" steps I got memory error. i.e. [DBT-50000] Unable to check for available memory
Earlier I selected 6GB for my SGA but after this error I had selected 1GB only. But still was facing this issue.

Here is the screenshot for the same.

After searching oracle docs for known issue, I found, it's a bug. One can find details about this bug on following oracle notes.

DBCA fails with errors: [FATAL] [DBT-50000] Unable to check for available memory in "Specify configuration option" (Doc ID 2631718.1)

Workaround for this issue is,

Run dbca with "-J-Doracle.assistants.dbca.validate.ConfigurationParams=false" like a bellow command,

$> dbca -J-Doracle.assistants.dbca.validate.ConfigurationParams=false

This will solve your issue and DBCA will run smooth, no issues thn.

ORA-65259: partial synchronization of application is disallowed

You have started application Sync but you find it is taking more time to complete this action. By mistake or purposely you did ctrl+c or stopped the application sync operation. Later point of time if you again start application syncing command, now you might end up hitting bug.

Check below listing,

alter pluggable database application MYAPP sync
*
ERROR at line 1:
ORA-65259: partial synchronization of application is disallowed

This issue occurs in two reasons,
1) If any other operation also going on in parallel with sync.
2) If previously Sync operation was killed or Stopped.

In both the cases, it makes PDBs in inconsistent state. Here suggestion is, always try to avoid cancellation on SYNC operation on application PDBs, no matter if it is taking long time.

Here are two workaround for this issue.

Workaround 1.
Work around for this issue is to apply for bug 24430427.
Patch can be found on https://support.oracle.com

Workaround 2.
Step 1. Create new PDB under application root.
Step 2. Synchronize PDBs with the application root
Step 3:- Upgrade application

DBConsole service won't start

The web-based Enterprise Manager tool was introduced in Oracle 10g and has significant functionality improvements over previous versions. How to solve an Enterprise Manager configuration issue, this article explains the first things you should look.

First, check the listener is running correctly.

lsnrctl status

If you get something like the following, your listener is not running and you need to start it using the "lsnrctl start" command.

$ lsnrctl status

LSNRCTL for Linux: Version 11.1.0.7.0 - Production on 13-MAR-2009 13:32:49

Copyright (c) 1991, 2008, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
Linux Error: 2: No such file or directory
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oel5-11g.localdomain)(PORT=1521)))
TNS-12541: TNS:no listener
TNS-12560: TNS:protocol adapter error
TNS-00511: No listener
Linux Error: 111: Connection refused
$

Next, check the Enterprise Manager service is running.

emctl start dbconsole

Assuming the console is already running, or fails to start, the next thing to do is to reconfigure Enterprise Manager. This is done using the Enterprise Manager Configuration Assistant (EMCA). For a first attempt, run the following command.

emca -config dbcontrol db -repos recreate

If it all goes well you would expect to see something like this.

$ emca -config dbcontrol db -repos recreate

STARTED EMCA at Mar 13, 2009 11:32:35 AM
EM Configuration Assistant, Version 11.1.0.7.0 Production
Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Enter the following information:
Database SID: DB11G
Database Control is already configured for the database DB11G
You have chosen to configure Database Control for managing the database DB11G
This will remove the existing configuration and the default settings and perform a fresh configuration
Do you wish to continue? [yes(Y)/no(N)]: Y
Listener port number: 1521
Password for SYS user:
Password for DBSNMP user:
Password for SYSMAN user:
Email address for notifications (optional):
Outgoing Mail (SMTP) server for notifications (optional):
-----------------------------------------------------------------

You have specified the following settings

Database ORACLE_HOME ................ /u01/app/oracle/product/11.1.0/db_1

Local hostname ................ oel5-11g.localdomain
Listener port number ................ 1521
Database SID ................ DB11G
Email address for notifications ...............
Outgoing Mail (SMTP) server for notifications ...............

-----------------------------------------------------------------
Do you wish to continue? [yes(Y)/no(N)]: Y
Mar 13, 2009 11:33:19 AM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /u01/app/oracle/cfgtoollogs/emca/DB11G/emca_2009_03_13_11_32_35.log.
Mar 13, 2009 11:33:20 AM oracle.sysman.emcp.util.PortManager isPortInUse
WARNING: Specified port 5540 is already in use.
Mar 13, 2009 11:33:20 AM oracle.sysman.emcp.util.PortManager isPortInUse
WARNING: Specified port 5520 is already in use.
Mar 13, 2009 11:33:20 AM oracle.sysman.emcp.util.PortManager isPortInUse
WARNING: Specified port 1158 is already in use.
Mar 13, 2009 11:33:20 AM oracle.sysman.emcp.util.PortManager isPortInUse
WARNING: Specified port 3938 is already in use.
Mar 13, 2009 11:33:20 AM oracle.sysman.emcp.util.DBControlUtil stopOMS
INFO: Stopping Database Control (this may take a while) ...
Mar 13, 2009 11:33:22 AM oracle.sysman.emcp.EMReposConfig invoke
INFO: Dropping the EM repository (this may take a while) ...
Mar 13, 2009 11:35:12 AM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully dropped
Mar 13, 2009 11:35:12 AM oracle.sysman.emcp.EMReposConfig createRepository
INFO: Creating the EM repository (this may take a while) ...
Mar 13, 2009 11:40:06 AM oracle.sysman.emcp.EMReposConfig invoke
INFO: Repository successfully created
Mar 13, 2009 11:40:11 AM oracle.sysman.emcp.EMReposConfig uploadConfigDataToRepository
INFO: Uploading configuration data to EM repository (this may take a while) ...
Mar 13, 2009 11:41:01 AM oracle.sysman.emcp.EMReposConfig invoke
INFO: Uploaded configuration data successfully
Mar 13, 2009 11:41:07 AM oracle.sysman.emcp.util.DBControlUtil configureSoftwareLib
INFO: Software library configured successfully.
Mar 13, 2009 11:41:07 AM oracle.sysman.emcp.EMDBPostConfig configureSoftwareLibrary
INFO: Deploying Provisioning archives ...
Mar 13, 2009 11:41:16 AM oracle.sysman.emcp.EMDBPostConfig configureSoftwareLibrary
INFO: Provisioning archives deployed successfully.
Mar 13, 2009 11:41:16 AM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Securing Database Control (this may take a while) ...
Mar 13, 2009 11:41:28 AM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Database Control secured successfully.
Mar 13, 2009 11:41:28 AM oracle.sysman.emcp.util.DBControlUtil startOMS
INFO: Starting Database Control (this may take a while) ...
Mar 13, 2009 11:42:31 AM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: Database Control started successfully
Mar 13, 2009 11:42:34 AM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: >>>>>> The Database Control URL is https://oel5-11g.localdomain:1158/em <<<<<< Mar 13, 2009 11:42:48 AM oracle.sysman.emcp.EMDBPostConfig invoke WARNING:
************************  WARNING  ************************  Management Repository has been placed in secure mode wherein Enterprise Manager data will be encrypted. 
The encryption key has been placed in the file: /u01/app/oracle/product/11.1.0/db_1/oel5-11g.localdomain_DB11G/sysman/config/emkey.ora.  
Please ensure this file is backed up as the encrypted data will become unusable if this file is lost.
*********************************************************** Enterprise Manager configuration completed successfully FINISHED EMCA at Mar 13, 2009 11:42:48 AM $

If this fails, then de-configure and configure Enterprise Manager.

If this fails, then de-configure and configure Enterprise Manager.

emca -deconfig dbcontrol db -repos drop
emca -config dbcontrol db -repos create

TX (Row Locks) TM (Table Locks) UL (User Lock)

PL/SQL lock timer

Concurrent Program/Reports Client Process Trace

Could not load application module 'oracle.apps.XXCUST.xxom.xxDOYOrders.server.XxOmDOYOrdersAM

Configuring MICR Font in XML Publisher

ORA-04063: rule set "APPLSYS.WF_JAVA_DEFERRED_R" has errors

Managing Configuration of Oracle HTTP Server and Web Application Services in Oracle E-Business Suite in 12.2

Load Users manually from Active Directory to OID in SSO

Load User accounts from SSO to EBS (R12) manually

Just an other RMAN scenario : RMAN-08137

Replication : Post Processing issue in Shareplex Target Instance

APP-SQLAP-10771: could not reserve record

set java options in setdomainenv.sh

APP-SQLAP-10771: Could not reserve record.

Some useful scripts

Workflow related useful scripts - $FND_TOP/sql

Article 5

Article 4

DBCA fails with errors: [FATAL] [DBT-50000] Unable to check for available memory

DBCA fails with errors: [FATAL] [DBT-50000] Unable to check for available memory

Article 3

ORA-65259: partial synchronization of application is disallowed

ORA-65259: partial synchronization of application is disallowed

Article 2

DBConsole service won't start