How is this Table Used ? Analysis of eTRM/Object Dependencies
The eTRM can be used to identify the other objects and products that use the table AP.AP_INVOICES_ALL.
However, it will be necessary to view several levels of dependencies (referenced by) :
Table AP.AP_IMVOICES_ALL is referenced (in Oracle E-Business Suite 12.2.n) by conditioning view AP.AP_INVOICES_ALL# and subsequently by synonyms AP.AP_INVOICES, AP.AP_INVOICES_ALL and triggers in the IGI, ITG, JAI and OIE modules.
And in turn synonyms AP.AP_INVOICES, AP.AP_INVOICES_ALL are referenced by objects (views, packages, triggers) in modules:
· Receivables (AR)
· Bills of Material (BOM)
· Cash Management (CE)
· Cost Management (CST)
· Transportation Execution (FTE)
· Financials Common Modules (FUN)
· Federal Financials (FV)
· Oracle Environmental Accounting And Reporting (GHG)
· Process Manufacturing Financials (GMF)
· Grants Accounting (GMS)
· Payments (IBY)
· Oracle iProcurement (ICX)
· Contract Commitment (IGC)
· Public Sector Financials International (IGI)
· Oracle Landed Cost Management (INL)
· Inventory (INV)
· Internet Procurement Enterprise Connector (ITG)
· Asia/Pacific Localizations (JA)
· European Localizations (JE)
· Regional Localizations (JG)
· Latin America Localizations (JL)
· Assets (OFA)
· Lease and Finance Management (OKL)
· Projects (PA)
· Project Manufacturing (PJM)
· Property Manager (PN)
· Purchasing (PO)
· Sourcing (PON)
· iSupplier Portal (POS)
· Public Sector Financials (PSA)
· Payables (SQLAP)
· General Ledger (SQLGL)
· Treasury (XTR)
· E-Business Tax (ZX)
A hierarchy of objects that reference table AP.AP_INVOICES_ALL can be obtained by using the following SQL (iteratively for a few levels):
SELECT *
FROM dba_dependencies
WHERE referenced_owner = 'AP' and referenced_name ='<object_name>';
What Access Paths Are Used - Analysis of AWR views
Analysis of the AWR views can be used to identify the best choice for partition key(s) and also analyze index usage.
Information on the content of AWR views is available in :
My Oracle Support Document "Performance Diagnosis with Automatic Workload Repository (AWR) (Document 1674086.1)"> 14 AWR Views
Oracle Database Reference > Part II Static Data Dictionary Views
Note that AWR retention period will limit the time period that can be reported (the default is 8 days), so it may be necessary to repeat the analysis several times to cover key periods such as Month End.
SQL similar to the following will identify the most common SQLs that reference AP_INVOICES_ALL and it's dependent views:
SELECT * FROM
(SELECT st.sql_id,
SUM(ss.executions_delta) total_execs
FROM dba_hist_sqlstat ss,
dba_hist_sqltext st,
v$database d
WHERE st.dbid = ss.dbid
AND ss.dbid = d.dbid
AND st.sql_id = ss.sql_id
AND
(st.sql_text LIKE '%AP_INVOICE%'
OR
st.sql_text LIKE '%ADS_DOCUMENT_NUMBERS_V%'
OR
....
OR
st.sql_text LIKE '%ADS_INVOICE_DETAILS%'
OR
....
OR
st.sql_text LIKE '%PA_PROJ_AP_INV_DIST_V%'
OR
....
OR
st.sql_text LIKE '%PO_SCC_DASHBOARD_V%'
OR
....
)
AND st.command_type != 47 -- exclude anonymous blocks
GROUP BY st.sql_id)
ORDER BY total_execs DESC;
Note: Predicates to select ranges of snapshots (snap_id) could be added.
Additional delta columns could be added to identify other execution statistics (such as fetches, parses, disk reads, buffer gets, rows processed, cpu time, elapsed time etc).
Be careful to use the delta columns, which contain the totals for the snapshot, rather than the total columns, which contain cumulative totals. Also, if the AWR has been exported from the production environment and imported into a test environment, then the dbid could be explicitly specified (rather than joining with the current database id on view v$database).
The following query will return the SQL text for each SQL ID.
SELECT sql_id, sql_text
FROM dba_hist_sqltext
WHERE sql_id = '<sql_id>';
The following will give an indication of the bind values used and may indicate any skew, which may be relevant to the choice of partitioning method.
SELECT sql_id, name, value_string, count(*)
FROM dba_hist_sqlbind
WHERE sql_id = '<sql_id>'
GROUP BY sql_id, name, value_string ORDER BY sql_id, name;
SQL similar to the following will show usage of AP_INVOICES_ALL indexes.
SELECT n.owner,
n.object_name,
SUM(r.logical_reads),
SUM(r.physical_reads),
SUM(r.physical_writes),
SUM(r.physical_read_req),
SUM(r.physical_write_req)
FROM dba_hist_seg_stat_obj n,
(SELECT ss.dataobj#,
ss.obj#,
ss.dbid,
SUM(ss.logical_reads_delta) logical_reads,
SUM(ss.physical_reads_delta) physical_reads,
SUM(ss.physical_writes_delta) physical_writes,
SUM(ss.physical_read_requests_delta) physical_read_req,
SUM(ss.physical_write_requests_delta) physical_write_req
FROM dba_hist_seg_stat ss,
v$database d
WHERE ss.dbid = d.dbid
GROUP BY ss.dataobj#, ss.obj#, ss.dbid) r
WHERE n.dataobj# = r.dataobj#
AND n.obj# = r.obj#
AND n.dbid = r.dbid
AND n.object_name like '%AP_INVOICES%'
AND n.object_type like '%INDEX%' -- include INDEX and INDEX PARTITIONS
GROUP BY n.owner, n.object_name
ORDER BY n.owner, n.object_name;
Note that this will not include custom indexes if their name does not include the string 'AP_INVOICES'.
This will indicate the main columns used to access AP_INVOICES_ALL in filters (and joins) as well as reporting index usage.
If index usage has been monitored for any of the indexes (by using the command 'ALTER INDEX <index> MONITORING USAGE') then this will be recorded on DBA_OBJECT_USAGE / V$OBJECT_USAGE, but it will not indicate the level of usage.
The following will identify the access methods used for AP_INVOICES_ALL on actual execution plans:
SELECT p.object_type,
p.object_owner,
p.object_name,
p.operation,
p.options,
COUNT(*) plan_count,
SUM(p.cost),
SUM(p.cpu_cost),
SUM(p.io_cost),
SUM(p.temp_space)
FROM dba_hist_sql_plan p,
v$database d
WHERE p.dbid = d.dbid
AND p.object_name like '%AP_INVOICE%'
GROUP BY p.object_type, p.object_owner, p.object_name, p.operation, p.options
ORDER BY p.object_type, p.object_owner, p.object_name, p.operation, p.options;
Unfortunately, it does not indicate the number of executions for each execution plan line.
Similarly the Active Session History (ASH) can be analyzed to give some idea of the level of activity for each access path (and index):
SELECT s.current_obj#,
o.owner,
o.object_name,
o.object_type,
s.sql_plan_operation,
s.sql_plan_options,
COUNT(*) samples,
SUM(s.tm_delta_time) sampled_time,
SUM(s.tm_delta_cpu_time) cpu_time,
SUM(s.tm_delta_db_time) db_time
FROM dba_hist_active_sess_history s,
dba_objects o,
v$database d
WHERE s.dbid = d.dbid
AND o.object_id = s.current_obj#
AND s.sql_plan_operation IS NOT NULL
AND o.object_name like '%AP_INVOICE%'
GROUP BY o.object_type, o.owner, o.object_name, s.sql_plan_operation, s.sql_plan_options, s.current_obj#
ORDER BY o.object_type, o.owner, o.object_name, s.sql_plan_operation, s.sql_plan_options, s.current_obj#;
This could be further grouped by the columns program, module and action (if these have been populated) to ascertain where the main access/workload is occurring.
When identifying columns that are suitable for partition keys, the most suitable are those that are present across multiple tables, where the tables are joined using that column. Columns with low numbers of distinct values are unsuitable in some cases and those with high skew (including NULLs) are usually poor choices. So the following SQL can assist in identifying the groups of tables that can be partitioned on the same column(s).
SELECT c.owner,
c.table_name,
c.column_name,
c.nullable,
c.num_distinct,
c.num_nulls,
t.num_rows
FROM dba_tab_columns c,
dba_tables t
WHERE c.owner = t.owner
AND c.table_name = t.table_name
AND t.owner = '<owner>'
AND c.column_name = '<column_name>'
ORDER BY t.num_rows DESC, c.owner, c.table_name, c.column_name;
Index Analysis
The usage of indexes can be ascertained from the SQLs on AWR views DBA_HIST_ACTIVE_SESS_HISTORY, DBA_HIST_SEG_STAT_OBJ and DBA_HIST_SQL_PLAN above.
Analysis of the statistics (DBA_INDEXES) on the indexes can indicate the size of the indexes (num_rows, leaf_blocks), the selectivity (distinct_keys, avg_leaf_blocks_per_key, avg_data_blocks_per_key) and the clustering_factor.
SELECT owner,
index_name,
distinct_keys,
num_rows
FROM dba_indexes
WHERE table_name = 'AP_INVOICES_ALL'
ORDER BY index_name;
OWNER INDEX_NAME DISTINCT_KEYS NUM_ROWS
----- ---------------- ------------- ----------
AP AP_INVOICES_N1 92345 763824184
AP AP_INVOICES_N10 23896523 47739768
AP AP_INVOICES_N11 1243 24567123
AP AP_INVOICES_N12 56734 56734
AP AP_INVOICES_N13 15634 49872
AP AP_INVOICES_N14 3 567819879
AP AP_INVOICES_N15 2563 10734
AP AP_INVOICES_N16 5 954786784
AP AP_INVOICES_N17 0 0
AP AP_INVOICES_N18 954786784 954786784
AP AP_INVOICES_N19 14532 15324
AP AP_INVOICES_N2 35476 954786784
AP AP_INVOICES_N20 949831672 954786784
AP AP_INVOICES_N21 68 954786784
AP AP_INVOICES_N3 141 954786784
AP AP_INVOICES_N5 6532 954786784
AP AP_INVOICES_N6 939921731 954786784
AP AP_INVOICES_N7 67342 954786784
AP AP_INVOICES_N8 17435216 954786784
AP AP_INVOICES_U1 954786784 954786784
AP AP_INVOICES_U3 49872 49872
Analysis of dictionary view DBA_IND_COLUMNS indicates which columns are in each index:
SELECT index_owner,
index_name,
column_name,
column_position
FROM dba_ind_columns
WHERE table_name = 'AP_INVOICES_ALL'
ORDER BY index_name, column_position;
INDEX_OWNER INDEX_NAME COLUMN_NAME COLUMN_POSITION
----------- --------------- -------------------------- ---------------
AP AP_INVOICES_N1 BATCH_ID 1
AP AP_INVOICES_N10 PO_HEADER_ID 1
AP AP_INVOICES_N11 PROJECT_ID 1
AP AP_INVOICES_N11 TASK_ID 2
AP AP_INVOICES_N12 VOUCHER_NUM 1
AP AP_INVOICES_N13 DOC_SEQUENCE_VALUE 1
AP AP_INVOICES_N14 GLOBAL_ATTRIBUTE1 1
AP AP_INVOICES_N15 PAID_ON_BEHALF_EMPLOYEE_ID 1
AP AP_INVOICES_N16 WFAPPROVAL_STATUS 1
AP AP_INVOICES_N17 VALIDATION_REQUEST_ID 1
AP AP_INVOICES_N18 VENDOR_ID 1
AP AP_INVOICES_N18 INVOICE_NUM 2
AP AP_INVOICES_N18 ORG_ID 3
AP AP_INVOICES_N18 EARLIEST_SETTLEMENT_DATE 4
AP AP_INVOICES_N18 INVOICE_DATE 5
AP AP_INVOICES_N19 CREDITED_INVOICE_ID 1
AP AP_INVOICES_N2 VENDOR_ID 1
AP AP_INVOICES_N2 INVOICE_TYPE_LOOKUP_CODE 2
AP AP_INVOICES_N2 PAYMENT_STATUS_FLAG 3
AP AP_INVOICES_N20 INVOICE_NUM 1
AP AP_INVOICES_N20 PARTY_ID 2
AP AP_INVOICES_N20 PAYMENT_STATUS_FLAG 3
AP AP_INVOICES_N21 ORG_ID 1
AP AP_INVOICES_N21 APPROVAL_READY_FLAG 2
AP AP_INVOICES_N21 VALIDATION_REQUEST_ID 3
AP AP_INVOICES_N21 PAYMENT_STATUS_FLAG 4
AP AP_INVOICES_N21 HISTORICAL_FLAG 5
AP AP_INVOICES_N3 PAYMENT_STATUS_FLAG 1
AP AP_INVOICES_N3 PAY_GROUP_LOOKUP_CODE 2
AP AP_INVOICES_N3 ORG_ID 3
AP AP_INVOICES_N5 INVOICE_DATE 1
AP AP_INVOICES_N5 ORG_ID 2
AP AP_INVOICES_N6 INVOICE_NUM 1
AP AP_INVOICES_N7 VENDOR_SITE_ID 1
AP AP_INVOICES_N8 CREATION_DATE 1
AP AP_INVOICES_U1 INVOICE_ID 1
AP AP_INVOICES_U3 DOC_SEQUENCE_ID 1
AP AP_INVOICES_U3 DOC_SEQUENCE_VALUE 2
Conclusion for AP_INVOICES_ALL
The primary performance concern is the length of time that batch processes are taking (e.g. load/interface, validation, payment and accounting) and the contention between different sessions/workers on these processes. Other means of reducing the contention on the interface and other batch processes have been carried out (e.g. striping, large extents to avoid High Water mark waits, SQL Tuning to reduce buffer busy waits etc), but there are still "hot blocks". These occur on the tables and on some of the indexes.
And of course there are downstream wait events caused by the delays (e.g. read by other session, enq: TX – contention etc.)
So a partitioning method of HASH is chosen to randomly and evenly distribute data across partitions to reduce the probability of "collisions" between multiple workers/sessions on "hot blocks".
The number of partitions should be high enough to avoid the contention. Too high a value will create additional administration and could cause performance issues on locally partitioned indexes. In this case a value of 8 is chosen. Note that all tables (of the same group), which share the same partition key and are often joined together by that key, should have the same number of hash partitions.
The analysis of AWR/eTRM has revealed that the largest number of filters and joins are on the column INVOICE_ID, which is a unique key.
So INVOICE_ID is chosen as the partition key. Because INVOICE_ID is unique (there is no skew) it will be distributed completely evenly across all 8 partitions.
It will also enable significant use of partition pruning and partition-wise joins.
Analysis of the AWR/eTRM has also revealed that table AP_INVOICES_ALL is frequently joined to tables AP_INVOICE_DISTRIBUTIONS_ALL and AP_INVOICE_LINES_ALL using INVOICE_ID. These tables could be candidates for hash partitioning too, using the same partition key: INVOICE_ID.
There are many other tables in Oracle Payables and other modules that contain the same column INVOICE_ID (e.g. Payables Localizations (JA, JE, JG, JL), Cost Management (CST), Federal Financials (FV), Payments (IBY), Public Sector Financials International (IGI), Assets (OFA), Projects (PA) etc). Some of these might be candidates for has partitioning by invoice_id, but it is more likely that they will not:
· Many will be much smaller tables and/or only be populated for a subset of invoices (e.g. AP.AP_HOLDS_ALL, AP.AP_MC_INVOICES, AP.AP_INVOICE_PREPAYS_ALL, AP.AP_TRIAL_BALANCE, Localization tables, Projects tables).
· Others will be temporary tables (e.g. AP.AP_DIST_LINE_GT) or may not be used by the customer (e.g. AP.AP_HISTORY_INVOICES_ALL, IGI.IGI_AP_INVOICES_ALL).
· The interface tables AP.AP_INVOICES_INTERFACE, AP.AP_INVOICE_LINES_INTERFACE contain INVOICE_ID and maybe candidates for HASH partitioning by INVOICE_ID, however they may not often be joined with AP_INVOICES_ALL and other tables.
· Other tables may contain INVOICE_ID and be populated for all invoices, but they may contain other columns that are more frequently used as joins/filters ( e.g. AP.AP_INVOICE_PAYMENTS_ALL (check_id, invoice_payment_id) and AP.AP_LIABILITY_BALANCE(ae_line_header_id, ae_line_id)).
The choice of whether to partition the indexes, locally or globally could be as follows :
Many of the indexes below will be less efficient (for queries) if locally partitioned. However, the prime concern is contention during the load (interface) and other batch processes, so local will be preferred
· AP.AP_INVOICES_U1 (INVOICE_ID) - This index is prefixed by the partition key, so local is the ideal choice. It grows monotonically (index insertions occur on the right edge) and has a high number of distinct keys (unique) so hash partitioning by the same partition key as the table is ideal. Local Partitioned (prefixed).
· AP.AP_INVOICES_U3 (DOC_SEQUENCE_ID, DOC_SEQUENCE_VALUE) - The columns in this index are not populated (at any stage) for the bulk of invoices on this system (which are imported from another source). So there is no contention during any of the batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N1(BATCH_ID) - This column is populated (for all invoices from the external source, i.e. 80%) during the load/interface and not updated after that. However, each batch, typically contains around 8,000 rows. They are likely to all be in the same blocks (the index grows monotonically), but they are likely to be spread across more than 15 blocks, so the overhead of accessing by batch_id on queries and during subsequent batch processes, when the index is partitioned locally, is not high. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N2 (VENDOR_ID, INVOICE_TYPE_LOOKUP_CODE, PAYMENT_STATUS_FLAG) - This index is populated for all invoices during entry/interface and updated during payment. There is an average of around 27,000 rows for each distinct key. The index does not grow monotonically, so those values are likely to be spread across a high number of blocks. There will be some vendors with relatively few invoices, but still the overhead of local partitions on queries will be relatively low. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N3 (PAYMENT_STATUS_FLAG, PAY_GROUP_LOOKUP_CODE, ORG_ID) - This index is populated for all invoices during entry/interface and updated during payment. There are around 6.8 million rows for each distinct key. The index does not grow monotonically, so those values are likely to be spread across a high number of blocks. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N5 (INVOICE_DATE, ORG_ID) - This index is populated for all invoices during entry/interface, it is not updated after that. It has a low number of distinct values, there are 146,000 values for each distinct key. It also grows monotonically. It is not extensively used and is mostly used to return a range of values. The overhead of local partitions will be low. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N6 (INVOICE_NUM) - This index is populated for all invoices during entry/interface, it is rarely updated after that. It has a very high number of distinct values, it is nearly unique. It is also populated for all invoices. Rows are usually added at the same points in the index (each of the large vendors uses ranges of invoice numbers). It is used extensively for queries and the overhead of local partitions would be high. Global Non-Partitioned.
· AP.AP_INVOICES_N7 (VENDOR_SITE_ID) - This index is populated for all invoices during entry/interface, it is not updated after that. This index has an average of around 14,000 rows for each distinct key. It does not grow monotonically. There will be some vendor_sites with relatively few invoices, but still the overhead of local partitions on queries will be relatively low. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N8 (CREATION_DATE) - This index is populated for all invoices during entry/interface, it is not updated after that. It has a high number of distinct values, there are only around 54 values for each distinct key. It also grows monotonically. It would be a candidate for Global Non-Partitioning. However, it is not extensively used and is mostly used to return a range of values. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N10 (PO_HEADER_ID) - The columns in this index are not populated (at any stage) for the bulk of invoices on this system (which are imported from another source). So there is no contention during any of the batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N11 (PROJECT_ID, TASK_ID) - The columns in this index are not populated (at any stage) for the bulk of invoices on this system (which are imported from another source). So there is no contention during any of the batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N12 (VOUCHER_NUM) - The column in this index is not populated (at any stage) for the bulk of invoices on this system (which are imported from another source). So there is no contention during any of the batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N13 (DOC_SEQUENCE_VALUE) - The column in this index is not populated (at any stage) for the bulk of invoices on this system (which are imported from another source). So there is no contention during any of the batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N14 (GLOBAL_ATTRIBUTE1) - This index has a very low number of distinct keys. There are a very high number of values (millions) for each key. There will be no discernible overhead from local partitioning. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N15 (PAID_ON_BEHALF_EMPLOYEE_ID) - The column in this index is not populated (at any stage) for the bulk of invoices on this system (which are imported from another source). So there is no contention during any of the batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N16 (WFAPPROVAL_STATUS) - This index has a very low number of distinct keys. There are a very high number of values (100,000s) for each key. There will be no discernible overhead from local partitioning. Local Partitioned (Non-prefixed).
· AP.AP_INVOICES_N17 (VALIDATION_REQUEST_ID) - This column is normally not populated. Global Non-Partitioned.
· AP.AP_INVOICES_N18 (VENDOR_ID, INVOICE_NUM, ORG_ID, EARLIEST_SETTLEMENT_DATE, INVOICE_DATE) - This index is populated for all invoices during entry/interface, it is rarely updated after that (just for prepayments). It has a very high number of distinct values, it is nearly unique. It does not grow monotonically. It is used extensively for queries and the overhead of local partitions would be high. Global Non-Partitioned.
· AP.AP_INVOICES_N19 (CREDITED_INVOICE_ID) - This column is only present for a low number of invoices. It will rarely be inserted to or updated during batch processes. Global Non-Partitioned.
· AP.AP_INVOICES_N20 (INVOICE_NUM, PARTY_ID, PAYMENT_STATUS_FLAG) - This index is populated for all invoices during entry/interface. It is updated after that (during payment). It has a very high number of distinct values, it is nearly unique. It does not grow monotonically. It is used extensively for queries and the overhead of local partitions would be high. Global Non-Partitioned.
· AP.AP_INVOICES_N21 (ORG_ID, APPROVAL_READY_FLAG, VALIDATION_REQUEST_ID, PAYMENT_STATUS_FLAG, HISTORICAL_FLAG) - This index has a very low number of distinct keys. Most distinct keys have a high number of values for each key. There will be no discernible overhead from local partitioning. Local Partitioned (Non-prefixed).