Adop Phase fs_clone is Failing on TxkADOPPreparePhaseSynchronize.pl

Error:

   [EVENT]     Performing CLONE steps...
   [PROCEDURE] [START 2019/09/11 10:21:32] CLONE Patch File System from Run File System
   [PROCEDURE] Calling: /u01/app/ebsint2/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl
   [EVENT]     Log: /u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/fs_clone_20150811_101857/ebsint2_ebsx01appd10
   [UNEXPECTED]Error occurred while executing perl /u01/app/ebsint2/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl

Cause:

A port was in use while the "adop phase=fs_clone" process was running.

Output of "adopscanlog -latest=yes" shows the port in use:
==========================================================
$ adopscanlog -latest=yes
Scanning /u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/ directory ...
/u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/adop_20150811_214930.log:
---------------------------------------------------------------------
Lines #(69-73):
  [PROCEDURE] Calling: /u01/app/ebsint2/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl
  [EVENT] Log: /u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/fs_clone_20150811_214930/ebsint2_ebsx01appd10
  [UNEXPECTED]Error occurred while executing "perl /u01/app/ebsint2/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl -contextfile=/u01/app/ebsint2/fs1/inst/apps/ebsint2_ebsx01appd10/appl/admin/ebsint2_ebsx01appd10.xml -patchctxfile=/u01/app/ebsint2/fs2/inst/apps/ebsint2_ebsx01appd10/appl/admin/ebsint2_ebsx01appd10.xml -phase=fs_clone -logloc=/u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/fs_clone_20150811_214930/ebsint2_ebsx01appd10 -promptmsg=hide"
  [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on ebsx01appd10
[PROCEDURE] [START 2019/08/11 21:51:23] Unlocking sessions table
Lines #(70-74):
  [EVENT] Log: /u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/fs_clone_20150811_214930/ebsint2_ebsx01appd10
  [UNEXPECTED]Error occurred while executing "perl /u01/app/ebsint2/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl -contextfile=/u01/app/ebsint2/fs1/inst/apps/ebsint2_ebsx01appd10/appl/admin/ebsint2_ebsx01appd10.xml -patchctxfile=/u01/app/ebsint2/fs2/inst/apps/ebsint2_ebsx01appd10/appl/admin/ebsint2_ebsx01appd10.xml -phase=fs_clone -logloc=/u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/fs_clone_20150811_214930/ebsint2_ebsx01appd10 -promptmsg=hide"
  [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on ebsx01appd10
[PROCEDURE] [START 2019/08/11 21:51:23] Unlocking sessions table
[STATEMENT] Unlocking ad_adop_sessions table for ebsx01appd10 with wait interval of 60 seconds and number of tries 2

/u01/app/ebsint2/fs_ne/EBSapps/log/adop/10/fs_clone_20150811_214930/ebsint2_ebsx01appd10/ADOPValidations_detailed.log:
-----------------------------------------------------------------------------------------------------------------------
Lines #(425-429):
  ONS Local Port available: Port Value = 6190
Checking ONS Remote Port on ebsx01appd10: Port Value = 6290
RC-50204: Error: - ONS Remote Port in use: Port Value = 6290
Checking ONS Request Port on ebsx01appd10: Port Value = 6590
  ONS Request Port available: Port Value = 6590

Solution:

Test the action plan below in a development instance and migrate accordingly:

1. Stop all the services from the RUN and PATCH file-system.
2. Make sure no Oracle processes are running via the following command: ps -ef
3. Confirm the port 6290 is free. For example: "netstat -nlp | grep 6290" should not show any attached process.
If some processes are still attached, reboot the OS.
4. Start the services from the Run filesystem.
5. Re-run "adop phase=fs_clone force=yes"