Thursday, September 24, 2020

Oracle apps adpatch fails

Many times , when applying patch using adpatch in oracle apps, the patch session session gets terminated because of network problems. The best way to continue the patch is to open a new putty session,start the adpatch again and select "YES" for continue from previous run.The adpatch utility automatically detects where it left processing last time using the .rf9 files in $APPL_TOP/admin/$SID/restart and continue from there. You don't have to re-apply the whole patch again!

However sometimes it does not work and gives the following error-->"AutoPatch error: The worker should not have status 'Running' or 'Restarted' at this point.

Telling workers to quit...

All workers have quit.

AutoPatch error: adfrjp(): Error running jobs in workers"

This is because of inconsistency between fnd_install_processes, ad_deferred_jobs and .rf9 files.

In this case if you check the status of workers using adctrl, and select option "1. Show worker status" it shows the worker status as Running. This is actually the cause of the problem.

Now when we try to manually quit the workers using option "3. Tell worker to quit" then it says "Control code already set to 'Quit' for worker xxx". However if we again check the status of workers now then it still shows them "Running". Now this is a serious problem. Here's how to resolve it--->

select option "4. Tell manager that a worker failed its job" and enter the worker number for which you see the status "Running". Now if you check the worker status then it will change to "Failed"

Now you can select "2. Tell worker to restart a failed job" and continue with ad patch session.

This resolves the problem.

Also there is a different scenario, where in EBS, while applying a patch using adpatch you get the following error->

ATTENTION: All workers either have failed or are waiting: FAILED: file xxxx on worker x. ATTENTION: Please fix the above failed worker(s) so the manager can continue

Here the best approach id to locate the worker log file and check it for the error. The worker log files are located in $APPL_TOP/admin/$SID/log.They are in format->adworkerxx.log where xx is 01,02...The error is clearly mentioned in the log file. If suppose you don't get any clue even after getting the error,then search on metalink for that error. After rectifying the error do the following->

Run adctrl using applmgr. Choose option 1 "Show Worker Status". Verify the worker number that has status failed. Go back to main menu and choose option "2. Tell worker to restart a failed job". Enter the number of worker which you want to restart.The status will change to "fixed,restart" and the patch will progress.

Thursday, August 6, 2020

SOA webservices not working after 12c ERP db upgrade

After upgrading  database from oracle 11g to 12c in Oracle Apps R12.1.3,  got a strange issue where SOA webservices- SOAP, REST  stopped.

The following url showed 404

http://host.domain:8000/webservices/SOAProvider/?testquery

The issue was resolved after adding  following line in sqlnet.ora
SQLNET.ALLOWED_LOGON_VERSION=8

Had to bounce Apache, oafm after that.

utlrp hangs after upgrade to 12c ERP database

We were performing database upgrade from oracle 11g to oracle 12c with Oracle Apps 12.1.3. We faced an issue where database upgrade took < 30 mins. However, it made huge number of invalids ~170K. Compiling these invalid objects by utlrp took  several hours.  Finally was able to resolve this issue. Here is what was done-

  • Dropped non-working invalids. Database was compiling many invalids and using dblinks as there were some views/synonyms which were pointing to remote databases. Since, the dblinks were not working, so utlrp was getting struck there.
  • Increased Job queue processes- In oracle 12c,  job_queue_processes was 25 and it was taking 1 hour to compile invalids. Increased  it to 50 and invalids got compiled in half the time- 30 minutes.