OEM Agent 13c ERROR Targets.xml was loaded with the wrong agent token

Encountering the “OEM Agent 13c ERROR Targets.xml was loaded with a wrong agent token” poses a critical challenge, preventing the OEM Agent from starting up. Despite seeking solutions from MOS Doc 1384366.1 for version 12c, additional steps were necessary to resolve the issue on OEM 13.4. Let’s delve into understanding and addressing this problem.

Error Targets.xml file loaded with a wrong agent token

So we encountered the following problem when attempting to restart an OEM Agent on a server. For context, the server was experiencing extreme load, due to a problem with the hardware and the Agent had crashed.

As the error states, there seems to be a problem with the AGENT_TOKEN in the Targets.xml file. The “targets.xml” file, serves as a repository of configuration data and metadata related to these monitored targets. It contains essential information such as target names, types, properties, and status. OEM manages this file, and regularly updates this file to reflect changes in the monitored environment, such as new targets being added, existing targets being removed or modified, etc.

Solution

The problem is that OMS has changed the TOKEN for the Agent in question, however, the information has not been synced with the OEM Agent. So to do so we need to modify targets.xml

Note: Oracle does not support manual editing of the targets.xml files unless the procedure is explicitly documented or you are instructed to do so by Oracle Support.

1: Ensure that the OEM Agent is down and clean up any leftover processes

emctl stop agent -force

$ps -ef | grep emagent

2: Backup your targets.xml and also old files from /agent_inst/sysman/emd/state/

3: Re-create target.xml with only the following contents

<Targets>
</Targets>

4: You should be able to restart the agent, and if you compare AGENT_TOKEN values you will see the new AGENT_TOKEN. However, do know that the Targets.xml is empty; try to run emctl config agent listtargets and it will not list any targets. So as a next step, we will resync the agent to populate the target information in the file

5: Block the Agent

Before you can resync the agent, you need to block it. If you do not you will encounter the error as below.

To block the Agent
Setup -> Manage Cloud Control -> Agent -> Select the Agent -> Click on Block button -> OK

6: Resynchronize the Agent

From the GUI Setup -> Manage Cloud Control -> Agent -> click on the Agent name link -> Agent Menu -> Resynchronization -> Click on the job name -> Be sure the job runs successfully

Remember to choose to unblock the Agent at the end of the resync job. You could alternatively use the cmd line to run the resync job, I tried but it failed, so then I tried it via the Console, and that worked well.

Voila, we are back in business! Although, it’s unclear why the Agent couldn’t update the new token in the targets.xml, getting it back via the documented procedures helped fix the problem.