Failed to get an exclusive lock, Unable to perform blackout operation. Please retry after some time

This issue is specific to the environment where the emctl blackout operation takes more than usual. i.e. More than the default of 60 seconds for the single execution. Hence it fails to get an exclusive lock within the 60 seconds window.

Snippet has being taken for orcl1 database;
$ cat <EM_INSTANCE_BASE>/em/<OMS_NAME>/sysman/log/emctl.log

4653462 :: Sat Sep 20 01:30:20 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 55 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:25 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 50 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:30 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 45 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:35 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 40 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:40 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 35 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:45 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 30 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:50 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 25 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:30:55 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 20 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:31:00 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 15 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:31:05 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 10 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:31:10 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 5 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:31:15 2014::Blackout oper: start, Name: orcl1 : Failed to acquire a lock, Give a try until 0 becomes zero, retflock = 0
4653462 :: Sat Sep 20 01:31:15 2014:: Blackout oper: start, Name: orcl1 : Failed to get an exclusive lock, Unable to perform blockout operation. Please retry after some time

The emctl.log file shows that the lock could not be acquired during the 60 seconds window for orcl1 database.

This failure results in unwanted alerts specially when database is down due to cold backup and OEM is unable to perform blackout.

Fix:

Edit the <ORACLE_HOME>\bin\commonenv.bat file and add the following line:

set EMAGENT_BLACKOUT_EXEC_TIME=120

On Unix, edit the commonenv file and add:

export EMAGENT_BLACKOUT_EXEC_TIME=120