Surprisingly, a popular question is about the possibility of using the Enterprise Manager (EM) Job System to replace customer’s numerous cron jobs. The answer is obviously YES! customers would have moved to use more enterprise-class job schedulers instead of cron. So, here is a quick post on how to get started with this conversion from cron to EM Jobs for some of our new users.

The script is scheduled in OEM as an “OSCommand” job and assigned to an OMS agent (as it needs to run an OS command calling EMCLI and passing it the script name and parameter needed.  The script must also be installed on the OMS servers in a place accessible by the OEM agent on these servers.
The EMCLI script should be placed on the OMS servers’ shared storage in order to allow access to the script from either node.

The OSCommand job in OEM is scheduled to call the EMCLI script and passing it the parameters needed to control the job specific to the environment or type of job.  This job can and most likely will be scheduled several times depending on the scheduling times, level of concurrency and other parameters specific to a set of targets. The parameters are listed below.

The parameters for the script are:

• Group Name
• Backup Job Prefix Name
• Level of Concurrency
• Debug level

The specific job for group-based backups works a follows:

• Called via an OSCommand using EMCLI
• Parameters include a top-level group
  1. The group and any subgroups in the hierarchy are traversed and all “oracle_database” targets are selected for potential backup.
     1. As each “oracle_database” is processed, the script will determine where the primary instance is located
  2. Once the primary database has been located, the script will create a new OEM job and execute that job
     1. The job name has a prefix that is controllable via a runtime parameter
     2. The job name will include the DBName as part of the job name

• The backup job created is an OSCommand job type that calls the “RMAN_BACKUP Script” and passes it the required parameters

1. As the script is starting backups for the of targets, it keeps track of currency and will pause execution of more jobs until there is room for more concurrency, as defined by the Concurrency runtime parameter
   1. The script queries the OEM job system for jobs with the backup prefix name and a running status (2)
2. Job continues until all targets have been processed.

Not all targets processed will have a backup started, for example, physical standby databases will be skipped, as the primary database will be backed up instead of the standby (Data Guard only mode).

Configuration information:

Wrapper script used as the main backup job. This is the job that is defined and scheduled for each group or tier.

dbadeeds:/scripts/dev>

oracle:OMS $ emcli @RMAN_DEEDS_GRP_DAILY.py Test-DEEDS-Grp nars01 2 0

Inside this RMAN_DEEDS_GRP_DAILY.py python script Calls below oscommand

oscommand=”variable.default_shell_command=/u01/app/oracle/rmanfull_bck.sh -sid ” + sidname + ” -nocat – parfile ” + “\n”

RMAN_DEEDS_GRP_DAILY.py

#Import all emcli verbs to current program
# Author : AEG & Sandeep R Narani
# Version v2 : Enchanced the parfile information

from emcli import *
import sys
from datetime import date
#from java.text import DateFormat
#from java.text import SimpleDateFormat
#from java.util import Calendar
import datetime
from java.lang import String
from jarray import array
#import javaArray
import jarray
import java.util.ArrayList as ArrayList
import time

# Things to do:
# 1) Load all target properties into a hash or easy lookup

# Arguments to Script (in order)
# ----------------------------------
# 1 = GroupName in OEM
# 2 = Backup Job Name Prefix, name prefix for all backup jobs created (one per DB)
# 3 = Level of concurrency for this tier
# 4 = debug level - 0 = info only, 1 = debug level output

# Process Command line Args

if (len(sys.argv) < 1 ):
print "Usage: ./emcli @script.py    "
#sys.exit(1)
exit(1)

dbglevel = 3

# Variables from ArgV
GroupName=sys.argv[0]
BackupJobPrefix=sys.argv[1]
Concurrency=sys.argv[2]

if (len(sys.argv) == 4):
dbglevel=int(sys.argv[3])
else:
# default debug level
dbglevel=0

# Global variables used in processing
SleepTime=90
SleepTime2=45
numrunningjobs=0
inc_a = 0
inc_b = 0
inc_c = 0

def log(linestr):
print linestr

def logdbg(linestr):
if dbglevel == 0:
print linestr

def logdbg2(linestr):
if dbglevel > 2:
print linestr

def print_target_details(target):
logdbg2("TargetName=" + target['Target Name'] + " TargetType=" + target['Target Type'])

def print_info(res):
logdbg2(' (func) - -----------------------------------')
logdbg2(' (func) - Results from Target query')
#logdbg2(' (func) - Number of targets:' + str(len(res.out()['data'])))
logdbg2(' (func) - EMCLI Errors :' + res.error())
logdbg2(' (func) - EMCLI Exit code :' + str(res.exit_code()))
logdbg2(' (func) - IsJson :' + str(res.isJson()) )

def return_error (res):
logdbg2(' (func) - EMCLI Errors :' + res.error() )
logdbg2(' (func) - EMCLI Exit code :' + str(res.exit_code()) )
return res.error()

def return_count (res):
logdbg(" (func) - Results from EMCLI call - NumberOfTargetsReturn:" + str(len(res.out()['data'])) )
return len(res.out()['data'])

def get_target_role_info (tgt_name):
sqltext0=" select distinct t.target_name,t.target_type,p.property_name, p.property_value,t.host_name "
sqltext1=" from sysman.mgmt$target t, sysman.mgmt$target_properties p "
sqltext2=" where t.target_guid = p.target_guid and t.target_name='" + tgt_name + "' "
sqltext3=" and t.target_type like 'oracle_database' and p.property_name='DataGuardStatus' "

longsql=sqltext0 + sqltext1 + sqltext2 + sqltext3
obj = list(sql=longsql)
return obj

def get_target_role_info_x (tgt_name):
sqltext0=" select t1.target_name,t1.target_type,t1.type_qualifier3,t1.type_qualifier4,t1.host_name,t1.property_name t1_property_name, "
sqltext1=" t1.property_value t1_property_value,t2.property_name t2_property_name, t2.property_value t2_property_value "
sqltext2=" from (select t.target_name, t.target_type, t.type_qualifier3, t.type_qualifier4, t.host_name, p.property_name, p.property_value "
sqltext3=" from sysman.mgmt$target t, sysman.mgmt$target_properties p "
sqltext4=" where t.target_name='" + tgt_name + "' and t.target_guid=p.target_guid and t.target_type like 'oracle_database' and p.property_name='DataGuardStatus' and p.property_value like 'Standby%' "
sqltext5=" order by target_name ) t1, "
sqltext6=" ( select t.target_name,t.target_type,t.type_qualifier3,t.type_qualifier4,t.host_name,p.property_name,p.property_value from sysman.mgmt$target t, sysman.mgmt$target_properties p "
sqltext7=" where t.target_name='" + tgt_name + "' and t.target_guid = p.target_guid and t.target_type like 'oracle_database' and p.property_name='orcl_gtp_lifecycle_status' "
sqltext8=" and t.type_qualifier4 like 'Primary' order by target_name ) t2 "
sqltext9=" where t1.target_name=t2.target_name and t1.target_type=t2.target_type and t1.type_qualifier3=t2.type_qualifier3 and t1.type_qualifier4=t2.type_qualifier4 and t1.host_name=t2.host_name "

longsql=sqltext0 + sqltext1 + sqltext2 + sqltext3 + sqltext4 + sqltext5 + sqltext6 + sqltext7 + sqltext8 + sqltext9
obj = list(sql=longsql)
return obj

def loaddict(dictobj,key,value,dictname):
logdbg2(" calling loaddict() add to dictionary--> " + dictname + " object... key-> " + key + " value --> " + value )
if key in dictobj:
logdbg2(" *** key (" + key + ") exists in " + dictname + " - append new data to existing values" )
existing = dictobj[key]
dictobj[key] = existing + ";" + value
else:
logdbg2( " *** key (" + key + ") exists in " + dictname + " - add new key/value entry " )
dictobj[key] = value
return dictobj

def gettargetproperties():
# Get a target's property value (returns a string value for the poperty value)
# Return-> {'PROPERTY_VALUE': 'DB12102', 'PROPERTY_NAME': 'DBName'}
sql0="select t.target_name,t.target_type,t.host_name,p.property_name, p.property_value "
sql1="from sysman.mgmt$target t, sysman.mgmt$target_properties p "
sql2="where t.target_guid=p.target_guid "
sql3="and t.target_type IN ('oracle_database','oracle_home','host') "
sql4="and ( property_name IN ('AdrBase','AdrHome','CDB','CPUCount','DBDomain','DBName','DBVersion','DataGuardStatus') "
sql5=" or property_name IN ('InstanceName','MachineName','OpenMode','OracleHome','SID','Version','VersionCategory') "
sql6=" or property_name IN ('orcl_gtp_department','orcl_gtp_lifecycle_status','orcl_gtp_target_version','INSTALL_LOCATION') )"
sql7=" or property_name IN ('CRSVersion','CRSVersionCategory','isSiha','NodeName') ) "
#sql4="and property_name='" + prop_name + "'"
long_sql = sql0 + sql1 + sql2 + sql3 + sql4 + sql5 + sql6
logdbg2(" running gettargetproperties() ")
obj = list(sql=long_sql)
logdbg2(" running --> objreturn = list(sql=long_sql) ")
for objline in obj.out()['data']:
logdbg2(" objline-> " + str(objline) )
#retpval = obj['PROPERTY_VALUE']
#retpname = obj['PROPERTY_NAME']
#logdbg2(" gettargetpropertyvalue - retpval->'" + retpval + "' retpname->'" + retpname + "'" )
return obj

def printdict (dobj,name):
cnt=0
logdbg2( " **** printing all key-values in dictionary->'" + name + "'" )
for key,value in sorted( dobj.iteritems() ):
cnt += 1
cnt2=0
logdbg2( " #" + str(cnt) + " = key->'" + key + "' value->'" + value + "'" )
# split the text in 'value'
kwords = key.split(";")
for word in kwords:
cnt2 += 1
logdbg2( " keysplit#" + str(cnt2) + " -->'" + word + "'")
vwords = value.split(";")
# for each word in the string
for word in vwords:
cnt2 += 1
logdbg2 ( " valsplit#" + str(cnt2) + " -->'" + word + "'" )
print "\n"

# --------------------------------------
# Start of Processing
# --------------------------------------

log ("\n *********** New Run ***********")
log ("\nStarting job for all database targets in group--> '" + GroupName)
log ("\nRuntime Arguments passed to job:\n")
log (" GroupName =" + GroupName)
log (" BackupJobPrefix=" + BackupJobPrefix)
log (" Concurrency =" + str(Concurrency))
log (" DebugLevel =" + str(dbglevel))
log ("")

# Connect to OMS URL and Login
# run emcli sync


set_client_property('EMCLI_OMS_URL','https://dbadeeds.com/em')
set_client_property('EMCLI_TRUSTALL','true')
log("Logging into the OEM system--> https://dbadeeds.com/em")
login(username='SYSMAN',password='Welcome123',force=True)


# Load Properties dict (loads all target properties into a dict obj

propdict={}
logdbg(" load 'propdict' with all target properties.....")
prop_array = gettargetproperties()
for lineitem in prop_array.out()['data']:
logdbg2(" lineitem (prop_array)--> " + str(lineitem) )
tname = str(lineitem['TARGET_NAME'])
thost = str(lineitem['HOST_NAME'])
tproperty = str(lineitem['PROPERTY_NAME'])
tvalue = str(lineitem['PROPERTY_VALUE'])
key = tname + ";" + tproperty
value = tvalue
propdict = loaddict(propdict,key,value,"propdict")


logdbg2( "\n")
logdbg2( "length of propdict{} --> " + str(len(propdict)) )
logdbg2( "")
logdbg2( "now print out key-values from each dictionary object")
printdict(propdict,"propdict")

# Now loop through list of targets found through group hierarchy

try:
log ("\nGetting all group members using get_group_members() to find all targets in the group hierarchy\n")
#targets_array = get_group_members(name='Omah-Internal Testing ENV-SoE-Grp',depth='-1')
targets_array = get_group_members(name=GroupName,depth='-1')
logdbg2(" loaded all targets from gorup query into --> targets_array ")
logdbg2(print_info(targets_array))
logdbg2 ("\n looping through all targets returned from group query......")
for tgt in targets_array.out()['data']:
downtarget="N"
# FInd and count total targets returned in group hierachy
logdbg2("\n**** new target line--> " + str(tgt) )
targetname = str(tgt['Target Type'])
inc_a += 1 
logdbg2("\n checking if newtarget->'" + str(tgt['Target Type']) + "' is an oracle_database")
logdbg2("\n targetname-->'" + targetname + "'")
if tgt['Target Type'] == 'oracle_database':
logdbg2("\n *** new Oracle DB found--> Target Name: " + tgt['Target Name'] + " is an " + tgt['Target Type'] + " - checking if primary" )
logdbg2(print_target_details(tgt))
logdbg2(' processing --> TargetName=' + tgt['Target Name'] + ' TargetType=' + tgt['Target Type'] )

# check on blackout
blackouttgt=tgt['Target Name'] + ":oracle_database"
blackoutstatus = get_blackouts( target=blackouttgt)
blackoutflag = "N"
blackoutflag = ""
logdbg(" checking blackout *** blackouttgt key --> " + str(blackouttgt) + " - blackoutflag->" + str(blackoutflag) )
for bstatusline in blackoutstatus.out()['data']:
logdbg2 (" bstatusline---> " + str(bstatusline) ) 
blackoutflag = "Y"

# check up or down target status
status = get_targets(targets='oracle_database')
for statusline in status.out()['data']:
tgt_status = str(statusline['Status'])
tgt_statusid = str(statusline['Status ID'])
if tgt['Target Name'] == statusline['Target Name']:
logdbg2(" ttgt_status_line--> " + str(statusline) )
logdbg2(" --> found statuline for target --> " + str(tgt['Target Name']) + " - " + str(statusline['Target Name']) )
logdbg2(" --> status is = '" + tgt_status )
logdbg2(" --> status is = '" + tgt_statusid )
if int(tgt_statusid) == 1:
logdbg (" target(" + statusline['Target Name'] + ") is UP --> " + tgt_status )
else:
logdbg (" target(" + statusline['Target Name'] + ") is DOWN --> " + tgt_status + " -- skipping" )
downtarget="Y"
break
#else:
# continue

# check to ensure database is not a standby (in readonly mode) 
# ------------------------------------------------------------

logdbg2(" CHecking DataGuardStatus compare to see if target is a 'Physical Standby'")
dgstatkey = str(tgt['Target Name']) + ";DataGuardStatus"
logdbg2(" check to see if dgstatkey->'" + dgstatkey + "' is in propdict......" )
if dgstatkey in propdict: 
logdbg2( " key-> '" + dgstatkey + "' found in propdict")
dgstatus = propdict[dgstatkey] 
else:
logdbg2( " key-> '" + dgstatkey + "' not found in propdict")
dgstatus = "None"

logdbg2(" DataGuardStatus compare dgstat->'" + dgstatus + "' == 'Physical Standby' target->'" + str(tgt['Target Name']) + "'" )
logdbg2(" if Physical Standby role - skip target .......") 
logdbg2(" DataGuardStatus is dgstatkey-> " + dgstatkey + " value-> " + dgstatus )
logdbg2(" DataGuardStatus compare dgstat->'" + dgstatus + "' == 'Physical Standby'" )

if ( downtarget == "Y" or blackoutflag == "Y"):
logdbg (" ***** continue to next tarrget....skipping")
downtarget="N"
continue

logdbg2("Runing SQL to collect role information (l_target_arry) (primary or standby) from OMR")
#logdbg(print_info(l_target_info))


l_target_info = get_target_role_info(tgt['Target Name'])
for target in l_target_info.out()['data']:
inc_b += 1
logdbg2(" now processing target#" + str(inc_b) + " (from l_target_info)--> " + str(target) )
#logdbg(" Tgt Info --> TargetName=" + target['TARGET_NAME'] + " TargetType=" + target['TARGET_TYPE'] + " Role=" + target['TYPE_QUALIFIER4'] + " TQ3=" + target['TYPE_QUALIFIER3'])


logdbg(" comparing dgstatus->'" + dgstatus + "' to 'Physical Standby'" + "for " + str(tgt['Target Name']) )
if dgstatus != 'Physical Standby':
inc_c += 1
logdbg2(" target -->" + str(target) )
logdbg2(" target is not a 'Physical Standby' - DataGuardStatus = '" + str(target['PROPERTY_VALUE']) + "'" ) 
logdbg2(" dgstatus-->'" + dgstatus + "'")
#
tn = target['TARGET_NAME']
new_tn = str(tn[0:10])
tt = target['TARGET_TYPE']
#tq4 = target['TYPE_QUALIFIER4']
hn = target['HOST_NAME']
pn = target['PROPERTY_NAME']
pv = target['PROPERTY_VALUE']
# Pull SID from propetty SID - canont use target name, unreliable
sidkey = tn + ";SID"
sidname = propdict[sidkey]
#sidparts=tn.split("_")
#logdbg (" sid is --> " + str(sidparts) ) 
logdbg (" target-->" + tn + " sidname is --> " + str(sidname) + " host-> " + hn ) 
#sid=str(sidparts[0])
#fn = target['T2_PROPERTY_NAME']
logdbg2(" getting LifeCycle_Status property for tn -> '" + tn + "'") 
ftkey = str(tn) + ";orcl_gtp_lifecycle_status"
ft = propdict[ftkey]
logdbg2(" got lifecycle_status ft-> " + str(ft) )
#ft = "TesT"
#ftc = "UN"
# COnvert LifeCYcle Status to two letter Code
if ft == 'MissionCritical':
ftc = "MC"
elif ft == 'Production':
ftc = "PR"
# Do the other thing
elif ft == 'Test':
ftc = "TS"
# Fall-through by not using elif, but now the default case includes case 'a'!
elif ft == 'Development':
ftc = "DE"
# Do yet another thing
elif ft == 'Staging':
ftc = "SG"
# Do yet another thing
else:
ftc = "UN"

logdbg2("#" + str(inc_c) + " TargetInfo-> TargetName=" + target['TARGET_NAME'] + " TargetType=" + target['TARGET_TYPE'] + " Role=" + target['PROPERTY_VALUE'] + " LCS=" + ft )
#testvar1 = "TestString" 
#testvar2 = str(testvar1[0:6])
#print "testvar1--> " + testvar1 + " testvar2--> " + testvar2 + " new_tn--> " + new_tn
log ("#" + str(inc_c) + " Preparing backup for -> Name=" +tn + " Type=" + tt + " sidname--> " + sidname)
# Get Date
timenow = datetime.datetime.now()
now=timenow.strftime("%Y-%m-%d_%H-%M")
#now=timenow.strftime("%m-%d-%y_%H")
# Create the Properties file
jobfilename=BackupJobPrefix + "-" + ftc + ".txt"
#my_file = open('RMAN_BACKUP Script_job.txt','w')
my_file = open(jobfilename,'w')
headerstr="\n#-- *** new job ** for " + tn + " : " + tt + " -> " + hn + " --\n"
my_file.write(headerstr)
#job_name="name=RMANBCKUP-" + tn + "_" + str(now) + "\n"
job_prefix=BackupJobPrefix + "-" + ftc
job_prefix_wildcard=BackupJobPrefix + "-" + ftc + "%"
job_name="name=" + BackupJobPrefix + "-" + ftc + "-" + new_tn + "_" + str(now) 
logdbg2(" job_prefix->'" + job_prefix + "' job_prefix_wildcard->'" + job_prefix_wildcard + "'")
logdbg(" -----> job_name = '" + job_name + "' for target = '" + tn + "'" )
my_file.write(job_name + "\n")
my_file.write("type=OSCommand\n")
my_file.write("targetType=host\n")
my_file.write("description=Weekly Backup Job for Database - Level 0\n")
my_file.write("owner=SYSMAN\n")
my_file.write("kind=active\n")
credentials="cred.defaultHostCred.:host=NAMED:SYSMAN:NC_HOST_ORACLE_SSH_GLOBAL_NONSUDO\n"
my_file.write(credentials)
targetlist = "target_list=" + hn + ":host\n"
my_file.write(targetlist)
oscommand="variable.default_shell_command=/u01/app/oracle/rmanfull.sh -sid " + sidname + " -nocat -parfile /u01/app/oracle/rman.par" + "\n"
my_file.write(oscommand)
my_file.write("schedule.frequency=IMMEDIATE\n")
my_file.close()
# Now create job to run backup for target
#-owner="SYSMAN" -input_file=property_file:"RMAN_BACKUP Script.txt" -appendtargets
log("#" + str(inc_c) + " Created job properties file to submit to OEM as job --> " + job_name)
# -------------------------------------------
# CHeck for Job concurrency nd hold up (loop) until concurrency has room for more jobs
# Use get_jobs to count runing jobs
# loop until running jobs is leve than Concurrency)
# -------------------------------------------
try:
logdbg2("\t(tryblock-1) Now checking number of running jobs --> staus_id=2 (running)")
#global numrunningjobs
numrunningjobs = 0
logdbg (" --> running jobsobj=get_jobs(name=job_prefix, status_ids='2') --> 5 is completed, 2 is running")
jobsobj=get_jobs(name=job_prefix_wildcard,status_ids='2')
returnv=jobsobj.error()
#print_info(jobsobj)
logdbg2("\t(tryblock-1) Ran get_jobs, now check return value")
#numrunningjobs = len(jobsobj.out()['data'])
try:
logdbg2("\t(tryblock-2) testing jobsobj.......")
logdbg2("\t(tryblock-2) returnv --> '" + returnv + "'")
if "Error" in returnv:
numrunningjobs=0
else:
numrunningjobs = len(jobsobj.out()['data'])
except NameError:
numrunningjobs = 0 
logdbg2("\t(tryblock-1) NumberOfRuninngJobs->" + str(numrunningjobs) + " : Concurrency-> " + str(Concurrency) )
except Exception:
logdbg ("in the Exception tryblock-1...")
(k, v)=sys.exc_info()[:2]
logdbg ("Type :", k, "\n", "error reason :" + v)
nval=numrunningjobs + int(Concurrency)
logdbg2("test--> " + str(numrunningjobs) + " >= " + str(Concurrency) + " nval-->" + str(nval) )
logdbg2("Calling IF to compare numrunningjobs to Concurrency ( " + str(numrunningjobs) + " > " + Concurrency + ")") 
if int(numrunningjobs) >= int(Concurrency):
timenow = datetime.datetime.now()
now2=timenow.strftime("%m-%d-%y %H:%M")
logdbg(" ** Job execution delayed, must wait, too many concurrent jobs running at --> " + str(now2) )
logdbg(" ** Starting loop to pause running more jobs numrunningjobs >= Concurrency ")
logdbg2(" testing while loop condition test based on--> numrunningjobs > Concurrent ( " + str(numrunningjobs) + " > " + str(Concurrency) + ")") 
while int(numrunningjobs) >= int(Concurrency):
# Date
timenow = datetime.datetime.now()
now2=timenow.strftime("%m-%d-%y %H:%M")
logdbg2(" (whileloop-start-sleeping-> " + str(SleepTime) + ") NumberOfJobs-> " + str(numrunningjobs) + " >= " + Concurrency + " Time: " + str(now2) )
time.sleep(SleepTime)
logdbg2("\t\t(whileloop) - checking number of running jobs again....")
#jobsobj=get_jobs(name="RMANBCK%",status_ids='2')
logdbg2("\t\t(whileloop) - running get_jobs")
jobsobj=get_jobs(name=job_prefix_wildcard,status_ids='2')
returnv=jobsobj.error()
logdbg2("\t\t(whileloop) - completed get_jobs")
#numrunningjobs = len(jobsobj.out()['data'])
logdbg2("\t\t(whileloop) ran get_jobs, now check return value")
try:
logdbg2("\t\t(whileloop) try-except --> testingreturnv from jobsobj.......")
logdbg2("\t\t(whileloop) try-except --> returnv --> '" + returnv + "'")
if "Error" in returnv:
numrunningjobs=0
else:
if dbglevel > 0:
#logdbg(print_info(jobsobj))
for job in jobsobj.out()['data']:
logdbg2("\t\t(whileloop) try-except--> runningJob-> " + job['Name'] + " : " + job['Job ID'] + " StatusCode-> " + job['Status ID'])
numrunningjobs = len(jobsobj.out()['data'])
except NameError:
dummy_xnumrunningjobs = 0
log(" ****** NumberOfRuninngJobs->" + str(numrunningjobs) + " : Concurrency-> " + str(Concurrency) )
else:
#Date
timenow2 = datetime.datetime.now()
now2=timenow2.strftime("%m-%d-%y %H:%M")
logdbg(" ** Run more jobs now: current jobs-> " + str(numrunningjobs) + " concurrency-> " + Concurrency + " at --> " + str(now2) )
#if dbglevel > 0:
#logdgb(print_info(jobsobj))
#for job in jobsobj.out()['data']:
# logdbg(" RunningJob-> " + job['Name'] + " : " + job['Job ID'] )

# Submit Job using Prop file
try:
log("#" + str(inc_c) + " Submitting job (for execution) using--> create_job(input_file=" + jobfilename + ") at " + str(now2) + " for --> " + tn )
propfile="property_file:" + jobfilename
logdbg (' --> create_job(input_file="property_file:RMAN_BACKUP Script_job.txt")')

# comment out create_job line to not create job in OEM - when testing
create_job(input_file=propfile)
log("#" + str(inc_c) + " **** Submitted job (for execution) using--> create_job(input_file=" + jobfilename + ") at " + str(now2))
log("#" + str(inc_c) + " **** Submitted job--> " + job_name + " (for --> " + tn + " sid->" + sidname + " on " + hn + ")" )
time.sleep(SleepTime2)
except:
print "Unexpected error: ", sys.exc_info()[1]
else:
logdbg(" *** skipping target--> " + target['TARGET_NAME'] + " --> standby DB in DG only mode dgstatus-> " + dgstatus + " \n")
else:
logdbg2(" ** Target is not an Oracle DB (skipping) target-> " + tgt['Target Type'] + " - " + tgt['Target Name'] + " ......skipping" )

#else:
# continue

log("Finished processing all targets (" + str(inc_a) + " in group '" + GroupName + "', started backups for " + str(inc_c) + " databases")
exit(0)
except:
#print "Unexpected error: ", sys.exc_info()[0], sys.exc_info()[1]
print "Unexpected error: ", sys.exc_info()[1]