Umar DBA

ORA-19511 Vendor specific error: Cannot find requested element ERR(-2)

2010-03-29T21:55:00.000-07:00

Applies to:
Oracle Server Enterprise Edition - Version: 10.2 to 10.2
This problem can occur on any platform.

Symptoms
Backup returns error:
RMAN-03009: failure of backup command on sbt_backup channel at 02/23/2009 13:28:40
ORA-19506: failed to create sequential file, name="c-3183778111-20090219-02", parms=""
ORA-27028: skgfqcre: sbtbackup returned error
ORA-19511: Error received from media manager layer, error text:
Vendor specific error: Cannot find requested element ERR(-2)
ORA-19600: input file is backup piece
(+ASM_FRA/ractest/autobackup/2009_02_19/s_679226354.411.679226355)
ORA-19601: output file is backup piece (c-3183778111-20090219-02)

Cause

HP data protector media manager issue. The HP data protector software causes the error. There
is a delay between backupsets which are send to the HP cell server. If the delay is to long the hp
cell server closes the session. The client did not recognise the close and fails with the transfer
of the data.

Cause justified by customer.

Solution
To implement the solution, please execute the following steps:

A solution for me was to set in the configuration file:

On Unix Plattform: /etc/opt/omni/server/options/global
On Windows Plattform: C:\Programme\OmniBack\Config\Server\Options\global

the parameter SmWaitForNewBackupClient to 2 minutes:

# SmWaitForNewBackupClient=WaitForInMinutes
SmWaitForNewBackupClient=2

There are further hints about the SmWaitForNewBackupClientSec parameter. Maybe this parameter also
helps.
# SmWaitForNewBackupClientSec=WaitForInSeconds

Reference

Metalink Doc [ID 785219.1]

Reinstating a Physical Standby Using Backups Instead of Flashback

2010-03-29T21:51:00.000-07:00

Applies to:
Oracle Server - Enterprise Edition - Version: 10.2.0.2.0 to 10.2.0.3
Information in this document applies to any platform.
Goal

After a Data Guard failover to a physical standby database, using flashback database is still the MAA recommendation to reinstate the original primary database since it is the simplest and fastest approach. However, if for some reason the flashback data is not available or flashback database was not enabled before the failure, you can use the steps provided in this document to reinstate the original primary database using existing backups instead of flashback database.
Solution

Oracle highly recommends utilizing flashback database to reinstate a physical standby after a Data Guard Failover. The procedure is simpler and safer. When using a controlfile restored from a backup instead of flashing back with the current controlfile, the backup controlfile may not reflect the same files as the data dictionary. However this can be avoided if a standby controlfile is created after each backup.

Prerequisites:
• Backups of all data files exist.
• Standby controlfile backups exist and were taken at the failed primary database using the ALTER DATABASE CREATE STANDBY CONTROLFLE command after each database backup.
• Database version is 10.2.0.2 or higher. Please refer to Alert Note 308698.1 and Note.368276.1.

Step 1: Retrieve STANDBY_BECAME_PRIMARY_SCN after Data Guard Failover
From the new primary:

SQL> select to_char(STANDBY_BECAME_PRIMARY_SCN) from v$database;

Step 2: Remove divergent archivelogs from the original, failed primary database

Remove any archivelogs created at the time of or after the Failover operation. The failed primary, if it was isolated from the standby, could have divergent archive logs not consistent with the current primary database. To ensure you do not ever apply these divergent archive logs, they must be deleted from your backups and flash recovery area. You can use the following RMAN command to delete the relevant archive logs from the flash recovery area.

RMAN> DELETE ARCHIVELOG FROM SCN STANDBY_BECAME_PRIMARY_SCN;

Step 3: Restore backup of the original, failed primary database

RMAN> RESTORE DATABASE;

Step 4: Startup standby database using the physical standby controlfile

If the standby controlfile backup needs to be restored, you will need to ensure the spfile or initialization files point to the standby control file and remount the new standby database.

SQL> startup mount

If a standby controlfile does not exist from the failed primary database, then follow steps 1 and 2.2 through 2.8 in Note 368276.1 to:

1. Clean up the flash recovery area
2. Create a new standby controlfile from the new primary database
3. Restart the new standby database with the new standby controlfile
4. Rename data files and log files

Step 5: Ship a new archive from the primary to the new physical standby database

SQL> alter system set log_archive_dest_state_n = ENABLE;
SQL> alter system archive log current;

From the primary, you can verify if the standby received the archive by querying:

SQL> select dest_id, dest_name, status, type, error, destination
2 > from v$archive_dest_status
3 > where dest_id = ;

Step 6: Restart managed recovery

SQL> recover managed standby database using current logfile through all switchover disconnect;

Step 7: Verify the new standby’s data files and control file are in sync

Once the standby is in sync with the primary, you can do a quick validation check to ensure all the data files are present. The following steps stop managed recovery and open the physical standby read only. The query checks if any data files found in the data dictionary do not match exactly the data files listed in the control file.

SQL> recover managed standby database cancel;
SQL> alter database open;
SQL> select file#, CRSCNBAS from file$
2> MINUS
3> select file#, creation_change# from v$datafile;

If no rows are returned, the data dictionary and controlfile metadata are in sync. Restart managed recovery as described in step 6.

References

Metalink Doc ID: [ID 416310.1]

NOTE:308698.1 - ALERT: Reinstating physical standby databases after a Data Guard Failover
NOTE:368276.1 - Steps to workaround issue described in Alert 308698.1
NOTE:416314.1 - Reinstating a Logical Standby Using Backups Instead of Flashback Database

How To Use RMAN For Backup And Restore In A Standby Database Environment

2010-03-29T21:43:00.000-07:00

RMAN can back up the standby database and its associated archived redo logs.

Standby backups of datafiles and archived redo logs are fully interchangeable

with primary database backups. In other words, you can run the RESTORE command

to restore a backup of a standby datafile to the primary database, and you can

restore a backup of a primary datafile to the standby database. The standby

control file and primary control file, however, are not interchangeable.

Prerequisites:

- The standby database is created and fully functioning.

- An Rman catalog is established. Use the same Rman catalog for the primary

database and the standby database. Do not create a separate Rman catalog for

the standby database.

- Do not register the standby database in the Rman catalog, only the primary

database is to be registered.

To take backup on the standby database:

- The standby database should preferably be in 'mount standby' mode, not in

'recover managed standby' mode. On ver. 9.0.1 the subsequent redo log

applications fail with ORA-01157 ORA-01110 if the database was in 'recover

managed standby' mode during backup, after which "alter database recover

managed standby database;" must be reissued.

- The standby database is the target database

- Connect to the Rman catalog

- Run backup

- On ver. 8.1.7, run 'resync catalog' after the backup

- Check with 'list backupset of database' that the new backupset is stored in

the Rman catalog

On ver. 8.1.7 it is necessary to manually resync the catalog in order to get

the backupset taken on the standby database stored in the Rman catalog. The

resync will do a partial resynchronization, this is updating the catalog with

the changed information in the control file.

To run restore on the standby database:

- The standby database is in 'mount standby' mode (not in 'recover managed

standby' mode)

- The standby database is the target database

- On ver. 8.1.7, connect nocatalog to Rman; do not connect to the Rman

catalog as the restore will be run against the primary database if connected to

the Rman catalog

- On rel. 9i, connecting to the Rman catalog as well as connecting nocatalog

works

- Run restore

To run restore on the primary database:

- The primary database is the target database

- Connect to the Rman catalog

- Check with 'list backupset of database' that the backupset taken on the

standby database is accessible

- Run restore

Reference:

Metalink Doc ID: 203980.1

For Oracle ver. 8.1.7:

Oracle8i Documentation Addendum Release 3 (8.1.7),

Chapter 7 Recovery Manager, Backing Up Files at the Standby Site Using RMAN

For Oracle ver. 9.0.1:

Oracle9i Recovery Manager User's Guide Release 1 (9.

0.1), Chapter 9 Making Backups and Copies with Recovery Manager, Backing Up

Files at a Standby Database Site with RMAN

For Oracle ver. 9.2.0.1:

Oracle9i Recovery Manager User's Guide Release 2 (9.

2), Chapter 9 Making Backups and Copies with Recovery Manager, Backing Up Files

at a Standby Database Site with RMAN

Creating a Data Guard Database with RMAN (Recovery Manager) using Duplicate Command

2010-03-29T21:12:00.000-07:00

PURPOSE

To understand and use the 9i Data Guard database features of Recovery manager

1. Summary of test environment:

The production database is "PROD".

The standby database is "STANDBY"

Testing was done on SUN hardware and SUN Solaris 2.8

2. Standby Restrictions:

Note the following restrictions involved when using the DUPLICATE

command to create a standby database:

-- The standby instance must be started but not mounted.

-- RMAN must be connected to the target database and to the auxiliary

instance. If desired, you can connect to the recovery catalog.

-- At least one auxiliary instance channel must be allocated in the RUN

block. You can allocate multiple auxiliary channels if needed.

-- Backups and copies on disk are available at the standby host with the

same path names as in the target host.

-- Backups on tape are accessible from the standby host.

-- Backups on disk are available at the standby host with the same

pathnames as in the target host.

-- If archived logs have not been backed up, then archived logs must be

available at the standby host with the same path names as in the target

host.

-- If RMAN recovers the standby database, then the checkpoint SCN of the

control file must be included in an archived redo log that is either

available at the standby site or included in an RMAN backup. For example,

assume that you create the standby control file and then immediately

afterwards archive the current log, which has a sequence of 100. You must

recover the standby database up to at least log sequence 100, or Oracle

signals ORA-1152 because the standby control file backup or copy was

taken after the point in time.

-- You cannot specify the SKIP READONLY and LOGFILE options of the

DUPLICATE command. These options are legal for a duplicate database but

illegal for the creation of the standby database.

-- You cannot use SET NEWNAME or SET AUXNAME to transform the filenames for

the online redo logs on the standby database. You cannot use the DUPLICATE

command to activate a standby database.

3. RMAN Standby database commands:

CURRENT CONTROLFILE FOR STANDBY -

makes a backup of the current control file that can be used with a standby

database. A standby control file can also be used as an ordinary control

file backup, so you can restore it in the target database if necessary.

DUPLICATE FOR STANDBY -

creates a standby database rather than a duplicate database. Specify this

keyword only when creating a standby database.

DORECOVER -

specifies that RMAN should recover the database after creating it. If you

specify an untilClause, then RMAN recovers to the specified point and

opens the database. If you do not specify this keyword, then RMAN

creates the standby database and then leaves it mounted.

4. Primary INIT.ORA setup:

db_name = PROD

instance_name = PROD

fal_client = PROD

fal_server= STANDBY

log_archive_dest_1 = 'LOCATION=/p01/app/oracle/product/9.0.1/dbs/arch'

log_archive_dest_2 = 'SERVICE=STANDBY LGWR ASYNC NOAFFIRM'

log_archive_format = %t_%s.dbf

log_archive_start = true

remote_login_passwordfile = EXCLUSIVE

service_names = PROD

Additional information on these parameters can be found in the Oracle 9i

Reference Manual Available at http://tahiti.oracle.com.

5. Standby INIT.ORA setup:

# Copy the init.ora from production to standby.

# Edit the initSTANDBY.ora to make neccessary changes:

db_name = "PROD" -- Must match the production database.

instance_name = STANDBY -- Instance identifier

lock_name_space = STANDBY -- Used when the standby or clone have the same

name as the production database being copied.

service_names = STANDBY -- Specifies the service names supported by the

instance.

fal_client = STANDBY -- Specifies the service name resolved from the

remote host to fetch archive logs to local.

fal_server = PROD -- Specifies the service name resolved on the

local host to request archive logs from remote.

db_file_name_convert = ('/p01/oradata/PROD',

'/u03/STANDBY/oradata')

log_file_name_convert = ('/p01/oradata/PROD',

'/u03/STANDBY/oradata')

# Use these parameters if you have a different filesystem structure on the

# standby. Then edit all the dump_dest and controlfile locations for the

# standby site.

standby_archive_dest = /p01/app/oracle/product/9.0.1/dbs/arch

standby_file_management = auto

REMOTE_LOGIN_PASSWORDFILE = EXCLUSIVE

- Create a password file for the standby database using orapwd command:

Create a password file 'orapwSTANDBY' file in the $ORACLE_HOME/dbs

directory and the password for internal "pwd".

orapwd file=$ORACLE_HOME/dbs/orapwSTANDBY password=pwd entries=5

6. Making a backup of the target for standby:

Connect to the target and the recovery catalog using Recovery manager:

setenv ORACLE_SID=PROD

rman nocatalog target /

Recovery Manager: Release 9.0.1.1.0 - 64bit Production

connected to target database: PROD (DBID=3261937922)

using target database controlfile instead of recovery catalog

Configure channels if you have not already. See the following Note for more

information:

Note 145178.1

RMAN 9i: Automatic Channel Allocation and Configuration

RMAN> Backup Database;

Starting backup at 09-APR-02

allocated channel: ORA_DISK_1

channel ORA_DISK_1: sid=7 devtype=DISK

channel ORA_DISK_1: starting full datafile backupset

channel ORA_DISK_1: specifying datafile(s) in backupset

including current controlfile in backupset

input datafile fno=00001 name=/p01/oradata/PROD/system01.dbf

input datafile fno=00002 name=/p01/oradata/PROD/undotbs01.dbf

input datafile fno=00005 name=/p01/oradata/PROD/example01.dbf

input datafile fno=00006 name=/p01/oradata/PROD/indx01.dbf

input datafile fno=00008 name=/p01/oradata/PROD/users01.dbf

input datafile fno=00003 name=/p01/oradata/PROD/cwmlite01.dbf

input datafile fno=00004 name=/p01/oradata/PROD/drsys01.dbf

input datafile fno=00007 name=/p01/oradata/PROD/tools01.dbf

channel ORA_DISK_1: starting piece 1 at 09-APR-02

channel ORA_DISK_1: finished piece 1 at 09-APR-02

piece handle=/p01/oradata/PROD/dbs/04dlfl6n_1_1 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:24:25

Finished backup at 09-APR-02

RMAN> Backup current controlfile for standby;

Starting backup at 09-APR-02

using channel ORA_DISK_1

channel ORA_DISK_1: starting full datafile backupset

channel ORA_DISK_1: specifying datafile(s) in backupset

including standby controlfile in backupset

channel ORA_DISK_1: starting piece 1 at 09-APR-02

channel ORA_DISK_1: finished piece 1 at 09-APR-02

piece handle=/p01/oradata/PROD/dbs/05dlfml6_1_1 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:22

Finished backup at 09-APR-02

RMAN> sql "Alter System Archive Log Current";

sql statement: Alter System Archive Log Current

RMAN> Backup filesperset 10 ArchiveLog all delete input;

Starting backup at 09-APR-02

current log archived

using channel ORA_DISK_1

channel ORA_DISK_1: starting archive log backupset

channel ORA_DISK_1: specifying archive log(s) in backup set

input archive log thread=1 sequence=3 recid=3 stamp=458742477

input archive log thread=1 sequence=4 recid=4 stamp=458742492

channel ORA_DISK_1: starting piece 1 at 09-APR-02

channel ORA_DISK_1: finished piece 1 at 09-APR-02

piece handle=/p01/oradata/PROD/dbs/06dlfmmt_1_1 comment=NONE

channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

channel ORA_DISK_1: deleting archive log(s)

archive log filename=/p01/oradata/PROD/dbs/arch1_3.dbf recid=3 stamp=458742477

archive log filename=/p01/oradata/PROD/dbs/arch1_4.dbf recid=4 stamp=458742492

Finished backup at 09-APR-02

RMAN> exit;

Recovery Manager complete.

7. Restoring the database to the standby location:

Copy the backup pieces to the remote server (same directory).

rman target / auxiliary sys/change_on_install@STANDBY

Recovery Manager: Release 9.0.1.1.0 - 64bit Production

connected to target database: PROD (DBID=3261937922)

connected to auxiliary database: PROD (not mounted)

RMAN> duplicate target database for standby dorecover;

Starting Duplicate Db at 09-APR-02

using target database controlfile instead of recovery catalog

allocated channel: ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: sid=17 devtype=DISK

printing stored script: Memory Script

{

restore clone standby controlfile to clone_cf;

replicate clone controlfile from clone_cf;

sql clone 'alter database mount standby database';

}

executing script: Memory Script

Starting restore at 09-APR-02

using channel ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: starting datafile backupset restore

channel ORA_AUX_DISK_1: restoring controlfile

output filename=/u03/STANDBY/oradata/control01.ctl

channel ORA_AUX_DISK_1: restored backup piece 1

piece handle=/p01/oradata/PROD/dbs/05dlfml6_1_1 tag=null params=NULL

channel ORA_AUX_DISK_1: restore complete

Finished restore at 09-APR-02

replicating controlfile

input filename=/u03/STANDBY/oradata/control01.ctl

output filename=/u03/STANDBY/oradata/control02.ctl

sql statement: alter database mount standby database

printing stored script: Memory Script

{

set until scn 266683;

set newname for datafile 1 to

"/u03/STANDBY/oradata/system01.dbf";

set newname for datafile 2 to

"/u03/STANDBY/oradata/undotbs01.dbf";

set newname for datafile 3 to

"/u03/STANDBY/oradata/cwmlite01.dbf";

set newname for datafile 4 to

"/u03/STANDBY/oradata/drsys01.dbf";

set newname for datafile 5 to

"/u03/STANDBY/oradata/example01.dbf";

set newname for datafile 6 to

"/u03/STANDBY/oradata/indx01.dbf";

set newname for datafile 7 to

"/u03/STANDBY/oradata/tools01.dbf";

set newname for datafile 8 to

"/u03/STANDBY/oradata/users01.dbf";

restore

check readonly

clone database

;

}

executing script: Memory Script

executing command: SET until clause

executing command: SET NEWNAME

Starting restore at 09-APR-02

using channel ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: starting datafile backupset restore

channel ORA_AUX_DISK_1: specifying datafile(s) to restore from backup set

restoring datafile 00001 to /u03/STANDBY/oradata/system01.dbf

restoring datafile 00002 to /u03/STANDBY/oradata/undotbs01.dbf

restoring datafile 00003 to /u03/STANDBY/oradata/cwmlite01.dbf

restoring datafile 00004 to /u03/STANDBY/oradata/drsys01.dbf

restoring datafile 00005 to /u03/STANDBY/oradata/example01.dbf

restoring datafile 00006 to /u03/STANDBY/oradata/indx01.dbf

restoring datafile 00007 to /u03/STANDBY/oradata/tools01.dbf

restoring datafile 00008 to /u03/STANDBY/oradata/users01.dbf

channel ORA_AUX_DISK_1: restored backup piece 1

piece handle=/p01/oradata/PROD/dbs/04dlfl6n_1_1 tag=null params=NULL

channel ORA_AUX_DISK_1: restore complete

Finished restore at 09-APR-02

printing stored script: Memory Script

{

switch clone datafile all;

}

executing script: Memory Script

printing stored script: Memory Script

{

set until scn 266683;

recover

standby

clone database

;

}

executing script: Memory Script

executing command: SET until clause

Starting recover at 09-APR-02

using channel ORA_AUX_DISK_1

starting media recovery

channel ORA_AUX_DISK_1: starting archive log restore to default destination

channel ORA_AUX_DISK_1: restoring archive log

archive log thread=1 sequence=3

channel ORA_AUX_DISK_1: restoring archive log

archive log thread=1 sequence=4

channel ORA_AUX_DISK_1: restored backup piece 1

piece handle=/p01/oradata/PROD/06dlfmmt_1_1 tag=null params=NULL

channel ORA_AUX_DISK_1: restore complete

archive log filename=/u03/STANDBY/oradata/arch/1_3.dbf thread=1 sequence=3

archive log filename=/u03/STANDBY/oradata/arch/1_4.dbf thread=1 sequence=4

media recovery complete

Finished recover at 09-APR-02

Finished Duplicate Db at 09-APR-02

RMAN> exit;

8. Put the Standby in Managed Recovery Mode:

On the Standby:

SQL> recover standby database;

# Recover all available logs

SQL> alter database recover managed standby database disconnect;

9. References:

Recovery Manager and Data Guard Documentation at http://tahiti.oracle.com

Note 118409.1 Creating a Standby Database using RMAN (Recovery Manager)

Note 145178.1 RMAN 9i Automatic Channel Allocation and Configuration

Note.149286.1 How To Create a Standby Database in Oracle8i

Maintenance Of Archivelogs On Standby Databases

2010-03-29T20:55:00.000-07:00

Applies to:

Oracle Server - Enterprise Edition - Version: 10.1.0.0 to 11.1
Information in this document applies to any platform.

Goal

Maintenance of archivelogs which are already applied on standby database in Oracle 10g

Solution

In Oracle 10g, RMAN has a specific configuration for archivelog deletion policy i.e.

ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY

This parameter can be set on primary and standby databases.

To enable automatic maintenance of the archivelogs on standby perform the below steps on standby database :

Step 1 : Enable the flash recovery area on the standby database so that all the archivelogs are shipped in the flash recovery area.

Example :

To enable Flash Recovery Area and to allow the archivelogs to be shipped to Flash Recovery Area set the below parameters

SQL> alter system set DB_RECOVERY_FILE_DEST='/disk1/FRA';

SQL> alter system set DB_RECOVERY_FILE_DEST_SIZE=80G;

SQL> alter system set LOG_ARCHIVE_DEST_1='LOCATION=USE_DB_RECOVERY_FILE_DEST';

Step 2 : Set the parameter Archive log deletion policy to applied on standby

Example :

RMAN> CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY;

Note : With the above settings whenever there is a space pressure in Flash Recovery Area all the archivelogs that are applied on standby database will be automatically deleted.

References

Oracle Metalink Doc ID 464668.1

Service Management

2009-12-17T22:53:00.001-08:00

This article introduces the concepts of services and service management in the following sections.

Introduction to Services

Modeling Services

Managing Systems

Monitoring Services

Diagnosing Service Problems

Introduction to Services

The critical and complex nature of today’s business applications has made it very important for IT organizations to monitor and manage application service levels at high standards of availability. Problems faced in an enterprise include service failures and performance degradation. Since these services form an important type of business delivery, monitoring these services and quickly correcting problems before they can impact business operations is crucial in any enterprise.

Service-level agreements are used to evaluate service availability, performance, and usage. By constantly monitoring the service levels, IT organizations can identify problems and their potential impact, diagnose root causes of service failure, and fix these in compliance with the service-level agreements.

Enterprise Manager Grid Control provides a comprehensive monitoring solution that helps you to effectively manage services from the overview level to the individual component level. When a service fails or performs poorly, Grid Control provides diagnostics tools that help to resolve problems quickly and efficiently, significantly reducing administrative costs spent on problem identification and resolution. Finally, customized reports offer a valuable mechanism to analyze the behavior of the applications over time.

Grid Control monitors not only individual components in the IT infrastructure, but also the applications hosted by those components, allowing you to model and monitor business functions using a top-down approach, or from an end-user perspective. If modeled correctly, services can provide an accurate measure of the availability, performance, and usage of the function or application they are modeling.

Defining Services in Enterprise Manager

A "service" is defined as an entity that provides a useful function to its users. Some examples of services include CRM applications, online banking, and e-mail services. Some simpler forms of services are business functions that are supported by protocols such as DNS, LDAP, POP, or SMTP.

Grid Control allows you to define one or more services that represent the business functions or applications that run in your enterprise. You can define these services by creating one or more service tests that simulate common end-user functionality. Using these service tests, you can measure the performance and availability of critical business functions, receive alerts when there is a problem, identify common issues, and diagnose causes of failures.

You can define the following service types: Generic Service, Web Application, and Aggregate Service. Web applications, a special type of service, are used to monitor Web transactions.

The following elements are important to understanding Grid Control’s Service Level Management feature:

Service: Models a business process or application.

Availability: A condition that determines whether the service is considered accessible by the users or not.

Service Test: The functional test defined by the Enterprise Manager administrator against the service to determine whether or not the service is available and performing.

System: A group of underlying components, such as hosts, databases, and application servers, on which the service runs.

Beacons: A functionality built into Management Agents used to pre-record transactions or service tests.

Performance and Usage: Performance indicates the response time as experienced by the end users. Usage refers to the user demand or load on the system.

Service Level: Operational or contractual objective for service availability and performance.

Root Cause Analysis: Diagnostic tool to help determine the possible cause of service failure.

Modeling Services

You can create a new target, called a service, to model and monitor your business applications from within Grid Control. While creating a service, you can define the availability, performance and usage parameters, and service-level rules.

Availability

"Availability" of a service is a measure of the end users’ ability to access the service at a given point in time. However, the rules of what constitutes availability may differ from one application to another. For example, for a Customer Relationship Management (CRM) application, availability may mean that a user can successfully log on to the application and access a sales report. For an online store, availability may be monitored based on whether the user can successfully log in, browse the store, and make an online purchase.

Grid Control allows you to define the availability of your service based on service tests or systems.

Service Test-Based Availability: Choose this option if the availability of your service is determined by the availability of a critical functionality to your end users. Examples of critical functions include accessing e-mail, generating a sales report, performing online banking transactions, and so on. While defining a service test, choose the protocol that most closely matches the critical functionality of your business process, and beacon locations that match the locations of your user communities. You can define one or more service tests using standard protocols and designate one or more service tests as "Key Tests." These key tests can be executed by one or more "Key Beacons" in different user communities. A service is considered available if one or all key tests can be executed successfully by at least one beacon, depending on your availability definition.

System-Based Availability: Your service’s availability can alternatively be based on the underlying system that hosts the service. Select the components that are critical to running your service and designate one or more components as "Key Components," which are used to determine the availability of the service. The service is considered available as long as at least one or all key components are up and running, depending on your availability definition.

Performance and Usage

You can define metrics to measure the performance and usage of the service. Performance indicates the response time of the service as experienced by the end user. Usage metrics are based on the user demand or load on the system.

Performance metrics are collected for service tests when the service tests are run by beacons. You can calculate the minimum, maximum, and average response data collected by two or more beacons. For example, you can monitor the time required to retrieve e-mails from your e-mail service in San Francisco, Tokyo, and London, then compare results. You can also collect performance metrics for system components, then calculate the minimum, maximum, and average values across all components. For example, you can monitor average CPU utilization, memory utilization, and disk I/O utilization across several hosts.

Usage metrics are collected based on the usage of the system components on which the service is hosted. For example, if you are defining an e-mail service that depends on an IMAP server, you can use the Total Client Connections metric of the IMAP server to represent the usage of this e-mail service. You can monitor the usage of a specific component or statistically calculate the minimum, maximum, and average values from a set of components. You can also set thresholds on the above metrics and receive notifications and alerts.

Setting Service-Level Rules

Service-level parameters are used to measure the quality of the service. These parameters are usually based on actual service-level agreements or on operational objectives.

Grid Control’s Service Level Management feature allows you to proactively monitor your enterprise against your service-level agreements to verify that you are meeting your needs for availability and performance within the service’s business hours. For service-level agreements, you may want to specify the levels according to operational or contractual objectives.

By monitoring against service levels, you can ensure the quality and compliance of your business processes and applications.

Monitoring Templates for Services

Administrators are often faced with the task of defining similar monitoring attributes or rules for many applications. The same set of rules are often applicable to different applications. This can be achieved through the Monitoring Templates feature in Grid Control. A monitoring template for a service contains definitions for one or more service tests, as well as a list of monitoring beacons. You can create a monitoring template from a standard service target, then copy this template to create service tests for any number of service targets and specify a list of monitoring beacons. This helps reduce the required configuration time where a large number of applications need to be monitored.

Managing Systems

A "system" is a logical grouping of targets that collectively hosts one or more services. It is a set of infrastructure targets (hosts, databases, application servers, and so on) that function together to host one or more applications or services.

In Enterprise Manager Grid Control, systems constitute a new target type. For example, to monitor an e-mail application in Enterprise Manager, you would first create a system, such as "Mail System," that consists of the database, listener, application server, and host targets on which the e-mail application runs. You would then create a service target to represent the e-mail application and specify that it runs on the Mail System target.

Note:

An Enterprise Manager "System" is used specifically to monitor the components on which a service runs. Many of the functions and capabilities for groups and systems are similar.

Creating Systems

Use the Create System pages to perform the following configuration tasks:

Select target components for a new system.

Define the associations between the components of the system using the Topology Viewer.

Add charts that will appear in the System Charts page. The charts represent the overall performance for the system or components of the system. Based on the target type of the components you select in the Components page, some charts are predefined.

Select a set of columns you want to appear in the System Components page and in the system’s Oracle Grid Control Dashboard.

Customize the refresh frequency and specify the format for viewing component status, alerts, and policy violations in the system’s Oracle Grid Control Dashboard.

Enterprise Manager provides a Topology Viewer for several applications. The Topology Viewer allows you to view the relationships between components, nodes, or objects within different Oracle applications. You can zoom, pan, see selection details and summary information, and evaluate aggregate components. Individually distinct icons are used for each object type, and standardized visual indicators are used across all applications.

You may want to create system topologies for a number of reasons:

Graphically model relationships

Identify the source of a failure

Perform visual analysis for high-level problem detection

When creating a system topology, you specify associations between the components in the system to logically represent the connections or interactions between them. For example, you can define an association between the database and the listener to indicate the relationship between them. Components are represented as icons, and associations are depicted as arrow links between components. After you have customized the topology to suit your needs, you can then view the overall status of the components in your system by accessing the System Topology page.

See Also:

"Create System Topology Page" in the Enterprise Manager online help

Monitoring Systems

Use the System pages to perform the following monitoring and administration tasks:

Quickly view key information about components of a system, such as outstanding alerts and policy violations.

View metric data for several time periods.

View summary information determined by the columns you configured when you created the system.

Perform administrative tasks, such as creating jobs and blackouts.

View the topology of system components, including the associations between them.

Monitoring Services

Monitoring a service helps you ensure that your operational and service-level goals are met. To monitor a service, define service tests that simulate activity or functionality that is commonly accessed by end users of the service. For example, you may want to measure a service based on a particular protocol, such as DNS, LDAP, and IMAP. To proactively monitor the availability and responsiveness of your service from different user locations, designate the geographical locations from which these service tests will be executed. Run service tests from specified locations using Enterprise Manager Beacons. You may also measure a service based on the usage of the service’s system components.

Services Dashboard

In Grid Control, service levels are defined as the percentage of time during business hours that a service meets the specified availability and performance criteria. Using the Services Dashboard, administrators can determine whether the service levels are compliant with business expectations and goals.

The Services Dashboard enables administrators to browse through all service-level-related information from a central location. The Services Dashboard illustrates the availability status of each service, performance and usage data, as well as service-level statistics. You can easily drill down to the root cause of the problem or determine the impact of a failed component on the service itself.

The following details are displayed in the Services Dashboard:

Availability: A measure of the end users’ ability to access the service at a given point in time. Service level agreements typically require a service be available at least for a minimum percentage of time.

Performance: Response time is a good measure of the performance experienced by the end users when they access the service. When the service performance is poor, the availability of the service may be affected.

Usage: Indicates end-user usage, or level of user activity, of the service.

System Topology

The System Topology page enables you to view the dependency relationships between components of the system. From the topology view, you can drill down to detail pages to get more information on the key components, alerts and policy violations, possible root causes and services impacted, and more.

Use the System Topology page, to get a quick overview of the status of your system’s components. The status indicators over each icon enable you to quickly assess which components are down or have open alerts. You can get more detailed information for any key component from this page.

Service Topology

Use the Service Topology page, to view the dependencies between the service, its system components, and other services that define its availability. Upon service failure, the potential causes of failure, as identified by Root Cause Analysis, are highlighted in the topology view. In the topology, you can view dependent relationships between services and systems.

Some data centers have systems dedicated to one application or service, while others have shared systems that host multiple services. In Grid Control, you can associate a single service or multiple services with a system, based on the setup of the data center.

Reports

Enterprise Manager provides out-of-box reports that are useful for monitoring services and Web applications. You can also set the publishing options for reports so that they are sent out via email at a specified period of time. Some of the reports that can be generated include Web Application Alerts, Web Application Transaction Performance Details, and Service Status Summary.

Notifications, Alerts, and Baselines

Using Grid Control, you can proactively monitor a service and address problems before users are impacted. Each service definition has performance and usage metrics that have corresponding critical and warning thresholds. When a threshold is reached, Grid Control displays an alert. There are a standard set of notification rules that specify the alert conditions for which notifications should be sent to the appropriate administrators. Apart from these standard sets of rules, you can define and set up schedules so that administrators are notified when the specified alerts conditions are met. For example, thresholds can be defined so that alerts are generated when a system is down, if the end user cannot login to an application, or if the online transaction cannot be successfully completed.

You can set up baselines for a specified period and use these baselines to evaluate performance. Statistics are computed over the baseline period for specific target metrics. You can use these statistics to automatically set metric thresholds for alerting, as well as to normalize graphical displays of service performance.

Service Performance

Grid Control provides a graphical representation of the historic and current performance and usage trends in the Performance and Usage Charts. You can view metric data for the current day (24 hours), 7 days, or 31 days. The thresholds for any performance or usage alerts generated during the selected period are also displayed in the charts. This helps you to easily track the performance and usage of the service test or system over time and investigate causes of service failure. Users can choose the default chart for the Services Home page; all performance and usage charts are available on the Charts page.

Use the Test Performance page to view the historical and current performance of the service tests from each of the beacons. If a service test has been defined for this service, then the response time measurements as a result of executing that service test can be used as a basis for the service’s performance metrics. It is possible to have multiple response time measurements if the service access involves multiple steps or the service provides multiple business functions. Alternatively, performance metrics from the underlying system components can also be used to measure performance of a service.

If performance of a service seems slow, it may be due to high usage of the service. Monitoring the service usage helps diagnose poor performance by indicating whether the service is affected by high usage of a system component.

Monitoring Web Application Services

Today’s e-businesses depend heavily upon their Web applications to allow critical business processes to be performed online. As more emphasis is placed on accessing information quickly, remotely, and accurately, how can you ensure your online customers can successfully complete a transaction? Are you certain that your sales force is able to access the information they need to be effective in the field?

The Web application management features complement the traditional target monitoring capabilities of Enterprise Manager Grid Control. Full integration with the Enterprise Manager target monitoring capabilities allows you to monitor the performance and availability of components that make up the applications’ technology environment, including the back-end database and the middle-tier application servers.

In Grid Control, you can define a Web application service to monitor Web transactions. This allows you to proactively monitor your e-business systems from the top down, and trace the experience of your end users as they enter and navigate the Web site. You can monitor the Web application service through the Services Dashboard, Topology Viewer, Charts, Reports, and more.

Additionally, you can monitor the end-user performance response times, which enables you to effectively manage your e-business systems and understand the impact of application service-level problems.

Transactions

Transactions are service tests that are used to test the Web application performance and availability. Important business activities for the Web application are recorded as transactions, which are used to test availability and performance of a Web application. A transaction is considered "available" if it can be successfully executed by at least one beacon. You can record the transaction using an intuitive playback recorder that automatically records a series of user actions and navigation paths.

End-User Performance Monitoring

The End-User Performance Monitoring feature enables you to measure the actual response time as experienced by the end users. When configured with Oracle Application Server Web Cache or Oracle HTTP Server/Apache HTTP Server, the End-User Performance Monitoring feature provides response time data generated by actual end users as they access and navigate your Web site.

You can track the response times for each user and all individual pages, allowing you to assess the end-user experience and address potential issues. You can also view the response times by individual visitor, domain, user-defined region, Web server, or a combination of these criteria. For example, tracking the response time of visitors ensures that critical customers, executives, and other important visitors are experiencing adequate response times.

You can set up Watch Lists of important URLs and view the response metrics of these critical pages at a glance. You can also use the Analyze feature to analyze the performance data stored in the Management Repository.

Diagnosing Service Problems

Grid Control offers you tools to help diagnose service problems, including Root Cause Analysis, Topology Viewer, and Web application diagnostics. If a service is unavailable or performing poorly, use these tools to determine the potential causes.

Root Cause Analysis

When a service fails, Root Cause Analysis returns a list of potential causes on the Service Home page. Potential root causes include failed subservices and failed key system components.

By default, Root Cause Analysis evaluates a key component’s availability status to determine whether or not it is a cause of service failure. You can specify additional conditions, or component tests, for Root Cause Analysis to consider. If a key component is unavailable, or if any of your component test’s conditions are not met, then this component is considered a possible cause of the service failure.

You can also specify additional conditions, or component host tests, for the host on which this key component resides. If Root Cause Analysis identifies the key component as a cause of service failure, the component’s host is then analyzed to see if it potentially caused the component, and therefore the service, to fail.

You can also access the Root Cause Analysis information from the Topology Viewer, which shows a graphical representation of the hierarchical levels displaying relationships between components. Red lines between the services and system components represent the associated failure. Follow these red lines to discover possible causes of failure.

Grid Control can also be integrated with the EMC SMARTS solution to detect network failures in Root Cause Analysis. When problems in the network are detected, you can use the SMARTS network adapter to query Root Cause Analysis information related to the hosts and IP addresses in the network.

Diagnosing Web Application Problems

When a Web application is unavailable, the Root Cause Analysis feature allows you to determine the causes of service failure. Apart from this feature, Grid Control provides tools to diagnose application performance degradation issues and pinpoint problem areas within the application stack. Comprehensive diagnostic tools enable you quickly drill down into the Oracle Application Server stack and monitor response times in various application server and database components.

Interactive Transaction Tracing

When the performance of a Web application is slow, you can trace problematic transactions as required using Interactive Transaction Tracing. You can record the transaction using an intuitive playback recorder that automatically records a series of user actions and navigation paths. You can play back transactions interactively and perform an in-depth analysis of the response times across all tiers of the Web application for quick diagnosis.

The Interactive Transaction Tracing facility complements the Transaction Performance Monitoring and End-User Performance Monitoring features by helping you diagnose the cause of a performance problem. This in-depth drill-down diagnostics tool enables you to trace the transaction path and performance across the application tiers, and helps identify the cause of performance bottlenecks. Using these diagnostic tools, you can quickly resolve application problems, thus reducing the mean-time to repair.

All invocation paths of a transaction are traced and hierarchically broken down by servlet/JSP, EJB, and database times to help you locate and solve the problem quickly. Once a problem is resolved, you can also run Interactive Transaction Tracing to reassure you that the problem has been satisfactorily repaired. In addition, you can use the SQL Statement Analysis link to view details.

Request Performance Diagnostics

Grid Control provides in-depth historical details on the J2EE and database performance of all URL requests. By examining the detailed J2EE and database breakdown and analyzing the processing time of a request, you can determine whether the problem lies within a servlet, JSP, EJB method, or specific SQL statement. Using this information, you can easily isolate the cause of the problem and take necessary action to quickly repair the appropriate components of your Web application.

Grid Control’s Request Performance Diagnostics feature is instrumental to the application server and back-end problem diagnosis process. Slowest URL request processing times and the number of hits are provided so that you can easily recognize where problem resolution efforts should be prioritized. Application administrators need to know how their J2EE and database components are performing, including the top JSPs and servlets by processing time and request rates so that they can identify how these components are affecting overall response times.

URL request processing time and load graphs provide you with information on the impact of server activity on response times. Analyzing the J2EE and database at the subcomponent level helps you make accurate decisions to tune or repair the appropriate elements of a Web application.

Easy to read graphs of URL request processing times by theOC4J subsystem allows you to quickly assess where the most time is spent. Further drill-downs bring you directly to in-depth URL request processing call stack details. You can correlate URL request times (EJB time, database time, and so on) to the underlying system component metrics.

What's New in Enterprise Manager 10.2

2009-12-17T22:48:00.001-08:00

Enterprise Manager Grid Control 10g Release 2 (10.2) includes many new features that add to your ability to monitor and manage your enterprise:

Grid Management Features

· System Monitoring:

Lights-out, comprehensive monitoring of all components of the Oracle ecosystem. New features include administrator control of metric collection schedules, Corrective Actions that enable automated responses to alerts, enhancements to SQL-based User-Defined Metrics, Monitoring Templates that enable administrators to apply standardized monitoring settings across their enterprise, and a rich set of out-of-box monitoring reports.

· Bare-Metal Provisioning:

Automatically deploy new fully configured systems, including operating systems and software images. Clone existing installations and software images (such as Oracle Cluster Ready Services (CRS), Real Application Clusters (RAC), and Oracle Application Server).

· Real Application Clusters Provisioning and Conversion:

Automatically configure RAC database instances to dynamically increase database capacity to meet your needs; convert non-RAC databases to RAC databases.

· Cloning:

Simplify deployment of Oracle solutions by cloning fully patched and tested Oracle homes to multiple hosts; clone database instances to create backups of important databases; create new single-node RAC by cloning an existing RAC Oracle home.

· Lights-Out Patching:

Quickly identify available patches for the components in your enterprise. Find out which have not been applied and which are critical. Bring those components up to the latest patch level with just a few clicks.

· Policy-Based Monitoring:

Apply consistent, proactive management practices across your enterprise using policies and management templates.

· Security Assessment:

Automatically assess the security vulnerabilities in your enterprise using the Critical Patch Facility, which provides notifications and a direct link to Oracle Support. Enhanced security rules harden the infrastructure database and host. Violation score reporting helps administrators prioritize security operations.

· Job System:

More powerful, feature-rich job system allows you to schedule and run multi-task jobs across multiple targets, as well as receive notifications.

· Group Management:

Enhanced group management features enable administrators to manage many components as one. Group pages roll up critical management information, such as overall status and group status history, open alerts and alert history, open policy violations and policy trend overview charts, and out-of-box group performance charts. Simplifies common administrative tasks, such as running jobs, executing custom SQL or host commands, and scheduling maintenance windows. A group "dashboard" enables proactive monitoring of status and alerts across all group members as they occur.

Service Management Features

· Services:

Define services to represent the business functions or applications that run in your enterprise. Measure the performance and availability of these services, receive alerts when there is a problem, identify common issues, and diagnose causes of failures.

· Service and System Topologies:

Easily define, visualize, and manage key business functions and the assets that they rely upon.

· Service Level Management:

Allows you to define service-level goals for availability and performance and more quickly resolve problems through rules, reports, and historical monitoring.

· Dashboards:

Offer "at-a-glance" monitoring of all critical indicators for your services and systems.

· Root Cause Analysis:

Diagnose service problems quickly using the Root Cause Analysis engine.

Enterprise Reporting Features

· Out-of-Box Reports:

Easy-access out-of-box reports for all Oracle components; reports on configuration, performance, service level, and audit.

· Graphical Report Creation:

Easy-to-use tool to create custom reports.

· Complete Repository Access:

Create reports using any data collected and stored in the Management Repository.

· Report Publishing:

Proactively deliver specialized management information to users, managers, and executives using out-of-box reports, custom reports, and secure publishing features, with views accessible by any reporting product.

Database Management Features

· Real Application Clusters Management:

Enhanced monitoring of RAC databases and clusters to include Interconnect monitoring, cluster cache coherency diagnostics, CRS monitoring, a graphical topology view; provision RAC deployments, convert single instances, and add nodes to existing clusters.

· Transportable Tablespaces:

Allows an administrator to move tablespaces between databases.

· Fast-Start Failover:

Allows Data Guard to rapidly and automatically fail over to the standby database without requiring manual intervention, significantly improving the degree of availability as well as the disaster resilience for the system infrastructure.

· Restore Points:

Guaranteed recovery of the database to a particular point in time when used with Flashback Database or Flashback Table.

· Enhanced Backup Management:

Provides a single view of all backups run against the databases in a domain.

· Access Advisor Enhancements:

Enhanced to expose more powerful features of the Access Advisor, including template support, tuning of a single statement, journaling, recommendation implementation status, and action implementation status.

· Automatic Segment Advisor:

Proactively identifies segments that have significant wasted space due to data fragmentation, which are good candidates for Online Segment Shrink. You no longer need to run Segment Advisor to determine what segments have data fragmentation.

· Online Segment Shrink Enhancements:

Extends Online Segment Shrink capability to large object (LOB) segments and index-organized table (IOT) overflow segments. Online Segments Shrink capability is now extended to all types of segments, further reducing the need to perform other more advanced reorganization methods such as Online Redefinition.

· Oracle Streams:

Share message information within a database or from one database to another. Control what information is put into a stream, how the stream flows from database to database, what happens to messages in the stream, and how the stream terminates.

· Adaptive Alert Thresholds:

Computes statistical alert thresholds using either static (user-defined) or dynamic (self-adjusting) baselines. Dynamic statistical baselines can significantly improve the accuracy of performance alerting, while reducing exposure to the "false positives" commonly incurred under fixed threshold schemes.

Application Server Management Features

· Extended Middle Tier Management:

Allows administrators to discover OracleAS Farms, OracleAS Clusters, Oracle HTTP Server High Availability Groups and OC4J High Availability Groups as composite targets and perform management and administration operations on them.

· Topology Views:

Provides graphical topology view of AS Farms, Clusters, and ASs, with current status, alerts and policy violations, and performance metrics. View by host, by request routing overview, or request routing details.

· Custom Aggregate Monitoring:

Define monitoring criteria for OracleAS Farms and Clusters using customizable aggregate metrics based on the statistical aggregation of individual metrics of cluster members. View aggregate metrics as an overlay of Òtop best,Ó Òtop worst,Ó or Òpeak thresholdÓ values for individual metrics.

· J2EE Application Monitoring:

Hierarchical J2EE application view at the OracleAS cluster level shows performance across the OC4J instances on which it is deployed. Ability to plug customer application JMX MBeans into Grid Control.

· Backup and Restore:

Supports full and incremental online backup of middle tier installations. Automatically recover instances to the last full backup, then apply incremental backups.

· OPMN Jobs:

Supports immediate or scheduled full process control (start, stop, restart) of OPMN managed components from Grid Control.

· Configuration Management:

Extended configuration management support for OracleAS deployments. Collect, search, track, compare, and report on all OracleAS hardware and software installation configurations. Maintain a central inventory of software configuration data of core components for all OracleAS instances.

· Provisioning "Gold" Images:

Save standard reference images of application server targets to the central Software Library and deploy to target locations. Standardize the deployments of the application server targets in your enterprise. Create and extend OracleAS Cluster environments by creating new clusters or adding nodes to existing clusters.

· Cloning:

Extended cloning support for different types of middle-tier installations, including WebCache, J2EE and WebCache, Portal and Wireless, Business Intelligence, Forms and Reports Services, and Business Intelligence and Forms.

Collaboration Suite Management Features

· Centralized Management:

Single target subtab for Collaboration Suite from which you can view your Collaboration Suite deployment(s) and navigate to specific components.

· Automatic Discovery:

Collaboration Suite server targets, such as SMTP servers and Calendar servers, are automatically discovered and monitored in Grid Control.

· Service Monitoring:

In addition to monitoring server components, Grid Control also monitors Collaboration Suite services from the end-user perspective.

· Custom Service Modeling:

Fast setup of services using a simple customization user interface.

· Topology Viewer:

Service topology view of your Collaboration Suite deployment showing relationships between different services in Collaboration Suite.

· Root Cause Analysis:

Detects the potential root causes of any service outage.

· Dashboards:

Services Dashboard for monitoring service availability, performance, usage, and service-level compliance. System Dashboard for monitoring exceptions in your Collaboration Suite system.

· Client System Analyzer:

Collects Windows client data and checks for minimum Collaboration Suite requirements.

Broader System Coverage

· Host Management:

Out-of-box operating system, hardware, and software monitoring of the hosts on which your applications and services reside. Increased coverage for hardware monitoring, process and file-level monitoring, remote command shells, and remote file editing.

· Enterprise Storage Reports:

Reports on allocation, utilization, roll-ups by location, line of business, application, and vendor.

· Support for Network Devices and Storage:

Monitor network and storage devices, such as F5 Server Load Balancers and NetApp Filer network storage.

· Support for Non-Oracle Middleware:

Support for discovery, performance monitoring, reporting, alerts, process control, and synthetic transactions for third-party application servers and application server clusters, such as IBM WebSphere and BEA WebLogic.

· External Product Integration:

Integrate external products using command line interface, notifications, and bi-directional messages.

· E-Business Suite Management:

Plug-in available to enable Grid Control to manage the Oracle Applications Manager, part of the E-Business Suite.

New Management Packs and Plug-ins

In addition to the six management packs available in the previous release, Enterprise Manager 10.2 offers eight new management options: three new standalone management packs and five system monitoring plug-ins.

· New Management Packs:

o Oracle Service Level Management Pack

o Oracle Configuration Pack for Non-Oracle Systems

o Oracle Provisioning Pack

· New Management Plug-ins:

o System Monitoring Plug-in for Hosts

o System Monitoring Plug-in for Non-Oracle Databases

o System Monitoring Plug-in for Non-Oracle Middleware

o System Monitoring Plug-in for Network Devices

o System Monitoring Plug-in for Storage

Database Management

2009-12-17T22:08:00.001-08:00

This article introduces the concept of database management in the following sections:

· Introduction to Database Management

· Monitoring Databases

· Administering Databases

· Maintaining Databases

· Additional Maintenance Features

· Monitoring Real Application Clusters

Introduction to Database Management

Database management involves the monitoring, administration, and maintenance of the databases and database groups in your enterprise. Enterprise Manager is the premier tool for managing your database environment.

With Enterprise Manager, you receive:

· A complete set of integrated features for managing Oracle Databases

· Unparalleled scalability that lets you manage a single database or thousands of instances

· An intuitive management product that leads the industry in ease of deployment and use

Database Control Versus Grid Control

Enterprise Manager provides two separate consoles that you can use to monitor your database: Database Control and Grid Control.

· Database Control is the Enterprise Manager Web-based application for managing Oracle Database 10g Release 2 (10.2). Database Control is installed and available with every Oracle Database 10g installation. From Database Control, you can monitor and administer a single Oracle Database instance or a clustered database.

· Grid Control is the Enterprise Manager console you use to centrally manage your entire Oracle environment. Within Grid Control, you access the multiple database targets using the Targets tab, then Databases.

Database Home Page as the Management Hub

The Enterprise Manager Database Home page shows important status and performance information about your database instance from a single source, including:

· A quick view of the status of the database and basic information about the database

· Relative CPU utilization of the Oracle host

· Amount of time the instance consumed using CPU and I/O, and the amount of time it consumed in bottlenecks

· Current response of the tracked set of SQL versus the reference collection response

· The number of Automatic Database Diagnostic Monitoring (ADDM) findings, the number of policy violations (Database Control only), and the Alert Log status

· Storage-related issues and recommendations for improved performance, and information about space violations and ADDM

· The most recent backup time and backup status

· Outstanding alerts and related alerts

· Rolled-up number of Oracle policy violations across all member targets in a system

· A quick view of database security

· Job executions that show the scheduled, running, suspended, and problem executions

Starting from the Database Home page, you can access additional details and management functions by drilling down through the user interface. In addition, the Database Home page provides a list of Related Links. From these links you can perform activities such as editing metric thresholds, analyzing job activity and metric collection errors, and accessing a number of advisors to help you improve the performance of your database.

See Also:

"Oracle Database Home Page" in the Enterprise Manager online help

Monitoring Databases

Comprehensive database monitoring enables you to identify the problem areas in your database environment that are degrading performance. After you have identified the areas to improve, you can tune your databases' performance using the Enterprise Manager administration capabilities.

Enterprise Manager uses data from the Automatic Workload Repository (AWR) to display performance information and initiate database alerts. The user interface provides several real-time performance charts and drill-downs for the targets you manage. Both aggregate and instance-specific performance statistics are displayed using color-coded charts for easier viewing. To help you identify the source of a problem and resolve it, you can click a legend link next to a chart to display a detail page that provides comprehensive information.

The basic work flow in monitoring performance is to first go to the Database Performance page, which provides a high-level, comprehensive overview of important performance indicators.

Assessing Database Performance

Database tuning is much easier and more effective when all the information for an in-depth performance diagnosis is available on a single screen. On the Database Performance page, which is accessible from the Database Home page, you can quickly view performance data for the instance and its underlying host. Additionally, you can analyze trends for all critical performance metrics and compare the trends to those of other database instances.

The most direct method for investigating and diagnosing performance problems from this page consists of accessing the Automatic Database Diagnostic Monitor (ADDM). After you start your Oracle database, the Automatic Workload Repository (AWR) begins taking snapshots of database activity once every hour by default, and ADDM executes when the snapshots are collected. ADDM uses these snapshots to perform a top-down analysis of your database activity and then provide tuning recommendations.

Another method for investigating and diagnosing performance problems consists of observing which of the wait classes beside the Average Active Sessions chart appears to be consuming an excessive amount of time, shown as a spike above the Maximum CPU line, then drilling down for more information. This provides a way for you to visualize the data for which ADDM has made its recommendations.

Either method assists you in diagnosing and resolving problems. The first automated method produces textual findings, whereas the second interactive method produces graphical findings.

The charts on the Database Performance page display current and recent metric information on a common time axis that enables visual correlation of metrics. These charts provide context-sensitive drill-downs where you can find additional details for rapid problem diagnosis.

Table 12-1 Performance Page Charts

Chart	Description
Host	This chart shows potential problems outside the database. The load average is a moving average of the run queue length. The run queue length indicates the level of contention for CPU time.
Average Active Sessions	This chart is the centerpiece of Oracle performance monitoring, and shows potential problems inside the database. It displays a profile of the amount of time sessions are either working or waiting to work in the database instance. The Maximum CPU line on the chart helps determine how much of the CPU resource is being utilized. Session wait time accumulates above the CPU line, and the ratio of wait time to CPU time indicates how efficiently the system is working. Specific wait classes accumulating time indicate where you should focus your tuning efforts.
Instance Disk I/O	This chart shows the total requests that represent the rate at which the database instance is issuing read/write requests.
Instance Throughput	The charts show any contention that appears in the Average Active Sessions chart

The Additional Monitoring Links section enables you to access several related pages. Among those, the following pages have real-time diagnostic capabilities:

· Top Activity

· Top Consumers

· Instance Activity

· Historical SQL

See Also:

"Performance Page" in the Enterprise Manager online help

Diagnosing Problems Interactively

You can manually diagnose problems using an interactive method by first investigating a wait class that you think may be problematic, then drilling down from there for either SQL details or session details. You can alternatively investigate all wait classes in one location and then drill down from there. The following sections explain these methods.

Investigating a Wait Class

Wait class drill-downs enable you to investigate a specific wait class where it appears that time is accumulating. You can then determine whether the problem is caused by one or several SQL statements that are accumulating an unusual amount of time, or whether the problem is caused by one or several sessions.

The wait class drill-downs are called Active Sessions Waiting pages. The pages are sourced from the Active Session History (ASH), which samples session activity every second. It continuously records what is happening, such as which sessions are using CPU and which sessions are waiting on I/O.

Select a wait class link beside the Average Active Sessions chart to obtain detailed information for the wait class. For instance, if you click User I/O, the Active Sessions Waiting: User I/O page appears.

By default, the largest consumers of resources appear at the top of the detail tables. Look for skewed activity for either Top SQL or Top Sessions. If an excessive accumulation of activity appears to originate from an SQL source, you can click its associated SQL ID to go to the SQL Details page, which shows the SQL statement and activity. If excessive accumulation appears to originate from a session source, you can click its associated Session ID to go to the Session Details page, where you can kill the session if needed. You can also view the wait events associated with the session.

See Also:

"Active Sessions Waiting Page" in the Enterprise Manager online help

Viewing SQL Details

The SQL Details page, displays the SQL statement you select in the Active Sessions Waiting page. You can also:

· Examine the activity of this SQL statement over time.

· View SQL-level statistics.

· Investigate the SQL plan.

· Access any prior Tuning Advisor run.

· Schedule a Tuning Advisor run.

See Also:

"SQL Details Page" in the Enterprise Manager online help

Viewing Session Details

The Session Details page, displays the wait events associated with the session you selected in the Active Sessions Waiting page. You can also:

· View the current values for metrics associated with the current session.

· View the list of currently open cursors in the selected session, including their hash value and SQL text.

· View sessions that are blocking other sessions.

See Also:

"Session Details Page" in the Enterprise Manager online help

Viewing Top Activity

The Top Activity page, is essentially the sum of all wait class drill-downs. The interface format is the same as the Active Sessions Waiting page, but rather than showing the average active sessions for a particular wait class, the Top Activity page shows the average active sessions for all wait classes. The page also shows the Top SQL and Top Sessions consumers for all wait classes. As with the Active Sessions Waiting pages, look for skewed activity for either Top SQL or Top Sessions.

See Also:

"Top Activity Page" in the Enterprise Manager online help

Diagnosing Problems Automatically

As mentioned in Assessing Database Performance, you can use the Automatic Database Diagnostic Monitor (ADDM) to automatically investigate and diagnose performance problems from either the Database Home page or the Database Performance page. Using regularly scheduled snapshots of the database activity, ADDM identifies the most resource-intensive components or operations, and determines whether or not these components or operations are acting as performance bottlenecks. If one or more problems have occurred, ADDM diagnoses these potential problems and provides advice, which may recommend that you run an advisor or change your database configuration.

You can view ADDM findings over a predetermined period of recent time or at the present time. You can view ADDM findings for the most recent time interval by clicking the ADDM Findings link in the Diagnostic Summary section of the Database Home page. You can also view ADDM findings for this interval by clicking the small ADDM icon below the Average Active Sessions chart in the Database Performance page. Either option displays the Automatic Database Diagnostic Monitor page. To determine ADDM findings at the present time, you can click the Run ADDM button on the Database Performance page.

By default, the database takes snapshots at 60-minute intervals. You can use the Automatic Workload Repository to change the snapshot interval, ranging from 10 minutes to 2 hours.

See Also:

"Performance Finding Details Page" in the Enterprise Manager online help

Using Additional Diagnostic Pages

Besides the primary diagnostic pages discussed above, there are other important secondary pages that can assist you in diagnosing and correcting performance problems. The following sections discuss these diagnostic pages:

· Top Consumers

· Instance Activity

· Historical SQL

Top Consumers

The Top Consumers link provides global summary information for the top database consumers of system resources. You can access detailed metrics data for a specific top consumer, such as sessions, services, modules, and clients. This enables you to pinpoint the most problematic areas on which to focus your database tuning efforts. Top Modules page of Top Consumers, where you can perform tasks such as enabling and disabling aggregation and SQL traces for the consumer. You can use an SQL trace to trace statistics of SQL statements, such as CPU time, elapsed time, and the Explain Plan.

See Also:

"Top Consumers Page" in the Enterprise Manager online help

Instance Activity

The Instance Activity link displays database activity for specific data about groups of metrics, such as cursors, sessions, and transactions. For instance, the Cursors metric group displays information about opened and session cursors along with authentications and parse counts. The legend links below the chart in the graphic view, or the name links in the table view, enable you to access the Top Sessions page for more detailed information.

See Also:

"Instance Activity Page" in the Enterprise Manager online help

Historical SQL

The Historical SQL (AWR) link displays statements stored in the Automatic Workload Repository (AWR) for a 24-hour period. This link is available in the Additional Monitoring Links section when you select the Historical data view.

The table at the bottom of the page shows an analysis of all SQL statements for performance and resource consumption. You can select a statement link to see SQL details (statistics, activity, SQL plan, and tuning information) for the statement.You can also run the SQL Tuning Advisor to receive recommendations for one or more statements.

See Also:

"Historical SQL (AWR) Page" in the Enterprise Manager online help

Using Other Performance Tools

Enterprise Manager offers several tools that provide additional assistance ranging from viewing the relationships between system components to collecting system statistics when database performance is degraded. The following sections discuss these tools:

· Topology Viewer

· Metric Baselines

· Memory Access Mode

· Hang Analysis

Topology Viewer

The Topology Viewer is available for the following database applications:

· Scheduler

· SQL Details

· SQL Explain Plans

· Real Application Clusters

· Cluster Databases

The Topology Viewer provides a graphical representation of your SQL steps as modeled in Enterprise Manager.

Metric Baselines

From the Database Performance page, you can set a metric baseline—a time period associated with a target that you can use as a reference for evaluating target performance . Statistics are computed over the baseline period for specific target metrics. You can use these statistics to automatically set metric thresholds for alerts, as well as to normalize graphical displays of system performance.

See Also:

"Metric Baselines Page" in the Enterprise Manager online help

Memory Access Mode

When the database server experiences performance issues, diagnostic queries could further impact the system. A new feature, called Memory Access Mode, assists you in diagnosing performance-related problems by collecting system statistics even when the database is either slow or hung.

Instead of relying on the normal SQL engine, the data is retrieved by accessing tables directly from the Shared Global Area (SGA). The data collection is quick and does not further impact a system that is already slow. Potentially, this allows for sub-second sampling of performance metrics without causing any additional load on a system.

You can disable the standard SQL Access Mode and switch to Memory Access Mode by clicking the Monitor in Memory Access Mode link in the Related Links section.

See Also:

"Database Performance Page" in the Enterprise Manager online help

Hang Analysis

A new feature, called Hang Analysis, enables you to diagnose locking issues that might either slow down a system or cause hanging. Normally, if a system is slow or in a hung state, the diagnostic queries are also either extremely slow or never return a result. This utility bypasses the typical query engine instead, and leverages the Oraclebug API to return results quickly even on systems that might be seemingly in a hung state.

The Hang Analysis page shows a visual map of the sessions that are either blocking or are blocked. A tree view of the sessions is displayed, and the problem session that is blocking other sessions is located at the root of the tree. Each session is shown color-coded, signifying how long the session has been blocked. Clicking on the session box brings up another page that shows session details. Using information from this page, you can cancel the problem session and return the system to its normal state.

You can access the Hang Analysis page by clicking the Hang Analysis link in the Additional Monitoring Links section.

See Also:

"Hang Analysis Page" in the Enterprise Manager online help

Administering Databases

Oracle Enterprise Manager effectively keeps your Oracle Databases available and running efficiently. Enterprise Manager can help database administrators perform everyday tasks. Specifically, it provides a graphical user interface for managing database storage structures and schemas. As with database monitoring, administration of the Oracle Database begins with the Oracle Database Home page. From this page, you can display an overview of your database properties and performance. However, you can also use the Administration section of the page to perform common administration tasks such as the following:

· Allocating system storage and planning future storage requirements for the database system

· Creating and managing primary database storage structures (tablespaces)

· Creating and managing primary objects (tables, views, indexes)

· Enrolling users and maintaining system security by controlling and monitoring user access to the database

· Backing up and restoring the database

Just as Enterprise Manager monitoring identifies problem areas in your database and database groups, you can administer your database using the Enterprise Manager administration tools. The administration tools allow you to manage database objects and initiate database operations inside an Oracle Database. The following sections provide an overview of the database administrative features available to you in Enterprise Manager.

Managing Storage Objects

As an administrator, you can use the administration tools in Oracle Enterprise Manager to optimize database performance. Using these tools, you can manage storage structures such as control files, tablespaces, datafiles, and archive logs. In addition to viewing, editing, and deleting these structures, you can also perform other functions such as making tablespaces locally managed, displaying the dependencies of a datafile, or backing up a control file to a trace.

Using Database Configuration Features

Oracle Enterprise Manager incorporates many features that help you to manage your database configuration. For example, you can use Enterprise Manager to manage the memory size of the system global area and program global area of your system. You can also use the Undo Management feature to provide a means of explicitly specifying the amount of undo information to retain and ensure that undo information is not overwritten.

You can create or edit initialization parameters for the current database, setting these parameters to specific values to initialize many of the memory and process settings of an Oracle Database instance. You can also display a list of database features, showing how often the features are used in the operation of the database. Usage information can be utilized by support groups and other organizations to gain knowledge about how the system is being used, and to help apportion resources as necessary.

Using Automatic Storage Management

Enterprise Manager uses Automatic Storage Management (ASM) to automate and simplify the layout of datafiles, control files, and log files. Database files are automatically distributed across all available disks, and database storage is rebalanced whenever the storage configuration changes. You can manage your entire Automated Storage Management environment through Enterprise Manger. Additionally, you can easily migrate an existing database to Automatic Storage Management on the same host by using the migration tool in Enterprise Manager. The process defines the migration request and submits a job to complete the migration process.

Converting Single Instances to Real Application Clusters

Converting to a Locally-Managed Tablespace

An added feature allows you to convert a dictionary-managed tablespace to a locally-managed tablespace, thereby simplifying space allocation, improving performance, and reducing reliance on the data dictionary.

Controlling Resources with Resource Manager

The Database Resource Manager controls the distribution of resources among various sessions by controlling the execution schedule inside the database. By controlling which sessions run and for how long, the Database Resource Manager can ensure that resource distribution matches the plan directive and hence, the business objectives. You can use the Database Resource Manager to automatically assign sessions to specific consumer groups by setting up mappings between session attributes and consumer groups. You can also map consumer groups in specified categories to users, client programs, modules, or services.

Resource consumer groups let you group user sessions together by resource requirements. Resource consumer groups are different from user roles; one database user can have different sessions assigned to different resource consumer groups. You can then use a resource plan to specify how the resources are to be distributed among various resource consumer groups.

Resource plans specify the resource consumer groups belonging to the plan and contain directives for how resources are to be allocated among these groups. Plan information is stored in tables in the data dictionary. Several views are available for viewing plan data. In addition to containing resource consumer groups, a plan can contain subplans. Use Enterprise Manager to manage all aspects of the resource plan.

Tracking Statistics to Improve Database Performance

The Workload Repository provides you with a mechanism for gathering database statistics for specific time intervals. You can use the Optimizer Statistics feature of Enterprise Manager to simplify the management of Optimizer Statistics operations such as gathering, restoring, deleting, locking, and unlocking statistics. Use these statistics to improve the performance of your SQL statements.

Using Oracle Scheduler

Oracle Enterprise Manager uses the Scheduler to enable database administrators and application developers to control when and where various tasks occur in the database environment. Using the Scheduler helps to improve the management and planning of these tasks. By separating a task into its component parts such as time, location, and database object, the Scheduler provides an easier way to manage your database environment. Database administrators can schedule and monitor recurring database maintenance jobs, such as backups or nightly data warehousing loads and extracts, and they can schedule job execution based on time or events.

With Enterprise Manager, you can enable and disable Scheduler jobs, alter the settings for existing jobs, start or stop current jobs, and view Scheduler information.

Working With Database Schemas

A schema is a collection of database objects comprised of logical structures that directly refer to the data in the database. Schema objects include structures such as tables, views, and indexes. These schema objects can be created and managed using the tools available in Oracle Enterprise Manager.

Managing Database Objects

Oracle Enterprise Manager provides a comprehensive set of tools that allows you to manage all aspects of database directory objects such as tables, indexes, and views. You can use the tools available in Enterprise Manager for fundamental tasks such as creating, editing, and viewing object properties, but you can also use Enterprise Manager for more comprehensive tasks such as running the Segment Advisor to evaluate a table for block and space usage and to determine whether space can be saved by shrinking highly fragmented segments. The space gained by the implementation of these recommendations is returned to the table.

Indexes are optional structures associated with tables that can be created to increase the performance of data retrieval. When you manage indexes in Enterprise Manager, you can perform functions such as shrinking segments to compact segments and free the recovered space to the current tablespace. Alternatively, you can also eliminate space problems by reorganizing space usage while changing the storage settings and location of the index.

Views are customized presentations of data in one or more tables or other views. In addition to creating, deleting, and managing views, you can also view the objects that are dependencies of the view. The Dependencies table shows you the Object Name and the Object Type that is dependent on the current view. Conversely, Enterprise Manager can also show you the objects on which the current view is dependent.

In general, use the Action menu on the Search page or Property page of any database object to see a list of the available functions you can perform on that object.

Enterprise Manager allows you to manage program structures, such as packages, package bodies, functions, and triggers in the same way. In addition to creating and viewing these elements, you can compile Java Sources, for example, or generate the data definition language (DDL) code for a specified object such as package. For more information about the various functions available for a specific object, you can refer to the online Help.

XML Database Features in Oracle Enterprise Manager

XML, eXtensible Markup Language, is the standard way to identify and describe data on the Web. Oracle XML DB treats XML as a native datatype in the database. Oracle XML DB offers a number of easy ways to create XML documents from relational tables. The result of any SQL query can be automatically converted into an XML document. Oracle also includes a set of utilities, available in Java and C++, to simplify the task of creating XML documents.

Managing Users and Privileges

Oracle includes security features that control how a database is accessed and used. Privileges and roles control user access to data and the types of SQL statements that can be executed. There are three types of privileges and roles: system privileges, object privileges, and roles. The system privilege is an Oracle-defined privilege usually granted only to and by administrators. It allows users to perform specific database operations. An object privilege is an Oracle-defined privilege that controls access to a specific object. A role can be created by users, usually administrators, and is used to group together privileges and other roles. This facilitates the granting of multiple privileges and roles to users.

Privileges and roles can be granted to other users by users who possess the power to do so. Oracle Enterprise Manager allows you to create and manage users, roles and profiles. You can also expire passwords and lock or unlock users by applying those actions against one or more users. When managing roles, you can display all users and roles assigned to the specified role by using the Show Grantees function.

Auditing is the monitoring and recording of selected user database actions. It can be based on individual actions, such as the type of SQL statement run, or on combinations of factors that can include name, application, time, and so on. Security policies can cause auditing when specified elements in an Oracle Database are accessed or altered, including content. You can set up and adjust audit settings easily within the Enterprise Manager interface. With Enterprise Manager, you can view the database audit configuration and manage audited objects, privileges, and statements. You can also view the content of the audit trail. Out-of-box, Enterprise Manager also provides audit reports to help you monitor successful and failed login attempts, as well as SYSUSER operations.

Managing Materialized Views

Materialized views are schema objects that can be used to summarize, compute, replicate, and distribute data. They are suitable in various computing environments such as data warehousing, decision support, and distributed or mobile computing. Enterprise Manager lets you create and manage materialized views and provides a set of additional tools that allows you to perform specific actions against the view. The Explain Materialized View feature helps you to determine what is possible with a materialized view by indicating whether a materialized view is fast-refreshable, what types of query rewrites you can perform with this materialized view, and whether PCT refresh is possible.

Oracle maintains the data in materialized views by refreshing them after changes are made to their master tables. The refresh method can be incremental (fast refresh) or complete. For materialized views that use the fast refresh method, a materialized view log keeps a record of changes to the master tables. You can use Enterprise Manager to advise you when a query can be rewritten. You can then take the appropriate action required to make a query rewrite. Query rewrite transforms a SQL statement expressed in terms of tables or views into a statement accessing one or more materialized views that are defined on the detail tables.

Materialized views can be refreshed either on demand or at regular time intervals. You can use the Action menu in Enterprise Manager to manually refresh a materialized view.

About Change Management

A dictionary baseline is an object containing a set of database definitions captured at a certain time. Baselines are stored internally in the Enterprise Manager repository, and are in a form that other Change Management applications can use. You can use Enterprise Manager to capture the database object definitions at a particular point in time and create reusable baseline scope specifications. By capturing a dictionary baseline, you can compare different database objects at different points in time and track changes applied to the database objects.

A dictionary comparison identifies differences in database object definitions between two databases, a database and a baseline, or two schemas within a single database/baseline. With Enterprise Manager, you can compare, view, and track differences between two sets of database object definitions at different points in time. The comparison result displays identical objects, objects/attributes that are different, and objects present either in the left or right source.

Using Advisors in Oracle Enterprise Manager

Advisors are procedures that you can invoke, or Enterprise Manager can invoke internally, that designate a specific object for analysis. An advisor can report on a variety of aspects of the object, and describe a recommended action for each condition worthy of user intervention. The advisor might report that the condition can be corrected by an automated task that it provides.

Some advisors also provide what-if analysis for specific situations. For example, the Undo Advisor provides an analysis of the impact of changing the retention period for undo records on the size of the undo tablespace. Additionally, the Memory Advisor graphically displays the impact on performance of changing the size of a component of the SGA.

You can invoke an advisor from the Advisor Central Home page that is displayed when you click the Advisor Central link under the Related Links heading on the Database Home page, or on other pages where it is listed. You can also invoke advisors in the context of recommendations from alerts. Advisors are powerful tools for tuning your database. In general, advisors produce more comprehensive recommendations than alerts, since alert generation is intended to be low cost and have minimal impact on performance. On the other hand, because Advisors are often user-invoked, they can consume more resources and perform more detailed analysis. Their automated analysis can provide more results than you might normally be able to produce manually in the time afforded you as part of regular operations. This analysis, along with the what-if capability of some advisors, provides vital information for tuning that cannot be procured from any other source.

Maintaining Databases

You can use Oracle Enterprise Manager to control the flow of data between or outside Oracle Databases. The following sections provide an overview of the functions available to you that can help you maintain your Oracle Database.

Using Backup

Backup of an Oracle Database generally refers to physical backup; protecting the files that make up your database. The files protected by the backup and recovery facilities built into Oracle Enterprise Manager include datafiles, control files, and archived redo log files. The backup mechanisms that work at the physical level protect against damage at the file level, such as the accidental deletion of a datafile or the failure of a disk drive. The focus in Oracle backup and recovery is generally on the physical backup of database files, which permit the full reconstruction of your database.

Enterprise Manager's physical backup and recovery features are built on Oracle's Recovery Manager (RMAN) command-line client. Enterprise Manager carries out its backup tasks by composing RMAN commands and sending them to the RMAN client. Enterprise Manager makes available much of the functionality of RMAN, as well as wizards and automatic strategies to simplify and further automate implementing RMAN-based backup and recovery.

Managing backups consists of two things: managing the backups themselves as they exist on disk or tape, and managing the record of backups kept in the RMAN repository.

Datafiles or archived redo logs copied at the operating system level can be cataloged and added to the RMAN repository, after which they can be used in data restore and recovery operations just as if they had been created with RMAN. Backup maintenance provided in Enterprise Manager includes the following:

· Viewing lists of backups (backup sets and image copies) recorded in the RMAN repository

· Cross-checking your repository, which marks as expired any backups that are in the repository but not accessible at the time of the crosscheck

· Deleting expired backups from your RMAN repository

· Deleting obsolete backups from the repository and from disk. Note that if you use a recovery area for your backup storage, many maintenance activities are reduced or eliminated because of the flash recovery area's automatic management of disk space.

Recovery

Media recovery using Enterprise Manager can be either complete recovery or point-in-time recovery. In complete recovery, all changes from the logs are applied and the database returns to its state at the time of failure. You can then reopen the database with no loss of data.

In point-in-time recovery, you can choose any System Change Number (SCN) between the time of the datafile backup and the last change in the redo logs, and apply only changes up through that SCN. You can thus return your database to any SCN (and thus any point-in-time) between the time of your backup and the most recent SCN in the redo logs. This technique can be used to recover from situations such as user errors that cause logical corruption in the database. Point-in-time recovery is sometimes referred to as incomplete recovery, since not all changes are applied.

Media recovery requires a control file, datafiles, and all online and archived redo logs from the time the datafiles were backed up. It is typically used only in the case of database failure.

Crash recovery is used to recover from a failure either when a single-instance database crashes, or all instances of a RAC database crash. Instance recovery refers to the case where a surviving instance recovers a failed instance in a RAC database. Datafile media recovery is used to recover from a lost or damaged current datafile or control file. Block media recovery is a technique for restoring and recovering individual data blocks while all database files remain online and available.

Flashback Recovery

Enterprise Manager's flashback features provide a range of physical and logical data recovery tools as efficient, easy-to-use alternatives to physical and logical backups. Flashback table allows you to revert a table to its contents at a time in the recent past; and flashback drop allows you to rescue dropped database tables. Neither requires advance preparation such as creating logical-level exports to allow for retrieval of your lost data, and both can be used while your database is available.

Flashback database lets you quickly recover an Oracle Database to a previous time to correct problems caused by logical data corruptions or user errors. If a flash recovery area is configured, then you can return the database to a prior time. Flashback table lets you recover tables to a specified point in time. You can restore table data along with all its associated attributes, such as indexes, triggers, and so on. This is done while the database is online by rolling back only the changes to the given tables. You can revert the table and its contents to a specific time or user-specified SCN. Use flashback table with flashback query and row versions to find a time to which the table should be restored.

The larger the flash recovery area, the more useful it becomes. Ideally, the flash recovery area should be large enough to hold two complete backup copies of your datafiles, plus any incremental backups and archive logs required to restore your database to any point in time during your recovery window.

Backup and Recovery Settings

You can use Enterprise Manager to configure a number of settings and policies that determine how backups are stored, which data is backed up, how backups perform, and how long backups are retained before being purged from the recovery area. You can also configure settings to improve backup performance. For disks, you can configure the default format for storing backups, the location on disk where backups are stored, and whether backup tasks are run in parallel for improved performance. Oracle backups to disk can be saved as image copies or backup sets. Backups to tape and similar media management devices must be stored as backup sets.

Managing Restore Points

You can use Enterprise Manager to create a restore point as a designated point in time to which you can restore your database. A restore point is a name associated with a past point-in-time of the database. You can flash back the database to a restore point if the required flashback logs and archived logs exist. A Guaranteed Restore Point is a restore point to which you can always flash back the database. Each restore point has a name and creation time. The restore points are sorted by creation time beginning with the most recent.

Overview of Data Guard

Oracle Data Guard ensures high availability, data protection, and disaster recovery for enterprise data. Data Guard provides a comprehensive set of services that create, maintain, manage, and monitor one or more standby databases to enable production Oracle Databases to survive disasters and data corruptions. Data Guard maintains these standby databases as transactionally consistent copies of the production database. Then, if the production database becomes unavailable because of a planned or an unplanned outage, Data Guard can switch any standby database to the production role, thus minimizing the downtime associated with the outage.

Data Guard can be used with traditional backup, restoration, and cluster techniques to provide a high level of data protection and data availability. With Data Guard, administrators can optionally improve production database performance by offloading resource-intensive backup and reporting operations to standby systems.

A Data Guard configuration consists of one production database and one or more standby databases. You can manage primary and standby databases using Enterprise Manager. You can use Enterprise Manager to monitor the status of a configuration as well as the online redo log file activity of the primary and standby databases. At the most basic level, the Data Guard Overview page for the configuration not only displays information about the configuration, but it also includes summary information about its databases.

Additional Maintenance Features

Enterprise Manager allows you to easily move data from files or databases into an existing database. You can export and import data, and you can clone databases using the tools available in Enterprise Manager. The following sections describe the features available in the Maintenance area of the Database Home page.

Exporting and Importing Features

The Export to Files feature enables you to move existing data in Oracle format to and from Oracle Databases. For example, export files can archive database data or move data among different Oracle Databases that run on the same or different operating systems. Using Export to Files, you can back up logical database objects while the database is open and available for use. It writes a read-consistent view of the database's objects to an operating system file.

Conversely, you can use Enterprise Manager to import the contents of a database, objects, and tables. You can also import the contents of a database by using the Import from Database feature. You can import an entire database, schemas within a database and the objects in the schemas, or one or more tables from within a schema. Use the Load Data from File feature to load data from a non-Oracle database into an Oracle Database.

Use the Monitor Export and Import Job feature to view the status of an import and export operation such as a full database export or a tablespace export. You can change the state of a job by suspending it, canceling it, or resuming it if it had previously been suspended. You can also increase the number of threads dedicated to the job and thus increase its resources.

Cloning Databases or Transporting Tablespaces

You can use the Enterprise Manager Clone Database tool to clone an Oracle Database instance to an existing Oracle home. After you have an Oracle Database instance in a known state, you may want to clone that database to another existing Oracle home.

Use the Transport Tablespaces feature to transport tablespaces between different machine architectures and operating systems. Transportable tablespaces entirely bypass the unload and reload steps. Using transportable tablespaces, Oracle datafiles (containing table data, indexes, and almost every other Oracle Database object) can be directly transported from one database to another. You can use the transportable tablespaces feature to move a subset of an Oracle Database and plug it in to another Oracle Database, essentially moving tablespaces between the databases.

Overview of Oracle Streams

Oracle Streams enables the propagation and management of data, transactions, and events in a data stream either within a database, or from one database to another. The stream routes published information to subscribed destinations. As users' needs change, they can simply implement a new capability of Oracle Streams, without sacrificing existing capabilities. Oracle Streams can stream data between databases, nodes, or blade farms in a grid and can keep two or more copies in sync as updates are applied. It also provides a unified framework for information sharing, combining message queuing, replication, events, data warehouse loading, notifications, and publish/subscribe into a single technology.

Oracle Streams provides a set of elements that lets users control what information is put into a stream, how the stream flows or is routed from node to node, what happens to events in the stream as they flow into each node, and how the stream terminates. By specifying the configuration of the elements acting on the stream, you can address specific requirements, such as message queuing or data replication.

Use the tools in Oracle Enterprise Manager to set up and manage your Oracle Streams environment.

Working with Software Configurations

The Oracle Management Agent on a host collects host configuration information for the host and database configuration information for the Oracle Databases on the host and client configuration information and communicates that information over HTTPS to the Oracle Management Service, which stores it in the Oracle Management Repository. Enterprise Manager lets you compare these configurations to determine differences between two or more hosts, clients or databases. The Generic Compare feature allows you to compare various types of current/saved configurations with one or more current/saved configurations.

Using this feature, you can compare the current configuration of a selected target type with one or more current configurations of other targets of the same type, or compare saved configurations with one or more saved configurations of the same or other targets. You can also compare saved configurations with one or more current configurations of the same or other targets, or compare a specific configuration with another configuration and list the differences immediately. Lastly, you can compare a specific configuration with another configuration and schedule the comparison as a job.

Using Database Software Patching

Use Enterprise Manager to simplify the patching of Oracle software on any host where an Oracle Management Agent is running, and to provide critical patch advisories. Enterprise Manager simplifies the process of patching Oracle software. Oracle Patch Advisories describe critical software patches for Oracle products. To help ensure a secure and reliable configuration, all relevant and current critical Oracle patches should be applied. To promote critical patch application, Enterprise Manager performs an assessment of vulnerabilities by examining the host configurations collected for your enterprise to determine the Oracle homes that require one or more critical patches to be installed. All the Critical Patch Advisories are listed with their corresponding Impact areas, a brief description of each advisory, the number of Affected Hosts, and Oracle homes for each advisory.

You can connect to OracleMetaLink via Enterprise Manager, perform a search, download the required patches/patch sets, and apply the patches. You can perform all the patching activities from the Patch Cache. That is, even when the OMS is not connected to OracleMetaLink via the Internet, you can perform a search, download, and apply a patch/patch set.

Monitoring Real Application Clusters

Oracle Real Application Clusters (RAC) provides a high-availability database environment spanning multiple hosts. Each cluster can be comprised of multiple cluster databases, each of which consists of multiple cluster database instances. A cluster database is available as long as one of its instances is available.Enterprise Manager provides performance pages to monitor all levels of a cluster environment, including the cluster, the cluster database, and the cluster database instances. Managing Real Application Clusters databases and instances is similar to managing single-instance databases. Using Enterprise Manager, you can perform various tasks, such as:

· Managing clusters, cluster databases, and cluster database instances

· Monitoring key performance statistics

· Performing administration and maintenance tasks

· Managing Real Application Clusters features, such as Cluster Managed Database Services

RAC enables each computer (or host) that is a member of the cluster to share access to the database. If one cluster host fails or is taken offline, the other hosts of the cluster continue operating, and the entire RAC database remains available for applications. This means that two or more computers with typical performance appear to applications as if they were a much more powerful computer.To increase performance, availability, and reliability of a two-host RAC database, you can add cluster hosts. Because data is not partitioned between hosts, adding hosts to the cluster does not create instability; instead, applications can run faster or support more users. The more hosts your RAC database has, the less the loss of any individual node affects the database.

Cluster Cache Coherency

Concurrent read and write activity on shared data in a cluster occurs frequently. Depending on the service requirements, this activity does not normally cause performance problems. However, when global cache requests cause a performance problem as indicated on the Cluster Database page, a successful strategy for performance tuning is to optimize SQL plans and the schema to achieve effective local cache hit efficiency and minimize I/O. To assist you in resolving the problem, the Cluster Cache Coherency page enables you to view cache coherency metrics for the entire cluster database, identify processing trends, and optimize performance for your Real Application Clusters environment.

See Also:

"Cluster Cache Coherency Page" in the Enterprise Manager online help

The charts have the following purposes:

· Global Cache Block Access Latency represents the end-to-end elapsed time or latency for a block request.

· Global Cache Block Transfer Rate shows the total aggregated number of data blocks received by all instances in the cluster through an interconnect.

· Global Cache Block Transfers and Physical Reads shows the percentage of logical reads that read data from the buffer cache of other instances through Direct Memory Access and from disk.

Cluster Interconnects

The Cluster Interconnects page enables you to view the current state of interfaces on hosts. You can use this page to monitor the interconnect interfaces, determine configuration issues, and identify transfer rate-related issues, such as excess traffic. This page helps determine the load added by individual instances and databases on the interconnect. Sometimes, you can immediately identify interconnect delays due to applications outside the Oracle Database.

See Also:

"Cluster Interconnects Page" in the Enterprise Manager online help

Cluster-Managed Database Services

Services are groups or classifications of applications that comprise business components that extend and respond to application workloads. Examples of services are Accounts Payable, Customer Relationship Management, and so on. Services in RAC enable continuous, uninterrupted database operations to support multiple services on multiple instances.

Services enable RAC to integrate cluster database resources into a single system image to optimize cluster manageability. This simplifies system deployment, testing, disaster recovery, and administrative overhead. With services, users connect to a database without regard for which instance executes the SQL application service.

You assign services to run on one or more instances, and alternate instances can serve as backup instances in case the primary instance fails. If a primary instance fails, Oracle moves the service from the failed instance to a surviving alternate instance.

Services enable you to model and deploy both planned and unplanned operations for all types of high availability or disaster recovery scenarios. During outages, RAC automatically restarts key components. Components that are eligible for automatic restart include instances, Oracle Net Services listeners, and the database as well as several database subcomponents.

You can create and edit services using the Create Services page in Enterprise Manager, which you can access from the Cluster Managed Database Services link in the Enterprise Manager Maintenance page.

See Also:

"Create Service Page" in the Enterprise Manager online help

Oracle Clusterware and High Availability

When you combine Oracle Clusterware and RAC, you can achieve excellent scalability and high availability. To maintain high availability, the Oracle Clusterware components can respond to status changes to restart applications and processes according to high availability rules you can specify in the Create Service page. Oracle Clusterware achieves high availability with the components in Table 12-2.

Table 12-2 Oracle Clusterware High Availability Components

Component	Description
Voting Disk	The voting disk manages cluster membership by using a health check, and arbitrates cluster ownership among the instances in case of network failures. RAC uses the voting disk to determine which instances are members of a cluster.
Oracle Clusterware Registry	The OCR maintains database and cluster database configuration information as well as configuration information about any cluster database within the cluster. The OCR also manages information about processes that Oracle Clusterware controls.
Application Programming Interface	Oracle Clusterware provides a high availability application programming interface (API) that you can use to manage applications or processes that run on single-instance Oracle databases or RAC databases. This enables you to provide high availability for all of your applications.

Application Server Management

2009-12-17T21:58:00.000-08:00

This article describes how you can use Enterprise Manager to manage the crucial components of your middle-tier Oracle Application Servers, which provide you with a platform for deploying your e-business Web applications.

Specifically, this chapter describes how Enterprise Manager can help you manage all aspects of your application server installations.

The Oracle Application Server management capabilities of Enterprise Manager are described in the following sections:

· Out-of-Box Management Using Application Server Control

· Centralized Management Using Grid Control

· Automated Monitoring and Alerts

· Diagnostics and Historical Analysis

· Monitoring Application Server Farms and Clusters

· Viewing Application Server Topology

· Complete Administration

Out-of-Box Management Using Application Server Control

Each Oracle Application Server 10g instance is installed with Application Server Control to manage that instance. Application Server Control provides Web-based management tools designed to monitor and administer application server instances, farms, and clusters. You can also deploy applications, monitor real-time performance, manage security, and configure the application server components.
Application Server Control relies on various underlying technologies to discover, monitor, and administer the environment.

Application Server Control consists of the Application Server Control console and its underlying technologies:

· Oracle Dynamic Monitoring Service (DMS)

· Oracle Process Management Notification (OPMN)

· Distributed Configuration Management (DCM)

· A local version of the Oracle Management Agent specifically designed to gather monitoring data.

For additional management functionality (for example, application service level management, deployments, historical data collections for performance trending alerts, and so on), you can use Enterprise Manager Grid Control.

Centralized Management Using Grid Control

While Application Server Control provides standalone management for one application server and its components, you can centrally manage all your application servers through a single window using Enterprise Manager Grid Control (Grid Control).

For example, if you have ten application servers installed on ten different hosts, you can use Grid Control to manage all these ten application servers. With the help of Management Agents deployed on each host, Grid Control automatically discovers the application servers on these hosts and begins monitoring them using default monitoring levels, notification rules, and so on.

Both Application Server Control and Grid Control have their own application server home pages that provide easy access to key information required by the administrators. The Application Server Home page on Grid
Control provides:

· Application server status, responsiveness, and performance data

· Resource usage for the application server and its components

· List of core components that were installed and configured for the application server, and links to their home pages

· Functionality to start, stop, and restart any of those core components

· Alerts and diagnostic drill-downs so you can identify and resolve problems quickly

· Links to Application Server Control for administration operations such as starting and stopping components, modifying configurations, and deploying applications

· Links to other pages in Grid Control that might be helpful in accomplishing your given task

Automated Monitoring and Alerts

Enterprise Manager provides a comprehensive set of features that facilitates automated monitoring and generation of alerts. The Oracle Management Agent on a host automatically discovers the Oracle Application Server targets on that host, and helps Enterprise Manager perform unattended monitoring of their status, health, and performance.

Enterprise Manager gathers and evaluates diagnostic information from these targets distributed across the enterprise, and an extensive array of application server performance metrics are automatically monitored against predefined thresholds.

For example, Enterprise Manager can automatically monitor:

· The CPU or memory consumption of the application server, including detailed monitoring of individual Java Virtual Machines (JVMs) being run by the server's Oracle Application Server Containers for J2EE (OC4J) instances.

· J2EE application responsiveness from the application down through individual servlets and Enterprise JavaBeans (EJBs).

· HTTP Server session volumes, connection duration, and error rates.

· Oracle Application Server Web Cache hit rates and volumes.

· Top servlets based on number of requests, maximum processing time, and highest average processing time.

If an Oracle Application Server or any of its core components go down, or if a performance metric crosses a warning or critical threshold, an alert is generated by Enterprise Manager and a notification is sent to you. Enterprise Manager supports notifications via e-mail (including e-mail-to-page systems), SNMP traps, and/or by running custom scripts.

When you receive an alert notification, Enterprise Manager makes it easy for you to investigate the problem and take corrective actions wherever required. For example, notification of excessive CPU consumption by OC4J may lead to investigation of the applications running in that container. By using the Top J2EE Applications tab of the Application Server Home page in Grid Control, you can quickly identify the highest volume or least responsive application. You can then drill down and diagnose application's servlets, Java Server Pages (JSPs), or EJBs to identify the bottleneck.

You can set up corrective actions to automatically resolve an alert condition. These corrective actions ensure that routine responses to alerts are automatically executed, thereby saving you time and ensuring that problems are dealt with before they noticeably impact the users.

You can also use monitoring templates to simplify the task of standardizing monitoring settings across your enterprise. You can specify the monitoring settings once and apply them to all Oracle Application Server targets. A Monitoring template defines all Enterprise Manager parameters you would normally set to monitor an Oracle Application Server target, such as:

· Target type to which the template applies.

· Metrics (including user-defined metrics), thresholds, metric collection schedules, and corrective actions.

When a change is made to a template, you can reapply the template across affected Oracle Application Server targets in order to propagate the new changes. You can reapply monitoring templates as often as needed.

Diagnostics and Historical Analysis

The following sections describe important management tasks, such as:

· Diagnosing Performance Issues with Top Reports

· Analyzing Historical Performance

Diagnosing Performance Issues with Top Reports

When you are troubleshooting performance problems, it can be helpful to know which servlets or JSPs are the most active. By using the Top Servlets or Top JSPs performance links of the Application Server Performance page Grid Control, you can identify the top Java servlets or JSPs running on the application server instance. You can then sort them to identify the servlets and JSPs by number of requests, maximum processing time, or highest average processing time.

Analyzing Historical Performance

As with all Enterprise Manager diagnostics, the application server diagnostic reports can be based on current or historical data. Application server metrics are collected and stored in the Management Repository, so you can analyze the data well after the situation has changed. For example, you can use historical data and diagnostic reports to research an application performance problem that occurred days or even weeks ago.

You can even provide a customized time period for which the data should be retrieved from the Management Repository. You can customize the time period for:

· Pre-defined range of the last 24 hours, last 7 days, or last 31 days

· Customized range of any number of days, weeks, months, or years

· Any start date and end date (such that the duration is not greater than 99 years)

Monitoring Application Server Farms and Clusters

Enterprise Manager provides a complete set of features for managing Oracle Application Server Farms (OracleAS Farm) and Oracle Application Server Clusters (OracleAS Cluster).An OracleAS Farm is a collection of OracleAS Clusters and application server instances that share the same Farm Repository.An OracleAS Cluster is a collection of application server instances with identical configuration and application deployment characteristics.

Using Grid Control, you can add OracleAS Farms, OracleAS Clusters and their members to Enterprise Manager for monitoring and centrally managing a set of application server instances and clusters.
Grid Control provides home pages for OracleAS Farms and OracleAS Clusters that help you monitor their overall health. The home pages provide information about the status and availability of all their targets, generated alerts, policy violations, configuration changes, and so forth.

Enterprise Manager also helps you study the High Availability grouping done for the members of an OracleAS Cluster. A High Availability Group is a group composed of similar individual components of application server instances clustered together in a DCM managed cluster. For example, an OC4J High Availability Group has a group of OC4J instances in an OracleAS Cluster.

You can also access the Oracle System Monitoring Dashboard and view the health of OracleAS Farms, OracleAS Clusters, OC4J High Availability Groups, and HTTP Server High Availability Groups. Oracle System Monitoring Dashboard presents information using intuitive icons and graphics that let you spot recent changes and quickly identify and respond to problems. You can customize the display attributes to match information requirements of managed targets, monitor status indicators for recent problems, and see new alerts that have been triggered since the dashboard was last viewed.

Along with managing OracleAS Farms, OracleAS Clusters, and High Availability Groups as targets, Enterprise Manager provides comparative statistics at each level for workload distribution and performance analysis. For example, the Metrics pages for OracleAS Farms, OracleAS Clusters, and HA Groups provide key aggregated metrics from all components in graphical form for performance analysis and component correlation. You can click any of these metric charts to drill down and see more detailed information.
Although Enterprise Manager provides some pre-defined metrics for all of these targets, the Metrics pages are completely customizable to the requirements of a particular deployment. You can include other metrics of your choice, remove existing metrics, or adjust the layout of the metrics as needed.

In addition to monitoring the health of your OracleAS Farms and OracleAS Clusters, you can also monitor the J2EE applications deployed and running across all OC4Js for the application server instances.

You can monitor the members of your OracleAS Farms and Clusters, and take administrative actions like starting, stopping, or restarting each of those members. You can perform administrative operations such as:

· Scheduling jobs to automate commonly-run tasks.

· Creating blackouts to perform scheduled maintenance on the targets.

· Viewing all the installations of the selected target types in your enterprise configuration. For example, you can view all the application server installations, all the database installations, and so on.

· Accessing a list of predefined search queries to search and retrieve information for the targets. For example, you can search all data sources for OC4Js, all deployed applications for OC4Js, all J2EE modules for OC4Js, and so on.

Using Grid Control, you can also view the topology for OracleAS Farms and OracleAS Clusters, the details of which are given in the following section.

Viewing Application Server Topology

Besides monitoring the performance of application servers, OracleAS Farms, and OracleAS Clusters, you can also view their topology to understand what application servers and components are running on which hosts, how these components are related to each other, and how requests are routed through different layers of the deployment. This kind of visualization of the data center enterprise topology helps administrators effectively monitor, manage, and validate the enterprise architecture.

Grid Control provides three different views of topology. Each view provides the overlay of some key metrics of components including current status, number of alerts and policy violations, and CPU/memory utilization performance metrics.

· Host View shows the physical view of deployment and visually shows the relationship between hosts and various components and instances hosted by them.

· Routing Overview shows the routing view of topology and provides an end-to-end view of wired component and application flow through them. The routing includes routing from OracleAS Web Cache to Oracle HTTP Server to OC4J to DB instances.

· Routing Details also shows the protocol and port used by various components to route requests to other components in the topology.

These routing views enable you to fix various topology configuration problems immediately. For example, if an OC4J instance is not accepting any user requests, the routing details view confirms if the front-ending Oracle HTTP Server instances are configured correctly to route user requests to that OC4J instance. OracleAS Web Caches, Oracle HTTP Server or OC4J components are displayed in the same box, if they provide redundancy for each other. For example, a set of OracleAS Web Caches are combined in the same box if all of them are routing to the same set of Oracle HTTP Server instances. Similarly, a set of OC4J instances are combined in the same box if they are routed by the same set of Oracle HTTP Server instances, and are also hosting the same set of J2EE applications. As the like-components are grouped together, the topology visual representation in routing overview or routing details view provides instant Root Cause Analysis for the service availability problems.

Complete Administration

Enterprise Manager provides a full set of features for performing Application Server administration, with Web-based interfaces for performing operations such as:

· Starting, stopping, or restarting any of the core components of the application server.

· Performing configuration management tasks, such as viewing and comparing configuration information. Refer to "Managing Configurations" in this chapter for more information.

· Accessing a list of predefined search queries to search and retrieve configuration information. Refer to "Managing Configurations" in this chapter for more information.

· Integrating application instrumentation in Enterprise Manager's event monitoring infrastructure. Refer to "Extensible Monitoring" in this chapter for more information.

· Staging or applying an interim patch or patch set, and/or cloning an application server's Oracle home to one or more hosts. Refer to "Cloning and Patching the Application Server Environment" in this chapter for more information.

· Performing scheduled backup and recovery. Refer to "Backup and Recovery of the Application Server Environment" in this chapter for more information.

· Creating blackouts to perform scheduled maintenance.

· Viewing the number of scheduled, running, suspended, and problem (stopped/failed) executions for all Enterprise Manager jobs submitted on the application server.

· Creating Application Server Farms and managing their components.

The following sections describe some of the administrative functions that can be performed.

Managing Configurations

Enterprise Manager provides a suite of configuration management functions that can be performed on Oracle Application Server and its components (OracleAS Web Cache, Oracle HTTP Server, and OC4J).
The Oracle Management Agent collects configuration information about Oracle Application Server targets from their respective configuration files, and communicates this information over HTTP/HTTPS to the Oracle Management Service, which stores it in the Management Repository. This information is periodically collected and updated while maintaining the audit of changes. Enterprise Manager's configuration management capabilities efficiently guide the users to desired configuration data in a particular component.

In addition to collecting and tracking hardware and software installations (binaries version number/patch level) of Oracle Application Server targets, Enterprise Manager also tracks configuration details of core components (OracleAS Web Cache, Oracle HTTP Server, and OC4J) of all Oracle Application Server instances. You can compare these configuration details and view the differences and similarities between the core components. You have the flexibility to compare two configurations in the Management Repository or two saved configuration files. You can also compare one configuration with multiple configurations or one configuration in the Management Repository with a saved configuration file.

Using Grid Control, you can search configurations across application servers and find configuration anomalies - whether they are a mismatch of an install/patch version of Oracle Application Server software, or they are a mismatch of software configuration data for the core components of Oracle Application Server. You can perform more intelligent searches to identify all the components hosting a particular application or other resources.

You can also perform some out-of-box searches. The Administration page in Grid Control provided for Oracle Application Server targets allows you to search one or more:

· Origin servers.

· Application servers with particular installation settings.

· Data sources used by the applications deployed across your enterprise configuration.

· J2EE applications deployed in a particular OC4J instance, application server instance, or host.

· Modules of J2EE applications deployed across your enterprise configuration.

· Application server ports across your enterprise topology.

Extensible Monitoring

Many administrators often require custom logic to be written to check for conditions specific to their application environments. Enterprise Manager allows integration of application instrumentation in Enterprise Manager's event monitoring infrastructure. If application developers expose application instrumentation using standards like JMX or Web Services operations, then you can build management plug-ins for the instrumentation using easy-to-use command line tools, and leverage Enterprise Manager's event monitoring system to monitor it. You do not have to edit any XM files or write any integration code to integrate such instrumentation in Enterprise Manager. Simply follow these procedures to integrate application defined instrumentation in Enterprise Manager:

· Use Command Line Interfaces that analyze MBean interfaces for JMX and WSDL for Web Services and create management plug-ins.

· Import Management Plug-in Archive in Enterprise Manager.

· Deploy Management Plug-in to Management Agents.

· Create Target-type instances for the target types defined in Management Plug-in Archive.

· Leverage Enterprise Manager's event monitoring system including monitoring templates, corrective actions, historical and real time metric views, alerts, customization of notification rules, and methods on events generated from application instrumentation metrics.

Cloning and Patching the Application Server Environment

Using Enterprise Manager's automated provisioning tools, you can ensure standardization in your data center and also significantly reduce the time spent on these tasks. To consistently maintain standardization in the topology, it is recommended that the new instances be added through ÒcloningÓ rather than Òinstall and configureÓ.

Cloning ensures that the new instance is installed and configured exactly like other instances in the enterprise topology. Cloning is the process of copying an existing installation to a different location while preserving its configuration and deployments. Enterprise Manager's cloning wizard automates the duplication of application server installations; specifically, the directories where the Oracle homes reside. Its "multicasting" capability also helps you create multiple clones on multiple target hosts in a single operation.

Using a direct link to Oracle MetaLink, Enterprise Manager proactively and regularly retrieves the list of critical patches that have to be applied on Oracle Application Server installations. Enterprise Manager also analyzes the data center environment and notifies you of patches that are applicable to their application server instances. All other patches can also be manually found in the context of a specific target. You can also automate the application of patches using robust job system infrastructure. You can apply the patches instantly or schedule the application in the maintenance window while backing out Oracle Application Server instances in the maintenance window.

Backup and Recovery of the Application Server Environment

Backup and recovery refers to the various strategies and procedures involved in guarding against hardware failures and data loss, and reconstructing data should there be a loss. A comprehensive backup strategy should involve a coordinated approach to backing up your entire application server environment, including the middle tiers and the Application Server Infrastructure Oracle homes.

Enterprise Manager helps you manage the backup and recovery of a single application server or a group of application servers.

Using Grid Control, you can:

· Schedule backups.

· Restore application server backups for recovery.

· Display the status of backup jobs.

· Display the status of recovery jobs.

· Configure the required settings for backup.

What is a Flash Recovery Area and how to configure it ?

2009-12-17T21:48:00.001-08:00

Subject:	What is a Flash Recovery Area and how to configure it ?
	Doc ID:	Note:305648.1	Type:	BULLETIN
	Last Revision Date:	06-OCT-2005	Status:	PUBLISHED

PURPOSE

-------

This document explains the basic design and configuration of a Flash Recovery Area.

SCOPE & APPLICATION

-------------------

DBA, Support.

What is a Flash Recovery Area and how to configure it ?

---------------------------------------------------------

The Flash Recovery Area is an unified storage location for all recovery-related

files and activities in an Oracle database. All the files you need to completely

recover a database from a media failure are part of the Flash Recovery Area.

Flash recovery extends the functionality of Oracle Managed Files to all recovery

related files (backup sets, image copies, and archived logs).  It also provides

automated space management, by deleting older files (based on a user specified

retention policy) to make room for newer ones. The user specifies only the

location of a Flash Recovery Area, and the amount of disk space that Oracle is

allowed to use for recovery related files. This feature is integrated with the

10G MMON feature, so that out of space conditions can be handled through the

standard Oracle monitoring framework.

Question : Why should we use a Flash Recovery Area?

As disk storage media is now competitive to tape with respect to purchase costs,

disk-based backup is an optimal and preferable storage mechanism. By using disks

for storage purposes, you gain significant benefits in terms of mass storage,and

you can randomly access your data in milliseconds rather than hours.  The

previously expensive downtime in traditional recovery is exchanged for rapid

data access and recovery times using cheap disk space.

The advantage that we have over tape is that tape is a sequential access device and disk is a random access

device. Hence the amount of time needed for restoring from the tape is eliminated

or reduced.

ASM (Automatic Storage Management) SUPPORT:

------------------------------------------

- RMAN can back up and restore files that are stored in ASM disk groups.In fact,

  RMAN is the only way to back up and restore ASM files.

- The Flash Recovery Area can be an ASM disk group.

- Backup sets and image copies can be created in ASM, either by automatically

  creating them in the Flash Recovery Area, or by explicitly specifying an ASM

  disk group for their location.

OMF (Oracle Managed Files) SUPPORT:

-----------------------------------

- Backup/Restore of OMF files is easier with RMAN as RMAN does not use

  the long OMF file names in any commands.

- Backup sets and image copies can be created as OMF files by creating them in

  the Flash Recovery Area.

- RMAN can be used to migrate existing files to OMF.

Files are stored in the Flash Recovery Area:

--------------------------------------------

The files in Flash Recovery Area are classified as permanent or transient.

-- Permanent Files

   The permanent files (assuming these are configured to be stored in the

   recovery area) are multiplexed copies of the current control file and online

   redo logs. These cannot be deleted without causing the instance to fail.

-- Transient Files

   Transient files include archived redo logs, datafile copies, control file

   copies, control file autobackups, backup pieces and flashback logs. Oracle

   manages these files automatically for deletion whenever space is required

   in the Flash Recovery Area. They are deleted once they become obsolete under

   the retention policy or have been backed up to tape. Any transient file in

   the flash recovery area once backed up to tape even if not deleted are

   internally placed on a file can be deleted list. Until there is a backup

   of the file on disk made to a teriary storage device it cannot be obsolete.

Initialization Parameters for Flash Recovery Area:

--------------------------------------------------

To enable the Flash Recovery Area, you must set the two initialization parameters:

-- DB_RECOVERY_FILE_DEST_SIZE :

   It is the disk limit, which is the amount of space the flash recovery area is

   permitted to use. The minimum size of the Flash Recovery Area should be at

   least large enough to contain archive logs that have not been copied to tape.

   Note: This value does not include certain kinds of disk overhead:

   -Block 0 or the OS block header of each Oracle file is not included in this

    size, so make sure to allow an extra 10% for this data when computing the

    actual disk usage required for the Flash Recovery Area.

   -DB_RECOVERY_FILE_DEST_SIZE does not indicate the real size occupied on disk

    when the underlying filesystem is mirrored, compressed, or in some other way

    affected by overhead not known to Oracle.

-- DB_RECOVERY_FILE_DEST:

   This initialization parameter is a valid destination to create the

   Flash Recovery Area. The destination can be defined as a directory,

   file system, or ASM disk group.

   Note: The Flash Recovery Area cannot be stored on a raw file system.

   DB_RECOVERY_FILE_DEST_SIZE must be set before DB_RECOVERY_FILE_DEST. The two

   parameters must be set together to enable the FRA. In a RAC database, all

   instances must have the same values for these parameters. Even though there

   are multiple nodes they all share the same controlfiles.

Restrictions on Initialization Parameters:

------------------------------------------

- You cannot use the LOG_ARCHIVE_DEST and LOG_ARCHIVE_DUPLEX_DEST parameters to

  specify redo log archive destinations. You must always use the

  LOG_ARCHIVE_DEST_n parameters in case you have configured flash recovery area.

- LOG_ARCHIVE_DEST_10 is implicitly set to USE_DB_RECOVERY_FILE_DEST if you

  create a recovery area and do not set any other local archiving destinations.

- Multiple database can have the same DB_RECOVERY_FILE_DEST only if the DB_NAME

  are different or if the DB_NAME is same (example the primary and standby

  database) then the DB_UNIQUE_NAME parameter must be different for the

  databases.(3)

- For RAC the location of Flash Recovery Area must be on a cluster file system,

  ASM or a shared directory configured through NFS. The location and disk quota

  must be the same on all instances.

Example :

  Note 275083.1 Flash Recovery Area in 10G

References

----------

Note 293418.1 ORA-19815 WARNING DB_RECOVERY_FILE_DEST_SIZE 100.00% USED

Note 268197.1 New Background Processes In 10g

Note 273015.1 HowTo: 10G Migrating to RAC using Data Guard

Note 305817.1 FAQ - Flash Recovery Area feature of 10G

Unable to Archive Due to flash_recovery_area full

2009-12-17T21:47:00.001-08:00

Subject:	Unable to Archive Due to flash_recovery_area full
	Doc ID:	Note:249452.1	Type:	PROBLEM
	Last Revision Date:	25-OCT-2005	Status:	REVIEWED

The information in this article applies to:

Use of the Oracle 10G database and flash recovery (recovery destination)  areas only.

This article is to help Support analysts workaround a specific problem.

This workaround should only be used following due consideration to how

it may effect the ability to recover and as such should not be made public

even after 10G goes production.

Symptom(s)

~~~~~~~~~~

Archiving fails due to the flash_recovery_area being full.

The alert log will show something like :-

Fri Sep 26 12:05:27 2003

ARC0: Evaluating archive   log 2 thread 1 sequence 240

ARC0: Archiving not possible: No primary destinations

ARC0: Failed to archive log 2 thread 1 sequence 240

Fri Sep 26 12:05:27 2003

Errors in file /u02/primary/admin/DB1/bdump/beta_arc0_24471.trc:

ORA-16014: log 2 sequence# 240 not archived, no available destinations

ORA-00312: online log 2 thread 1: '/u02/primary/oradata/DB1/redo02.log

The archive trace file will show something like :-

*** SESSION ID:(11.1) 2003-09-26 11:47:00.270

ORA-19809: limit exceeded for recovery files

ORA-19804: cannot reclaim 9261056 bytes disk space from 2147483648 limit

*** 2003-09-26 11:47:00.272 49832 kcrr.c

ARC0: Error 19809 Creating archive log file to '/u02/primary/flash_recovery_area/BETA/archivelog/2003_09_26/o1_mf_1_241_%u_.arc'

*** 2003-09-26 11:47:00.273 48728 kcrr.c

kcrrfail: dest:10 err:19809 force:0 blast:1

Change(s)

~~~~~~~~~~

You will only see this if you are using a flash_recovery_area as set by the

initialisation parameter

db_recovery_file_dest    = /u02/primary/flash_recovery_area

And you use the allowed spce in that area as specified by ..

db_recovery_file_dest_size= 2147483648

Cause

~~~~~~~

We register all the information about what we place in the flash recovery area in

the rman repository/controlfile.  If we determine that there is not sufficient space

in the recovery file destination, as set by dest_size then we will fail.

Just deleting the old backups/archive logs from disk is not sufficient as it's the rman repository/controlfile

that holds the space used information.

Fix

~~~~

There are a couple of possible options.

1) Increase the parameter db_recovery_file_dest_size, which is dynamic.

SQL> alter system set db_recovery_file_dest_size=XG; (larger amount)

2) Stop using the db_recovery_file_dest by unsetting the parameter.

   ( This assumes you never really wanted to use this option )

3) Remove the Entries from the rman repository/Controlfile

   The removal is desribed in the RMAN documentation but this is a quick and

   dirty way if you don't have an rman repository - but could endanger your ability

   to recover - so be careful.

   a) delete unwanted archive log files from disk ( rm /del )

   b) connect to rman

   c) rman crosscheck archivelog all - marks the controlfile that the archives have been deleted

   d) rman delete expired archivelog all - deletes the log entries identified above.

You should then find archiving resumes OK.

Important Note

~~~~~~~~~~~~~~

When using ASM for your flash recovery area (with normal or high redundancy),

make sure to take into account the miirrored space when setting the

db_recovery_file_dest_size.  For example, if the ASM disk group space is 160GB,

then available space is only 80GB when using normal redundancy.  In this case,

it is recommended that the db_recovery_file_dest_size be set to 10% less than

80GB.  The 10% gives cushion for orphan files and OSD block headers.

References

~~~~~~~~~~~

Oracle10i Backup and Recovery Basics Release 1 (10.1)