Backup and Restore for mySAP Business Suite
Best Practice for Solution Management Contents
APPLICABILITY, GOALS, AND REQUIREMENTS ............................................................................... 3 BEST PRACTICE PROCEDURE AND VERIFICATION ........................................................................ 4 Procedure...................................................................................................................................... 4 1. Getting Started with Designing a B&R Concept ............................................................. 4 2. Preliminary Tasks ......................................................................................................... 11 3. Performing B&R for Individual System Components .................................................... 12 SAP R/3 ................................................................................................... 15 SAP CRM / EBP Server........................................................................... 16 SAP BW / SAP SEM ................................................................................ 16 SAP APO ................................................................................................. 17 SAP APO liveCache ................................................................................ 17 SAP APO Optimizer................................................................................. 18 SAP EM (Event Manager) ....................................................................... 19 SAP Workplace Server ............................................................................ 19 SAP Knowledge Warehouse ................................................................... 19 SAP J2EE Application Server / InQMy Application Server...................... 19 SAP ITS ................................................................................................... 20 Web server............................................................................................... 21 SAP IPC................................................................................................... 22 SAP IMS .................................................................................................. 23 TREX Search Engine............................................................................... 23 SAP Mobile Client.................................................................................... 24 SAP Mobile Application Studio ................................................................ 25 SAP Mobile Application Repository ......................................................... 25 SAP Mobile Workbench (Development Station)...................................... 26 SAP BDOC Modeler ................................................................................ 26 SAP Communication Station ................................................................... 26 SAP Business Connector ........................................................................ 27 CAT Server .............................................................................................. 27 BackWeb Server ...................................................................................... 28
Best Practice: Backup and Recovery for mySAP Business Suite
2
Requisite Catalog Server......................................................................... 28 SAP Enterprise Portal Server .................................................................. 29 SAP EP Unification Server ...................................................................... 30 SAP EP Knowledge Management ........................................................... 30 SAP Content Server ................................................................................ 31 SAP Content Server Cache ..................................................................... 31 SAP Gateway........................................................................................... 31 Drag & Relate Servlet .............................................................................. 32 4. Performing Consistent Landscape Backups / Landscape Copies................................ 33 5. Managing an Incomplete Recovery .............................................................................. 44 Verification................................................................................................................................... 54 FURTHER INFORMATION ................................................................................................................... 54 Example of a B&R Concept for a mySAP CRM System Landscape................................ 54 Background Information and References ......................................................................... 59
Best Practice: Backup and Recovery for mySAP Business Suite
3
Applicability, Goals, and Requirements
To ensure that this Best Practice is the one you need, consider the following goals and requirements.
Goal of Using this Service
Use this service to guide you in creating a backup and restore (B&R) concept for your multicomponent mySAP Business Suite solution landscape. This best practice covers, for example: • • • B&R for individual components Consistent Landscape Backups Incomplete recoveries
This best practice can be used to create a customer specific backup & restore concept for all kinds of mySAP Business Suite Solution Landscapes, although most examples of this document refer to mySAP CRM. Additional components that may not be explicitly listed under “Procedure” can easily be analyzed with respect to their backup requirements using the background information given throughout this document.
Staff and Skills Requirements
To apply this Best Practice you need a team consisting of • • System administrators, consultants or technical persons charged with planning, setting up, or administrating a mySAP Business Suite system landscape Application developers or consultants with knowledge of your applications and business processes.
Duration and Timing
The initial B&R concept should be created during the implementation of a mySAP Business Suite solution and must be finalized before the start of production. Creating your B&R concept may take around two weeks, depending on the complexity of your solution landscape and business scenarios. Verifying, adapting, and testing the concept should be an ongoing process during the complete lifecycle of your mySAP Business Suite implementation.
How to Use this Best Practice
• Use the information given under “Getting Started …” to provide you with the necessary background information for setting up a backup & restore concept for your mySAP Business Suite system landscape. Complete the analysis described under "Preliminary Tasks" to gather all required information about your system landscape. Design your B&R concept using the following sections under "Procedure". Verify the success of your use of this Best Practice as described under "Verification"
• • •
Example B&R Concept for a mySAP CRM Landscape
For a sample B&R concept for a mySAP CRM landscape, see Example of a B&R Concept for a mySAP CRM Landscape.
Best Practice: Backup and Recovery for mySAP Business Suite
4
Best Practice Procedure and Verification Procedure
1. Getting Started with Designing a B&R Concept
A B&R concept is not only influenced by the system components actually in use but also by many other customer- or business process-specific factors of an implementation of mySAP Business Suite. This is why a B&R concept must always be tailored individually to suit the demands of each specific implementation. To design a backup and restore concept, many topics need to be considered, for example: • • • • • Which system components and which data need to be backed up Which backup methods shall be used How is data consistency between the system components maintained Is a Consistent Landscape Backup needed and in what situations What are the impacts of an incomplete recovery and which steps need to be taken
The following sections guide you through these questions. Section 1.1 provides background information on database and application data consistency after a restore. It also looks at the need for a consistent backup of the whole landscape from different angles. Section 1.2 describes the requirements a B&R concept must fulfill. Section 1.3 lists types of backup relevant data of software components while section 1.4 discusses different backup strategies for these types of data. Finally, section 1.5 indicates the aspects that must be covered by a good B&R concept.
1.1 General Aspects
In contrast to the traditional SAP R/3 system, mySAP Business Suite Solutions use several SAP component systems to implement cross-system processes. Data is no longer held centrally in a single SAP R/3 system but is rather distributed between several SAP systems. Data is often held redundantly whereas each system might also hold originals of some pieces of data. Data transfer between the systems is automated and must ensure that data is always consistent between the participating systems. There may be different input sources for the same type of data in the same system (for example, sales orders in the SAP R/3 system can originate from the Internet -via the SAP CRM server- or can be entered directly in the SAP R/3 system). From a technical point of view, data may be held in databases provided by different database vendors or data may even be held directly in files without using a database at all. There is no common ‘checkpoint’ between the component systems and thus no common point of consistency because data is constantly and automatically exchanged between systems. To find out what requirements a B&R concept must fulfill, let us first have a look at a typical database restore scenario. Normally a database restore will consist of two steps: Restore of a backup and subsequent log recovery. By applying the logs to a restored database, the database can always be recovered to the exact state at the time of a crash. If systems exchange data, this is sufficient to ensure data consistency after the restore. All committed transactions are recovered while all open transactions are rolled back. In conjunction with the tRFC-protocol used for data exchange, this ensures a consistent cross-system state after a restore. For additional information on data consistency during normal operation, see also Data Consistency During Normal Operation. Supposing that a failure requires a restore of one system, it is thus absolutely sufficient to restore only this single system using a backup of this system. (Prerequisite: The system can be recovered to the crash point in time – we will come back to that later). So the answer to the most common question when talking about a backup concept for a multi-system landscape "Do we need a consistent backup of all our systems?” is "For normal operation: No!” It is
Best Practice: Backup and Recovery for mySAP Business Suite sufficient to backup each system individually because a restore will also be done for just an individual system. To analyze this further, let us now look at the question "What is a consistent backup of a system landscape?”
5
Consistent Backup of a Complete System Landscape
A consistent backup of a complete system landscape is a backup that includes all systems of the landscape and that provides a consistent state of all data over all systems. That means that by restoring the complete landscape using this consistent backup, the data is consistent between all systems. Cross-system data consistency means for all replicated data that the corresponding original data exists in the corresponding system and that they are identical. For all data that should be replicated to other systems the data must already be replicated or will be replicated after the restore (for example, if the RFC queues have not yet been processed they will be processed after the restore). On the other hand, a consistent backup cannot reflect any arbitrary point in time like a database backup of a single system can. So when restoring a Consistent Landscape Backup after a crash, the landscape will not reflect the current state at the time of the crash but typically the state of some point in time well before the crash (the time when the backup was taken). Note: It is generally not possible to do a consistent restore of two systems by doing point-in-time recoveries of both systems to the same point in time. Because there is no common, synchronized time for all systems, there will usually be some modifications that were already done by one system but not by the other system, thus leading to an inconsistent state. As we have seen above, a consistent backup is not needed in case of a failure of a single system. Now many people object and say: "But we need a consistent backup in case that we cannot recover a system to the most current state”. This leads us to the question of an incomplete recovery.
Best Practice: Backup and Recovery for mySAP Business Suite with a possible incomplete recovery must always be made on site by weighing the factors ‘data loss’, ‘acceptability of inconsistencies’ and ‘downtime’ (time needed to fix problems before business operation can go on).
6
A B&R concept must include a strategy for dealing with exceptional situations like an incomplete recovery. Section 5 will further deal with this topic and analyze different possibilities of how to proceed in that case.
Use of a Consistent Backup
From what has been said above, the question arises, "Why would we then need a consistent backup at all?” A Consistent Landscape Backup may, even if not intended for use after an incomplete recovery, still be useful for other purposes. It can increase security by providing an additional option for the case of disaster recovery (a disaster that requires a restore of the whole system landscape like for example, the destruction of the data center). It can help to create a consistent copy of the whole landscape for example for building up a consistent test environment (for example, for upgrade or conversion tests). A consistent backup can also serve as a fallback point before an upgrade or before and during data migration. Section 4 will present different methods for creating consistent backups of a complete system landscape.
1.2 Characteristics of a Good B&R Procedure
The backup strategy for a customer’s system landscape mainly depends on his recovery requirements. Depending on the recovery scenarios for different error situations, appropriate backup procedures must be determined. Most error scenarios only require the restore of one single system. So most important for a backup strategy is to have a B&R procedure in place for each single system in a system landscape that prevents data loss and thus ensures data consistency between the system components. This usually means that a backup must enable system recovery exactly up to the crash point. Database systems automatically guarantee this property (otherwise this would be an incomplete recovery). For other components based on file systems this must be analyzed individually (see section 1.4 and section 3). When restoring a backup of a non-database system, special care must be laid on a consistent state after the restore. The backup strategy must take this fact into account. Apart from application data the software and configuration files of the system components must also be included in a B&R concept. The next important question is whether a Consistent Landscape Backup is required as a part of the backup concept. The answer mainly depends on the strategy that is chosen for proceeding after an incomplete recovery of one component. If the answer is ‘No, we would not use a Consistent Landscape Backup in this case’, the main task of a backup concept must be to describe procedures on how to deal with data consistency issues that may come up in case of an incomplete recovery. If the answer is ‘Yes, we could restore all systems using a Consistent Landscape Backup’, the backup concept must deal with procedures for creating such a Consistent Landscape Backup. The larger the system landscape, the more demanding these tasks will be; either describing repair strategies for inconsistent data or creating a Consistent Landscape Backup. If Consistent Landscape Backups are not to be used after an incomplete recovery, their use for other purposes as described above should still be considered in a B&R concept. Another important factor of a restore is speed. As the restore process must usually meet some kind of service level agreement that regulates system availability, the backup method must enable a fast restore and recovery process, which meets these requirements. But backup runtime may also be important for the decision on the backup strategy. A fast backup can reduce the impact on production. To almost completely avoid an impact on the productive system, special backup solutions like split-mirror backup can be implemented. These solutions exploit storage technologies that mirror disks to another storage system where a backup can be taken. Please ask your hardware vendor for more information on this kind of scenarios.
1.3 Types of Backup Relevant Data
The backup method and requirements of a specific system component mainly depends on the type of data that the component holds.
Best Practice: Backup and Recovery for mySAP Business Suite In a mySAP Business Suite System Landscape we have to deal with different kinds of data as a candidate for backups: • • • • • Data managed by databases Data in files which are managed directly by the applications Software of the applications Configuration files of the software components Log files of the software components
7
For each of the above kinds of data different backup methods and concepts can be used. In the following we will refer to the first two kinds as ‘application data’ or ‘business data’ to distinguish them from the application software, configuration files and log files. The backup method then also depends on the following factors: • Does the component hold original data (which means the component is the leading system for some pieces of data) or replicated data? When speaking of replicated data, we mean both data that is merely replicated from another system and data that is derived from another system’s data. The time needed to replicate data: Depending on the time it takes to restore replicated data from the originals it might not be necessary to backup the replicates. The type of data The type of system managing the data
• • •
In mySAP Business Suite landscapes we have system components that do not hold any application data, system components whose application data is managed by a database system and system components that manage their application data by themselves. Based on these factors, we developed a categorization of system components that you will find in the appendix.
1.4 B&R Alternatives Depending on Data Type
In this section we look at backing up system components depending on the type of data they contain.
B&R of Application Data
In general, all systems that hold originals of application data must be backed up carefully to avoid the loss of important business information. On the other hand systems that hold only replicated data could be rebuilt from the original data sources so there might be no need to do a backup. Often it is not clear at a first glance whether a component only holds replicated data or if there are also original pieces of data. So the type of data must be carefully analyzed for each system component before the decision for this component’s backup strategy can be made. The backup method to be used for the component then depends on the type of data management: It can be a database and log backup in case of database-managed application data or a file system backup in case of file-based data managed by the application itself. In all cases aspects of data consistency between the system components must be regarded for system components that exchange data. For replicated data there can be three alternatives concerning the backup strategy: No Backup This means that the data must be completely re-replicated in case of the components failure. This will be equivalent to an initial data load (initial download) for this component. This alternative could be used if the replication speed is sufficient to fulfill the time limits required of a restore procedure. Data consistency between the components will be automatically ensured. An important prerequisite for this alternative of course is that the component does not hold any original data at all (which would be lost after the initial load). Regular backup of the data using an appropriate backup method (database or file system backup, full or delta backup). In case of a failure this backup can be restored.
Best Practice: Backup and Recovery for mySAP Business Suite Special attention must be paid to the fact that after a restore, the data must be consistent to the state of the other components. Restore Using Multiple Instances
8
This means that in case of multiple installations of the same component holding the same data, each one could serve as a data backup of the other. This scenario may for example be applicable if there are multiple identical servers for scalability or high availability reasons. If one instance fails, the data could simply be copied back from another instance. This method requires at least two instances of a component to hold exactly the same data. It is not applicable if similar instances of a component hold different parts of data (for example, if a distributed product catalog is implemented on two servers). This alternative of course requires that the data format between the instances is compatible and that it is possible to copy the data onto another instance and make it available there. Data consistency between the systems should then be automatically ensured (but this must be analyzed carefully when using this alternative). A backup might then only be needed if all instances of the same type are destroyed (but in this unlikely case the data could be downloaded from the original system again, even if replication speed is considered insufficient under normal circumstances).
An exception to this is represented by the data of the mobile client databases in mySAP CRM. Although at some time they hold original data (the data that was entered during the day) there is no need for a special backup if the data is uploaded to the CRM server on a very regular basis. If the mobile client database is destroyed, it can be rebuild by downloading all relevant data from the CRM system. Still all data that has been entered on the mobile client and not yet been uploaded to the CRM server will be lost.
B&R of Software and Configuration Data
Apart from business critical application data the system and application software itself may be worth being backed up. This can prevent from a new installation in case all or parts of the software have been destroyed. As a new installation will always be possible, this backup is not mandatory. A general recommendation is to backup the application software at least once after it has been installed or after it has been upgraded. Note that restoring a backup of the system or application software is generally only possible if it is restored to exactly the same hardware. Installing the software often requires a large amount of configuring and customizing. So saving this kind of information may also be important to provide a fast restore and to avoid time and effort after possible failures of a component. As the configuration may change more often that the software, the configuration files should generally be backed up on a regular basis or each time the configuration has been modified. The following items should be considered when planning a B&R strategy for software and configuration files: • • • • • • Operating system DBMS software SAP software and file systems Log files (SAP and other) Software of other system components (file systems, configuration files, log files) Files used for data transfer or interface implementation (for example, ftp-files)
Note: Instead of identifying all relevant files and configuration entries, a full file system backup (and registry backup on Windows platforms) will usually be the easiest option for backing up software, configuration data, and log files.
Best Practice: Backup and Recovery for mySAP Business Suite example the actual system landscape design, the hardware and software in use, the amount of data, the implemented processes, the distribution of original and replicated data onto the various system components, the replication intervals, special service level agreements and so on. This easily indicates that it is not possible to provide a general B&R concept here. This document describes the general ideas of a B&R concept for a distributed system landscape and provides the information needed to set up a B&R concept for a specific mySAP Business Suite implementation. If available it shows different alternatives that can be used to backup individual system components. The actual decision then mainly depends on the possibilities available on site and the time needed for a restore in case of a failure.
9
Apart from the decision how to backup each individual system component the question of whether a consistent backup of the complete landscape is needed for special purposes (see 1.1 or 4) must be answered. It must be determined what it shall be used for, which technique shall be used to create the backup and when or how often it shall be taken. A B&R concept must also take into account possible procedures in case of an incomplete recovery of one of the system components. A strategy for dealing with incomplete recoveries is mainly influenced by the factors ‘data loss’, ‘inconsistencies’ and ‘downtime’, all of which should be minimized. It is necessary to evaluate to which degree each of the factors' influence could be tolerated in case of an incomplete recovery. Each project must set up a description of what needs to be done from an application point of view if specific application data is lost in one of the systems. This part of a backup concept involves a lot of application and process specific knowledge. The implementation of a backup concept for a distributed system landscape thus always requires that people from the application knowing the customer’s business processes tightly work together with technical persons and system administrators. A B&R concept should be documented in detail and should include the following topics: 1. General information • Components used in the business scenarios • Short description of the business processes • Leading system (original system) for each business object • Systems in which replications of each business object are stored • Data flow description for the main business objects • Availability requirements for the system landscape and for each component • Applicable service level agreements (SLAs) and their restrictions on backup, restore and maintenance times • Backup infrastructure (Backup hardware & software) 2. Backup • Backup procedure for each system component • Backup schedule (frequency, level (full or incremental) • Scheduling procedure • Control mechanisms to check backup status • Tape expiration dates • Backup of development and test systems
Best Practice: Backup and Recovery for mySAP Business Suite 3. Restore / Recovery • Description of error scenarios with their respective solutions • Detailed restore procedure for each component • Dependencies of a component’s restore (possible side effects on other components)
10
• Handling of exceptional situations, like an incomplete recovery of a component • Documentation on usage of additional tools/reports for consistency checks 4. Additional security measures • Hardware redundancy • Database consistency checks • Tape checks • Recovery training schedule and test procedures 5. Special backup procedures (Consistent Landscape Backup; system copies) • When needed? • How often? • Backup procedure • Restore procedure A very important factor for the operability of a restore in case of an emergency is thorough testing and training of administrators on restore scenarios and restore procedures. A test plan should include regular • • • • Restore and recovery tests Application of database logs to ensure correct recovery Database consistency checks to ensure database consistency Restore tests for non-database components
When talking about a B&R concept, other important security measures that increase system reliability and availability should be taken into account. On hardware and storage level this includes RAID protection of disks, redundant hardware paths for accessing data, different devices for database logs and data, additional offline copies of the data and the implementation of High Availability solutions. On database level special care should be taken to secure the database logs using log mirroring on different disk controllers and storing the log files twice on disk and twice on tape. For the case of a restore it might be worthwhile thinking of alternatives that avoid a restore on the actual production landscape. Doing the restore on another system while preserving the original production system has the advantage that the risks involved with a failing restore can be avoided. After the restore was finished successfully there are two possibilities how to repair the production system. If only a few objects were destroyed, these could be repaired from the restored copy without having to discard the original production system (possibly production could even go on during the whole restore procedure). If the original system was completely unusable, production could continue on the restored copy.
Best Practice: Backup and Recovery for mySAP Business Suite
11
2. Preliminary Tasks
Different components have different B&R requirements. Before proceeding to create your B&R concept, you should identify and describe the following: • • The components used in your specific business scenario The availability requirements for the system landscape and each component, depending on the applicable service level agreements (SLAs) and their restrictions on backup, restore and maintenance times The data flow between these components The core business processes exchanging data between system components The business objects which are exchanged between system components The leading system (original system) for each business object The systems in which replications of each business object are stored
• • • • •
This information will be needed to decide on the backup requirements for backing up each individual system component. The information about business processes and business objects is required to identify possible sources and objects of data inconsistencies in the rare case of an incomplete recovery of a system component. This serves as the basis for defining and describing procedures for such scenarios (which can be Business Recovery or the restore of a Consistent Landscape Backup).
Best Practice: Backup and Recovery for mySAP Business Suite
12
3. Performing B&R for Individual System Components
This section lists ways of performing B&R for the individual components of a mySAP Business Suite system landscape. It includes the relevant backup methods, intervals, and recovery procedures for software and data. At the end of this Best Practice you will find a short Example of a B&R Concept for a mySAP CRM Landscape. Before discussing each individual component in turn, below is a table summarizing the main information for individual components. The Backup method given in the table only refers to the backup of application data of each component. Additionally, backups of software, configuration, and log files need to be considered (file system backups)! Note: Instead of restoring a file system backup, the software of a component or of the complete server can usually be installed completely new in case a serious error occurred. So backing up the software and operating system of a server may be obsolete. The decision whether to backup server and software should mainly be based on an evaluation of the cost of the backup and the duration of a restore compared to a new installation. For a server hosting only one software component requiring few configuration work, a new installation may be the easiest option while for a server hosting multiple software components, a complete restore may be faster than a new installation of all the software.
Note: Although many components used by mySAP Business Suite Solutions are listed in the following, it is not possible to always provide a complete listing. To help you analyze the backup requirements for components not explicitly listed here we developed a classification scheme. Together with the general concepts of chapter 1, this scheme allows to integrate such components into your backup & restore concept. The category of each system component is also given in the table below. The classification scheme and general backup method for components of each category can be found in Categorization of System Components.
Component SAP APO
Used in the mySAP Solution
Category XI
Application Data Type Orig. / Repl.
Backup Method for Application Data Database and log backup; file system backup (full and/or incremental) liveCache 7.2: checkpoint and synchronous logging liveCache 7.4: SAPDB database and log backup --3rd party product. Follow guidelines of vendor. -----
mySAP SCM mySAP CRM
mySAP EBP mySAP SCM
SAP APO liveCache
X
Orig. / Repl.
mySAP CRM
mySAP EBP
SAP APO Optimizer BackWeb Server
mySAP SCM
II
---
mySAP CRM – Mobile Sales mySAP CRM 2.0– Mobile Sales mySAP EBP
Database and log backup; file system backup (full and/or incremental) ---
SAP Content Server Cache SAP CRM / EBP Server
III / V
Repl.
XI
Orig. / Repl.
Database and log backup; file system backup (full and/or incremental) File system Backup or No backup*
Drag&Relate Servlet SAP EM (Event Manager)
III
Repl.
mySAP SCM
XI
Orig. / Repl.
Database and log backup; file system backup (full and/or incremental) File system backup, Database and log backup File system backup Database and log backup Database and log backup ----File system backup
SAP Enterprise Portal Server SAP EP Knowledge Management SAP EP Unification Server SAP Gateway HTML Export Service SAP IMS
mySAP EP 5.0
IX / X / VI
Orig.
mySAP EP 5.0
IX / X
Orig.
mySAP EP 5.0 mySAP CRM – Internet Sales Knowledge Warehouse mySAP CRM – Internet Sales Knowledge Warehouse
VII II I III / IV
Orig. ----Repl.
InQMy Application Server SAP IPC 2.0B
mySAP CRM 3.0 mySAP CRM 2.0B
VI III / V
Orig. Repl.
File system backup Database and log backup or No backup*
Best Practice: Backup and Recovery for mySAP Business Suite SAP IPC 3.0 mySAP CRM ≥ 3.0 **Standalone IPC SAP ITS mySAP CRM 2.0– Internet Sales mySAP EBP mySAP Workplace Knowledge Warehouse Employee Self Service SAP J2EE Application Server SAP Knowledge Warehouse mySAP CRM ≥ 3.1 Knowledge Warehouse II VIII / XI --Orig. --Database and log backup; file system backup (full and/or incremental) ------Database and log backup --VI / VII** II --Orig. File system backup Database Backup** ---
14
Knowledge Workbench KPRO Transport server KPRO Web server SAP Mobile Application Repository SAP Mobile Application Studio SAP Mobile Client
Knowledge Warehouse Knowledge Warehouse Knowledge Warehouse mySAP CRM 3.0 – Mobile Sales Development mySAP CRM 3.0 – Mobile Sales Development mySAP CRM – Mobile Sales
II II II VII
------Orig.
II
---
X
Orig. / Repl.
Exception: No backup needed, regular upload to SAP CRM is sufficient Database and log backup. File system backup. Database and log backup; file system backup (full and/or incremental) Database and log backup or No backup *
SAP Mobile Workbench SAP R/3
mySAP CRM 2.0 – Mobile Sales Development Most solutions
Best Practice: Backup and Recovery for mySAP Business Suite SAP SEM mySAP SEM XI Orig. / Repl. Database and log backup; file system backup (full and/or incremental) --File system backup
15
SAP Show TREX (Search Engine)
Knowledge Warehouse mySAP CRM ≥ 3.0 – Internet Sales mySAP EP Knowledge Warehouse
I III / IV
--Repl.
Web server
mySAP CRM – Internet Sales mySAP EBP mySAP Workplace
III / IV / (VI)
Repl. / (Orig.) Orig. / Repl.
File system backup or No backup* Database and log backup; file system backup (full and/or incremental)
SAP Workplace Server
VIII / XI
* For more information on when to skip backup see following sections on the individual components We now explain each individual component in more detail.
Best Practice: Backup and Recovery for mySAP Business Suite Type: • • • • • • Database backup, either full or incremental, depending on the database software Log backup File system backup, either full or incremental Database backup daily Log backup several times a day or frequently, as logs are filled File system backup regularly, frequency depends on importance of interface and log files
16
Interval:
Restore Procedure: • • • Restore of file systems Database restore Database recovery by applying logs
Dependencies: In case of a complete restore without data loss there is no impact on any of the other components. In case of data loss see section 5.
SAP CRM / EBP Server
Classification The SAP CRM system is the leading system for application data, for example, business partner relations, activities, catalog definitions. It also holds replicated data. It exchanges data with other components frequently. SAP CRM is based on SAP WebAS. Dependencies exist to multiple other components: There are dependencies to components, which are also based on a SAP WebAS (SAP R/3, SAP BW, SAP APO). For some objects exchanged between these components SAP CRM is the leading system, for others it only receives data replicates. And there are dependencies to components, which do not include a SAP WebAS (SAP ITS, SAP IMS, Web server). For data objects exchanged with these components, the SAP CRM server usually is the leading system. Software Backup See Software Backup of SAP R/3. Data Backup See Data Backup of SAP R/3. Restore See Restore of SAP R/3.
SAP BW / SAP SEM
Classification The SAP BW system receives data from other SAP systems. This data is in the first step just replicated data. Based on this data SAP BW calculates new data. Such originals in BW could (theoretically) be rebuilt anytime from the data replicated from the other systems. In a mySAP CRM solution, SAP BW may also be the leading system for application data like ‘top N product proposals’ or marketing and campaign data. SAP SEM is the leading system for some application data that cannot be derived from other data. In contrast to SAP BW data, this original data is created online in SAP SEM and cannot be recalculated from other data extracted from other systems.
Best Practice: Backup and Recovery for mySAP Business Suite
17
SAP BW exchanges data with other components frequently, and is based on SAP WebAS. Dependencies exist to multiple components (SAP CRM, SAP APO, SAP R/3), which are all based on a SAP WebAS as well. Software Backup See Software Backup of SAP R/3. Data Backup See Data Backup of SAP R/3. Restore See Restore of SAP R/3.
SAP APO
Classification The SAP Advanced Planner and Optimizer consists of several components itself, the SAP APO system, the SAP liveCache and optionally the SAP APO Optimizer. The SAP APO system is a component, which receives data from other systems like a SAP R/3 system but which is also the leading system for some application data of its own. The SAP APO system is based on SAP WebAS. In a mySAP SCM or mySAP CRM scenario, dependencies exist to multiple other components like SAP R/3 and SAP CRM systems (called APO external consistency) that are based on SAP WebAS as well. Strong dependencies also exist to SAP liveCache (called APO internal consistency). For more information on Backup for SAP APO and its components, please have a look at the separate guides and Best Practices on APO/liveCache. These can be found at http://www.service.sap.com/scm, mySAP SCM Technology. These guides contain detailed information about SAP APO and SAP liveCache backup, recovery, and data consistency. Software Backup See Software Backup of SAP R/3. Data Backup See Data Backup of SAP R/3. Restore See Restore of SAP R/3. Dependencies: In case of a complete restore of the SAP APO database without data loss, there is no impact on external data consistency with other systems. In case of data loss see section 5. On the other hand, APO internal data consistency between SAP APO and SAP liveCache may be endangered even in case of a complete restore of the SAP APO database. Such inconsistencies may arise if active transactions in SAP liveCache are still committed when the APO database is no longer available due to the crash requiring a restore. SAP offers tools for checking SAP APO internal (and external) data consistency (see link given above).
SAP APO liveCache
Classification SAP liveCache receives data from the SAP APO system but also builds up original data that is maintained only in liveCache. The SAP liveCache has data dependencies with the SAP APO system and the SAP APO Optimizer. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes.
Best Practice: Backup and Recovery for mySAP Business Suite Data Backup The backup method available for SAP liveCache data depends on the live cache version in use.
18
With SAP liveCache 7.2, the concept of checkpoints in combination with synchronous application logging is implemented (application logging is not available on older releases of SAP APO). This has several disadvantages concerning the impact of the checkpoint on liveCache operation, the duration of a restore and the security of data as logging is not available for all liveCache objects. For more information on backup procedures for liveCache 7.2 see the respective documentation and the best practice “APO Backup and Availability” at http://www.service.sap.com/scm, mySAP SCM Technology. With release 7.4, SAP liveCache is based on a regular SAPDB database that provides database backups and physical logging for all changes to persistent data. So a backup for SAP liveCache 7.4 is done basically the same way as for other databases. Type (for SAP liveCache 7.4): • • • • • • Database backup, either full or incremental Log backup File system backup, either full or incremental Database backup daily Log backup several times a day or frequently, as logs are filled File system backup regularly, frequency depends on importance of interface files and log files
Interval:
Restore Procedure (for SAP liveCache 7.4): • • • Restore of file systems Restore SAP liveCache data files Recover SAP liveCache database logs
For more information on recovery of liveCache 7.2 see corresponding documentation and the best practice “APO Backup and Availability” at http://www.service.sap.com/scm, mySAP SCM Technology. Dependencies: In case of a complete restore of SAP liveCache 7.4 without data loss, there is no impact on external or internal data consistency of SAP APO (internal consistency is ensured because commits are always issued in SAP liveCache first). In case of data loss see section 5. A restore of SAP liveCache 7.2 cannot avoid data loss if synchronous application logging is not used or if applications are used, which are not included in synchronous logging (for example Demand Planning).
Best Practice: Backup and Recovery for mySAP Business Suite
19
SAP EM (Event Manager)
Classification The SAP Event Manager is installed as an add-on to SAP Web Application Server 6.20. It can be installed as an add-on to SAP R/3 or standalone. SAP EM is based on SAP WebAS. SAP EM exchanges data with various systems of a mySAP Business Suite System landscape. Software Backup See Software Backup of SAP R/3. Data Backup See Data Backup of SAP R/3. Restore See Restore of SAP R/3.
SAP Workplace Server
Classification The SAP Workplace Server collects user roles from the component systems and builds the role based and personalized portal Web page. As it can be used for central user administration, it holds original data of users and profiles. It is based on SAP WebAS. Software Backup See Software Backup of SAP R/3. Data Backup See Data Backup of SAP R/3. Restore See Restore of SAP R/3.
SAP Knowledge Warehouse
Classification The SAP Knowledge Warehouse is an SAP R/3 system with the Knowledge Warehouse Add-On. It is the leading system for application data. It exchanges data with the SAP Content Server and the KPRO web server. It is based on SAP WebAS. Software Backup See Software Backup of SAP R/3. Data Backup See Data Backup of SAP R/3. Restore See Restore of SAP R/3. Dependencies: Dependencies exist to SAP Content Server and KPRO web server.
Best Practice: Backup and Recovery for mySAP Business Suite these applets are not included in SAP’s change and transport system, they must be backed up to avoid a loss of program code. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades or configuration / code changes. Data Backup
20
Regular backup of Java Applets. If the current version of Java applets is always available on a test or development system, this backup may not be required. Type: File system backup of J2EE installation directory, either full or incremental. Interval: On a regular basis, depending on frequency of changes to the code. Restore Procedure: • • Restore host from file system backup. Alternative: New installation. Restore most recent backup of Java applets or transfer Java applets from test/development system.
Dependencies: None
SAP ITS
Classification The SAP Internet Transaction Server (SAP ITS) does not have any application data itself, only configuration data files. These configuration files are kept in file systems. Part of these files, the HTML templates, the flow and service files, are connected to the correction and transport system and thus also kept in the associated SAP CRM server. Maintenance of the files takes place on the SAP CRM server from which they are distributed to the SAP ITS. Software Backup It is recommended to take a full file system backup of the host after installation and configuration of the SAP ITS. After applying changes to the configuration files, which are kept locally on the system, it is also recommended to take a full backup. Such changes may occur during tuning measures. The backup should include all libraries, registry entries etc. HTML templates, flow, and service files are changed on a more regular basis and can be backed up separately from the software. There are three possibilities: The first possibility is not to back up these files at all, since they are automatically backed up together with the database of the associated SAP CRM server. If you do not use the Development Workbench (Transaction SE80) and the Correction and Transport System (CTS) on the associated SAP CRM development server to maintain these files, it is of course necessary to back them up on a regular basis, for example, daily. To speed up the restore, it is of course possible to use a file system backup. Whether this option is faster then re-distributing the files from the SAP CRM server highly depends on the amount of data. If choosing this option, please take care that a backup of these file systems is done after each redistribution started in SE80 on the server. While the backup is running, no new distribution may be started. To ensure this, the ftp publishing service can be stopped. A file system backup is also not necessary, if a second SAP ITS (for example, for high availability reasons) for the same SAP CRM server exists, from which the lost file systems can be copied. Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes.
Best Practice: Backup and Recovery for mySAP Business Suite Data Backup There is no application data that needs to be backed up. Restore Procedure: • • Restore host from file system backup. Alternative: New installation. Re-distributed HTML templates flow and service files from associated SAP R/3 server; alternatively restore last file system backup. If a second SAP ITS for the same scenario exists, files can also be copied from the second SAP ITS.
21
Dependencies: HTML templates, flow- und service files need to have the same version as the files on the SAP CRM server to match the called function modules within the flow.
Web server
Classification Multi-media files for the catalog are kept in local file systems of the web server. The originals of these files are kept in the associated SAP CRM server or, if used, in the knowledge provider. Additionally there may be static HTML pages, cgi scripts, and other files in the web server’s directories, which are not needed for the SAP business scenarios but for the Internet site. If there are no static application data in the local file systems, the need for a backup depends on the time needed to replicate and whether this time is sufficient in case of a restore. Software Backup We recommend taking a full system backup to make sure that all libraries, registry entries etc. are included. Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup From a SAP point of view a backup of the multi media files is not necessary, since these are also kept in the database of either the SAP CRM server or the knowledge provider. Taking a backup of these files on a regular basis is nevertheless an option, if the process of initially replicating a catalog is too time-consuming. In this case, the file systems containing the files need to be backed up on a regular basis, at least after each update replication of the catalog, since the name of these files referenced on the HTML pages may change. If there are files located on the web server, which are not needed for the SAP business scenarios but for the website itself, a backup of these files needs to be included into the backup strategy as well. A file system backup is also not necessary, if a second web server (for example, for high availability reasons) exists, from which the lost file systems can be copied. Type: File system backup of mime file directory, either full or incremental. Interval: On a regular basis, depending on frequency of mime file distribution. Restore Procedure: • • Restore host from file system backup. Alternative: New installation. Depending on the backup strategy chosen for multi-media files, there are the following possibilities: o Distribution of the multi-media files is done together with the initial replication of the catalogs. Using this procedure the files of the SAP Index Management Service (SAP IMS) will also be updated. Restore the most recent backup of the data. In this case it is essential to check that no update replication has taken place since the date of this backup.
Best Practice: Backup and Recovery for mySAP Business Suite o If there is a second web server, for example, used for high availability reasons, there is also the possibility to copy the files from this server.
22
Dependencies: Multi-media objects and files need to have the version of the last update replication, which also distributed the catalog contents to the SAP IMS.
SAP IPC
Classification SAP IPC’s concept for data access and data storage is depending on the IPC release you are using. IPC 2.0B stores application data in a local SQL server database. This data is replicated from the associated SAP R/3 system. Replication time in general is quite long, so that replication is not an option to be used in case of a restore. IPC 2.0C does not store any application data locally. Pricing and configuration information is requested via RFC from the SAP CRM server and kept in main memory. IPC ≥ 3.0 can be installed with and without local database. If it is used like IPC 2.0C without any local data, there may still be user exits used for pricing which are stored locally. These user exits need to be backed up from the file system. For IPC 3.0 you also have the option of installing it for standalone use with SAP Internet Sales R/3 Edition, with other SAP applications or with a third party order software. This option requires a local database to store IPC’s data, which will be replicated for example from an SAP R/3 system. Software Backup We recommend taking a full system backup to make sure that all libraries, registry entries etc. are included. Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup Type: • SAP IPC 2.0B: Database backup and log file backup. If there is for high availability or performance reasons a second SAP IPC with the same application data, one might consider using these two servers to back up each other and not taking a database backup at all. SAP IPC 2.0C: No data backup is needed, because data is dynamically requested from the SAP CRM server. SAP IPC ≥ 3.0: No data backup needed when installed without local database. In this case all data is dynamically requested from the SAP CRM server and stored only in memory. Standalone installation with local database: Database backup and log file backup. If there is for high availability or performance reasons a second SAP IPC with the same application data, one might consider using these two servers to back up each other. Additionally, customer specific user exits for pricing need to be backed up by file system backup. Alternatively user exits could be restored from a test or development system. Daily database backup / Continuous log backup. Regular file system backup of user exits depending on frequency of changes.
• •
Interval: • •
Restore Procedure: Restore host from full system backup. Alternative: New installation. SAP IPC 2.0B only: Restore database from most recent backup and apply all application logs. Alternatively, if there is a second SAP IPC with the same pricing information, you may also use a backup from this database.
Best Practice: Backup and Recovery for mySAP Business Suite
23
SAP IPC ≥ 3.0, standalone installation only: Restore database from most recent backup and apply all database logs. Alternatively, if there is a second SAP IPC with the same pricing information, you may also use a backup from this database. Restore user exits from file system backup or from a test system. Dependencies: SAP IPC 2.0B only: Pricing information needs to have the same version as on the SAP R/3 system. Make sure delta download from the SAP R/3 is stopped while restoring and recovering the database. SAP IPC ≥ 2.0C: Data cached on IPC needs to have the same version as on the leading system. So after a restore of the leading system, the IPC cache may need to be initialized. SAP IPC ≥ 3.0, standalone installation only: Data needs to have the same version as on the leading system. Make sure delta download from the leading system is stopped while restoring and recovering the database.
SAP IMS
Classification The Index Management Service (IMS) consists of two components: the SAP IMS program and the R/3 Standalone Gateway. Both components do not hold any specific data, neither application nor configuration data. The configuration for communication takes place on the associated SAP CRM server. The associated search engine needs to be installed on the same host, for more information see TREX Search Engine. The things said for TREX also apply to former versions of the search engine used with SAP IMS (e.g. Verity). Software Backup It is not necessary to backup the software of the SAP IMS, the gateway or the search engine as long as the installation CDs are available. But it is nevertheless an option to take a full system backup after the installation is done, to reduce the amount of time needed for recovery in case of a system crash. This full system backup must make sure to include all libraries, registry entries etc. Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup The SAP IMS program and the R/3 gateway do not hold any application data. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: None.
TREX Search Engine
Classification The TREX search engine can be used by several mySAP Solutions. On the search engine host, data is kept in local file systems in a search engine depending format (index files). These index files can be rebuilt any time from the original data sources. Depending on the amount of data, which need to be indexed, a backup might be necessary to speed up recovery time in case of a system crash, if rebuilding the indexes would take too long. Software Backup It is not necessary to backup the software of the search engine. But it may be an option to take a full system backup to reduce recovery time. Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes.
Best Practice: Backup and Recovery for mySAP Business Suite Data Backup The search engine data should be backed up using a file system backup to speed up recovery time after a crash. Instead of a data backup a second host with a search engine holding the same data could be used to recover the data from. Type: File system backup. Interval: After each initial or update replication of index files. Restore Procedure: • • Restore host/software from file system backup. Alternative: New installation. Depending on the backup strategy chosen for index files, there are the following possibilities: o o o Rebuild the index files for the search engine from the original data sources. Restore search engine files from most recent backup. This is only possible if there has not been an update of the index files since the last backup was taken.
24
If a second search engine server with the same indexed data (identical index files) is installed, you can also copy the files from this server.
Dependencies: CRM: Search engine index files must match the most recent version of catalog data from the SAP CRM server. Enterprise Portal Knowledge Management: Search Engine index files must match the most recent version of indexed content. Knowledge Warehouse: Search engine index files must match the most recent version of knowledge warehouse contents.
Best Practice: Backup and Recovery for mySAP Business Suite
25
Starting the extract of data on the administration console of the SAP CRM server. Once the extract is finished and the outbound queue on the SAP CRM server is filled, the initial download to the client can be started. To reduce time needed for extract and initial download, the "Laptop Provider Tool” can be used (see separate documentation).
SAP Mobile Application Studio
Classification The Mobile Application Studio is used in mySAP CRM ≥ 3.0 for adapting the mobile sales application. All development is stored in the Mobile Application Repository. The Mobile Application Studio itself does not store any data. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup There is no application data that needs to be backed up. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: None.
SAP Mobile Application Repository
Classification The Mobile Application Repository is used in mySAP CRM ≥ 3.0 to store development objects of the mobile application development. This data is stored in an SQL Server database. Released development objects (change lists) are transported from the development landscape to the test and production landscapes using SAP’s change and transport system. Change lists are attached to transports. They should always be transported separately and not be mixed in the same transport together with other SAP objects. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup Type: Database and log backup of the Mobile Application Repository database, both in the development and production landscape. Interval: Regular database backup, continuous log backup. Restore Procedure: • • Restore host from file system backup. Alternative: New installation. Restore repository database from backup and apply logs. Change lists for the production environment could instead also be re-imported into the database from their corresponding transports. But this procedure requires in-depth knowledge of the transfer mechanisms for change lists.
Best Practice: Backup and Recovery for mySAP Business Suite
26
SAP Mobile Workbench (Development Station)
Classification The mobile workbench of mySAP CRM 2.0 holds application data in its own database. The development workbench is the prototype of the mobile client application. In case of data loss on the development workbench, this needs to be taken into account before distributing software from the development workbench again. Software Backup Type: File system backup, either full or incremental. Registry Backup on Windows platforms Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup Additionally to the data kept in the database, the DLLS developed for the client application are also part of the application data and need to be backed up as well. Type: • • Database backup and log backup. File system backup, which includes the mobile client DLLs regularly.
Interval: Daily database backup. Log backup several times a day or frequently, as logs are filled. Restore Procedure: • • • Restore host from file system backup. Alternative: New installation. Restore database from backup and apply logs. Restore mobile client DLLs from most recent file system backup.
Dependencies : The application software itself is distributed from the development workbench to the mobile clients and thus versions on both sides must match.
SAP BDOC Modeler
Classification The BDOC modeler of mySAP CRM 2.0 merely consists of the software itself, which is installed on the host. All application data is kept in the associated SAP CRM server database. Software Backup In general, no backup of the BDoc modeler software is needed, since it can be installed from the original installation CD again. The BDoc modeler is only used for development, so it is not a business critical component. Data Backup There is no application data that needs to be backed up. Restore Procedure: Re-Install BDoc modeler from installation CDs. Dependencies: None.
Best Practice: Backup and Recovery for mySAP Business Suite
27
security issues or incomplete recovery of the CRM server because they allow determining the clients that logged on to the server in a specific time interval. Software Backup A full system backup after installation and configuration changes is recommended, to make sure, that all registry entries and libraries are included. If you are aware of all configuration entries and changes that have been made on the communication station since installation, re-installing the software is also an option. Type: File system backup, either full or incremental, registry Backup on Windows platforms Interval: After installation and software or configuration changes. Data Backup There is no application data that needs to be backed up. Nevertheless, the log files of the communication station (TransferService.log) should be backed up because they may be valuable for analysis reasons. Type: File system backup of log file directory. Interval: Regularly. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: None.
SAP Business Connector
Classification The SAP Business Connector merely consists of the software itself, which is installed on the host and configuration information. The configuration information includes tuning parameters (for example, maximum session numbers), user and user group information. There is no application data. Software Backup A full system backup after installation and configuration changes is recommended, to make sure, that all registry entries and libraries are included. If you are aware of all configuration entries and changes that have been made on the SAP Business Connector host since installation, re-installing the software is an option also. Type: File system backup, either full or incremental, registry Backup on Windows platforms. Interval: After installation and software or configuration changes. Data Backup There is no application data that needs to be backed up. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: None.
CAT Server
Classification The CAT (CRM Application Tools) Server is a software component that provides support for special functionality in CRM Online. It does not store any application data. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes.
Best Practice: Backup and Recovery for mySAP Business Suite Data Backup There is no application data that needs to be backed up. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: None.
28
BackWeb Server
Classification The BackWeb server is a 3rd party product. It keeps data in its own database, either a Microsoft SQL server or an oracle database. The information, which is provided to the clients, is fetched from the Internet, so it is reproducible any time. Of high importance is the backup of the configuration data, which decides which data is being fetched from the Internet and which user needs to get which documents. Software Backup Backup software and configuration data. Data Backup Regular database and log file backup Restore • • Restore software and configuration data. Restore data from database backup.
Requisite Catalog Server
Classification The Requisite catalog server holds replicated data only, either from an SAP system or a supplier. This information can be replicated in case of a system crash. If replication time is too slow because of high data volumes, it is recommended to backup the database. Software Backup Type: File system backup, either full or incremental, registry Backup on Windows platforms Interval: Regularly. Data Backup Type: Database backup and log backup. Interval: Daily. Restore Procedure: • • Restore host from file system backup. Recover database either by restoring the backup and applying logs or by replicating data from SAP system and/or supplier.
Dependencies: Catalog database needs to have the same version of catalog information as the SAP CRM server.
Best Practice: Backup and Recovery for mySAP Business Suite
29
SAP Enterprise Portal Server
Classification The Enterprise Portal Server consists of two parts, the middleware application and the persistence layer. The middleware application consisting of the Portal WebServer and the iView Server uses the persistence layer to store its data. The persistence layer uses three different data locations to store persistent data: • • • the Portal Content Directory (PCD), a File system used for role definitions and content to role or user assignment, the Portal Repository, a MSSQL or Oracle database for content metadata and user personalization the Portal LDAP, an Enterprise Portal specific addition to an LDAP directory used for user to role assignments and user mapping
Data dependencies which must be regarded during backup and restore exist between all components of the persistence layer (see below). In addition to these parts of the persistence layer, the iView Server file system is used for storing iView code and iView personalization. iView code should also be backed up, especially if modified by the customer. Software Backup Type: • for all components: File system backup, either full or incremental, registry backup on Windows platforms.
Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup Type: • • • • PCD: File system backup, either full or incremental. Portal Repository: Database & log backup. Portal LDAP: Using LDAP specific backup tools or file system backup. iView Server FS: File system backup, either full or incremental.
Interval: Regularly, depending on frequency of changes. Restore Procedure: • • • • Restore host from file system backup. Alternative: New installation. Restore PCD and iView Server file system from most recent file system backup. Restore Portal (and possibly Corporate) LDAP from LDAP or file system backup. Restore Repository database from backup. If all components of the Portal server are to be restored consistently from an online backup taken as described in the document below, no log recovery may be done (which means that some data may intentionally get lost for the benefit of a consistent state between components). If only the repository database needs to be restored (for example due to a database crash which did not affect all other components), all database logs are to be recovered to avoid data loss.
Dependencies: In case of a restore of all or one component of the persistence layer, data consistency between PCD, EP Repository and Portal LDAP must be regarded. See documentation ‘Backup & Restore SAP Enterprise Portal’ at “http://service.sap.com/ep ->product information-> enterprise portal 5.0-> how-to–guides” for more information, especially for a risk analysis of possible data inconsistencies with online file system backups of PCD and LDAP.
Best Practice: Backup and Recovery for mySAP Business Suite
30
SAP EP Unification Server
Classification The Enterprise Portal Unification Server is used for access to external applications and handling of drag & drop functionality in Enterprise Portal. It consists of the Unification WebServer, the Unification Server, the Unification Repository and Unifiers for different applications to be accessed. Only the Unification Repository , a MSSQL database, is used to store the information needed for executing drag& drop relations and for accessing external applications. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup Type: Database backup and log backup. Interval: Regularly, depending on frequency of changes. Restore Procedure: • • Restore host from file system backup. Alternative: New installation. Restore database from database and log backup.
Dependencies: None.
SAP EP Knowledge Management
Classification The Knowledge Management platform of Enterprise Portal consists of the Components Content Management and TREX. The component TREX is described separately in this document. Content Management uses the KM Service (software only) and two locations for data storage: • • The KM Database on MSSQL or Oracle for storing content metadata and access control lists The KM Repository, a File system for storing configuration and KM iViews.
The content which is managed by the Knowledge Management platform is stored in various external locations like file systems, web servers, databases etc. These repositories containing the actual content are not further regarded here. If this content is managed by your company, a backup should of course be considered too. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes. Data Backup Type: • • KM Database: Database & log backup. KM Repository: File system backup, either full or incremental.
Interval: Regularly, depending on frequency of changes. Restore Procedure: • • • Restore host from file system backup. Alternative: New installation. Restore KM repository from file system backup Restore KM database from database and log backup.
Best Practice: Backup and Recovery for mySAP Business Suite
31
SAP Content Server
Classification The content server used by SAP Knowledge Warehouse holds application data in its own database. Software Backup Type: File system backup, either full or incremental, registry Backup on Windows platforms Interval: Regularly. Data Backup Type: Database backup and log backup. Interval: Daily. Restore Procedure: • • Restore host from file system backup. Recover database either by restoring the backup and applying logs or by replicating data from SAP system and/or supplier.
Dependencies: SAP R/3 with Knowledge Warehouse Add-On: The information in the content server must match the administrative information in the Knowledge Warehouse. In case of data loss on the content server, this needs to be corrected.
SAP Content Server Cache
Classification The content server used by SAP Knowledge Warehouse cache holds replicated data from the Content Server only. Software Backup Type: File system backup, either full or incremental, registry Backup on Windows platforms Interval: Regularly. Data Backup Type: Not necessary. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: Content Server: Reload the Cache Server from the Content Server.
SAP Gateway
Classification SAP Gateway is a software component without application data. Software Backup Type: File system backup, either full or incremental, registry backup on Windows platforms. Interval: On a regular basis, at least after installation and software upgrades/configuration changes.
Best Practice: Backup and Recovery for mySAP Business Suite Data Backup There is no application data that needs to be backed up. Restore Procedure: Restore host from file system backup. Alternative: New installation. Dependencies: None.
32
Drag & Relate Servlet
Classification The Drag & Relate Servlet holds information on data objects in the component system that can take part in Drag&Relate actions. This data is extracted from the Business Object Directory (BOR) of the corresponding component system. Software Backup Type: File system backup, either full or incremental, registry Backup on Windows platforms Interval: Regularly. Data Backup Type: Not necessary, extraction speed is usually sufficient. Restore Procedure: • • Restore host from file system backup. Alternative: New installation. Extract data from Business Object Directory.
Dependencies: Data must be consistent with BOR of corresponding component system.
Best Practice: Backup and Recovery for mySAP Business Suite
33
4. Performing Consistent Landscape Backups / Landscape Copies
A consistent backup of two or more systems, which exchange data with each other, is a backup that ensures application data consistency between the components. If such a Consistent Landscape Backup is restored on all components, the application data is logically consistent in the whole system landscape. A Consistent Landscape Backup can be used to create copies of the landscape, for example for test environments. It is also useful to create a Consistent Landscape Backup as a fallback point before doing large system changes such as upgrades or data migrations. As mentioned in section 1.1 it is usually not necessary to create a Consistent Landscape Backup during standard operation. A Consistent Landscape Backup cannot be achieved by performing a point in time recovery of all involved components. This is due to the fact that there might be differences in the local machine time, which make it impossible to recover the components to the exact same point in time without having inconsistencies, because some transactions have been committed on one system but not on the other. In section 4.1 only components that hold original data are included in a Consistent Landscape Backup. Annotations on the dependent middleware components, which hold only replicated data, are given in section 4.2.
4.1 Creating a Consistent Landscape Backup
A Consistent Landscape Backup can be achieved on one of the following three levels: the application level, illustrated by the R/3 instance in figure 4.1, the database level or the storage level. The storage level is drawn in a separate box, as there usually is a separate storage management software in place. This section explains the different possibilities for each level. Another way to achieve a Consistent Landscape Backup is to install all components of the landscape in one single database (Multiple Components in One Database). This option is also shown below.
SAP CRM
SAP R/3
SAP Instance CRM Online R/3 Adapter RFC
SAP Instance Plug-In
Database Instance
Database Instance
Storage Mgmt Data
Storage Mgmt Data
Figure 4.1: Possible levels for creating a Consistent Landscape Backup Note: All options on application level require downtime of at least some systems of the landscape while creating the Consistent Landscape Backup. In contrast to that do all options on database or storage level focus on ways of creating Consistent Landscape Backups online, without downtime of the involved systems.
Best Practice: Backup and Recovery for mySAP Business Suite
34
Application level
On application level, we can achieve data application consistency by stopping the communication between all involved components. The only way to stop communication between the components is to stop the application. This does not necessarily include the database, which can be kept online. There are several variations, which have different effects on downtime: • • • Stop of all applications during backup Stop of all but one application during backup Stop of all but one application during log backup
Stop of all applications during backup The easiest method from a handling point of view is to stop all involved components and take an offline backup. To achieve consistency from an application point of view it is not necessary to stop the underlying databases as well. By keeping the databases up a performance gain is achieved, because the database buffers are kept in place.
SAP CRM
SAP R/3
SAP Instance CRM Online R/3 Adapter
SAP Instance Plug-In
Database Instance
Database Instance
Storage Mgmt Data
Storage Mgmt Data
Figure 4.2: Stop of all applications Advantages • • • • • Easy to implement and handle Database buffers are preserved Downtime of several hours of involved systems needed (no 7x24 operation possible) Performance loss due to buffer resets Stop of batch processing
Best Practice: Backup and Recovery for mySAP Business Suite
35
Stop of all but one application during backup To achieve application data consistency, it is also possible to stop all but one (the most important) application. There is no incoming communication from the stopped applications, outgoing communication will be buffered in the database tables of the running application and be processed later on. This solution might be considered if the system that will stay online mostly does asynchronous communication. If there is a lot of synchronous communication that relies on the availability of the other systems this alternative is not applicable because there will be lots of communication errors. A good example for using this alternative is the mobile sales/mobile service or the contact center scenario, as these scenarios might make it possible to identify times during which no one is using the SAP CRM server.
SAP CRM
SAP R/3
SAP Instance CRM Online R/3 Adapter
SAP Instance Plug-In
Database Instance
Database Instance
Storage Mgmt Data
Storage Mgmt Data
Figure 4.3: Stop of all but one application Advantages • • • • • • • • All databases can stay online; buffer contents of databases are preserved Most important application is running (allowing 7x24 operation) Easy to implement and handle Not suitable for scenarios requiring 7x24 operation, for example, internet sales Administrator activity needed to check RFC status Downtime of several hours Performance loss due to buffer reset Stop of batch processing
Disadvantages On running system: On stopped systems:
Best Practice: Backup and Recovery for mySAP Business Suite
36
Stop of all but one application during log backup To reduce downtime of the involved systems, the two procedures from above can be modified as follows (also see figure 4.4): 1 2 3 Start an online backup on all involved systems. The backups should be started in a way, that the end of the backups are close to each other. Once all systems have finished their online backups, stop either all or all but one application. Take a backup of the logs, which have been filled since the online backup.
The Consistent Landscape Backup then consists of the tapes of the online backup plus the log backup. The application data is consistent after the applications are stopped (step 2). At this point, all communication between the systems is stopped.
Backup running
System 2 finished
System 3 finished Shutdown n-1 systems and start log backup
System 1 finished System 1 Online Backup System 2 Online Backup System 3 Online Backup
Delta Data Log Backup
Delta Data
Log Backup
Delta Data
Log Backup
Systems synchronized Consistent backup
Figure 4.4: Stop of all but one application during log backup Advantages • • • • • • • • • All databases can stay online; buffer contents of databases are preserved Most important application is running (allowing 7x24 operation) Reduced downtime on all other applications Less queue contents to be processed after startup Not suitable for scenarios requiring 7x24 operation, for example, internet sales Administrator activity needed to check RFC status Performance loss due to buffer reset Stop of batch processing Procedure is more difficult to handle and thus more liable to errors
Disadvantages On running system: On stopped systems:
Best Practice: Backup and Recovery for mySAP Business Suite
37
Database level (Coordinated Suspend)
Using the "suspend write” capability of the database management systems, it is possible to create a point of consistency without stopping the application (Split Mirror Backup). Scenarios using this feature are already used for the backup of single systems. By issuing a suspend write command to the database, all modifications and, depending on the database system used, all read accesses are blocked. This causes the application to “freeze” for a short time, but it is not aborted. From the moment on that a database is in the suspend write mode, all outgoing and incoming communication is stopped, because communication always includes write access to the database. If there is communication, which only needs read access to the remote database, then this does not cause any inconsistencies either. Since no data is modified on the remote database, there can’t be any inconsistency from an application point of view. The following table lists the commands which implement the "suspend write" feature on the different databases. DB system DB2 UDB S/390 DB2 UDB open INFORMIX ORACLE suspend WRITE SET LOG SUSPEND set write suspend for database onmode –c block ALTER DATABASE ... ... BEGIN BACKUP ALTER SYSTEM SUSPEND SQL Server via Virtual Device Interface
(dbcc freeze_io (<db>) may not be used for forward recovery)
resume WRITE SET LOG RESUME set write resume for database onmode –c unblock ALTER SYSTEM RESUME ALTER DATABASE... ... END BACKUP via Virtual Device Interface
(dbcc thaw_io (<db>) may not be used for forward recovery)
Best Practice: Backup and Recovery for mySAP Business Suite By recovering all systems to the memorized point in time a consistent copy can be created.
38
Note: To implement this, it must be possible to get the current log record from each database and all participating database systems must support a point-in-time recovery to that specific log point.
Start re-sync
Start suspend
Suspend System 2
Resume Systems
Re-sync finished
Suspend System 1
Split
Fast disk copy
System 1 System 2
All systems suspended Start suspend Suspend System 2
Start backup
Backup finished
Suspend System 1
Resume Systems
Pointin-Time
System 1 System 2
All systems suspended
Figure 4.5: Coordinated Suspend on database level Advantages • • Low performance impact (no buffer resets) Very short time needed for system freeze
Disadvantages • • • Implementation needs specific knowledge Process gets more complicated the more systems are involved Process gets more difficult the more different databases are included
Note: The above procedure has been tested and verified by SAP in several different projects in cooperation with its partners. It has not yet been implemented in a productive customer environment and needs to be tested before use. The procedure might vary depending on hardware vendor and database software in use. SAP is currently preparing Best Practices on how to implement this procedure. Please check http://service.sap.com/atg for more information.
Note: Due to the nature of the commit protocol used for transactions between SAP APO and SAP liveCache, this procedure can currently not guarantee data consistency between SAP APO and SAP liveCache.
Best Practice: Backup and Recovery for mySAP Business Suite
39
Storage level
On storage level, some storage systems offer Consistent Split or Consistency Group technology, which allows to create a consistent (atomic) image of all storage volumes belonging to the system landscape. This is, depending on the storage hardware used, possible either if the application data of all involved systems (e.g. SAP R/3 and SAP CRM) is stored on the same storage subsystem or even if the application data is distributed over several storage subsystems. Consistent Split on one storage subsystem Steps: 1. Re-synchronize mirror 2. Split mirror (consistent split of all involved volumes!) 3. Resume write activity The mirrors will now contain a consistent landscape copy.
SAP CRM
SAP R/3
SAP Instance CRM Online R/3 Adapter RFC
SAP Instance Plug-In
Database Instance
Database Instance
Storage Mgmt Data SAN/NAS
Figure 4.6: Consistent Split on storage level Advantages: • • • • Low impact on performance Can be done several times a day (depending on synchronization time) File systems can be included in backup Read access to data still possible
Note: This procedure has not yet been implemented in a productive environment and needs to be tested before use. The procedure may vary depending on storage hardware in use and may not be available on all storage systems. Please ask your storage partner for possible solutions.
Note: This procedure does not permit to do database forward recovery on the crash images of the databases. So this procedure may be used for consistent system copies but not as a backup of an individual database system.
Best Practice: Backup and Recovery for mySAP Business Suite
40
Note: Due to the nature of the commit protocol used for transactions between SAP APO and SAP liveCache, this procedure can currently not guarantee data consistency between SAP APO and SAP liveCache.
Best Practice: Backup and Recovery for mySAP Business Suite
41
Consistency Groups To overcome the limitation of having one storage system for all involved systems, the concept of Consistency Groups can be used. A Consistency Group spans multiple storage systems and allows grouping arbitrary volumes into a consistency group. A split command issued to a consistency group will stop write access to all disk partitions belonging to this group so the mirror volumes will contain a consistent image of all involved volumes and of all applications using these volumes. Steps: 1. Re-synchronize mirror 2. Split mirror (consistent split of all involved volumes!) 3. Resume write activity
SAP CRM
SAP R/3
SAP Instance CRM Online R/3 Adapter RFC
SAP Instance Plug-In
Database Instance
Database Instance
Storage Mgmt Data
Storage Mgmt Data
Consistency Group
Figure 4.7: Consistent split using consistency groups
Note: The above procedure has been tested and verified by SAP in different projects in cooperation with its partners. The procedure might vary depending on storage hardware in use and may not be available on all storage platforms. Please ask your storage partner for possible solutions. SAP is currently preparing Best Practices on how to implement this procedure. Please check http://service.sap.com/atg for more information.
Note: This procedure does not permit doing database forward recovery on the crash images of the databases. So this procedure may be used for consistent system copies but not as a backup of an individual database system.
Note: Due to the nature of the commit protocol used for transactions between SAP APO and SAP liveCache, this procedure can currently not guarantee data consistency between SAP APO and SAP liveCache.
Best Practice: Backup and Recovery for mySAP Business Suite
42
Multiple Components in One Database
Another option for creating a Consistent Landscape Backup is to install all involved databases in a single database management system. When using this installation type, every backup of the database is a Consistent Landscape Backup. This is schematically shown in figure 4.8. For productive systems, this option will probably mostly not be practicable because it imposes some restrictions on scalability, flexibility and upgrade possibilities. The following things must be taken into account when considering this option: • • • • • • • • All components are affected by a crash or a restore A component’s performance correlates to that of all other components Resource consumption may be quite challenging On most database platforms, it is not possible to do a point-in-time recovery for only one of the installed components Database administration cannot be done independently On most database platforms, upgrades cannot be done with logging switched off OLTP (e.g. SAP R/3, SAP CRM) cannot be installed together with OLAP (e.g. SAP BW) applications because of different parameterization Middleware components are still not included in the Consistent Landscape Backup
A multiple components in one database installation is not a solution that is generally applicable for all installations. Its use is probably limited mostly to test, demo and development environments. A productive use may be considered for systems with very strong consistency requirements in an environment that provides sufficient performance capacity (e.g. in combination with parallel database systems). Please note: This feature is currently only available under controlled availability. For further information on single database installations, see also http://www.service.sap.com/onedb.
Best Practice: Backup and Recovery for mySAP Business Suite
43
4.2 Dependent components
In the previous sections, we only looked at possibilities for creating a consistent image of several databases. For backing up or copying a complete system landscape, other software components need to be regarded as well. Using a Consistent Landscape Backup to set up a test environment If you’re planning to use a Consistent Landscape Backup to set up a test environment, it is recommended to install and configure all other software components once on the designated hosts and configure them for connection to the associated SAP systems (like SAP CRM, SAP R/3, SAP BW and SAP APO) of the test landscape. On some components some adjustments are needed. The necessary procedures are described in the Best Practices for System Landscape Copy, which are available from the SAP service marketplace. To initially copy or refresh application data (replicated data) of these components of the test landscape, you can simply distribute or re-distribute the data from the corresponding SAP systems (if replication time is not an issue). If replication takes too long, you can also use a backup taken from the productive instance of these components. Nevertheless do we recommend to only restore application data of such components, not configuration data. The Backups of these components must be consistent to the backup of the SAP systems included in the Consistent Landscape Backup. As data of such components is usually updated only at regular intervals, it should be possible to easily find a matching backup that includes all changes, which had been propagated so far. Using a Consistent Landscape Backup for a point-in-time recovery of the whole landscape If a restore of the complete production landscape is needed (for example, to reset the state of the landscape to the state before an upgrade started), full system backups of the middleware components are needed as well. These backups must be consistent to the backup of the SAP systems included in the Consistent Landscape Backup. Before an upgrade for example, all components of the production landscape should be backed up by offline backups done during a common downtime.
Best Practice: Backup and Recovery for mySAP Business Suite
44
5. Managing an Incomplete Recovery
This section explains how to manage an incomplete recovery. It shows the consequences that arise if one component of a system landscape cannot be recovered to the crash point in time and evaluates possible alternatives in dealing with such a situation. In addition to merely describing backup and restore procedures, a B&R concept must also describe strategies on how to deal with data loss or data inconsistencies after an incomplete recovery. Because this requires a lot of application specific knowledge, people from the business departments and system administrators must work together closely on managing incomplete recoveries. Note: Before doing an incomplete recovery of a productive system please open a call at SAP and check if the situation can be solved using SAP note 434645. Please also see SAP note 434647 for important information when doing a point-in-time recovery.
5.1 Indications and Consequences
The logging mechanisms of database systems enable you, after a crash, to recover system data to the point in time at which the crash occurred. After the recovery, the data reflects the exact state at the time of the crash. The RFC techniques used to exchange data in a mySAP Business Suite system landscape together with these logging mechanisms ensure that all data is in a consistent state after one of the component systems had to be restored. But under rare circumstances a recovery to the crash point in time might not be possible. If a system cannot be recovered to the crash point in time we speak of an incomplete recovery. An incomplete recovery may be caused by a database restore without applying the logs, by a point-intime recovery to an earlier point in time or by a point-in-log recovery to a log, which is not the current log. Reasons for an incomplete recovery of a database system may be corrupt log files, destroyed tapes or a severe logical corruption of the database. In the following we will use the terms ‘incomplete recovery’ and ‘point-in-time recovery’ interchangeably. An incomplete recovery of one component of a system landscape always causes data to be lost and in most cases also causes data inconsistencies between the systems. Both will usually have a serious impact on business operation. A point-in-time recovery in a multi-system landscape should therefore be avoided as long as any other solution seems possible.
Wrong transports were imported: Other handling errors caused data loss or inconsistencies: Replicated data is lost or inconsistent:
Best Practice: Backup and Recovery for mySAP Business Suite For more information on how to avoid a point-in-time recovery in special situations see SAP note 434645.
45
As for database-managed data an incomplete recovery might also occur for application-managed data. If the data is backed up regularly and the latest backup turns out to be unusable, an older backup might have to be restored and the data might thus be inconsistent with other systems. But it is important to note that if a component only holds replicated data, an incomplete recovery should never be done because the data can always be re-replicated from the system holding the originals.
5.3 Alternatives for Performing an Incomplete Recovery
Incomplete recovery of a system component always means data loss for at least this component. In general there are three possibilities of how to deal with this situation. (1) Only the affected system is restored doing a point-in-time recovery (2) A consistent backup of the complete system landscape is restored (3) The other systems are also restored to the point-in-time that the affected system had to be restored to These alternatives mainly differ with respect to the factors ‘data loss’, ‘inconsistencies’ and ‘downtime’, all of which should be minimized. These factors must be carefully weighted when deciding on one of these alternatives for a B&R concept. Figure 5.1 gives a schematic representation of the amount of data loss and the number of inconsistencies with each of these alternatives for two system components. Note: To diminish the impact of data loss and to provide options to at least partly recover lost data we generally recommend always preserving a copy of the original system when doing a restore. This requires preventive actions, which allow copying any system to some spare hardware (for example, using storage technology). This copy may then be used to track and recover lost data after the restored system goes online again.
/ Last Recovery of for - Alternatives Incomplete Consistent Backup OLTPCRM
Last Synchronization Point
Point-in-time
CRM
I. Point-in-time restore for one system
I
SAP CRM CRM OLTP
Data lost Inconsistencies
ess Busin er y Recov
II. Restore of a consistent landscape backup
II
SAP CRM OLTP
Data lost Data lost
III. Point-in-time restore for all systems
III
SAP CRM OLTP
Data lost Inconsistencies Data lost
ess Busin y cover Re
Figure 5.1: Data loss caused by an incomplete recovery of the SAP CRM system
Best Practice: Backup and Recovery for mySAP Business Suite Alternative (1): Point-in-time recovery for one System
46
Doing a point-in-time recovery to a point in time that is as near as possible to the crash and only for the affected system while keeping all other systems as they are, restricts data loss to only this system and also keeps it to the absolute minimum. But the amount of data inconsistencies is the biggest, which might have a severe impact on business operation. On the other hand, this option offers the chance to reconstruct lost data from the data, which is still available in the other untouched systems. As the errors caused by inconsistencies might get worse the longer they stay uncorrected, the correction based on the data available in the other systems should be done as fast as possible. Identification and correction of inconsistencies (which is also called Business Recovery) can only be done with a very good knowledge of the business processes and data objects. For some system combinations, SAP offers tools that help to identify and fix cross-system inconsistencies. This alternative only causes downtime for one system. All other systems can continue running, unless this is prevented by the amount and kind of inconsistencies. Keeping the other systems running while doing the restore does not increase the number of inconsistencies because all new data that needs to be exchanged will be queued until the system comes up again. Alternative (2): Restore of a Consistent Landscape Backup Data inconsistencies can be completely avoided by restoring a consistent backup of the system landscape. But this also implies data loss in all systems, not only in the originally affected system. And since with the current techniques available for doing Consistent Landscape Backups (as described in section 4) a consistent backup of the whole system landscape is only available for a few specific times of the day, the data loss will usually be much bigger than actually required (with the exception of the ‘multiple components in one database’ solution that can provide a consistent backup for any time of the day). Additionally, this option will cause downtime on all systems. This alternative often becomes more and more problematic, the more time has elapsed since the error occurred (because data loss will become the bigger the longer it takes until the error is recognized). But although data of the recovered system components will be consistent there might still be other components that cannot be restored this way. So there might still be cross-system inconsistencies even after the restore of a Consistent Landscape Backup. This holds for example for the mobile client computers in mySAP CRM Mobile Sales. In case of a point-in-time recovery of all other system components, the mobile clients will still possess the newest data, which will then be inconsistent to the rest. This shows that it is very important to identify all such dependencies when planning a B&R concept. Alternative (3): Point-in-time recovery for all Systems The third alternative could be to first do the required point-in-time recovery for the affected system and then do point-in-time recoveries for all other systems of the landscape to the same point in time. This would keep data loss smaller than with the second alternative and would not cause as many inconsistencies as the first alternative. But because time is not synchronized between the systems, there will probably be some inconsistencies between the systems, which have to be dealt with during Business Recovery. Since there will be data loss on all systems, there is no chance to salvage data lost on the affected system (unless a copy of the original systems is preserved). This option will cause downtime on all systems. Assessing which Alternative to Use A B&R concept must definitely include concepts for dealing with incomplete recovery of different system components. Which of the above alternatives is applicable for a specific customer and a specific system component depends mainly on the degree to which each of the factors' influence could be tolerated in case of an incomplete recovery. The following questions can help with this decision: Is data loss acceptable? To which degree? Can data be recovered from data in other systems? Can operation continue with partly inconsistent data? What tools are available for Business Recovery? Table 5.1 summarizes the most important aspects distinguishing the three alternatives.
Best Practice: Backup and Recovery for mySAP Business Suite Alternative: Factor: Data loss Inconsistencies Downtime (1) Point-in-time recovery for one system Minimum, only 1 system affected Most inconsistencies Only 1 system down, downtime depends mainly on amount of logs (2) Restore a consistent Backup Maximum, all systems affected No inconsistencies All systems down, downtime possibly quite short (depending on method used) (3) Point-in-time recovery for all systems Medium, all systems affected Little inconsistencies All systems down, downtime depends mainly on amount of logs
47
Table 5.1: Tradeoffs of the three alternatives for an incomplete recovery As we have seen in section 1.1, a restore of a consistent backup in case of an incomplete recovery of one system component will often not be appropriate. Instead of losing a business document completely, it may usually be preferable to keep at least part of this information although it is no more consistent across all systems. Especially in case of an incomplete recovery of a ‘depending’ system holding mainly replicated data (e.g. SAP CRM up to release 2.0C) the leading system (e.g. SAP R/3) should not be set back. So if an incomplete recovery of a system component cannot be avoided and if the upcoming inconsistencies can be tolerated on short term and be fixed based on the data available in the other systems, we generally favor alternative (1) because • • • • • Data loss is kept to the absolute minimum Downtime is kept to a minimum Downtime on other systems is avoided Business can go on (although some processes may of course be affected) Restore of other systems can cause additional problems and errors (with the other alternatives)
The main steps of a restore using alternative (1) are: 1. Keep all other systems running and restore only the affected system 2. If needed, initialize components holding only replicated data to reflect the current state 3. Analyze and fix inconsistencies on a project basis (Business Recovery) 4. Reconstruct lost data from unaffected systems or from a system copy taken before the restore
Best Practice: Backup and Recovery for mySAP Business Suite only a few minutes more are lost, the damage might be less than the problems caused by the inconsistencies after a point-in-time recovery of one system.
48
Alternative (2) may also be an option for a ‘one database’ installation that can do a consistent restore to any arbitrary point in time. But still the data loss will be bigger than with alternative (1) because data is lost in all system components instead of just one. A consistent backup could also be used for restore in a landscape where the leading system has to do an incomplete recovery. Instead of accepting inconsistencies it might be preferable to also restore the depending systems if they do not hold other important original pieces of data. An example may be an SAP R/3 – SAP BW environment. Although original SAP BW data will be lost, most of it could probably be rebuilt after the restore.
5.4 Examples for Specific System Combinations
Using some examples we want to show in this section, • • • • what can be the impact of an incomplete recovery on other system components, how this situation can be analyzed with respect to the appropriate steps to take what aspects should be documented in a B&R concept and what kind of tools for Business Recovery are available by SAP
Incomplete Recovery of the SAP CRM Server
Impact on SAP R/3 System If in case of a restore the SAP CRM system cannot be recovered to the most current time, there will be data loss in the SAP CRM system. Apart from the fact that original data in SAP CRM will be lost (for example, orders that have not been transferred to the leading SAP R/3 system) there will also be inconsistencies to the SAP R/3 system because some pieces of data that were replicated between both systems will no longer be available in the SAP CRM system. The following alternatives can be considered in case of an incomplete recovery of the SAP CRM system: 1. Restore the SAP CRM system to the latest possible point in time while keeping the SAP R/3 system running. After the restore, the inconsistencies should be fixed on a project basis. There are tools available to assist with this, see below for more details. 2. Restore a consistent backup of both systems and thus also reset the SAP R/3 system to a former state Since the most important data is still available in the leading SAP R/3 system, we would prefer alternative (1.) and would keep the SAP R/3 system running. Alternative (2.) would impose downtime also onto the SAP R/3 system and would cause much more data loss than alternative (1.). Figure 5.2 depicts both alternatives for further comparison.
Best Practice: Backup and Recovery for mySAP Business Suite
49
Last Synchronization Point / Last Consistent Backup
Point-in-time for CRM
CRM
I
SAP CRM OLTP
Inconsistent OLTP data Lost data (e.g. CRM online)
Rebuilt data
Lost data (not Lost data yet replicated)
II
SAP CRM OLTP
Lost data
Figure 5.2: Alternatives after incomplete recovery of the SAP CRM system Here are some more details concerning alternative (1): • • The SAP CRM system will be restored as far as possible, while the SAP R/3 system is kept running There will be data of the SAP CRM system that is definitely lost. This can be o Data that only exists in SAP CRM and that is not replicated to other systems (like activities, attributes to the product master, business partner relations, catalogs and their associated products) or Data that was entered in the SAP CRM system and not yet replicated to the SAP R/3 system (like orders from Internet Sales). If it is possible to track such lost data, it could later on be re-entered into the SAP CRM system manually.
o
•
Some data that is lost in SAP CRM will still be available in SAP R/3. Such inconsistencies between the SAP CRM and SAP R/3 system must be analyzed and fixed. mySAP CRM 2.0 With CRM 2.0 there is a generic compare tool available (see SAP Note 363602). The compare tool can be used to compare objects from SAP R/3 with the consolidated database of the SAP CRM system. It does not show objects that only exist in the SAP CRM system. The inconsistencies might then be fixed by downloading missing objects from SAP R/3 to SAP CRM using the standard download mechanism. Please note that the document numbers in SAP CRM after the download may be different from the original numbers in SAP CRM because the download order may be different and the numbers will be newly assigned. Note: If the compare tool for inconsistencies between SAP R/3 and SAP CRM is executed while one of the systems is running (or if the queues are not empty), new or altered data might show up as a difference just because replication has not yet been finished. To eliminate objects that were included in the list mistakenly because they were just being processed, the compare tool could be run twice and both lists could be compared to identify ‘true’ inconsistent objects.
Best Practice: Backup and Recovery for mySAP Business Suite
50
Starting with mySAP CRM 3.0 SP 11, the Data Integrity Manager (DIMa) is available for comparing objects in SAP R/3, CRM Online and CRM Consolidated Database. See SAP note 531217 for a list of currently supported objects. In addition to these tools, SAP may have other tools available for Business Recovery, so please contact SAP when doing a Business Recovery. In case of an incomplete recovery of SAP CRM, one further alternative might come into ones mind: "Do a complete initial download from SAP CRM, without doing a SAP CRM restore”. This would mean that all SAP CRM data that has not been replicated to SAP R/3 is lost (since start of production!). Since this may affect a large amount of data as well as all customizing of the SAP CRM system we don’t consider this to be a practicable way. Indeed, this alternative would result in a completely new installation of the SAP CRM server because an initial download does not delete obsolete or inconsistent objects from the SAP CRM database. Impact on Other System Components Data loss in the SAP CRM system will probably not only cause inconsistencies with the SAP R/3 system but also with other depending systems of the web middleware, which hold replicated data from the SAP CRM system. After an incomplete recovery of the SAP CRM system, all other system components must also be regarded to re-establish data consistency. Of course no actions need to be taken if they are still in a consistent state (which means that there were no changes affecting any of these components made during the period that was lost). The following example applies to mySAP CRM 2.0: • SAP ITS: Since some flow and service files managed by the SAP CRM server may be lost, the corresponding transports must be imported from the development system. All objects that are referenced by the flow must also be recreated on the SAP CRM system. If there is no development system, flow and service files could also be uploaded from the SAP ITS to the SAP CRM server. SAP IMS/Search Engine: The version of the catalog files must match the version maintained by the SAP CRM server. This might no more be true after an incomplete recovery of the SAP CRM server. In that case, the catalog files must newly be downloaded from SAP CRM. Lost catalog data must then be recreated in SAP CRM, an upload of the catalog from SAP IMS into SAP CRM is not possible. Web server: The multimedia files on the Web server are tightly coupled to the SAP IMS files and must also match the version on the SAP CRM server. They must therefore be downloaded if this data was affected by an incomplete recovery. Lost objects must be recreated in SAP CRM, for example, by loading the objects from their original data sources or from a backup on the Web server taken before the download. SAP IPC (with mySAP CRM ≥ 2.0C): The cache of the SAP IPC will need to be reloaded if the cached data reflects a version different from the restored version in SAP CRM. After restarting the IPC the cached information will again match the data on the incompletely recovered SAP CRM system. Mobile clients: Chances that the lost period after an incomplete recovery of SAP CRM does not affect many mobile clients are quite high as data between these components is exchanged only during limited periods of time. The less the lost period on the CRM server, the fewer mobile clients will be affected. There is no mechanism to synchronize the mobile client in case of data loss on the SAP CRM server. Data coming from the mobile clients, which was lost in the SAP CRM system, could possibly be uploaded to the SAP CRM server by repeating the upload process manually (fill the outbound queue of the client by doing a dummy update on all objects that shall be resent). Data that was replicated to the mobile clients but which is no more available on the SAP CRM server cannot automatically be deleted on the mobile clients (the SAP CRM system cannot send ‘delete’-requests because it does not know of the obsolete data on the clients). It is usually not possible to initialize all mobile clients doing an initial download because of its long duration and the offsite location of the mobile clients. To determine the clients that logged on to the CRM server during the lost period, it might be helpful to back up the log files of the communication station as well (TransferService.log). Information on site id, date and time is kept in this log file.
Best Practice: Backup and Recovery for mySAP Business Suite • SAP BW: Evaluations might be based on data that is no more available in SAP CRM. Depending on the business needs, correcting actions might need to be taken
51
Incomplete recovery of SAP R/3 system
The SAP R/3 is the leading system for most business data. Data loss due to an incomplete recovery means, that all replicates residing in other systems loose their referencing data object. This will probably lead to errors during some business processes relying on these references. The following alternatives can be considered in case of an incomplete recovery of the SAP R/3 system: 1. Restore the SAP R/3 system to the latest possible point in time while keeping the other systems running. After the restore, inconsistencies with other systems must be fixed during Business Recovery. Business Recovery with SAP CRM 2.0: There is no compare tool available to find out what is available in SAP CRM and what is missing in the SAP R/3 system. The compare tool described in the last section can only identify lost changes (Changes to data that already existed before the point of the restore and which was changed afterwards can be identified because the data in SAP CRM will be newer after the incomplete recovery). Lost data of the SAP R/3 system can possibly be reconstructed from the data’s replicates in other systems. Business Recovery with SAP CRM 3.0: Starting with mySAP CRM 3.0 SP 11, the Data Integrity Manager (DIMa) is available for comparing objects in SAP R/3, CRM Online and CRM Consolidated Database. See SAP note 531217 for a list of currently supported objects. Business Recovery with SAP APO: The SAP APO and SAP liveCache may contain data that is no more available in SAP R/3. SAP offers tools to check and repair external consistency between SAP R/3 and SAP APO. See SAP note 425825, the documentation at “http://service.sap.com/scm , mySAP SCM Technology, Consistency Checks” and the Best Practice “Data Consistency Between R/3 and APO” for more details. In addition to the above tools for Business Recovery, SAP may have other tools available, so please contact SAP when doing a Business Recovery. 2. Restore a consistent backup of all systems and thus also reset all other systems to a former state. This will cause data loss and downtime for all systems. (Attention: apart from the replicates, the other systems may also hold original data that would be lost!) 3. Reset other systems to the same or to a slightly earlier point in time than SAP R/3 (point-intime recovery for all other systems). This will cause data loss and downtime for all systems and still leave certain inconsistencies between the systems, which should be fixed on a project basis. For mySAP CRM, these inconsistencies could be identified by the compare tool mentioned before in section ‘Point-in-time recovery of the SAP CRM Server’ for alternative 1 (if the SAP CRM system was restored to a state shortly before the SAP R/3 system’s state). The following figure depicts all three alternatives for further comparison.
Best Practice: Backup and Recovery for mySAP Business Suite
52
Last Synchronization Point / Last Consistent Backup
Point-in-time for OLTP
OLTP
I
SAP CRM CRM OLTP SAP CRM OLTP SAP CRM OLTP
Inconsistent CRM data
II
III
Figure 5.3: Alternatives after incomplete recovery of the SAP R/3 system
Impact on Other System Components Data loss in SAP R/3 also has an impact on other system components of a mySAP Business Suite system landscape. In a mySAP CRM 2.0 solution for example, the following components may be affected: o SAP IPC (up to SAP CRM 2.0B): Pricing information is replicated from SAP R/3 to the SAP IPC database. In case of an incomplete recovery of the SAP R/3 system, the pricing information on the SAP IPC might differ from the information in SAP R/3 (also for the SAP CRM server). The SAP IPC should then be newly loaded to reflect the current situation in SAP R/3. SAP BW: Evaluations might be based on data that is no more available in SAP R/3. Depending on the business needs, correcting actions must be taken.
•
Incomplete recovery of SAP APO
Data loss in the SAP APO system destroys the internal and external consistency of SAP APO. Internal consistency means data consistency between the SAP APO database and the SAP liveCache while external consistency means data consistency between SAP APO and the dedicated SAP R/3 systems. As the SAP APO database mainly holds replicated data from other systems, a point-in-time recovery of only the SAP APO database followed by Business Recovery will usually be the preferred alternative if an incomplete recovery cannot be avoided. See SAP note 425825, the documentation at “http://service.sap.com/scm , mySAP SCM Technology, Consistency Checks” and the Best Practice “Data Consistency Between R/3 and APO” which provide information on available compare tools to check and repair the internal and external consistency of SAP APO data.
Best Practice: Backup and Recovery for mySAP Business Suite
53
Incomplete recovery of Other Components
Incomplete recovery of a component that holds only replicated data in a file system or that has only configuration data but no application data can be caused by the following situations: • • The File system can only be restored to a point where some subsequent changes of the data or of the installed components are lost. The server running these components must be installed completely new.
If, in the first situation, it is possible to identify and reload or redo the lost changes, only these need to be reapplied to reach a consistent state. Otherwise or in the second situation, a consistent state can always be achieved by a complete download of the data (regardless of the duration of the download). An incomplete recovery of a component that has original data but that does not exchange this data with any other component is not critical for data consistency. All components which hold replicated and original data and which exchange data with other components of a system landscape (category IX, X and XI), are potentially critical in case of an incomplete recovery of this component or any other component. For each such component it is necessary to carefully analyze possible impacts and to carefully deliberate the actions that are to be taken in case of an incomplete recovery (similar to the considerations done in the previous sections).
Best Practice: Backup and Recovery for mySAP Business Suite
54
Verification
After you execute this Best Practice, check whether you produced the desired results as follows: 1. Check your B&R documentation for completeness 2. Test backup and restore procedures on test systems
Further Information
Example of a B&R Concept for a mySAP CRM System Landscape
This is an example for a possible structure of a B&R concept, showing a mySAP CRM 2.0B landscape. Please note that all information given in this example, especially dates and frequencies, are fictional and need to be determined in relation to your specific landscape. Scenario Internet Sales and Mobile Sales. Since the Internet Sales scenario is used, no time for an offline backup is available. System Landscape:
Communication Station
MTS DCOM Connector
Windows NT 4.0 Server
APO Communication Station
MTS DCOM Connector
Windows NT 4.0 Server
Best Practice: Backup and Recovery for mySAP Business Suite
55
B&R Concept 1. General information • Components used in the business scenarios Components used: SAP CRM, SAP R/3, SAP BW, SAP APO, SAP ITS, SAP IPC, SAP IMS (SAP Gateway, Search Engine, SAP IMS), Web server, SAP Communication Station • Short description of the business processes Internet Sales: Sales order creation including availability check Mobile Sales: Sales order creation, activity maintenance ... • Leading system (original system) for each business object Sales orders: Leading system SAP R/3 Activities: Leading system SAP CRM ... • Systems in which replications of each business object are stored Sales orders: Replicated to CRM, APO, BW … • Data flow description for the main business objects Orders created in SAP CRM are transferred to SAP R/3. Activities created on mobile clients are transferred to SAP CRM. ... • Availability requirements for the system landscape and for each component, depending on the applicable service level agreements (SLAs) and their restrictions on backup, restore and maintenance times
Best Practice: Backup and Recovery for mySAP Business Suite • Handling and impact of exceptional situations, like incomplete recovery of a component Impact on the system components (data loss, data inconsistencies) Steps to do before, during and after the Incomplete recovery of a component Procedures and tools for Business Recovery or for Restore of a Consistent Landscape Backup ... • Documentation on usage of additional tools/reports for consistency checks Description of functionality and usage for customer-developed reports; evaluation and description of usage of standard reports from SAP for comparison. 4. Additional security measures • Hardware redundancy SAP CRM, SAP R/3, SAP BW, SAP APO: Hard disks protected by RAID 1
56
To achieve high availability, there are two communication stations, which are addressed by an IP router. This applies also to the web server. Load balancing on the AGate is done by the internal load balancing algorithm of the SAP ITS. • • • Database consistency checks Database specific check tools scheduled once a week; evaluation of log files. Tape checks Readability tests scheduled once a week plus additional test restores every 3 months. Recovery training schedule and test procedures Practice of restore procedures every 3 months (SAP-basis and non-SAP-basis components). 5. Special backup procedures (Consistent Landscape Backup; system copies) • When needed? Before data migration and software upgrades of leading systems. For test landscapes no 100% consistent copy is needed (the copy to the test landscape thus can be done by doing point-in-time recoveries for each component). • • • How often? On demand. Backup procedure Offline backup. Restore procedure Restore offline backup.
Best Practice: Backup and Recovery for mySAP Business Suite
57
Example Backup Schedule From these prerequisites, the following example backup schedule is derived. The full system backups of the middleware components once a month are scheduled to prevent the loss of the installation backup due to tape expiration and reuse dates. For these components no data backup is scheduled, because all components exists twice due to high availability reasons and serve as backup for each other. If both systems fail, it is always possible to restore software and configuration files from a full system backup and download data from the leading system. A file system backup can be an incremental backup as well. Please also keep tape expiration dates in mind when scheduling backups. Component SAP R/3 Supported Operating System Different OS Backup Procedure Standard database backup Log backup File system backup SAP CRM Different OS Standard database backup Log backup File system backup SAP BW Different OS Standard database backup Log backup File system backup SAP APO Different OS Standard database backup Log backup File system backup SAP liveCache 3.0 Checkpoint activated; online backup of checkpoint data Synchronous logging enabled, archive log area in use Web Middleware I: SAP ITS, SAP IMS, SAP IPC Web Middleware II: SAP ITS, SAP IMS, SAP IPC SAP Communication Station I SAP Communication Station II Web server I Web server II Windows NT Full system backup Once a month Windows NT Full system backup Once a month Windows NT Full system backup Once a month Interval Daily Continuous Daily Daily Continuous Daily Daily Continuous Daily Daily Continuous Daily Every six hours
Best Practice: Backup and Recovery for mySAP Business Suite
58
Restore and Recovery In the following table you’ll find example restore procedures in case of failure of the different components, based on the backup schedule above. Component SAP R/3 /SAP CRM/SAP BW/SAP APO Type of Failure Single table corrupted Restore Procedure The following two options can be considered: • If table contains redundant data only, rebuild it (contact SAP to find out if this is possible) • Restore on test system and copy appropriate data to production Database corrupted File system corrupted Hardware failure SAP liveCache 3.0 Server crash Restore and recover database from database backup; apply all logs Restore file system from backup Restore file systems from backup; restore and recover database from backup; apply all logs Recover SAP liveCache from checkpoint and logs Restore host from full system backup; copy relevant files from secondary server (index files, flow and service files, HTML templates); restore SAP IPC database from second SAP IPC database Restore from second server Restore from second server Restore host from full system backup Restore files from full system backup or Re-install software Web server I/II File system which contains catalog data corrupted Hardware failure; all data lost Copy data from second web server
Web Middleware Hardware failure; all I/II data lost
File system with data corrupted SAP IPC database corrupted SAP Communication Station I/II Hardware failure; all data lost SAP Communication Station Software deleted/corrupted
Restore host from full system backup; copy application data from second web server
Best Practice: Backup and Recovery for mySAP Business Suite
59
Background Information and References
Data Consistency During Normal Operation Distributed processes and data exchange between systems must ensure that data in all participating systems is in a consistent state at all times. This includes that all distributed processes must be tolerant against system failures. In a mySAP Business Suite System Landscape data is exchanged using the Remote Function Call (RFC) technique. Mainly two types of RFCs are used for data exchange: Synchronous RFC and transactional RFC. With synchronous RFC (sRFC), the caller of a remote function always waits for the answer or result from the target system. Thus data consistency is always guaranteed because the remote operation is either executed completely or the calling process receives an error code and must take appropriate actions. This is true both for application errors in the target system or if the target system is not available at all. Transactional RFC (tRFC) and queued RFC (qRFC) are both executed asynchronously while the calling process continues without waiting. qRFC is a variant of tRFC that maintains the order in which the functions are executed in the target system and thus can be used for serialization of tRFCs of different SAP LUWs (Logical Unit of Work). The tRFC protocol ensures for both tRFC and qRFC that the function is executed exactly once in the target system. This is also guaranteed if the connection cannot be established the connection is terminated during the execution or during data transfer the target system crashes during the RFC execution the sending system crashes somewhere in the middle of processing the tRFC protocol
Best Practice: Backup and Recovery for mySAP Business Suite instances, short-term inconsistencies may come up. If such inconsistencies cannot be tolerated, appropriate measures must be taken to ensure simultaneous updates.
60
Categorization of System Components Based on the type of application data a component holds we introduce a categorization scheme for system components that can be used to analyze the backup requirements of any system component and to easily determine an appropriate backup method for this component. The categorization can be done using the flowchart in the following figure. While many components used by mySAP Business Suite Solutions are already described in chapter 3 of this document, this categorization can be used for new or customer-specific components not included here. The following table then lists all categories together with their most important properties and required backup methods. That table also gives examples for components belonging to the different categories. For categories III to XI, a backup of the software and configuration data as listed for Category I or II is also needed.
Best Practice: Backup and Recovery for mySAP Business Suite Categorization of system components:
Categorization of a System Component
61
? ?
Only software, no application or configuration data ? Category I Only software and configuration data, no application data ? Category II Component has application data
? ?
Only replicated data ? Replication speed sufficient in case of a restore? Category III B&R strategy required
?
Not database managed ? Category IV Category V
Component has originial data, B&R strategy required
? ?
Standalone system, no data exchange ? Not database managed ? Category VI
?
No SAP Basis ? Category VII Category VIII Systems exchange data, data consistency after restore must be secured
Best Practice: Backup and Recovery for mySAP Business Suite Categories of system components Category Properties Suggested Backup and Recovery Methods II Only software and configuration information, no application data No backup, new installation in case of a restore or Initial software backup after installation and upgrade Backup of log files Backup after changes have been applied or No backup, New installation and configuration in case of a restore Backup of log files No data backup needed
62
Example (Taken From mySAP CRM) BDOC-modeler
I
Only software, no configuration or application data
SAP Gateway Comm. Station SAP Business Connector SAP IPC (2.0C)
III Only replicated application data, replication time is sufficiently small for a restore -
Data: Backup of software, configuration, log files
SAP IMS / Search Engine * SAP IPC (2.0B) * Webserver * SAP ITS SAP IMS / Search Engine * Webserver *
IV
Only replicated application data, backup recommended because replication time is too long data not managed by a DBMS
Data: Application specific file system backup or Multiple instances
Backup of software, configuration, log files V Only replicated application data, backup recommended because replication time is too long data managed by a DBMS Data: Database and log backup or Multiple instances SAP IPC (2.0B) * Catalog Server
Backup of software, configuration, log files VI Original application data, standalone system, data not managed by a DBMS Data: Application specific file system backup Webserver *
Backup of software, configuration, log files VII Original application data, standalone system, data managed by a DBMS, not based on SAP WebAS Original application data, standalone system, based on SAP WebAS Data: Database and log backup Backup of software, configuration, log files Data: Database and log backup, application log backup (e.g. job logs in file system) Standalone SAP R/3
Best Practice: Backup and Recovery for mySAP Business Suite
63
Category
Properties
Suggested Backup and Recovery Methods Data: Application specific file system backup, data consistency with other systems must be regarded
Example (Taken From mySAP CRM)
IX
Original application data, data exchange with other systems, data not managed by a DBMS
Backup of software, configuration, log files X Original application data, data exchange with other systems, data managed by a DBMS, not based on SAP WebAS Data: Database and log backup, data consistency with other systems must be regarded SAP liveCache SAP Mobile Workbench
Backup of software, configuration, log files XI Original application data, data exchange with other systems, based on SAP WebAS Data: Database and log backup, application log backup (e.g. job logs in file system), data consistency with other systems must be regarded SAP R/3 SAP CRM SAP APO SAP BW
Backup of software, configuration, log files * Component may fall into different categories because of different usage
Best Practice: Backup and Recovery for mySAP Business Suite
64
Feedback and Questions
Send any feedback by formulating an SAP customer message to component SV-GST. You can do this at http://service.sap.com/message.