SAP CBTA Advanced - How to make use of Conditions, Loops

PREREQUISITES
The looping capability are planned to be shipped with SAP Solution Manager 7.1 SP13
Alternatively you can implement following notes in advance:

USE-CASE FOR LOOP FUNCTIONALITY:
A test script may need to perform actions against an unknown number of entries in a table. The script may therefore need to:
•    Start at first row and check if there is an entry
•    If entry exists perform one or more actions on the current row
•    Continue with next row

REQUIRED DEFAULT COMPONENTS: DO, EXIT_DO, LOOP

Keyword: DO
It can be used to iterate over several steps. It defines where the loop starts.
•    It must be used together with the LOOP keyword which defines where the loop ends.
•    The EXIT_DO keyword must be used as well to determine when to stop the loop.

The CounterName parameter provides the name of the iteration counter. This counter is incremented automatically at runtime while iterating over the included steps. The actual value of the counter can be retrieve using the regular token syntax.
For instance, when CounterName is set to "COUNTER" its value can be reused in the subsequent steps using %COUNTER% (or $COUNTER$ for specific situations where the percent character is ambiguous).

If you plan to use nested loops please make sure to declare a different counter names.

Component Parameters

CounterName: Specifies the the name of the iteration counter.

Keyword: EXIT DO
It must be used within a loop that has been defined using the DO and the LOOP keywords. The EXIT_DO keyword interrupts the loop as soon as the condition is met.
A typical use case is to check the value of iteration counter that has been declared via the CounterName parameter of the DO keyword.
For instance, when CounterName is set to "COUNTER" its value can be checked using the %COUNTER% token.

Component Parameters
LeftOperand
•    Specifies the value of the left operand that is to be checked.
Operator
•    Specifies the boolean operator to use.
The operators supported are the ones below:
o    = for "Equal to"
o    < for "Less than"
o    > for "Greater than"
o    <= for "Less than or equal to"
o    >= for "Greater than or equal to"
o    <> for "Not equal to"
o    {contains} for "Contains"
o    {startsWith} for "Starts with"
o    {endsWith} for "Ends with"
An additional operator is supported when testing WEB applications (i.e.: applications running in the browser):
o    {matches} for checking whether the value matches a regular expression. The regular expressions are expressed using the .NET syntax.
RightOperand
•    Specifies the value of the right operand that is to be compared with the left operand.

Options
The options parameter lets you perform some adaptations or conversions of both the left and right operand before comparing them.
The supported options are:
•    /u (for uppercase) - Both values are converted to upper-case before being compared
•    /t (for trimmed) - Both values are trimmed before being compared
•    /i (integer) - Both values are converted to an integer before being compared
•    /f (float) - Both values are converted to a float (or double) before being compared
•    /b (bool) - Both values are converted to a Boolean before being compared

Keyword: LOOP
It defines the end of the loop and must be used together with the DO keyword which defines where the loop starts.

 
EXAMPLE – PROCESS LINE ITEMS IN SALES ORDER
The following scripts was created for transaction VA02 (Change Sales Order) to add shipping information for each line item of an existing sales order.

SAP CBTA(Component Based Testing)

With SAP Solution Manager 7.1 SP07, Test Management has got a new meaning ! Test Automation Framework using Component Based Test Automation provide automation in Testing under Test Option 1.

It provides E2E Testing Approach within SAP Solution Manager. This document explains the benefits of using CBTA.

Easy Installation:

• Check the Compatibility Matrix to install the correct CBTA version depending on your SolMan Version
• Install ST-TST 300 Component considering the Compatibility matrix
• CBTA front-end to be installed on the computer which is required to carry out the Recording
• Technical Users created in the SolMan and Managed system
• Enable Scripting in SolMan and SUT(Managed System)

Components of CBTA:

System Data Container: Contains information regarding the System Under Test (SUT)

Test Script: Recording of the Transaction

Test Data Container: Information about Test Data is mentioned in the TDC

Now you are ready to explore the easy to record Test automation tool !

Following diagram explains the Activity flow in CBTA :

 

From the Business Process Structure, a Test Configuration is created in the Test Composition Environment (TCE).

Using CBTA, the Test Script is recorded which is stored back in SolMan.

The Test Data is assigned from the Test Data Container.

Final TEst Configuration is stored at the Business Process for the particular Transaction

Advantages of CBTA:

• The biggest advantage of CBTA is easy edit of Test Scripts. The Test scripts are divided into Screen Components and Default Components.
• Multiple SUTs can be tested as the information present in System Data Container
• Test Scripts can be re-used with different Test Data Container
• Check point can be added while the Test recording to include a validation step
• Test Report shows the results with screen shots

The CBTA Test Configuration is linked with the Transaction code in the Business Process Structure maintained in the Project. Thus there is a connection established between the Business Process and Test Scope. Once the Test Scripts are ready in CBTA, Test Plan can be created with the traditional approach of Test Management and we can include the CBTA Test Configuration. Test Execution takes place from SolMan to Managed system and finally the Test results can be evaluated in the Test Evaluation Phase.

CBTA provides the environment to take advantage of Test Automation for SAP GUI and CRM Web UI transactions. This combination of Solution Manager and CBTA is the ultimate solution where SAP Solution Manager can be used for Test Planning, Test Evaluation and Reporting and CBTA is used to create Test Scripts to automate Testing. CBTA provides a value add to Test Management Methodology of SAP Solution Manager.

As CBTA is well integrated with SAP Solution Manager, we can use the CBTA Test scripts to Create TBOMs (Technical Bill of Material) for the Transactions. These TBOMs can be used to carry out the Change impact Analysis for the Business Process.

I would like to say that CBTA has made the life of Test Engineers easy with less effort to setup and having no extra cost incurred to carry out Test Automation.

 

HP UFT/QTP ReplayType importance

For few of the objects, though the objects reference available and able to recognize in applications and perform script execution without any issues, it doesn't perform expected action in the application.

Example: There is button in the application
Object is available in Object Repository and we are able to highlight it
During script execution, it executes without any issue but on clicking button it is able to move to next screen(in fact it is not clicking on button).

In order to handle such issues, UFT has an option use ReplayType which will change the execution with mouse/keyboard control(this means similar to manually taking control with hand)

Below line of the script can be used for this.

Setting.WebPackage("ReplayType") = 2

If you want to change it again then below line of script can be used.

Setting.WebPackage("ReplayType") = 1

Effective Way of SAP Automation Testing

As SAP ERP is huge in nature, every software development group tests its products, as the delivered product contains defects. Test Engineers strive to catch them before release of the product with the best manual testing processes but they end up in vain. Automated Testing is the best way to test the product with increase in efficiency. Automated tests are created and can be easily repeated to perform the tasks impossible with manual testing. Automated testing is considered critical for big software organizations or too expensive for small companies to implement. Automated software testing improves the accuracy, test coverage within short interval of time at low cost. Test automation has specific advantages for improving the long-term efficiency of a software team’s testing processes at low cost and time.

Many organizations use SAP software in their various departments to take care of everyday business processes. The variety of configurations and extensions possible increases the need for testing such business processes and mission-critical applications before releasing new SAP functionality to potentially thousands of users.

Why we have to carry out Automation Testing on SAP applications?
In this dynamic and competitive industry, immediate implementation of crucial business can help us to reduce cost and increase market share.
It’s true that test automation tools are designed to dramatically cut costs, manual labor, and shorten testing duration.

Challenges faces by Organizations in Typical SAP Releases
  
•    Limited time and budget for development and QA cycle
•    Huge number of test cases for execution to ensure robust testing for critical business processes
•    Large number of changes in system with each release of an application
•    Distributed and diverse legacy applications
•    Risk associated with interfaces involved
•    Manual testing requires large number of testing team, results in higher number of errors, provides less test coverage, offers inconsistent results and is time-consuming

An organization dealing with the above challenges can benefit greatly from automated regression testing if it is implemented properly using the right strategy, processes, guidelines and approach.

How Automation can address these Challenges

•    Reduced cost of failure by early detection of bugs and issues during testing
•    Reduced cost, by completing the QA cycle quicker
•    Automated testing is consistent and repeatable
•    Automation routines can be executed for data setups like creation of master data & variants
•    Reporting and critical defect detection inputs for impact analysis
•    Automated tests can extract variable data from one application and pass it to a test to run in another module, ensuring different application modules and interfaces work well together
•    Once created and benchmarked, the automation suite needs minimum manpower to execute tests during each cycle, eliminating the need for experts in all SAP modules
•    Automation is the way to cost-effectively and consistently achieve adequate test coverage. It provides proof of compliance to law with strict quality regulations and audit trail standards
•    Automation ensures reliability of mission-critical business processes like financial month-end closings, bank reconciliation, production planning, sales order entry, etc
•    As new versions of R/3 are implemented and “hot-packs” applied, automated tests can be re-executed for regression testing for critical business processes to be performed without error
•    Test suite can be used for regression testing while legacy system software, networks, server hardware, and communication components are upgraded to prevent a negative impact
•    Automation ensures uniformity in the testing process each time a test is executed
•    Scripts can be executed at night or on weekend’s unattended – huge value in time crunch
•    Test scripts can run independently with minimal human intervention in multiple sessions
•    Given the challenges, it is clear that automated regression testing is not a luxury but a necessity. Having identified these challenges and decided that automation is the way to go, it becomes imperative to develop the correct roadmap. The first step in this path is choosing the right test tool and automation framework.
Know how to automate SAP tests
Testing SAP releases and upgrades is a challenging, time consuming and complex activity. Many organizations would like to automate their SAP testing but often don’t know the best way to approach this.
Usually this results in the effort required to maintain automated Test Scripts being as high as that required to maintain SAP itself which is why over 70% of test automation initiatives fail to meet their anticipated Return on Investment. There are a number of reasons for this: Lack of experienced automation testers, custom automation frameworks that do not create the necessary efficiencies, incorrect choice of test tool, and management overhead are a few but there are many more. Our solution overcomes these challenges in an innovative and unique way.
Approach:
•    Is proven to be a faster and more efficient method to automate tests
•    is based on a world leading platform
•    increases the coverage and speed of regression testing your existing functionality
•    reduces the likelihood of mistakes being released into your production system
•    eliminates the need to waste time and money creating a custom automation framework
•    provides consistent, low level tests step documentation for the automated tests
•    Integrates with ALM or Test Management tools

Choosing the right framework:
Key success of automation is choosing the right framework for the tool to get maximum benefits with less time.
Component/Function/Method Driven: built up with various reusable functions and called in test scripts on need basis
Pros: Quick & easy to implement, popular, supported by HP UFT, SAP TAO, SAP CBTA, Worksoft Certify, and other tools.
Cons: Not as robust as Keyword Driven framework and coding expert is required
Keyword Driven: requires pre development of keywords and selecting keywords to make test script ready
Pros: Robust framework, can be deployed independent of test tool and application under test
Cons: Detailed steps for each test condition need to be documented, more initial time required

Hybrid Driven: most frameworks evolve into the hybrid framework – a combination of the above two frameworks which brings together their strengths and mitigates their weaknesses

Component/Function/Method driven framework is best for SAP applications and it is quick and easy to implement with maximum ROI with short span of time
Frameworks:
Design robust automation framework which uses industry available best framework features and made Hybrid automation framework ready.

Identifying the best Test management tool:
Test management tool should be open to integrate with internal and external automation tools to execute entire project management, it may be adding Cycles, releases, requirements, manual and automation test scripts, results and reports, defects and dashboards..etc.
Ideally HP ALM (formally known as HP Quality Center) best for test management tool

Identifying right business scenarios for regression automation testing
Analyzing the system along with SMEs and stack holders and finding out key business scenarios where it is required to execute such scenarios repetitively on each release is an important activity before kicking off automation testing.
•    Is automation feasible for the selected scenarios/ test cases?
•    Test cases are required to execute repetitively on each release/cycle multiple times
•    No manual intervention during script execution
•    Able to run script independently or in batch
•    Always ROI focused
•    Identify areas of ERP that have been customized as customizations can have an adverse effect on functionality and performance that must be tested with every build
•    Concentrate testing on interfaces with communication between SAP & legacy system
•    Scenarios/transactions which need to be tested with different data sets/values
•    High path frequency: Scenarios/process paths used with high degree of frequency when the software is running in full production. Examples include creating customer records, invoicing and other high volume activities where software failures can occur frequently.

Performing Feasibility analysis (FA) and Proof of Concept (PoC):
Carrying out FA and PoC is really required for any automation testing project. It will definitely provide immediate feedback on our automation testing capabilities and success ratio of our automation regression pack which we supposed to automate
E2E Test Management with Solution Manager:
With SAP Solution manager we can successfully carry out End to End test management as show in below picture. This includes various automation tools called CBTA, HP UFT/QTP, Worksoft Certify, SAP TAO..etc
If we are committed to use CBTA, SAP TAO then it is recommended to use Solution manager, as it doesn’t add additional cost to us

Similar kind of test management can be performed in HP ALM too, so based on project nature we can pick up test management tool.
Easy to identify changes in SAP system for automation:
Transporting mechanism happens in SAP system in vary controlled way, each and every change is linked to executable object and added to transport, so that at the end of transport, we come to know changes/additions happens in new SAP application
To control the transport and objects we have SAP BPCA (Business Process Change Analyzer) in it. So it is recommended to perform transports with BPCA to track the changes in system. BPCA has an option to find out which test scripts got impacted because of certain transport, and this will give us direct solution where to fix the script

Benefits with automation approach:
•    Good amount of returns on our investments
•    Saving substantial amounts of staff time previously spent on running regression tests
•    Improving the quality of services by catching data-related defects before production
•    Speeding up testing and application deployment cycles to create greater business agility
•    Fostering collaboration between IT and business experts to streamline technology projects
•    Double/Tripling the percentage of business processes automated
•    Operational efficiency
•    Reduced risk of application failure
•    Improved customer experience
•    Foster time to market

Key Considerations for Developing Automation Scripts Architecture for SAP
•    To automate all critical business end-to-end scenarios, each scenario is split into test conditions, and each test condition broken down into unique SAP T-Codes that must be executed
•    Each unique SAP transaction is scripted as a reusable action under the Common Reusable library.
•    Each reusable function/action is called in different scenarios. All the required parameters to execute the reusable function/action are passed through the driver script
•    Error handling, common functions also built into reusable common library; included in each script
•    Using test management tool, test sets are created for each business scenario; test conditions are added to test set to complete testing & validation of one end-to-end business process
•    All scenarios required to be tested in each regression cycle are set for execution using individual test sets driven by test management tool
•    The structure of having of one test script per test condition provides great modularity and ease of maintenance and debugging. If a test condition fails, it is easy to detect which test condition and corresponding test script in the complete business scenario caused the issue and broke the execution
•    Separate the test data from the scripts. Maintain all the required Master & Transactional data in separate files. These files input the required data to the scripts during execution
•    Estimation & planning of effort required to completely build the regression test suite becomes easy. Calculate the effort required to build script for each unique T-Code, common reusable library functions, error handling functions, developing of calling scripts and integration of individual test conditions into scenarios
•    Proper coding & commenting guidelines should be documented; all team members must follow the same
•    Peer review process to ensure all scripts are well reviewed
•    Ensure all the required test data (Master and Transactional data) are current and valid.
•    Perform dry run to integrate and run scripts to test a complete business transaction
•    Proper documentation for each test script
Preparing Test Suites with batch of automation test scripts
It is always good practice to create batches of test scripts as below, so that you will quickly get to know the feedback of the system quality
•    Sanity pack
•    Integration Pack
•    Regression Pack
•    CR Specific Pack
Generating Test Results:
Generating user friendly test results plays an important role here, as test results files can be used as evidences for future purpose.
Summary Results as part of A2 framework with QTP                 Detailed Results Generated by TAO and CBTA
 
Script Health Checkup:
We have to consider Automation testing as typical application development process; otherwise we may end up with failure at the end. As part of this, making sure scripts are working fine on regular intervals is essential’
•    Scripts evaluated and reviewed against checklist of following parameters periodically to ensure regression suite is up to date and ready during regression testing of the SAP release:
•    Apart from maintenance, scripts are executed periodically in batches overnight and test results analyzed. In case of failures, scripts are updated to correct errors.
•    Test data for each test script valid and current
•    Test environments (SAP, other systems, databases & interfaces) up and running
•    All required changes suggested incorporated and script up to date
•    Test environment. setup, connectivity, tool license, configuration and tools available
•    Test set and test scripts pulled in Test Director before execution starts
•    Test cases are up to date and scripts reflect current steps as per test case
•    Required results & defects reporting templates in place
•    Access to SAP system & transactions and other systems/databases available
•    ‘All clear’ in test results report and no issues identified with scripts during dry run