Posts Tagged ‘multi-tool automation’

It was more than just validation. It was ensuring nation’s pride.

Monday, August 9th, 2010
A large petroleum major was rolling out a specialized solution to ensure that fleet
tracking solution to ensure zero pilferage during transport. The solution consisted of a
plethora of technologies (GPS, GSM, Web, Mapping) and our role was to ensure that the
final solution is indeed risk-free for deployment.
With the launch date in the next few weeks, we got cracking on applying HBT to extract
the cleanliness criteria from the business and technical specifications outlined in the
tender document.The cleanliness criteria consisted of multiple aspects- deployment
environment correctness, cleanliness of software, clean working of hardware/software
interfaces and finally the ability to support a large load, volume with real-time
performance.
We identified the potential types of defects that spanned the entire spectrum of
hardware and software. The first step was to understand system development process
and identify senior consultants visited the vendor’s facility to assess the people and
processes used to develop the system. This provided a clear picture of what to expect
and the work that lay ahead of us.
Post our understanding of the development system, we developed a scientific strategy
and the evaluation scenarios. A variety of tests were identified – individual feature
validation, simulating various business use-cases, understanding load limitations and
performance evaluation of the system.
Now we were ready to validate the final system in the data center. The first cut of the
solution was used to develop a set of automated scripts for large scale load/stress/
performance testing. The system was populated with large data representing a real life
system. Vehicles were fitted with the vehicle mounted unit. We were ready to roll now.
The various vehicles were set in motion in various terrains, at various speeds and
mapping of the fleet on the India map was validated. We simulated large number of
vehicles with data arriving from the simulators at a high rate to ensure that
performance was indeed real-time.
In addition the deployment environment was validated, configurations checked, legality
of the software verified. We also verified that the solution integrates with customer’s
SAP database as well.
Bugs popped up and were fixed. We recommended changes in the system capacity,
pushed the vendor to close all the critical, high and medium priority defects before
providing a qualitative feedback on the solution and the potential risks. Once satisfied
our customer’s investment was safe, we gave a go-ahead to rollout the solution.

A large petroleum major was rolling out a specialized solution to ensure that fleet tracking solution to ensure zero pilferage during transport. The solution consisted of a plethora of technologies (GPS, GSM, Web, Mapping) and our role was to ensure that the final solution is indeed risk-free for deployment.

With the launch date in the next few weeks, we got cracking on applying HBT to extract the cleanliness criteria from the business and technical specifications outlined in the tender document.The cleanliness criteria consisted of multiple aspects- deployment environment correctness, cleanliness of software, clean working of hardware/software interfaces and finally the ability to support a large load, volume with real-time performance.

We identified the potential types of defects that spanned the entire spectrum of hardware and software. The first step was to understand system development process and identify senior consultants visited the vendor’s facility to assess the people and processes used to develop the system. This provided a clear picture of what to expect and the work that lay ahead of us.

Post our understanding of the development system, we developed a scientific strategy and the evaluation scenarios. A variety of tests were identified – individual feature validation, simulating various business use-cases, understanding load limitations and performance evaluation of the system.

Now we were ready to validate the final system in the data center. The first cut of the solution was used to develop a set of automated scripts for large scale load/stress/performance testing. The system was populated with large data representing a real life system. Vehicles were fitted with the vehicle mounted unit. We were ready to roll now.

The various vehicles were set in motion in various terrains, at various speeds and mapping of the fleet on the India map was validated. We simulated large number of vehicles with data arriving from the simulators at a high rate to ensure that performance was indeed real-time.

In addition, the deployment environment was validated, configurations checked, legality of the software verified. We also verified that the solution integrates with customer’s SAP database as well.

Bugs popped up and were fixed. We recommended changes in the system capacity, pushed the vendor to close all the critical, high and medium priority defects before providing a qualitative feedback on the solution and the potential risks. Once satisfied our customer’s investment was safe, we gave a go-ahead to rollout the solution.

We demystified the automation puzzle. Relentless validation tamed!

Monday, June 28th, 2010

A large global provider of BI solutions has a product suite that runs on five platforms supporting thirteen languages with each platform suite requiring multiple machines to deliver the BI solution. The entire multi-platform suite is released on single CD multiple times a year.

The problem that stumped them was “how to automate the final-install validation of multi-platform distributed product”. They had automated the testing of the individual components using SilkTest, but they were challenged with “how to unify this and run off a central console on various platforms at the same time”.

Considering each platform-combination took about a day, this required approximate two months of final installation build validation, and by the time they were done with this release, the next release was waiting! This was a relentless exercise, consuming significant QA bandwidth and time, and did not allow the team to do things more interesting or important.

The senior Management wanted single-push-button automation -identify what platform combination to schedule next, allocate machine automatically from the server farm, install and configure automatically, fire the appropriate Silk scripts and monitor progress to significantly reduce time and cost by lowering QA bandwidth involved in this effort. After deep analysis, in-house QA team decided this was a fairly complex automation puzzle and required a specialist! This is when where we were brought in.

After an intense deep-dive lasting about four weeks, we came up with a custom master-slave based test infrastructure architecture that allowed a central console to schedule various jobs onto the slaves, utilizing a custom developed control & monitoring protocol.  The solution was built using Java-Swing, Perl, Expect and adapters to handle Silk scripts. Some parts of the solution where on Windows platform while some on UNIX.  This custom infrastructure allowed for scheduling parallel test runs, automatic allocation of machines from a server farm, installing appropriate components on appropriate machines, configuring them and finally monitoring the progress of validation through a web console.

This test infrastructure enabled a significant reduction of the multi-platform configuration validation. The effort reduced from eight weeks to three weeks. We enjoyed this work simply because it was indeed a boutique work fraught with quite a few challenges. We believe that this was possible because we analyzed the challenging problem from wearing a development hat and not the functional test automation hat.