The customer in focus provides data integration software and services that empower organizations to access, integrate, and trust all its information assets, giving organizations a competitive advantage in today’s global information economy. As the independent data integration leader, the customer has a proven track record of success helping the world’s leading companies leverage all their information assets to grow revenues, improve profitability, and increase customer loyalty.
The customer had a large base of automated test scripts (1300) in Rational Visual Test (VT) and a single license of the Visual Test on a dedicated machine – a risky affair considering that the Visual Test is no more supported and relies on a older version of Windows. They decided to de-risk this to support the newer versions of products and also get rid of limitations existing in the existing test script suite. Some of the limitations of existing automation suite were: (1) Manual dependency to get the scripts run in suite against any new build released,(2) Non-existence of tool support for Visual Test (3) Limitations with respect to support on other flavors of windows operating system and support for internationalization (i18N) (4) Limitations in the framework to extend the scripts with new functional changes and (5) Difficulty in building competency on Visual Test. The management decided to migrate these VT scripts to Borland Silk Test Suite.
Now came the challenges that we had to solve. The test suite was large with only the script available and no documentation on the test cases. They were keen that the performance of the new suite in SilkTest be considerably enhanced. Technically the test suite was intensely data-driven with huge data set to drive the test suite. The automation run was l-o-n-g, ranging from 24-36 hours, necessitating that a robust recovery mechanism be in place to ensure uninterrupted runs with minimal baby-sitting. Since it was a new investment, the management was keen that the automation framework is indeed flexible so that new test cases could be added to the suite quickly. Finally, the suite had to cater to the different language packs of the product.
Phew – It was real challenge “scaling the peak”, and we had our share of issues of encountering bad weather, storms, landslides, but heck we made it! As always, the journey was arduous, but the rush of adrenalin after reaching the peak was great. We must confess that the journey would not been possible without the wholesome support and cooperation from the customer – Thank you.
Now the details of the climb! We had to analyze the large VT suite to understand the structure, flow, data inter-relationship and the finer nuances. Remember that we only had the scripts machine to try and understand! The key learning points and the action items were:
1. Build an effective data-driven mechanism to provide the flexibility to add and maintain test data in external SQL tables.
2. Implement robust delivery mechanism to enable the scripts to run uninterrupted for long durations, upwards of 24 hours.
3. Support for internationalization to enable testing of English and Japanese language packs via external language property files.
4. The ability to add new test cases to the existing framework to support new features and application changes with 50% less effort.
We approached the problem of scaling the peak, by firstly going through a strong intellectual process of technical problem analysis and devising a library-based & data-driven framework and subsequently putting together a factory-driven approach to rapidly code the scripts. Once we architected the custom framework, we identified with the “good principles of development”, such as avoiding global variables, avoiding hardcoded information, level & depth of documentation, language coding conventions, and finally object referencing and de-referencing strategies. The development process was iterative with multiple milestones identified and acceptance criteria clearly identified.
A skeletal team of architects and specialists got cracking on the problem, making the first move to built a flexible and robust automation framework. They also commenced development of common library components, that will be used to by the larger extended team later. Once the architecture custom framework was in place, coding standards were enforced, and then these activities of coding the framework and the library components were done. At this point, a larger team was assembled, each of them was assigned a certain set of scripts to be converted from VT to SilkTest. The act of coding the newer SilkTest scripts was individually done on developer machines, code-reviewed and then later integrated on a test machine, and tested by running this on the target application. We did encounter a stormy climb, with myriad integration issues popping up, each of them was solved and we continued to make good progress.
The D-day came and we were delighted to hoist the flag on the peak! We had covered good ground, generating approximately 50,000 LOC with about fifth of that constituting the framework level component code. The cool air at the peak was refreshing and sweet! - We had reduced the cycle time by approximately 80% i.e. from FIVE days to just ONE day, were able to long runs of 24 hours without issues, switched language pack with ease and were able to add a new set of 120 test cases quickly enough. In the subsequent two months of intense usage only about five issues were reported, which were fixed.
This was a unique project, where we had to migrate automation code from one commercial tool to another with constraints of documentation and machine availability. It was indeed a pleasure to work with a demanding customer, who worked closely with us to help us understand the product and the VT automation, and also making available a dedicated integration machine with the lone VT server machine.
We have always enjoyed challenges, and we thank the customer for giving us the opportunity and reposing trust in us. This is the fun-part of being a test boutique!