Posts Tagged ‘guarantee software’

It was more than just validation. It was ensuring nation’s pride.

Monday, August 9th, 2010
A large petroleum major was rolling out a specialized solution to ensure that fleet
tracking solution to ensure zero pilferage during transport. The solution consisted of a
plethora of technologies (GPS, GSM, Web, Mapping) and our role was to ensure that the
final solution is indeed risk-free for deployment.
With the launch date in the next few weeks, we got cracking on applying HBT to extract
the cleanliness criteria from the business and technical specifications outlined in the
tender document.The cleanliness criteria consisted of multiple aspects- deployment
environment correctness, cleanliness of software, clean working of hardware/software
interfaces and finally the ability to support a large load, volume with real-time
performance.
We identified the potential types of defects that spanned the entire spectrum of
hardware and software. The first step was to understand system development process
and identify senior consultants visited the vendor’s facility to assess the people and
processes used to develop the system. This provided a clear picture of what to expect
and the work that lay ahead of us.
Post our understanding of the development system, we developed a scientific strategy
and the evaluation scenarios. A variety of tests were identified – individual feature
validation, simulating various business use-cases, understanding load limitations and
performance evaluation of the system.
Now we were ready to validate the final system in the data center. The first cut of the
solution was used to develop a set of automated scripts for large scale load/stress/
performance testing. The system was populated with large data representing a real life
system. Vehicles were fitted with the vehicle mounted unit. We were ready to roll now.
The various vehicles were set in motion in various terrains, at various speeds and
mapping of the fleet on the India map was validated. We simulated large number of
vehicles with data arriving from the simulators at a high rate to ensure that
performance was indeed real-time.
In addition the deployment environment was validated, configurations checked, legality
of the software verified. We also verified that the solution integrates with customer’s
SAP database as well.
Bugs popped up and were fixed. We recommended changes in the system capacity,
pushed the vendor to close all the critical, high and medium priority defects before
providing a qualitative feedback on the solution and the potential risks. Once satisfied
our customer’s investment was safe, we gave a go-ahead to rollout the solution.

A large petroleum major was rolling out a specialized solution to ensure that fleet tracking solution to ensure zero pilferage during transport. The solution consisted of a plethora of technologies (GPS, GSM, Web, Mapping) and our role was to ensure that the final solution is indeed risk-free for deployment.

With the launch date in the next few weeks, we got cracking on applying HBT to extract the cleanliness criteria from the business and technical specifications outlined in the tender document.The cleanliness criteria consisted of multiple aspects- deployment environment correctness, cleanliness of software, clean working of hardware/software interfaces and finally the ability to support a large load, volume with real-time performance.

We identified the potential types of defects that spanned the entire spectrum of hardware and software. The first step was to understand system development process and identify senior consultants visited the vendor’s facility to assess the people and processes used to develop the system. This provided a clear picture of what to expect and the work that lay ahead of us.

Post our understanding of the development system, we developed a scientific strategy and the evaluation scenarios. A variety of tests were identified – individual feature validation, simulating various business use-cases, understanding load limitations and performance evaluation of the system.

Now we were ready to validate the final system in the data center. The first cut of the solution was used to develop a set of automated scripts for large scale load/stress/performance testing. The system was populated with large data representing a real life system. Vehicles were fitted with the vehicle mounted unit. We were ready to roll now.

The various vehicles were set in motion in various terrains, at various speeds and mapping of the fleet on the India map was validated. We simulated large number of vehicles with data arriving from the simulators at a high rate to ensure that performance was indeed real-time.

In addition, the deployment environment was validated, configurations checked, legality of the software verified. We also verified that the solution integrates with customer’s SAP database as well.

Bugs popped up and were fixed. We recommended changes in the system capacity, pushed the vendor to close all the critical, high and medium priority defects before providing a qualitative feedback on the solution and the potential risks. Once satisfied our customer’s investment was safe, we gave a go-ahead to rollout the solution.

Can we guarantee cleanliness of a software?

Wednesday, July 15th, 2009

Couple of weeks ago, I conducted a Thought Leadership workshop on an absolutely new topic “Hypothesis-based testing”. It was conducted in Bangalore at ”The Chancery Hotel” and was well received. Subsequently, I did a 90 minute lecture in Ford Chennai to the QA audience and the feedback that I received was “All the participants thoroughly enjoyed this thought provoking talk to engineer clean software. It was more value adding and was different from the other normal monthly talks”.

So what is “Hypothesis-based testing” and how is it connected to STEM™ 2.0? Let us start from the fundamentals – “Can we guarantee that the released software is clean?” Most often the answer is an emphatic NO! The reason for this is that we do not wish to believe that the final software will have no defects. Hey, wait a minute…. Is this what guarantee means? Not really. As a consumer, we expect the products that we buy to be guaranteed. That is, we expect that it has been well validated, and should ensure a great experience when we use it. In the extreme case when it does fail, we expect fast turnaround via a responsive support system, resulting in a quick fix or replacement. Our customers expect the same too from us as producers of software. Hence guarantee means that we are extremely clear of the probability of the failure and the risk therefore. Once we are sure of this, we should be able to guarantee the cleanliness of software.

What does it take to do this?  If we can guarantee the process of evaluation of software, then we can guarantee that cleanliness of the delivered software. Now what does it take to do this? How can one guarantee the process of evaluation? Most often, we like to believe that experience of the test staff is what will make this possible. But this is simply not enough.  If we can scientifically analyze the process of evaluation, rationally justify why we did the various activities and the outcomes, and yet do not find any anomalies, then we can inclined to think that the process of evaluation is guaranteed to produce clean software. Hence guarantee can be achieved if we can explain the means to achieve the outcome. For example if we can scientifically explain our test strategy, scientifically prove the completeness of test cases etc, we can indeed guarantee cleanliness. The trouble is that the industry does not seem to have rational and scientific form of thinking/method.

STEM™ 2.0 is a method that allows for scientific thinking and disciplined implementation to ensure that the delivered software can indeed be guaranteed to be clean. The central core of STEM™ is establishing a clear goal and then performing activities that will indeed get us to the goal. In STEM™, the goal translates to “Uncover these potential defects”. These potential defects are hypothesized and all the later activities are about proving that the hypothesized defect(s) do/do-not exist.

Hypothesis-based testing is therefore an approach to testing that is about establishing a hypothesis and proving/disproving them. In Hypothesis-based testing, the key tenet is to establish a hypothesis that these potential defects may exist, and then proving/disproving their existence. Hypothesis-based testing is powered by STEM™ 2.0! I hope you see the connection now!

Think – when you go the doctor with ailments, he/she hypothesizes potential problems based on your symptoms, performs diagnostic tests to confirm the hypothesis and then prescribes the treatment regimen. When you go on a holiday, you hypothesize needs and accordingly pack items. Whenever you have a goal-focused, you always hypothesize the future situation(s) and then up with a list of activities. Hypothesis-based testing is the application of this thinking to uncovering defects in software. Hypothesis-based testing is not a normal term, it is invented in STAG! Hypothesis based testing is powered by STEM™ 2.0.