- "Happy About
Global Software Test Automation"
Subject: Oct 2006 eZine: "Happy
About Global Software Test Automation"
Framework® Institute eZine: Your Link to Business Strategy
2006 *5,500 subscribers* Volume 8, Issue 4
Online at http://ValueFrameworkInstitute.org/publications.html
This Issue online at http://ValueFrameworkInstitute.org/Oct2006/
In this issue, I
wanted to share an excerpt from the recently released book "Happy
About Global Software Test Automation." Please let me know
what you think. Learn more below or at that Happy
About page discussing the book.
In this book, we described software testing and the top pitfalls
of manual software testing, test automation, and outsourcing/offshoring of software
testing. We have also described some actions that you can take to improve each
of these areas. Driving toward the solution, we presented the Global Test Automation
strategy that integrates manual software testing, test automation, and global
resource strategies to maximize the benefits of software testing while minimizing
the costs. We also elaborated on its benefits.
We would like to leave you
with a list of the top ten executive takeaways from this book.
Top 10 Executive Takeaways:
view of software testing and QA will increase revenue and decrease cost.
engineering and testing strategy starts from the top.
must budget and adequately fund testing and QA as a separate line item.
need to create visibility into the process.
- Metrics for visibility
are not rocket science, but they need to be established quantitatively and qualitatively.
- Recognize that manual testing is unavoidable, but that you need to
encourage automation when and wherever possible while applying the Automation
- Although automation solves the speed problem, it's not a
- Global software test automation can save money and
time and also provide around-the-clock productivity.
- Plan first, execute
secondthe Global Test Automation strategy should come first. The strategy
then integrates automated testing programs and global testing resources.
critical on staffingdont settle for second-class quality and testing
staff, and dont treat them as second class.
the rest of this chapter, we elaborate on these takeaways.
#1 An executive view of software testing and QA will increase revenue and
The ultimate benefits of effective software testing are
increasing your revenue and decreasing your expenses. Both of these benefits directly
improve your companys profitability. Some of the key internal values that
drive this are the following:
in the consistency and dependability in quality through the visibility into the
quality level of the product under development throughout the development lifecycle
time on development, less time on maintenance
utilization of resources and budget due to on-time delivery
can be done if it can be done cheaper
can be done and delivered faster if can be done faster
an effective test and QA strategy, you can gain confidence in your software products
on a number of fronts. You can be confident that your software is of a consistent
quality with each release, and that your customers can depend on its level of
quality. This is achieved through visibility into the quality level of your companys
products under development throughout the development lifecycle. Visibility is
a key benefit of an effective test and QA strategy, and it directly enables you
to be confident in your products.
When your test and QA
strategy are effective, your team can spend more time on development and less
time on maintenance. This is because bugs are either avoided or found early in
the process so that isolating and fixing them are not onerous tasks. The later
in the process that bugs are found, the more work is required to correct them
and maintain your customers loyalty. The more time your development staff
spends fixing bugs and addressing short-term workarounds for the customers that
are hitting the bugs, the less time they have to spend on developing new features
to make you more competitive and profitable.
and QA strategies also enable an effective utilization of your resources and budget.
They enable an on-time delivery of your products, and therefore avoid cost and
schedule overruns. When you can complete the new product development project on
budget and on time, you can avoid problems associated with needing additional
resources beyond the project plan to complete the project.
test and QA strategies enable you to do more to make you more competitive and
increase your revenue. They do this by enabling your company to save money on
new development projects. These savings can then be invested right back into your
company to enable you to do more. Also, the time savings can be utilized for other
projects that will also make you more competitive and increase your revenue.
when your test and QA strategy are effective, you wont have surprises. You
wont hear about your customer finding bugs at critical times needing fast
fixes and workarounds. You wont be woken up first thing in the morning on
your first day of a badly needed vacation with a crisis from the office due to
a major customer finding a critical bug. No surprises is a good thing, and can
help you to get a better nights sleep.
#2 Quality engineering and testing strategy starts from the top.
for improvements in quality engineering and testing strategies have to start from
the top. The executive team must have a solid understanding of the quality cost
concept. In addition, they must understand their own organizations quality
cost model including the data associated with it. The executive team must also
understand that testing and QA are not synonymous. The executive that heads up
the quality engineering efforts should be fully educated in the intricacies of
the organization and testing activities, and must be fully accountable for the
quality of the delivered product as well as educating the executive team in these
If the executive team isnt driving the quality
initiatives towards improvement, they may actually be hindering it. A highly skilled
and knowledgeable development and test team may be prevented from reaching their
full quality potential due to misguided management decisions that lack a focus
Takeaway #3 You must budget and
adequately fund testing and QA as a separate line item.
Testing and quality
assurance are interrelated with development, but are actually different disciplines
that require a different focus and a level of independence from one another. They
should have separate budgets so that the test and QA functions arent squeezed
by cost overruns in development to the point that they cannot adequately perform
their function. The way they operate and the short-term and long-term business
impacts of their output are also sufficiently different that their ROI is measured
By analogy, consider the roles of sales and
marketing. These are often spoken together in a single phrase as if they are part
of the same organization and function. But, we know that they are separate functions
with separate disciplines and a separate focus. They are generally given separate
budgets. Likewise, research is also different than product development, and is
generally given a separate budget. To be effective and provide improvements in
quality, testing and QA must have its own budget separate from product development.
#4 You need to create visibility into the process.
With the visibility
that a well-run test organization provides to management, you wont have
surprises. This will give you confidence in the product and service that you deliver
to your customers. Software testing and quality engineering are still immature
disciplines that have not been studied as extensively as they should be. This
makes visibility all the more important for you to maintain confidence. You need
to capture the data you need regarding your products quality and associated
development and test activities so that you can understand where you are and set
appropriate goals for where you want to go.
#5 Metrics for visibility are not rocket science, but they need to be established
quantitatively and qualitatively.
Measurability is the key to visibility.
The challenge is to know that what you are measuring is valid and useful, and
having confidence in the integrity of the data. By and large, the testing discipline
lacks an effective bookkeeping infrastructure. Management of quality engineering
and testing should be metrics-driven. The numbers obtained from appropriate, valid,
and trustworthy measurements should drive quality improvement initiatives. A valid
metrics model is critical to the success of these efforts.
#6 Recognize that manual testing is unavoidable, but that you need to encourage
automation when and wherever possible while applying the Automation 5% Rule.
when you have a good test automation program in place, you still need to do some
manual testing. The usability testing, for example, requires human involvement.
However, manual testing is not the solution for short-cycle, high-volume test
challenges. A powerful test automation strategy is required for these applications.
For these applications, manual testing has the following drawbacks:
#7 Although automation solves the speed problem, it's not a silver bullet.
solves the speed problems in a short-cycle and high-volume test environment. But
automation does present some challenges and problems of its own. The key to success
in automation is focusing your resources on test production rather than test automation.
Focus on improving the quality and quantity of your tests, not on automating the
tests. Apply the 5% rule:
most critical thing for a successful test automation program is the test methodology.
No matter what tool you use, without a solid methodology in place, it wont
be effective. The tools you select should then support the implementation of your
test methodology to help you succeed.
The benefits of a
successful test automation strategy include the following:
Produce higher quality releases.
Improve Test/QA communication.
Halve testing costs.
for early and frequent testing.
and continued engineering costs.
use of resources.
Improve customer confidence
drivers of these benefits are the following:
effective test strategy provides visibility into the quality of the software at
an early stage, enabling effective management to improve the quality before the
product reaches customers hands. Whereas an effective object-oriented software
development methodology provides reusable software modules, an effective test
automation methodology provides reusable test modules. This helps drive scalability,
which enables automated tests to be scalable as the product and test requirements
grow. It also helps drive maintainability. A change in the application that requires
a change in the test modules will only require a change in a small number of reusable
modules, and all the high-level tests utilizing these modules will receive the
change for free. This is infinitely more maintainable than the situation in which
each test is fully independent from other tests, and a change has to be made to
every test that makes use of a changed element in the product individually.
#8 Global software test automation can save money and time and also provide
Outsourcing/offshoring provides a cost advantage
due to lower labor rates, but these must be weighed taking other factors into
account that add to the cost of outsourcing. These extra factors include communication
and travel costs, training costs, and higher local management overhead, for example.
The around-the-clock test production benefit requires a good working management
process to be effective. These benefits require a serious commitment on the part
of the local management and significant management oversight.
#9 Plan first, execute secondthe Global Test Automation strategy
should come first. The strategy then integrates automated testing programs and
global testing resources.
The Global Test Automation strategy should come
first. The methodology is the key to success. The tools then must be selected
to support the methodology, and the global resources chosen to work with the methodology
and tools. These three factors must be integrated to be successful. This is the
key to the success of the Global Test Automation strategyintegrating methodology,
tools, and global resources together synergistically.
#10 Be critical on staffingdont settle for second-class quality
and testing staff, and dont treat them as second class.
the key to any organizations success. You want high-quality people on your
test team, just like you want high-quality people on your development team and
your management team. Dont settle for second-class quality staff. Choosing
staff with the appropriate skill sets and aptitudes for testing vs. development
is an important aspect of this. Likewise, dont treat your test staff as
second class. The test team is critical for the release of quality products. Your
companys bottom line depends on that as much as it depends on the most innovative
software from the development team. If you want to keep quality people on your
test staff, they need to be treated as well as quality people on your development
team. You need to get away from situations where test positions are seen as an
entry level to development positions down the road, and therefore, in a lower
In practice, there is much more
that can be said and much more technical detail that could be explored. We hope
that this book has given you a fresh perspective on a strategy for integrating
test methodologies, tools, and global resources to greatly improve your software
products quality through effective testing.
you enjoyed this chapter.
About Global Software Test Automation" contains
a management discussion of the changing world of software testing. Testing problems
are difficult, expensive, and hard to understand. Most software companies have
bugs escape the testing cycles and reach customers. How can some of the inherent
problems with software testing be eliminated? This book addresses this fundamental
issue and helps the reader understand the high-level elements necessary to better
execute software test automation and outsourcing initiatives. Pick up this book
to get the answer. You can do so at Happy
About, Amazon or other online and offline bookstores.
Mitchell Levy, Publisher &
CEO, Happy About
Chief Strategy Office, The Value Framework Institute
publisher @ happyabout.info, 408-257-3000
About Global Software Test Automation: A Discussion of Software Testing for
Although your team may
know about test automation, have they ever executed efficiently to meet the goals
of faster delivery, better quality, and saving money? If not, this book is a must
read. After reading the 160 pages (125 for the eBook), you will be able to:
Describe the need for software testing and the shortcomings of the current paradigms.
the hidden costs and problems of offshoring the testing function.
the models that work and understand if they are worth incorporating into your
you need to do to have it done "right
| || |
Discounted to ($16.96)
| || |
Reviews of "Happy About
Global Software Test Automation"
is complex but I'm tired of finding bug after bug that a 5th grader wouldn't have
turned in. Virtually every technical product these days includes a lot of software.
It's a rare engineer that can write nearly perfect code. Methodical and thorough
testing of software is the key to quality products that do what the user expects.
Read this book to learn what you need to do!"
Steve Wozniak, Wheels
of Zeus, CTO
"In theory, test automation
is supposed to be a silver bullet to increase test coverage and improve quality;
offshoring is supposed to drastically cut costs. In reality, many organizations
struggle with both, and don't see significant gains despite extensive efforts.
This book clearly presents the challenges of test automation and a practical way
for organizations of any size to overcome them to realize significant time and
cost savings in their software testing effort. Every executive responsible for
development and testing of software should become familiar with these ideas!"
Martin, Vice President, Product Strategy, PSS Systems
by itself is not enough.This book provides you testing strategies to stay ahead
of the competition and maximize your investment."
Robert S. Alvin,
CEO and Chairman, Netline Corporation
continued advances in development techniques and technologies, software quality
problems are as pervasive as ever. Software testing teams are under tremendous
pressure to test more complex systems with the same or fewer resources, and corporate
managers are always looking to shave costs by leveraging offshore testing. This
book does a great job of highlighting the fundamental challenges of software testing
today, and then presents a thoughtful solution for leveraging test automation
and offshoring to meeting your organizations quality goals."
Au, Vice President, Engineering, Centrify Corporation
isn't just about technical decisions. Finally, this book is the first that offers
a practical business case for effective test automation."
Hatam, President, Application Services, Moyo Group
is one of the must read books by software executives. It goes over the major concepts
and best practices of software testing in an efficient and effective manner. At
Sun, we already use some of the best practices described in this book and are
planning to adopt the remaining best practices."
Satya Dodda, QA
Director, J2EE App Server, Sun Micro Systems, Inc.
Nguyen is CEO, President, and Founder of LogiGear and is responsible for the
company's strategic direction and executive business management. He's been a leading
innovator in software testing, test automation, testing tool solutions and testing
education programs for the last two decades.
Nguyen is coauthor of the top-selling book in the software testing field, Testing
Computer Software (Wiley, 2nd ed. 2002) and other publications including Testing
Applications on the Web (Wiley, 2nd ed. 2003). His experience over the past two
decades includes leadership roles in software development, quality, product and
business management at Spinnaker, PowerUp, Electronic Arts, Palm Computing and
other leading companies. A frequent speaker at industry events and a contributor
to many industry publications, Nguyen also teaches software testing at LogiGear
University, and at the University of California Berkeley Extension and Santa Cruz
Extension in San Francisco and Silicon Valley.
Hackett is co-founded LogiGear in 1994 and leads the company's LogiGear University
training operations division, setting the standard in software testing education
programs for many of the world's leading software development organizations. Mr.
Hackett is coauthor of the popular Testing Applications on the Web (Wiley, 2nd
ed. 2003), and has helped many clients produce, test and deploy applications ranging
from business productivity to educational multimedia across multiple platforms
and multiple language editions. His clients have included Palm Computing, Oracle,
CNET, Electronics for Imaging, The Learning Company, and PC World.
to co-founding LogiGear, Mr. Hackett managed QA teams at The Well, Adobe Systems,
and PowerUp Software. He holds a Bachelor of Science in Engineering from Carnegie-Mellon
K. Whitlock is currently a Program Manager at Digidesign, a division of
Avid Technology, Inc. Prior to this, he was Director of Optical Systems Research
and Business Development at RSoft Design Group, Inc., where he initiated and led
the development and commercialization of several optical communication system
simulation software packages including LinkSIM, ModeSYS, and OptSim 4, which won
the Lightwave OFC/NFOEC 2005 Attendees Choice Award. He has also secured and served
as Principal Investigator on federally funded SBIR, STTR, and NIST ATP research
Dr. Whitlock earned his BS, MS, and PhD all in Electrical
Engineering from the University of Illinois at Urbana-Champaign. Dr. Whitlock
has co-authored over 30 technical papers and articles. He is a Sr. Member of the
IEEE and Chair of the Santa Clara Valley chapter of IEEE LEOS.
Money Back Guarantee:
Our books offer a 100% money back
guarantee. Pick up one today and see if you agree with our other customers that
have enjoyed our books.
for our bi-monthly eZine
Enter your e-mail address below and click
Happy About writes and publishes books for corporations the utililze case studies,
testimonials and war stories from those that have "Been There and Done That!".
Check our books or white
us if you are a company that would like us to write a book for you or an author
that would like to submit a book.
lonely at the top! There can be no better network than one of your peers to
share your issues and get suggested solutions. CEOnetworking
is a membership organization based in Silicon Valley that puts 6-8 CEOs
in a room once a month to talk about issues and brain-storm ideas. These highly-facilitated
meetings are geared toward helping CEO's and their companies achieve success.
It is a quick way of creating a peer-based virtual board that does not have a
vested interest in your company. Not only will you help your company reach the
next level, you'll have the chance to give feedback, share advice, and watch other
companies succeed. Let us know if you'd like to test-drive a meeting.
more at "CEOnetworking", contact us about
membership by e-mail (firstname.lastname@example.org)
or phone Mitchell Levy, or Thomas Hong, at 408-404-6758.
Valley Executive Business Program
The Silicon Valley Executive
Business Program is a unique program that explores business from a CEO's perspective.
Students learn from and understand what CEOs are looking for in their people and
organizations, especially from C-Level Staff (CXOs). A key element of the program
is its practical vs. theoretical focus.
more at "SiliconValleyPace.com", get
info by e-mailing (email@example.com)
or calling 408-257-3000.
SHARING THE VALUE FRAMEWORK® INSTITUTE
Please forward this eZine to
friends and colleagues who will find it valuable. For copyright purposes, we must
ask that it be distributed in its complete form. View past issues and signup to
receive it at this location: http://ValueFrameworkInstitute.org/Publications/
This newsletter is 100% opt-in. You are receiving it because you have requested
to receive it in the past. If you no longer wish to receive it, please let us
know either on-line http://ValueFrameworkInstitute.org/signup.htm
or via e-mail eZine.cancel
Executive Producer, Mitchell Levy (info
to the main eZine Page (http://ValueFrameworkInstitute.org/Publications/)
Back to this issue: