Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality
On July - 11 - 20105 COMMENTS
Product Description
“This book fills a huge gap in our knowledge of software testing. It does an excellent job describing how test automation differs from other test activities, and clearly lays out what kind of skills and knowledge are needed to automate tests. The book is essential reading for students of testing and a bible for practitioners.”
—Jeff Offutt, Professor of Software Engineering, George Mason University
“This new book naturally expands upon its predeces… More >>
Implementing Automated Software Testing: How to Save Time and Lower Costs While Raising Quality
Wanna know what’s hot in test automation? Just do a Google search on the term “Implementing Automated Software Testing” (IAST), and you’ll find out quickly. With approximately 3 million search results, this is clearly a very popular term, but not only that, it is now the title of a new book written by Elfriede Dustin, Thom Garrett and Bernie Gauf. We at the Automated Testing Institute aren’t sure if the title is coincidently identical to one of the most popular test automation terms, or if it was a genius marketing ploy to get more attention drawn to the book; whatever the reason, it is definitely aptly named. This successor to Automated Software Testing (AST) – a 1999 book also coauthored by Dustin, and also a term with approximately 30 million Google search results (wink, wink) – wastes no time in picking up where its predecessor leaves off. AST largely focuses on how test automation may fit into the overall software development and testing lifecycles, IAST lends more of its attention to the test automation effort itself and the details of managing an automated software testing effort.
IAST does an excellent job covering the following Automation Body of Knowledge version 1 (ABOK v1) skill categories (visit [....] for more information on the ABOK):
* Skill Category 1: Automation’s Role in the Software Testing Lifecycle (STLC)
* Skill Category 2: Test Automation Types and Interfaces
* Skill Category 3: Automation Tools
* Skill Category 4: Test Automation Framework Design
* Skill Category 5: Automation Framework Design
* Skill Category 12: Automated Test Reporting
This review will therefore describe the book and discuss how these categories are addressed. IAST is divided into three major segments:
* Part I: What Is Automated Software Testing and Why Should We Automate? (Chapters 1 – 4)
* Part II: How to Automate: Top Six Keys for Automation Payoff (Chapters 5 – 10)
* Appendices (Appendix A – D)
Part I does an excellent job addressing Skill Category 1 of the ABOK in that it addresses the `what’ and `why’ of test automation. Chapter 1 sets the tone of the book with a “unified” definition of test automation; a definition that also works to distinguish test automation from manual software testing. This chapter, along with its reference to Appendix B, also addresses ABOK Skill Category 2 with a discussion of the “typical testing types that lend themselves to automation”. Chapters 2 through 4, round out Part 1 of IAST, and continue to further address ABOK Skill Category 1 with topics including: reasons for automating, making the business case for test automation with return on investment (ROI) calculations, and how to avoid typical test automation pitfalls. The latter half of Chapter 4 also addresses ABOK Skill Category 3, by discussing tool evaluation and selection. This discussion is aided by Appendix C which expands into tools and tool requirements not only for automating tests, but also for automation of processes such as requirements management, defect tracking and security testing.
Part II of IAST graduates from the `what’ and `why’ of test automation and delves into the `how’ of test automation. Not `how’ in terms of how to develop scripts, but `how’ in terms of how to create, implement and monitor an automated test framework; and these are items found in Skill Categories 4, 5 and 12 of the ABOK. Part II of IAST begins with Chapter 5, which discusses how to define requirements for your test automation effort. Chapters 6 and 7 build on this by discussing how to take these requirements and develop a compatible strategy and framework. Chapters 8 and 9 cover the next logical steps of defining metrics to track test automation progress, and the implementation of the automated software test framework processes. Then finally, Chapter 10 closes the main portion of the book with a broad discussion of the skills required for test automation implementation.
In summary, IAST is strong with respect to test automation from a macroscopic level. This makes it a great resource for managers, leads, and anyone that is responsible for or will take part in defining and implementing a test automation effort. [Review originally posted in the August 09 Issue of the Automated Software Testing Magazine - [...]]
Rating: 5 / 5
My professional background had been as a Software Engineer and Manager in application development; however, I recently became the Manager for my company’s automation and performance testing team. I wanted to get a good overview of implementing automated testing, and this looked like the best of the books available for my purpose. I considered the author’s previous book Automated Software Testing, but it was written in 1999. I wanted something that would talk about more current tools available so this 2009 offering seemed to better suit my needs.
Like most technology books, this book is written in a very organized manner. The first four chapters are a good overview of the what and why of automated testing along with information about developing a business case and common myths. The section on the business case is fairly involved on how to compute ROI for automated testing. You may be able to get by with something simpler than this, but it’s a good starting point. The remaining six chapters give more details for executing an automated software testing effort from requirements and tools to processes and staffing guidelines. I found the chapters on automated software testing process and staffing guidelines the most helpful. The process recommendations are lightweight, but I agree with the authors that testing automation *is* software development.
The authors write from a perspective of a defense contractor, and this is important to understand. In this environment projects are typically standalone and large in nature, but this will not be the case for all readers. I work in the IT department of a for profit company, and my automated software testing team operates in a shared service model to support the highest priority projects. Whereas a defense contractor typically buys hardware, software, tools, etc. for each program as a part of their bid, my team uses a consistent development stack and reuses a consistent hardware environment. We may add tools or hardware as new situations come up, but things are fairly stable overall. We also have different titles, roles, and responsibilities than those defined in chapter 10. These differences don’t change the applicability of the concepts, but it does require me to translate the application from the defense contractor mindset.
The appendices give additional checklists and some detailed information on tools in the marketplace. The tool information will become dated soon, but it’s probably good for another year or so. The authors also give a lot of links to web sites throughout the book, and I like it when readers are pointed to additional information for continued learning.
Overall, I couldn’t ask for a lot more given what I was looking for. I am now in a better position to work with the experience people on my team and be confident in my ability to understand the key issues and considerations. Those looking for more hands on information may be left wanting. There are not a lot in the way of examples, but this is difficult to do without slanting the book toward specific tools. I think that the authors assume a certain level of experience for the software developers who will be doing the actual implementation, and they assume that they can translate the concepts into code. Please feel free to ask questions in the comments section if there is an area that I have not addressed.
Overall: B
Rating: 4 / 5
Implementing Automated Software Testing is a systematic examination of the why and how of large scale automation of big, complex systems.
While the back cover says “Whether you’re a test professional, QA spec ialist, project manager, or developer, this book can help you bring unprecendented efficiency to testing – and then use AST to improve your entire developement lifecycle”, I’m not sure this book is for everyone.
New QAers or testers, those involved with testing smaller-scale systems, or those involved in projects with smaller budgets and shorter timelines might find the book’s recommendations overwhelming. It seems clear to me that this book is best for those who are charged with determining a test automation approach for very large scale, long-term systems, with large budgets, such as those used by the Department of Defense and larger commercial efforts.
From the Preface:
“We at IDT have identified a boilerplate solution, strategies, and idea, all provided in this book, that can help increase the chances of your automated testing success.”
I agree. All of the authors and all of the contributing authors are IDT employees. This is IDT’s boilerplate solution. As with all boilerplate solutions, it may not fit your specific situation.
If it does, you will find a wealth of information. And even if it doesn’t you can still find useful nuggets of information about how some of these “big-project” teams automate their tests. For example, I particularly liked Chapter 4 – Why Automated Software Testing Fails and Pitfalls to Avoid. I believe this chapter would be useful for anyone about to embark on a test automation project.
Rating: 3 / 5
(Full disclosure: I got this book for review at no cost for me)
“Implementing Automated Software Testing” walks the reader through the each of the phases necessary for going from a manual testing environment to an automated one. The book is written mostly in general terms, making it useful for most environments, regardless of the type of applications being tested.
The book reads like a compilation of tips accumulated over the course of years of experience by the authors. It can almost be described as vaccination: If you don’t read the book you might not make all the mistakes it tries to prevent, but if you do read it, you’ll instinctively know how to make good decisions and what to do next.
Content is organized in a very logical manner. It starts defining why AST is beneficial and how to introduce it to the current environment, including presenting the business case. From there it goes to the main subject, implementation, followed by results measurement, and personnel roles in the project.
Overall, the book is very well written. Points are made quickly and in simple terms. The page layout and fonts make the book easy to read.
The book is a very valuable tool for managers and team leads of software QA. If read before starting an AST project, it will save the reader from many of the common mistakes made by many.
Rating: 5 / 5
Testing accounts for a large percentage of the time and cost of any new software development, and using AST – automated software testing – developers and testers can make the most of testing lifecycles, reducing costs. This comes from three leading experts in AST who explain the process, review its pros and cons, and use their experience in the defense and commercial world to help users make realistic business decisions. A ‘must’ for professionals involved in QA or project management as well as for software engineers, this is a pick for computer and business collections alike.
Rating: 5 / 5