Richard Bornet is the creator of the Scenario Testing methodology, and co-author of Scenario Tester. He as been involved in automated testing for the past 5 years, and now describes, at a greater level of detail, the methodology of Scenario Testing introduced in our last discussion. We join Richard at his office.

Q. Mr. Bornet, thanks for spending time with us again. In our last discussion you introduced Scenario Tester, and explained what it offers to QA. Today we've asked you to explain more about the process, and Scenario Tester in detail.

Thanks again... it's a privilege. As I was explaining previously, automated testing efforts often fail. They fail because testers don't want to be coders, test data is too complicated to create and control, and scripts become unmanageable.

Scenario Tester solves the first two problems and makes scripting easier and faster to do and much easier to maintain.

Q. But is it straightforward?

Extremely. Scenario Tester mirrors the way people test. It is organized to maximize people strengths. Remember that most testers don't want to become professional programmers, but still scripts need to be created and maintained. We have put order to the whole process and at the same time simplified it, so testers can successfully use the power of automated testing.

Q. If testers don't want to be programmers, why not hire programmers or consultants?

Programmers generally don't want to be automated testers and they usually don't know the business. When programmers control the automated testing process, only a very limited number of people get to use automated testing. That isn't good. A QA department may have many testers and you really want to give these testers the ability to use the power of automated testing because they are the ones who really need it.

The question of hiring consultants is an interesting one. We have seen lots of scripts that consultants have written, many were very well written, some not so well; but the one thing all these scripts had in common was a high degree of complexity. One thing to remember is that consultants are temporary, and they will eventually leave and hand over the scripts to someone internal. This internal person is almost never an experienced programmer.

Q. So what happens then?

Let me answer that by recounting some of the experiences I had from teaching automated testing for five years. In my classes I would have the individuals who had to take over these scripts. Most of these individuals were testers, and many of them were quite worried that they were not up to the task. They would bring me the scripts. The sad part was that in many cases the scripts were just too complicated and the individuals would not pick up enough in the five-day course to maintain those scripts. I knew that either the automated testing effort would die or the worried individual in front of me would be found wanting. Both were not pleasant outcomes.

Having taught for many years I resolved that I would come up with a process that allowed all testers to use the power of automated testing without having to script and that scripts would be simple and easy to maintain. This became the Scenario Tester approach.

Q. So how does the Scenario Tester approach work?

Again, the goal is to have the testers create and execute their tests without having to script. This goal is achieved through Scenario Tester. Here testers enter their test data and expected results. They then execute their tests and look at the results - no scripts, just their test data.

To do this the application-under-test is "mapped" into Scenario Tester. I'll explain how we do this later. This allows the testers to enter their test data simply and quickly. We like to call it testing by filling in the blanks. It's that simple.

In order for the tester to do this, two things have to take place:

1.	Scenario Tester needs to be set up.
2.	Scripts need to be built which use the data entered in Scenario Tester. These scripts will input values into screens and test for expected results.

Q. So the testers use Scenario Tester, but you still need someone to set up the scripts?

That's correct. In the beginning, you will need individuals to set up Scenario Tester and create the scripts. This may be an individual or team of individuals. You may use internal people or hire consultants.

Then you will move to the maintenance phase where you will have many testers using Scenario Tester to test, and one individual (we usually find that is enough) to maintain the scripts.

But now let's concentrate on how to set up Scenario Tester and how to build the scripts.

Q. How many people are needed to do it ?

This can vary, and depends in part on how fast you want to get the job done! Anyone with an elementary background in IT can perform the tasks. Being somewhat computer savvy or knowledgeable in the business typically helps. But these are usually the people we are dealing with.

In most companies we have people with ordinary computer skills set up all the scripts with no problems. These are individuals who, even though they are not professional coders, enjoy learning and working with scripts. We have had most of these people up and writing scripts very quickly, often in less than a day, and this includes teaching them what they need to know about the automated testing tool.

Q. How is it possible to get people using the automated testing tools in a day, when formal training for the tools is often 3 to 5 days?

The majority of our scripts are very straightforward. They are very logical and well-documented. There is a consistency to the scripts, which allows users to pick up how to script extremely quickly. We find that once people understand what we are doing, they say, "That's pretty simple" and then go ahead and just do it.

Q. Who works with you in the phase where you build the scripts?

Typically we'll have a person or two very familiar with the business operations of the application, and we'll have a person or two who is programmatically inclined. I don't think we've ever had a single application test team of more than six people, including ourselves. More often it is one coder and one tester.

Q. I've heard that some companies have test teams of 50 people or more.

Well, that's usually for more than one application. And a number of people may be doing tasks such as documentation or reviews at any given time. But when it comes down to building the tests and running them, we just find that too many cooks spoil the broth. We prefer a small focused team, from 2 to 6 people.

Q. Is Scenario Tester absolutely essential in order to perform scenario based testing?

You might be able to get away with using Excel for a small application. Handling multiple Excel spreadsheets for large applications can dramatically increase the complexity, and therefore increase the risk of failure. It definitely takes it out of the reach of most testers.

Many of the automated test tools have spreadsheet type features to assist in controlling the data. These are often even more complicated and definitely in the realm of programmers.

But it goes further than this. Scenario Tester provides many, many features that just make script development around scenarios so much easier. For example, one structure often encountered in applications is tables or lists. There are special methods of handling them that Scenario Tester supports. What happens when you need to add new controls or fields to the data, across multiple scenarios? I can go on . . .

Finally, Scenario Tester is the tester's tool. We concentrated on making the application extremely useable. From the tester's point of view, storing data in multiple spreadsheets is not very user friendly. Lack of usability can kill an automated testing effort. Testers have universally commented about how simple it is to use Scenario Tester. The goal can be summarized in the following expression: "Users will use useable software in a useful way".

Q. So what needs to be done to set up Scenario Tester?

To set up Scenario Tester, you need to do the following:

1.	Someone who knows the application well (business user, application expert or tester) needs to identify the major scenario components. This will determine what screens/windows need to be handled by Scenario Tester and the scripts.
2.	Based on the above, a scripter creates the input and output scripts that enter data into and validate data from the app.
3.	A programmer optionally works with a scripter to develop routines that make it easier to script or handle the application. For some applications, some complex code may need to be written and you will need the services of an experienced coder.
4.	Field names and values are entered into Scenario Tester.
5.	The master script is built by the scripter, which reads the scenarios and sends them to the input and output scripts.
6.	Scenarios are fine-tuned, and then fully deployed.

This diagram shows the relationships among these steps.



Q. Can you go into each step in more detail for our readers to gain a better understanding?

Sure. Figuring out how you want to break down the application is the first step. You need to get a feel for how the application is put together, and how you will want to organize the testing of this application.

The first thing to do is to look over the application. How is the application organized? How many screens does it have? What is the navigational flow of the application? What functions can I perform in the application? Where do we find our expected results? What screens do we want to test? How do we know that a specific test was successful?

You need to get a feel for how the application is put together, and how you will want to organize the testing of this application.

As you can see in the following picture, Scenario Tester has three distinct sections:

1. List of scenarios
2. List of screens
3. List of fields



A "scenario" is an event you are testing, for example "create a new customer".

By the way, for any RUP [ed. Rational Unified Process] readers out there, a scenario can be equated to a full or partial use case, depending on how involved you want to make it.

A "screen" is a screen or window or component or function in your application, or it could be a number of screens/windows, more about this below.

A "field" is a field in your application.

The trick is to find the right way to break down the application: what screens/modules/components do you want, and how do you want to group them.

Q. Can you elaborate on this?

Let's take a specific example. Say you have an application which manages contract information. There are several modules: create new contract (10 screens/windows), apply payments (3 screens/windows) and terminate contract (2 screens/windows). This is a total of 3 modules and 15 screens. The question is how to organize this. There are two obvious ways of breaking down the application: by module or by screen. You will have to decide which way makes the most sense. In this case the decision will probably be guided by the navigational flow. You need to be able to specify whether a "screen" will be used or not within the scenario. Do you need to do this at the individual screen level or can it be at the module level?

Let's take another example. You have an application that manages mutual funds. The main functions of the application are handling transactions that buy stocks, sell stocks, perform various reporting functions, run batch jobs, create extracts for customers. Each one of these is a major component of the application. It may make sense to break down this application by these components.

In other applications, you may breakdown by function. There is no formula to determine which is the best way to break down an application; it depends on the app.

Whatever way you break down the application, it is the tester in Scenario Tester who will indicate in the List of Screens whether that screen, window, module, component or function will be executed.

For each one of these screens, there will be associated data. For example a window may have a series of controls where you enter data. The specific information for those controls is entered in List of Fields.

Q. Ok, after you've decided on one approach, what's next?

Scenario Tester can duplicate any application under test. The question arises, do you script the whole application or do you do it in stages with the testers entering scenarios after each stage?

We usually script in stages. We figure out the most important functionality and script that part of the application first. Then we add additional parts of the application.

We often script the most important scenario with the actual test data. In this way, when we have finished scripting that portion of the application, the testers have an important scenario to use in their testing immediately.

Now we are ready to script the screens/modules/components/functions ("Screens") for inputs and outputs.

Q. What's the point of input and output scripts?

Input scripts are scripts which input data into screens: input scripts open screens, type text into text boxes, click on radio buttons, enable or disable check boxes, select values from lists and combo boxes. In short, the scripts mimic all the actions a user typically takes when inputting or editing data.

Output scripts are scripts which pull data from screens and test them against the expected values that have been entered in Scenario Tester.

Although there is no technical reason why you must separate input scripts from output scripts, this is a convenient and recommended approach. We will take this approach here.

Since we have separated input from output, we will need separate scripts for each. This often means that each screen/module has two scripts associated with it. In Scenario Tester under List of Screens there will also be two entries: one for input and one for output.

Generally there is one input or output screen per screen. That's the connection.

Q. Is there a trick to creating input scripts?

The way we usually create an input script is to turn on the "record" function of the automated testing tool and click on each object on the screen. This is to get the recognition strings for the objects. Some test tools store the recognition string internally and not in your scripts and you need to use the record function, so the test tool can create an entry for those objects.

Then we alter those scripts to work with Scenario Tester based on very specific standards. Sometimes we alter the scripts manually; other times we have routines, which build the scripts to our specification.

What you end up with is a series of scripts, based on a very specific set of standards. Our standards have features that make your scripts run efficiently, include sending all your inputs to the test log. We also force you to comment your scripts extremely well, so they will be easy to understand and maintain.

Here are some sample inputs for the various tools.

Each one of these scripts uses the same logic: