Activity: Obtain Testability Commitment

Examine testability needs

Study the test automation architecture and test interface specifications to gain a good understanding of the test implementation and assessment needs. In particular, understand the constraints that these needs will place on either the software engineering process, or the software architecture and design.

Assess impact and prioritize

Study the testability needs and perform basic impact analysis in terms of the impact on the test effort of not having the need met. Also consider some basic analysis of the potential effort required by the development team to investigate and provide a solution for the need. For each need, identify potential alternative solutions that would have less impact the development teams.

Using this information, formulate a prioritized list that places foremost the needs that have a large impact on the test effort if they are not met, yet have no alternative solution. Do this to both avoid wasting valuable development resources on less essential testability needs, saving this opportunity for the really important ones.

Define testability benefits

By asking the development team to develop software with specific provision for the test effort, you will be adding further requirements and constraints to the development effort; that essentially equates to more work and additional risk and complexity for the development team. Some development teams will view designing for testability as outside the scope of their responsibility. In other cases, the testability needs will have to compete for the development resources against customer needs and requirements that will usually be given more priority. As such, you need to "sell" the benefits of the testability needs to the project manager, software architect and other development team stakeholders.

Formulate an analysis of the benefits of each testability need you want to obtain commitment for. Research papers, article and studies that support the value of your testability need, and make use of ROI statistics where available. Think of the benefits in terms of the value provided to the development team; what useful evaluation information will you be able to provide to them that could not be provided without this need being met? How will this make it easier or more efficient for you to give the development team timely, accurate, in-depth or useful feedback during each build cycle? Does this need provide the development team with a useful feature that can be used in their own test effort or in future diagnosis of software failure? In the case of competition against customer needs, consider ways you can show that providing a solution to the testability need earlier will provide additional opportunities for customer requirements to be supported in subsequent build cycles.

Identify and engage testability champions

Given that you will be imposing potentially additional work or risk on the development team, you should identify and engage with those influential stakeholders who have the ability to approve or mandate the support of testability. Do this as soon as possible, before actively promoting the testability needs you want supported.

The three most important stakeholders are the software architect, the project manager and the customer representative. Spend time with the software architect and promote the value of creating a software architecture that supports testability. Spend time with the project manager and promote the benefits of testability in terms of test team productivity and fast turn around on evaluation information. Encourage the customer to place value on a quality product being delivered.

Promote testability needs and benefits

It's important to promote testability needs in the right way. Each combination of project manager, development team and customer stakeholders has a different social dynamic and culture, and it's important to be sensitive to that when you promote testability needs. As general heuristics, don't mount a formal testability "campaign" if the team is relatively laid-back and informal; and don't use an informal approach in a high-ceremony project.

In some cases, a collaborative "brainstorming" session is a useful presentation format, where the need is presented as a challenge to the development team, and they are encouraged to identify creative solutions to meet the testability need(s). This encourages their ownership of the solution and fosters a feeling of partnership in the effort.

Timing is also important for this step. As a general rule, you should try to identify and promote the most important testability issues as early as possible, generally during the Elaboration, and where possible the Inception phase. When testability issues are raised in these early stages of the project, the team is typically smaller and is more receptive to change. It's also easier to include these needs in the evolving design as minimal rework is usually required.

A good place to identify testability needs and present them in a positive and less "official" manner is to have the test team offer their services in evaluating proof-of-concept activities and in evaluating the selection of third-party components for use in the development effort. In particular, the involvement of test teams during database component selection, UI control or component selection, middleware components etc. means that testability issues can be used as one aspect of the component selection criteria. For example, in many cases development teams will have minimal concern over which UI widget library to make use of; if one library is more testable than another, the development team will be happy to select the more testable widget library.

If you've had trouble identifying or engaging with testability champions, you may need to consider either an approach that introduces the changes more incrementally making them potentially less risky and smaller blocks of effort, or you may have to escalate the most important testability needs as critical project issues that prevent the test effort from being successful until they are resolved. In the latter case, we recommend you carefully consider all your options before deciding on this course of action.

Gain commitment to support and maintain testability

It's important to ensure the testability needs are regarded in the same way as any other requirement or constraint placed on the development effort. You need to be assured that the testability features made available today will not be abandoned tomorrow.

In some cases, attempting to gain this commitment may result in the development team refusing to develop or support the testability needs. While this can be disheartening, it's better to be aware of this situation and deal with the reality of it at as early as possible; it's much worse to have spent extensive time and effort developing a test implementation that the development team then abandon uncommitted support for.

Advocate the resolution of testability issues

While the development team may agree to provide the necessary support for the testability needs of the test effort, it's important that you take an active interest in the design, implementation and completion of this work. Don't simply abandon concern because the development team have agreed to address the testability needs or have begun work on a solution; you need to ensure that an appropriate solution is developed in a timely manner.

Make yourself and the other test team staff readily available to answer the development teams questions, and offer to evaluate the prototypes as soon as they are built. Offer constructive feedback and show enthusiasm for the effort the development team have put into helping meet your needs. Offer to have your key staff attend or facilitate design workshops for the more complex testability needs, but guard against your team being overbearing and controlling the solution space of the design process for the developers.

Where issues arise and you feel they are not getting adequate attention, or being addressed with the necessary haste, raise your concerns with the software architect and project manager. Have the project manager log an issue on the project issue list if appropriate.

Evaluate and verify your results

Now that you have completed the work, it is beneficial to verify that the work was of sufficient value, and that you did not simply consume vast quantities of paper. You should evaluate whether your work is of appropriate quality, and that it is complete enough to be useful to those team members who will make subsequent use of it as input to their work. Where possible, use the checklists provided in RUP to verify that quality and completeness are "good enough".

Have the people performing the downstream activities that rely on your work as input take part in reviewing your interim work. Do this while you still have time available to take action to address their concerns. You should also evaluate your work against the key input artifacts to make sure you have represented them accurately and sufficiently. It may be useful to have the author of the input artifact review your work on this basis.

Try to remember that that RUP is an iterative process and that in many cases artifacts evolve over time. As such, it is not usually necessary-and is often counterproductive-to fully-form an artifact that will only be partially used or will not be used at all in immediately subsequent work. This is because there is a high probability that the situation surrounding the artifact will change-and the assumptions made when the artifact was created proven incorrect-before the artifact is used, resulting in wasted effort and costly rework. Also avoid the trap of spending too many cycles on presentation to the detriment of content value. In project environments where presentation has importance and economic value as a project deliverable, you might want to consider using an administrative resource to perform presentation tasks.

Rational Unified Process