What are some ways to test an architecture?

Question

How does one test an architecture? Are there test-related things that can be done while the architecture is still being pounded into shape?

I plan to iterate but I don't want to wait to have an entire architecture done (even if it's very rough) before I start thinking about testing.

Update: I'm using the word architecture the way it's used in Code Complete.

In other words, at this second, my architecture is a pile of paper showing how various blobs interact (e.g. one sheet of paper shows a legacy system that talks to a facade that parses/slices-and-dices, which in turn talks to the main website application).

Later, I'll flesh it out to the point of public classes/methods. And after that, I'll write out private method signatures and poke at algorithms, though I'm already looking into them for high-level things such as classifying text.

Answer 1

Usually, "testing an architecture" means testing an applicative and technical architecture (as opposed to business and functional one, which is not so much "tested" as "validated").
And this is not "unit testing" or "continuous integration" or TDD either, but more a black-box approach for testing a all system (composed from many modules) from start to finish.

You can indeed start designing not one, but several testing policies: when they concern "an architecture", they are often "system-wide" tests that implies a dedicated infrastructure (network, server close to the deployment target).

So, when designing your first "architectural" tests, you can take into account:

• what kind of test you want to make first (regression, stress-tests, uat -- user acceptance tests --)?
• what kind of traceability you want between those tests and the initial set of requirements?
• what kind of deployment process you need to automate in order to easily and quickly set up or update a testing platform (especially if there is a base involved, how will you deal with schema upgrade, and testing data?).

All of those topics can be addressed while the architecture is refined, in order to support more efficiently the development effort as soon as it begins.

Answer 2

A SOFTWARE ARCHITECTURE-BASED TESTING TECHNIQUE

This dissertation presents a practical, effective, and automatable technique for testing architecture relations at the architecture level. It also presents a proof-of-concept tool to generate test requirements.

http://cs.gmu.edu/~offutt/rsrch/JinDiss.pdf

Answer 3

Your archecture is a design document right?

Inspections are what you will want.

Inspections can catch bugs just as well as testing. Inspections will catch "bugs" sooner.

Put the documents in front of a group of knowledgeable people. Ask them to read it and to tell you where there are issues. Rework your document and get it back to them. Continue this process until it is completely clear to your knowledgeable audience what you are doing and all are convinced it will work.

Hopefully your document is written in a way where your readers can understand the interactions and see the workings even though they are abstract.

Answer 4

Design Rule #1: Don't assume anything is easy, thinking a system will be easy usually means the requirements or scope of what has to be coded hasn't been thought through yet.

Design Rule #2: Don't design something and then hand it off to developers and tell them it's easy when you haven't actually coded or worked with the technology yet.

Design Rule #3: New technology is not a silver bullet that makes all of your problems go away. All new technologies have integration challenges and often many bugs that have yet to be fixed by the vendor.

Bearing these rules in mind, proceed as follows:

Define the technical assumptions, unknowns, and new technologies being introduced or considered.

Validate any unproven assumptions - i.e. something you or the team has not worked with before. Reading a white paper or vendor press release doesn't count. Make sure it works before building your system on it if its something that hasn't been done before or is being used in new way.

Build a hardcoded throw-away prototype to verify system interactions and unknowns can be solved with a minimum of workarounds. Drop any part of your plan that isn't working and find another way to do it before it becomes a black hole that consumes your architecture.