Release It! - Chapter 5.7 Test Harness

It is really difficult to get Integration Points to fail in the ways you need them to when testing.  Especially, when you are trying to test against abnormally, should never happen, conditions.  A Test Harness should be devious and attempt to leave scars on the calling system. It substitutes for any Integration Point services.  The author provides a whole set of network related scenarios that the Test Harness should support, such as setting up a socket connection but never sending any data.  A Test Harness runs as its own server and is free to do whatever it needs to in order to simulate the numerous ways a service can break. Don't be tempted to add failure simulation code to your application. One idea is to have the Test Harness listen on different ports for different bad behavior -- port 80 might be the "send back one packet of data every 30 seconds" port while 22 might be the "accept connections but never reply" port.  It allows for reuse of a test harness. You probably want your Test Harness to log what it is doing in case that app you are testing dies silently.  The Test Harness can be as flexible and pluggable as you want and is likely a good use of company resources.

• emulate out-of-spec failures - try and fake all of the messy, real-world failures that probably haven't been accounted for in your APIs.
• stress the caller - throw slow responses, no responses or garbage and see how your application behaves.
• leverage shared harnesses for common failures -- you don't need a seperate harness for each application because many of the failure modes you'll be testing for apply to many types of applications.
• supplement, don't replace, other testing methods - unit, integration
  and functional test verify functional behavior.  The Test Harness verify
  non-functional behavior while maintaining isolation from the remote
  systems.

I wonder if open source test harnesses exists?  It would be nice not to have to start from scratch but then again, a harness is going to be specific to your application.