Simulators are often used in place of real products and are designed to behave like these products.
However, they present problems that slow testing progress and entail significant costs.
Imagine being the provider of one of these subsystems, having to simulate the dependencies…
Using simulators adds tremendous complexity to a testing environment. The simulator must not only be part of the environment, it must be completely integrated in the environment. Before testing can even begin, numerous issues must be dealt with, including:
When setting up a new lab the simulator must be automatically installed and configured
Version control must be established for the simulator. When does the simulator need to be updated? Can the update be automated for all users?
How stable is the simulator? Will it run consistently, or will it need to be restarted frequently?
Increasingly complex signaling requires well-designed test cases and careful setup of simulators. Often, simulators simply don't have the means to handle intricate testing scenarios - and tailoring a solution will require far too much time and too many resources, if this can be done at all.
Every time you add new functions to your product, the simulator must be updated to support them. What may seem at first like a relatively small addition can cause havoc in schedules and budgets, and seriously hamper product development.
Configuration complexities mean that the simulator will probably be difficult to control during test execution.
Most simulators will not allow you to do all the negative testing you need.
Most simulators will have difficulty in accurately simulating timing problems.