An exercise needs to represent an industrial environment, but the original software cannot be made available. The commissioning team still wants participants to investigate events, make decisions and defend the systems in the scenario.
The next decision is not simply which substitute product to install. First, establish which functions participants need and which observations will support a meaningful exercise. That gives the engineering team a bounded problem to solve and the evaluator a clear account of what the result can demonstrate.
This is particularly important for operational technology, or OT: systems concerned with monitoring or controlling physical operations. An exercise may represent a selected part of that operation. Its scope should say exactly what is implemented and what is assumed.
An anonymised example from an expert's previous experience
This example comes from an expert's previous experience. It was not an ObsidianCorps engagement.
The expert contributed story development, architecture, attack facilitation and evaluation to an exercise involving government and energy-sector contexts, including SCADA and OT systems. Licences for the original software were unavailable. Custom software was built to reproduce selected functions and logs needed for the exercise.
That is the confirmed account. It does not establish that the software was a complete replica or a validated digital twin. It also does not establish participant outcomes, performance improvements or the accuracy of any physical-process model. No such results were supplied in the interview.
The useful lesson concerns an engineering choice: decide which behaviour the exercise requires, then choose how to provide it within the available software and resources. The remaining sections propose a method for making that choice. They do not add undocumented details to the historical example.
Start with the participant task, not the product interface
An application screen may look familiar while providing none of the behaviour required for a technical task. Conversely, a simplified implementation may allow the relevant action and generate the evidence needed to discuss it.
Write down what participants must do. Must they notice a state change, investigate access to a service, interpret a sequence of events or decide whether a containment action is appropriate? These are illustrative questions, not the confirmed tasks from the previous exercise.
For each task, identify three requirements: the function participants interact with, the evidence they can observe and the consequence that helps them understand their action. Then ask whether a simplified implementation preserves those relationships.
For example, a hypothetical investigation may require participants to relate an account action to a changed application state. If the exercise supplies only a screenshot announcing the change, it can prompt a discussion. If the objective is to demonstrate an investigation using system evidence, the environment needs appropriate observable behaviour as well.
Describe fidelity by function
“High fidelity” is too broad to be an acceptance criterion on its own. A simulation might represent selected application functions, selected log events and a simplified operational consequence at different levels of detail.
Use a function-by-function description. It should be possible for the commissioner, developer and evaluator to agree on what each part does without assuming that the whole original system has been reproduced.
| Question |
What to record in a proposed design brief |
| What can the participant do? |
The selected action or function |
| What changes as a result? |
Implemented state or represented consequence |
| What can the participant observe? |
Relevant logs, interface behaviour or facilitator information |
| What has been simplified? |
Missing features and deliberate assumptions |
| What cannot be concluded? |
Limits on technical or operational interpretation |
This is a recommended worksheet. The actual functions and log fields in the expert's previous exercise have not been specified, so the table should not be filled with invented historical details.
Compare the available implementation choices
Software availability is one constraint among several. The selected approach must also fit the exercise objective, build effort and observable evidence. Consider the choices before committing to a detailed technical scenario.
| Approach |
When to investigate it |
Questions that still need answering |
| Original software available for lawful exercise use |
The required task depends on its specific behaviour |
Are the necessary features and use rights available in the exercise environment? |
| Another available application |
A comparable selected function may serve the objective |
Do differences change the participant's task or the meaning of the evidence? |
| Bounded custom software |
A defined set of functions and logs is needed |
Can those functions be implemented, checked and maintained within scope? |
| Exercise material or a facilitator |
The objective concerns reasoning about a represented event |
Is a discussion sufficient, or would this remove a required practical task? |
This table recommends design questions rather than a universal preference. More code does not automatically produce a better exercise. A simpler representation may be appropriate when the objective is a decision; an implemented function may be essential when the objective is practical defence.
Where custom implementation is appropriate, treat it as software engineering with a defined scope. Specify the required behaviour and acceptance checks, rather than describing the deliverable as a replacement for the unavailable product. Software-use rights and any permissions for reference material must be established for the actual project.
Build observations that support the objective
An exercise can produce many logs without giving participants useful evidence. Begin with the reasoning they need to perform. Which event indicates an action? What lets them distinguish one explanation from another? What observation supports or contradicts their proposed conclusion?
Recommend checking consistency between the implemented function and the available records. If an action changes a state, the evidence should reflect the behaviour the exercise claims to expose. If some events are deliberately invisible, identify that limitation in the exercise design.
Do not infer that custom logs have the same semantics as the original product's logs. If exact semantics matter to the task, that is a requirement to investigate and substantiate. Otherwise, explain that the exercise uses selected representative behaviour.
The wider cyber range environment design also matters. Accounts, network paths, services, story events and evidence need to agree. A carefully built substitute function will not rescue a scenario whose other components contradict it.
Keep physical consequences and technical observations distinct
An exercise may describe an operational consequence that the software does not model. That can be useful for exploring decisions, provided the distinction is clear to the exercise team and reflected in evaluation.
For example, a facilitator might describe the operational significance of a service becoming unavailable. If that consequence is an assumption, record it as an assumption. Do not later report that the software validated the behaviour of the physical operation.
The same discipline applies to defensive actions. A participant may demonstrate that a selected access control changed the behaviour of the exercise application. This does not by itself establish the effect of the same action across a complete industrial environment.
If the required objective depends on behaviour that the available simulation cannot represent, change the scope or investigate a more appropriate testing arrangement. The exercise design should expose that mismatch before participants arrive.
Rehearse and document the limits
As a recommended acceptance step, run the intended participant task against the actual exercise build. Confirm that the selected function works, the expected observations are available and the consequence is consistent with the story.
Then test the interpretation. Ask an evaluator who did not write the implementation to explain what the evidence supports and what remains uncertain. This can reveal assumptions that were obvious to the developer but unavailable to participants.
Record the simplified functions, unavailable features, represented consequences and known limitations in a short scope note. Include the note in the evaluation context so a successful task is not turned into a broader claim about competence or operational readiness than the evidence supports.
The expert's preference for learning over winning is useful here. A bounded simulation can create a purposeful challenge if it gives participants the right decisions and observations. Difficulty alone does not establish quality, and lack of access to the original software should not become an unexplained obstacle to learning.
ObsidianCorps offers cyber range design and technical development expertise across the exercise lifecycle. Tell us which functions your OT exercise needs, what software is available and what participants should demonstrate. Contact us.
