Some of the most dangerous PLC behaviour appears after the original fault has already occurred.
A motor trips. Communication disappears. Control power fails. A drive reports a fault, or the safety system removes permission to operate.
The equipment stops.
At that moment, the control system must decide what happens next.
Without structured recovery logic, old commands may remain stored, outputs may return when permissives recover, and an interrupted sequence may continue from a state that no longer matches the physical machine.
The fault itself may be handled correctly.
The recovery may not be.
Why Fault Recovery Is So Difficult
Normal operation usually follows an expected sequence:
Ready
↓
Start request
↓
Startup validation
↓
Equipment starts
↓
Normal operationFault recovery begins from a less predictable condition.
After a trip or interruption:
- Actuators may no longer be in their expected positions.
- Pneumatic pressure may have decayed.
- Drives may have lost their internal state.
- Remote I/O may still be reconnecting.
- Analog values may be stale or invalid.
- Product may remain inside the machine.
- Internal sequence latches may still be active.
- Operators may have moved equipment manually.
The PLC cannot safely assume that the condition before the fault still exists afterward.
A recovered machine should therefore behave like a controlled startup—not an uncontrolled continuation.
A Safe Recovery Structure
A professional recovery sequence can follow this structure:
Fault detected
↓
Enter faulted state
↓
Remove or control commands
↓
Confirm fault cause is gone
↓
Validate reset request
↓
Recheck communication and permissives
↓
Enter recovery state
↓
Verify physical machine condition
↓
Return to ready state
↓
Require normal startupEach stage has a specific purpose.
Skipping one stage may allow the machine to move before the controller has confirmed that recovery is complete.
The Faulted State
When a fault is detected, the sequence should enter a clearly defined fault state.
For example:
Machine_State := FAULTED;The faulted state should define:
- Which output requests are removed
- Which outputs may remain active
- Whether a controlled stop must finish
- Which fault information is retained
- Whether manual recovery is permitted
- What conditions are required before reset
- What state follows a successful reset
It should not simply clear random latches throughout the program.
A structured fault state provides one authoritative description of the machine’s condition.
Not Every Fault Requires the Same Response
Faults can require different stopping and recovery behaviour.
Possible categories include:
Immediate stop faults
These require rapid removal of ordinary control commands.
Examples may include:
- Motor overload
- Drive trip
- Critical pressure loss
- Mechanical jam
Controlled shutdown faults
Some processes need an orderly stop.
The machine may need to:
- Stop material feeding
- Clear a conveyor
- Reduce speed
- Close a valve
- Complete cooling or purge operation
Communication faults
The response depends on which data was lost and how important it is.
A communication failure may require:
- Immediate stop
- Controlled shutdown
- Blocking new starts
- Holding a limited operating state temporarily
Safety-system stops
Emergency-stop and guard-related functions must be handled through the required safety-rated architecture.
The standard PLC may receive status from the safety system and coordinate normal recovery, but ordinary PLC logic must not replace the safety function.
Fault Detection Should Be Latched Clearly
Some faults disappear as soon as the output turns off.
For example:
- Motor runs.
- Pressure falls.
- Low-pressure fault stops the motor.
- Pressure immediately recovers.
- The original fault condition disappears.
Without fault memory, maintenance personnel may arrive and find no active cause.
A fault latch preserves the event:
IF Low_Pressure_Condition THEN
Low_Pressure_Fault := TRUE;
END_IF;The latch should clear only when the required reset conditions are satisfied.
IF Low_Pressure_Condition THEN
Low_Pressure_Fault := TRUE;
ELSIF Reset_Request
AND Pressure_Recovery_Valid THEN
Low_Pressure_Fault := FALSE;
END_IF;The active fault has priority over reset.
Reset Is Not Fault Recovery
Pressing RESET should not automatically make the machine ready to run.
A reset request usually means:
The operator acknowledges the fault and requests that the control system evaluate recovery.
It does not prove:
- The physical cause has disappeared
- Communication data is fresh
- Actuators are correctly positioned
- Process pressure has recovered
- Old commands have been cleared
- Automatic operation is safe
A strong sequence separates:
Fault acknowledgement
Fault clearing
Recovery validation
Ready state
Start commandThese are different actions.
Reset Must Not Mean Start
Combining reset and restart creates dangerous operator expectations.
The operator may press RESET only to clear an alarm, yet a retained run request can immediately become active.
A safer sequence is:
Fault condition removed
↓
Operator presses RESET
↓
Reset conditions checked
↓
Machine enters RECOVERY
↓
Recovery completed
↓
Machine enters READY
↓
Fresh START request requiredThe RESET button restores eligibility for operation.
The START button requests operation.
Validate the Reset Request
A reset should be accepted only when the original fault and required recovery conditions are healthy.
For example:
Reset_Valid :=
Reset_Request
AND NOT Fault_Condition
AND Safety_System_Healthy
AND Communication_Healthy
AND Device_Data_Valid;Where appropriate, the conditions should remain stable for a defined time.
Recovery_Conditions continuously healthy
for validation period
→ Reset permittedThis prevents a fault from clearing during a one-scan signal flicker.
Never Allow Reset to Override an Active Fault
Weak logic may use separate set and reset instructions:
Fault condition → SET Fault_Latched
Reset button → RESET Fault_LatchedIf both are active, the final result may depend on execution order.
A more explicit structure gives the fault priority:
IF Fault_Condition THEN
Fault_Latched := TRUE;
ELSIF Reset_Request
AND Recovery_Conditions_Valid THEN
Fault_Latched := FALSE;
END_IF;The system cannot be reset while the fault remains present.
Retained Commands and Unexpected Motion
Retentive memory can preserve values through power loss or CPU restart.
This is useful for:
- Production totals
- Recipes
- Setpoints
- Calibration data
It is dangerous when applied carelessly to:
- Motor run commands
- Valve movement requests
- Automatic-cycle latches
- Temporary bypasses
- Sequence-complete bits
Consider this sequence:
Motor running
↓
Power failure
↓
Output turns off
↓
Retained run latch remains true
↓
Power returns
↓
Drive becomes ready
↓
Motor starts unexpectedlyThe PLC may be executing exactly as programmed.
The unsafe behaviour comes from an undefined recovery strategy.
Movement requests should normally return to a defined safe condition after startup unless automatic restart has been specifically assessed and designed.
Clear Requests Without Destroying Diagnostics
At fault entry, the program may need to clear:
- Pending start requests
- Automatic run requests
- Temporary manual commands
- Sequence transition requests
- Maintenance overrides
However, diagnostic information should remain available.
Preserve:
- First fault
- Previous machine state
- Active sequence step
- Output commands
- Field feedback
- Process measurements
- Communication status
- Timestamp
Clearing every internal bit immediately may remove the evidence needed to understand the event.
Do Not Resume From an Unknown Sequence State
A retained sequence state may no longer match the physical equipment after a fault.
For example, the PLC may remember:
State = FILLING_TANKDuring the interruption:
- The inlet valve closes through its fail-safe action.
- Product continues draining.
- The operator manually moves another valve.
- The level signal becomes invalid.
- Pneumatic pressure decays.
Resuming directly from FILLING_TANK would assume that all required physical conditions remain unchanged.
A stronger recovery sequence compares stored state with actual feedback.
When the condition cannot be validated, the machine should enter:
RECOVERY_REQUIREDrather than continuing automatically.
A Dedicated Recovery State
Recovery should be represented as an explicit operating state.
For example:
FAULTED
↓
WAITING_FOR_RESET
↓
RECOVERY_REQUIRED
↓
VALIDATING_POSITIONS
↓
READYDuring recovery, the program may:
- Keep automatic outputs blocked
- Re-establish communication
- Confirm fresh analog data
- Validate actuator positions
- Restore utilities
- Remove incomplete product
- Permit controlled manual movement
- Rehome motion systems
- Require operator confirmation
The exact actions depend on the machine.
The important point is that recovery is a controlled phase—not a shortcut back into production.
Manual Recovery Must Be Restricted
Some faults require maintenance personnel to move equipment manually.
Manual recovery may allow:
- Jogging a conveyor
- Opening or closing a valve
- Returning a cylinder home
- Clearing material
- Repositioning an axis
These actions should be restricted by:
- Selected maintenance mode
- Appropriate authorization
- Hold-to-run controls where required
- Valid interlocks
- Clear HMI indication
- Reduced speed or limited movement where applicable
Manual movement should not silently advance the automatic sequence.
After recovery movement is complete, the PLC should verify the resulting machine state before returning to READY.
Revalidate Every Important Permissive
Recovery logic should not reuse the permissive result that existed before the fault.
Each important condition should be checked again.
Examples include:
- Safety system healthy
- Drive communication established
- Drive ready for operation
- Remote I/O data valid
- Air pressure stable
- Valve positions confirmed
- Cooling flow available
- Analog values current and plausible
- Downstream equipment ready
For networked values, validate both the value and its quality:
Remote_Pressure_Permissive :=
Remote_Communication_Healthy
AND Remote_Data_Valid
AND Remote_Pressure > Minimum_Pressure;A retained healthy value is not proof that the current process condition is healthy.
Communication Recovery Is Not Immediate Readiness
A device may become connected before its process data is usable.
A typical sequence can be:
- Network connection returns.
- Device appears online.
- Old or default data becomes visible.
- Device completes initialization.
- Fresh cyclic data begins updating.
- Ready status becomes valid.
Recovery logic should distinguish between:
Device_Connected
Device_Communicating
Device_Data_Valid
Device_ReadyTreating the first connection bit as complete readiness can create false permissives and unsafe startup transitions.
Analog Recovery Requires Fresh Data
Analog values also require validation after recovery.
A filter may contain:
- An old retained value
- Zero after initialization
- A default substitute value
- A slowly recovering measurement
Before using the value, confirm:
- Channel healthy
- Signal inside valid electrical range
- Fresh input received
- Scaling configuration valid
- Filter initialized
- Process value plausible
- Required stability period completed
For example:
Pressure_Recovery_Valid :=
Pressure_Channel_Healthy
AND Pressure_Data_Fresh
AND Pressure_Signal_Valid
AND Pressure_Filter_Initialized
AND Pressure_Stable;Recovery Timeouts
Every recovery action should have a defined timeout.
Examples include:
- Drive communication must return within 10 seconds.
- Valve must reach its recovery position within 5 seconds.
- Pressure must recover within 30 seconds.
- Axis homing must complete within 60 seconds.
Without timeouts, the machine may remain in recovery indefinitely.
A useful structure is:
IF Recovery_Condition_Complete THEN
Next_State := READY;
ELSIF Recovery_Timeout THEN
Recovery_Fault := TRUE;
Next_State := FAULTED;
END_IF;The timeout alarm should identify the exact failed action.
Avoid generic messages such as RECOVERY FAILED when the PLC knows that a specific valve or drive did not respond.
Prevent Recovery Loops
Poorly designed logic can repeatedly switch between fault and recovery states.
For example:
- Fault clears.
- RESET sends the machine to recovery.
- One unstable permissive drops.
- Machine returns to fault.
- Permissive returns.
- RESET remains active.
- Machine enters recovery again.
This creates repeated transitions and confusing alarms.
Prevent this by:
- Requiring a new reset edge
- Validating conditions before leaving the faulted state
- Using hysteresis or stabilization where appropriate
- Clearing pending reset requests
- Recording the reason recovery failed
A held RESET button should not repeatedly attempt recovery without deliberate design.
Fault Priority During Recovery
A new fault may occur while recovery is already in progress.
The logic should define priority clearly:
IF Critical_Stop THEN
Next_State := SAFE_STOPPED;
ELSIF New_Fault_Active THEN
Next_State := FAULTED;
ELSIF Recovery_Timeout THEN
Next_State := FAULTED;
ELSIF Recovery_Complete THEN
Next_State := READY;
END_IF;Normal recovery completion must not override a newly detected fault in the same scan.
Separate Fault Cause From Machine State
The machine state and fault code should be separate.
For example:
Machine_State = FAULTED
Fault_Code = DRIVE_2_COMMUNICATION_LOSTThe state describes what the machine is doing.
The fault code explains why it entered that state.
Useful records include:
Current_State
Previous_State
First_Fault_Code
Current_Faults
Fault_Time
Faulted_Sequence_Step
Recovery_Failure_CodeThis structure allows the machine to remain in one faulted state while preserving detailed diagnostics.
Record the First Fault
One failure often creates several secondary alarms.
For example:
- Drive communication fails.
- Motor stops.
- Flow disappears.
- Pressure falls.
- Downstream equipment stops.
- Sequence timeout activates.
By the time maintenance arrives, five alarms are active.
The first fault was the communication loss.
A first-out recorder should capture:
- Original fault
- Time
- Machine state
- Output request
- Process values
- Communication status
- Operator mode
This prevents troubleshooting from beginning with secondary consequences.
Controlled Recovery Example
A simplified state-machine structure could look like this:
CASE Machine_State OF
RUNNING:
IF Critical_Fault THEN
Previous_State := RUNNING;
Machine_State := FAULTED;
ELSIF Stop_Request THEN
Machine_State := STOPPING;
END_IF;
FAULTED:
Motor_Run_Request := FALSE;
Automatic_Mode_Allowed := FALSE;
IF Fault_Cause_Removed
AND Reset_Rising_Edge
AND Reset_Conditions_Valid THEN
Machine_State := RECOVERY_REQUIRED;
END_IF;
RECOVERY_REQUIRED:
Motor_Run_Request := FALSE;
Automatic_Mode_Allowed := FALSE;
IF New_Fault THEN
Machine_State := FAULTED;
ELSIF Recovery_Timeout THEN
Recovery_Fault := TRUE;
Machine_State := FAULTED;
ELSIF Communication_Validated
AND Positions_Validated
AND Process_Data_Validated THEN
Machine_State := READY;
END_IF;
READY:
IF Fresh_Start_Request
AND Start_Permissives_Validated THEN
Machine_State := STARTING;
END_IF;
END_CASE;The recovered machine returns to READY rather than directly to RUNNING.
Output Ownership During Faults
Fault logic should not write physical outputs from several unrelated locations.
Instead, the sequence generates requests, and one device-control block owns the final command.
For example:
Motor_Run_Request :=
Machine_State = RUNNING;Then:
Motor_Command :=
Motor_Run_Request
AND Motor_Run_Permissive
AND NOT Motor_Fault
AND NOT Stop_Required;Finally:
Physical_Motor_Output := Motor_Command;This prevents the recovery routine, manual routine and automatic sequence from competing for the same output.
Emergency-Stop Recovery
Resetting an emergency-stop device or safety controller must not automatically restart ordinary machine movement.
A typical control sequence may require:
Emergency-stop released
↓
Safety system reset
↓
Safety status healthy
↓
Standard PLC confirms machine recovery
↓
Operator issues normal START commandThe exact behaviour depends on the safety design and machine risk assessment.
The standard PLC should treat restored safety permission as one recovery condition—not as a start command.
Power-Recovery Behaviour
Power restoration should enter a defined startup or recovery state.
The program should decide:
- Which values remain retained
- Which commands are cleared
- Whether the previous state is trusted
- Whether field positions must be checked
- Whether automatic operation is blocked
- Whether operator reset is required
A common safe approach is:
On PLC startup:
Clear movement requests
Clear temporary bypasses
Block automatic mode
Enter INITIALIZING
Validate I/O and communication
Validate process data
Check machine positions
Enter READY_FOR_RESETAutomatic restart may be appropriate in some continuous processes, but it must be deliberately engineered and supported by the relevant risk assessment.
Bypasses Must Not Short-Circuit Recovery
A maintenance bypass may allow normal permissive logic to appear healthy.
For example:
Valve_Permitted :=
Valve_Open_Feedback
OR Valve_Bypass;During recovery, this can hide the fact that the valve position is unknown.
Recovery should preserve the actual condition:
Valve_Feedback_Healthy := Valve_Open_Feedback;
Valve_Bypass_Active := Valve_Bypass;The machine may block automatic recovery while a critical bypass remains active.
A bypass should never allow the controller to pretend that physical validation has succeeded.
HMI Recovery Information
A useful recovery screen should show:
- Current machine state
- Original fault
- Active faults
- Conditions blocking reset
- Conditions blocking recovery
- Required manual actions
- Active bypasses
- Communication status
- Field-position status
- Recovery timer
- Whether a fresh START will be required
Avoid a generic message such as:
Machine not ready
A better message is:
Recovery blocked: discharge valve not confirmed closed.
Clear information reduces repeated reset attempts and unnecessary bypassing.
Recovery Tests That Should Be Performed
Commissioning should deliberately test:
- Motor trip while running
- Drive fault during startup
- Communication loss during a transition
- Remote I/O reconnecting late
- Power failure while outputs are active
- PLC restart with retained sequence data
- Emergency-stop activation during automatic operation
- Safety reset while process permissives remain invalid
- RESET held while a fault clears
- Recovery timeout
- Manual actuator movement during fault
- Analog data returning slowly
- Bypass active during recovery
- New fault appearing during recovery
- Output feedback not returning after reset
For every test, confirm:
- The machine enters the correct fault state
- Output requests are controlled correctly
- The first fault is preserved
- Reset is rejected when conditions are invalid
- Recovery does not skip startup validation
- Old run requests do not restart equipment
- A fresh START is required where intended
- HMI diagnostics explain the blocked condition
Final Thoughts
Fault recovery should never be treated as the removal of an alarm bit.
A fault can leave the physical machine, internal sequence and process data in different states. Simply clearing the fault may allow old commands to become active before the system has confirmed that recovery is complete.
Professional recovery logic should define:
- Fault states
- Output behaviour
- Reset requirements
- Permissive revalidation
- Communication recovery
- Physical-position checks
- Timeouts
- Manual recovery actions
- Safe return to startup
The recovered machine should not continue blindly from where it stopped.
It should return through a controlled path, confirm that the process is trustworthy and require the appropriate deliberate command before normal operation resumes.
A safe recovery behaves like a structured startup—not an uncontrolled continuation.
