Editing Step Functions (section)

== Error Handling ==

The error handling in the '''Indici-State-Machine''' is designed to ensure robust processing and debugging of Indici files. Below are the key mechanisms used:

=== Retries ===
* '''Purpose:''' Automatically retries tasks to handle transient errors.
* '''Where Implemented:''' 
  * Copy Indici Files
  * Load Data into indici_staging Table
  * Deduplicate Indici Data
* '''Retry Rules:''' 
  * Errors handled: `Lambda.ServiceException`, `Lambda.AWSLambdaException`, `Lambda.SdkClientException`, `Lambda.TooManyRequestsException`
  * Retry mechanism:
    * Interval between retries starts at 1 second.
    * Maximum of 3 attempts.

=== Error Catching ===
* '''Purpose:''' Captures errors to enable conditional handling and logging.
* '''Where Implemented:''' 
  * Each task contains a `Catch` block to handle `States.ALL` errors.
* '''Error Storage:''' Errors are saved under `$.ErrorInfo`.

=== Error Logging ===
* '''Purpose:''' Logs errors for debugging and analysis.
* '''LogGroup:''' All logs are stored in the '''stateMachine''' LogGroup, with the following streams:
  * '''ErrorLogs:''' Captures general errors during state machine execution.
  * '''SuccessLogs:''' Logs the successful completion of tasks.
  * '''EmptyFilesLogs:''' Logs specific errors when a file is empty.

* '''Lambda Function Used for Logging:''' `stateMachine_LogHandler`  
* '''Write Empty File Error Log:''' 
  * Triggered if the error cause matches `'''File is Empty!'''`.
  * Logs to the '''EmptyFilesLogs''' stream in the '''stateMachine''' LogGroup.
  * After logging, the state machine transitions to the '''Success State'''.
* '''Write General Error Log:''' 
  * Logs all other errors to the '''ErrorLogs''' stream in the '''stateMachine''' LogGroup.
  * Error details are extracted from `$.ErrorInfo.Cause`.

=== Failure Handling ===
* '''Fail State:''' 
  * Indicates that the state machine could not complete successfully.
  * Includes a generic error name ("StateMachineError") and the specific cause.

=== Error-Specific Decision Making ===
* '''Check Error Type State:''' 
  * Inspects `$.ErrorInfo.Cause` to determine error type.
  * Routes execution to:
      '''File is Empty''': If StringMatches *File is Empty!*, transitions to '''''Write Empty File Error Log'''''.
      '''Missing sql_dd attribute''': If StringMatches *No sql_dd attribute found*, transitions to '''''No sql_dd Data Load'''''.
      '''Default''': All other errors go to '''''Write General Error Log'''''.

=== Flow for Handling Errors ===
# Each task attempts execution.
# If a transient error occurs, retries are triggered.
# Errors are caught and stored in `$.ErrorInfo`.
# The `Check Error Type` state evaluates the error and routes to the appropriate logging step.
# Errors are logged in CloudWatch under appropriate streams within the '''stateMachine''' LogGroup using the `stateMachine_LogHandler` Lambda function.
# If an empty file error occurs, the state machine logs it and transitions to the '''Success State'''. For all other errors, it transitions to the '''Fail State'''.