Read

The Read Stage can be used to easily read a single source. For more complex read operations, use the Merge Read Stage.

Reference Parameters

If a reference that supports parameters is used in this stage the parameters can be dynamically driven using the incoming event with the following syntax: {{event.value}}

This syntax can be combined with static values, for instance index_{{event.metadata.breakupIndex}} if the breakup index was “10”, this would result in “index_10”.

For more information on using the event variable see: Using Transform Expressions (Note that event is the only variable is accessible from this stage)

Success / Failure

The Read Stage provides alternative paths for successful and failed reads. This allows for pipelines to have special handling for read failures.

A failed read occurs if the source fails to read, and a single event is sent to the Failure path. The event includes the original event being read, as well as a readResults metadata value that contains the status of the read. The metadata looks as follows:

{
    reference: string   // the source reference name
    quality: string     // good or bad for success/failure
    reason: string      // the error message for the failed read
}

If the Failure path is not connected, the stage produces an error. If the source is successfully read, the event takes the Success path.