Stages

All pipeline stages include the following:

• A required Name field
• An optional Stage Description field, which can be used to describe the stage’s purpose or inform LLM tools

See below for additional information for each pipeline stage type:

• Transform
  • JSONata
  • JavaScript
• Common
  • Breakup
  • Filter
  • Flatten
  • Model
  • Model Validation
  • Size Buffer
  • Timed Buffer
• Control
  • Delay
  • Error
  • For Each
  • Return
  • While
  • On Change
  • Switch
  • Subpipeline
• Parse
  • CSV
  • JSON
  • Parquet
  • XML
• Format
  • CSV
  • GZip
  • JSON
  • Parquet
  • XML
  • Zip
• I/O
  • Read
  • Merge Read
  • Smart Query
  • Write
  • Write New