output command

data management

slug: reference-output-command

Output Command

Purpose

Manages the routing and storage of processing results from DAZL steps, providing a unified mechanism to direct output to various destinations. Acts as the central output management system for the entire DAZL pipeline.

When to Use

• Specify where step results should be stored or sent
• Name datasets for later reference in the pipeline
• Direct output to different storage mechanisms
• Create named reports in the HTML environment
• Save data to external systems like databases or APIs
• Control the persistence and accessibility of step results
• Create complex workflows with multiple output destinations

How It Works

1. Each DAZL step can include an output parameter to specify its output destination
2. The output command processes this parameter and routes the result appropriately
3. Handles multiple output destination types through a consistent interface
4. Supports both simple string references and complex configuration objects
5. Automatically determines appropriate destinations based on result type
6. Generates default names when no output is explicitly specified

Output Specification

Simple Reference (String)

output: customerSegments  # Stores in $work['customerSegments']

Full Configuration (Object)

output:
  target: target_identifier
  type: array|html|sql|api
  options: {}  # Type-specific options

Supported Output Destinations

Work Arrays (type: array)

• Target: Name to store result in the DAZL work environment
• Options: None required
• Default behavior: Used for data-oriented step results
• Example:

  output:
  target: filteredCustomers
  type: array

HTML Reports (type: html)

• Target: Name to store in the HTML report collection
• Options: None required
• Default behavior: Used for visualization-oriented step results
• Example:

  output:
  target: customerDashboard
  type: html

SQL Storage (type: sql)

• Target: Table name or stored procedure
• Options:
  • connection: Database connection identifier
  • mode: insert, update, upsert, or replace
  • key_columns: Columns used for updates/upserts
  • batch_size: Number of records per batch operation
• Example:

  output:
  target: customer_segments
  type: sql
  options:
    connection: analytics_db
    mode: upsert
    key_columns: [customer_id]

API Endpoints (type: api)

• Target: API endpoint URL or endpoint identifier
• Options:
  • method: HTTP method (POST, PUT, etc.)
  • headers: HTTP headers
  • transform: Transformation function for output data
  • auth: Authentication details
• Example:

  output:
  target: "https://api.example.com/v1/insights"
  type: api
  options:
    method: POST
    headers:
      Content-Type: application/json
      Authorization: "Bearer ${API_TOKEN}"

Automatic Output Type Detection

The output manager uses the step's outputType property to determine the appropriate destination:

• outputType: 'work' → Routes to work arrays
• outputType: 'html' → Routes to HTML reports
• Other output types are handled according to explicit configuration

Default Naming

When no output is specified, the system generates names automatically:

• For work outputs: _result_N (where N is a sequential number)
• For HTML outputs: _report_N (where N is a sequential number)

Example Usage

In a Filter Step with Work Output

filter:
  dataset: salesData
  where: "region = 'North'"
  output: northRegionSales

In a Chart Step with HTML Output

chart:
  dataset: productSales
  type: bar
  x_axis: product
  y_axis: revenue
  output:
    target: productPerformance
    type: html

In a Calculate Step with SQL Output

calculate:
  dataset: customerMetrics
  assign:
    lifetime_value: "total_purchases * avg_order_value * 1.5"
    churn_risk: "days_since_last_order > 90 ? 'high' : 'low'"
  output:
    target: customer_metrics
    type: sql
    options:
      connection: crm_db
      mode: update
      key_columns: [customer_id]

Related Documentation

• work-environment - How datasets are stored in DAZL
• html-reports - Creating and managing HTML reports
• sql-connection-configuration - Setting up database connections for SQL output
• api-configuration - Configuring API endpoints for data delivery
• generateWorkArrayName-function - Algorithm for automatic work array naming