attributes

data management

slug: step-attributes

Attributes Step

Purpose

Defines, manages, and enhances the Physical Data View (PDV) metadata for datasets, controlling how fields are interpreted, formatted, and displayed. Serves as the central metadata management system for the DAZL pipeline.

When to Use

• Define display formats for columns (date formats, number styles, etc.)
• Add labels and descriptions to technical field names
• Set up special visualization formats (progress bars, ratings, etc.)
• Auto-detect field types from data values
• Apply formatting to data values for display
• Remove unwanted fields from the PDV
• Prepare data for visualizations and reports
• Standardize field definitions across multiple datasets

How It Works

1. Takes the current PDV metadata and enhances it with new field definitions
2. Can automatically detect field types from data if requested
3. Allows removing fields from the PDV
4. Optionally applies formatting to data values using defined formats
5. Maintains a comprehensive metadata layer that other steps can use
6. Integrates with the showFmt system for display formatting

Parameters

Optional

• fields (object) - Field definitions to add or update:
  • Field name as key
  • Definition object as value, containing:
  • type - Data type (text, number, date, etc.)
  • label - Human-readable field label
  • options - Type-specific formatting options
• detect (boolean) - Auto-detect fields from data (default: false)
• apply (boolean) - Apply formatting to data values (default: true)
• remove (array) - Fields to remove from PDV

Field Types and Formatting

The attributes step supports various field types with specialized formatting:

• text - Standard text display
• number - Numeric values with formatting options:
  • format: comma, dollar, percent, words, short
  • decimal: Number of decimal places
• date - Date values with format options
• time - Time values with format options
• checkbox - Boolean values displayed as checkboxes
• progress - Progress bar visualization
• trafficlight - Color-coded status indicators
• rating - Star or icon-based rating display
• email - Email addresses with mailto links
• phone - Phone numbers with tel links
• url - URLs with hyperlinks
• markdown - Markdown-formatted text
• And many other specialized formats (see showFmt function)

Input Requirements

• Any dataset with a PDV structure (or empty PDV to build from scratch)
• Field definitions matching the showFmt formatting capabilities

Output

Data

• Original data, or formatted data if apply is true

PDV

• Enhanced PDV with new or updated field definitions

Extras

• attributes_applied - Timestamp when attributes were applied
• fields_defined - Total count of fields in the PDV

Example Usage

Basic Field Definitions

attributes:
  fields:
    customer_id:
      type: text
      label: Customer ID

    revenue:
      type: number
      label: Total Revenue
      options:
        format: dollar
        decimal: 2

    last_purchase:
      type: date
      label: Last Purchase Date
      options:
        format: Y-m-d

Auto-Detection with Custom Overrides

attributes:
  detect: true
  fields:
    satisfaction:
      type: rating
      label: Customer Satisfaction
      options:
        icon: star
        max: 5
        color: #FFD700

    completion:
      type: progress
      label: Project Completion
      options:
        min: 0
        max: 100

Formatting Application

attributes:
  fields:
    status:
      type: trafficlight
      label: Account Status
      options:
        min: 0
        max: 100
        threshold1: 0.33
        threshold2: 0.66
        color1: #FF0000
        color2: #FFFF00
        color3: #00FF00

  apply: true

Related Documentation

• showFmt-function - Display formatting implementation details
• PDV-structure - Physical Data View metadata structure
• print step - How formatted data is displayed in tables
• format-options - Complete list of formatting options for each field type