Data Processing Model
(Redirected from data processing workflow)
Jump to navigation
Jump to search
A Data Processing Model is a process model that is a systematic pattern for performing data processing tasks on data items.
- Context:
- It can (typically) involve Data Input Handling and Data Output Generation.
- It can (often) require Data Quality Control and Process Monitoring.
- ...
- It can range from being a Manual Data Processing Workflow to being a Semi-Automated Data Processing Workflow to being an Automated Data Processing Workflow.
- It can range from being a Simple Data Processing Workflow (e.g., single transformation) to being a Complex Data Processing Workflow (e.g., multi-stage pipeline).
- It can range from being a Sequential Data Processing Workflow to being a Parallel Data Processing Workflow.
- ...
- It can be managed by a Data Processing System.
- It can include data sequences, data conditions, and data branches.
- It can involve:
- Data Processing Tasks: Individual transformation steps.
- Data Dependencies: Input/output relationships between steps.
- Processing Conditions: Rules determining flow control.
- Data Quality Checks: Validation steps.
- Error Handling Steps: Recovery and correction procedures.
- ...
- Example(s):
- ETL Workflows, such as: Data Extraction Workflow or Data Loading Workflow.
- Data Cleaning Workflows, such as: Data Normalization or Duplicate Removal.
- Data Transform Workflows, such as: Format Conversion or Data Aggregation.
- Data Integration Workflows, such as: Data Merging or Data Consolidation.
- Data Analysis Workflows, such as: Statistical Analysis or Pattern Detection.
- ...
- Counter-Example(s):
- Data Collection Workflows, which gather rather than process data.
- Data Storage Workflows, which archive rather than transform data.
- Data Visualization Workflows, which display rather than process data.
- Data Annotation Workflows, which label rather than transform data.
- See: Data Pipeline, Process Management, Data Engineering.