Data Orchestrator#

The Data Orchestrator provides access to a dashboard to view and configure the data orchestrations currently in use by the system. The Data Orchestrator is a set of tools that define functions that should be applied on a given dataset, applying transformation workflows and producing assets that can be read into a common storage space, while using an external orchestrator designed for developing and maintaining data assets such as tables, datasets, machine learning models and reports.

This is currently achieved by using Dagster ⧉, an orchestrator tailored to streamline the entire data development lifecycle, supporting workflows from local experimentation and unit testing to integration testing, staging, and production deployment.

You can see the execution of each job and the progress it is currently showing:

You can also see, among others, the assets produced by the orchestrator: