scheduler

Description

Allow custom executors subclassed from existing (CeleryExecutor, CeleryKubernetesExecutor, etc.) to use default CLI commands to start workers or flower monitoring.

Use case / motivation

Currently, users who decide to roll their own custom Celery-based executor cannot use default commands (i.e. airflow celery worker) even though it's built on top of existing CeleryExe

Issue from the Dagster Slack

[dagster_shell] defer environ copy to solid run time

This issue was generated from the slack conversation at: https://dagster.slack.com/archives/C01U954MEER/p1623181007055200?thread_ts=1623181007.055200&cid=

there is the following example:
// runs./jobs/worker-8.jsat midnight (once) { name: 'worker-8', timeout: 'at 12:00 am' },

in my usecase i am running a job every 30 minutes that queries an api and dynamically adds jobs to be executed once.
My question is what happenes to the jobs in the queue after they are executed. Will finished jobs be cleared out automat