BenchMark Quizzes by MDFT Pro

Configure Notebook Execution Order in DAG

You work for MDFT Pro, a well-known training agency that processes student data using Fabric notebooks. Claire, a Data Engineer at MDFT Pro, is building a master orchestration notebook called DataProcessor that coordinates the execution of three data transformation notebooks. The three notebooks must execute in a specific order: first, ValidationNotebook cleans and validates raw student data; second, EnrichmentNotebook adds calculated fields and lookups; finally, AggregationNotebook creates summary tables for reporting. Claire has created a DAG configuration using the runMultiple function to orchestrate these notebooks, but currently all three notebooks are executing in parallel because no dependencies are defined. The DAG configuration specifies a concurrency of 0 (unlimited parallelism) and lists the notebooks in this order: EnrichmentNotebook (execute notebook 1), AggregationNotebook (execute notebook 2), and ValidationNotebook (execute notebook 3). Claire needs to modify the DAG to ensure the notebooks execute sequentially in the correct order: ValidationNotebook first, then EnrichmentNotebook, then AggregationNotebook.

Which two actions should Claire perform to achieve the correct execution sequence? Each correct answer presents part of the solution.

Choose all correct answers from the options below.

Explanations for each answer:

Move the declaration of Notebook_03 to the top of the Directed Acyclic Graph (DAG) definition is correct. Moving Notebook_03 to the top of the DAG definition helps establish it as the starting point for execution. When combined with proper dependencies, this ensures Notebook_03 executes first, followed by the dependent notebooks in the correct sequence.
Add dependencies to the execution of Notebook_02 is correct. Adding dependencies to Notebook_02 is essential to control execution order. By specifying that Notebook_02 depends on Notebook_01 (which in turn depends on Notebook_03), you create a dependency chain that ensures the notebooks execute in the correct sequence: Notebook_03 → Notebook_01 → Notebook_02.
Add dependencies to the execution of Notebook_03 is incorrect. Notebook_03 should be the first to execute and shouldn't have dependencies on other notebooks in this DAG. Adding dependencies to Notebook_03 would prevent it from starting first and would create the wrong execution order.
Change the concurrency to 3 is incorrect. Changing concurrency to 3 would allow up to three notebooks to run simultaneously, but it doesn't control execution order. Without dependencies, all three notebooks would still attempt to run in parallel. The current concurrency value of 0 (unlimited) isn't the issue; the lack of dependencies is.
Split the Directed Acyclic Graph (DAG) definition into three separate definitions is incorrect. Splitting the DAG into separate definitions would eliminate the ability to manage dependencies between notebooks within a single orchestration flow. This approach would require external orchestration (like a data pipeline) and wouldn't leverage the DAG's built-in dependency management capabilities.
Move the declaration of Notebook_02 to the bottom of the Directed Acyclic Graph (DAG) definition is incorrect. The order of declaration in the DAG JSON doesn't determine execution order; dependencies do. Without explicit dependencies defined, all notebooks run in parallel regardless of their position in the JSON array. Only dependency definitions control the execution sequence.

Learn more about notebook orchestration:

Notebook Orchestration with DAG

Practice Exam

Configure Notebook Execution Order in DAG

Discuss this question on social media: