BenchMark Quizzes by MDFT Pro

Schedule Pipeline Executions at Specific Times

You work for MDFT Pro, a well-known training agency that manages student enrollment data. Claire, a Data Integration Developer at MDFT Pro, has created a data pipeline called EnrollmentProcessor in the StudentData workspace. The pipeline processes CSV files from Azure Blob Storage and loads them into a lakehouse named StudentRecords. Two files are generated each day by the company’s registration system: MorningEnrollments.csv (created at 1 PM with morning class registrations) and EveningEnrollments.csv (created at 4 PM with afternoon and evening class registrations). Claire needs to configure the pipeline to automatically process MorningEnrollments.csv at 2 PM each day and EveningEnrollments.csv at 5 PM each day, ensuring the data is loaded shortly after the files are created. The solution should be simple to configure and maintain without requiring additional code or external orchestration tools.

What should Claire use to configure the pipeline to run at these specific times?

Choose the correct answer from the options below.

Explanations for each answer:

A data pipeline schedule is correct. A data pipeline schedule is the correct choice because it allows you to configure multiple daily execution times (up to 10 times per day) directly in the pipeline settings. You can set one schedule to run at 2 PM and another at 5 PM, minimizing development effort without requiring custom code or additional components.
An activator is incorrect. Activators are used for event-driven scenarios where actions are triggered based on conditions or events in real-time data streams. For scheduled execution at specific times, a pipeline schedule is simpler and more appropriate than setting up event monitoring.
A job definition is incorrect. Job definitions are used for Apache Spark jobs and are designed for scheduling Spark workloads, not data pipelines. They don't provide the built-in scheduling capabilities for pipeline orchestration activities.
A data pipeline trigger is incorrect. Data pipeline triggers are used for event-driven execution (like file arrival events or workspace events), not for time-based scheduling. For running at specific times each day, a pipeline schedule is the appropriate solution.

Learn more about running and scheduling pipelines:

Pipeline Scheduling

Practice Exam

Schedule Pipeline Executions at Specific Times

Discuss this question on social media: