BenchMark Quizzes by MDFT Pro

Share Spark Session Between Notebooks

You work as a Data Engineering Coordinator for Mark at MDFT Pro, a well-known training agency. MDFT Pro operates a Fabric workspace named Workspace1 that contains a notebook named Notebook1, which processes student enrollment data and course analytics using Apache Spark. Mark has asked you to create a new notebook named Notebook2 in the same workspace to perform additional analysis on the same data. To improve efficiency and enable collaboration between the two notebooks, you need to ensure that Notebook2 can attach to the same Apache Spark session as Notebook1, allowing them to share data and computational resources without duplicating the Spark context setup.

What should you do?

Choose the correct answer from the options below.

Explanations for each answer:

Enable high concurrency for notebooks is correct. High concurrency mode allows multiple notebooks to share the same Spark session, enabling collaboration and resource sharing between Notebook1 and Notebook2. This is the specific feature designed for sharing Spark sessions across notebooks.
Enable dynamic allocation for the Spark pool is incorrect. Dynamic allocation automatically adjusts the number of executors based on workload, but it doesn't enable sharing of Spark sessions between notebooks. Each notebook would still have its own separate session.
Change the runtime version is incorrect. Changing the runtime version affects which Spark libraries and features are available but does not enable session sharing between notebooks. Both notebooks would still run in separate Spark sessions.
Increase the number of executors is incorrect. Increasing executors provides more parallel processing capacity but doesn't allow notebooks to share the same Spark session. Each notebook would still have its own isolated session with its allocated executors.

Learn more about high concurrency mode in Fabric notebooks:

Notebook High Concurrency Mode

Practice Exam

Share Spark Session Between Notebooks

Discuss this question on social media: