BenchMark Quizzes by MDFT Pro

Transform String Column to Integer in PySpark DataFrame

You are working for MDFT Pro, a well-known training agency. You are building a solution using a Fabric notebook. You have a Spark DataFrame assigned to a variable named df. The DataFrame returns four columns.

You need to change the DataFrame as follows:

A string column named ‘age’ must be transformed to integer
The code must return a DataFrame that includes all four columns

Which PySpark code should you use to achieve the goal?

Choose the correct answer from the options below.

Explanations for each answer:

df.select("age", withColumn("age").transform("int")) is incorrect. The select method is used to choose columns, but the syntax is incorrect. The withColumn method should be used to transform columns, and transform is not a valid PySpark method for type conversion.
df.withColumn("age", col("age").cast("int")) is correct.
df.cast("age", selectExpr("transform(age, int)")) is incorrect. The cast method is not a DataFrame method in PySpark. The withColumn method should be used to transform columns, and the cast method should be called on the column reference, not on the DataFrame.
df.transform("age", withColumn("age").cast("int")) is incorrect. The transform method is not a valid DataFrame method in PySpark. The withColumn method should be used directly on the DataFrame to transform columns.

Learn more about the withColumn() function in PySpark:

WithColumn Function

Exam Preparation Quiz

Transform String Column to Integer in PySpark DataFrame