A machine learning engineer is trying to perform batch model inference. They want to get predictions using the linear regression model saved at the pathmodel_urifor the DataFramebatch_df.
batch_dfhas the following schema: customer_id STRING
The machine learning engineer runs the following code block to perform inference onbatch_dfusing the linear regression model atmodel_uri:

In which situation will the machine learning engineer??s code block perform the desired
inference?

1. A. When the Feature Store feature set was logged with the model at model_uri
2. B. When all of the features used by the model at model_uri are in a Spark DataFrame in the PySpark
3. C. When the model at model_uri only uses customer_id as a feature
4. D. This code block will not perform the desired inference in any situation.
5. E. When all of the features used by the model at model_uri are in a single Feature Store table

Correct Answer: A
The code block provided by the machine learning engineer will perform the desired inference when the Feature Store feature set was logged with the model at model_uri. This ensures that all necessary feature transformations and metadata are available for the model to make predictions. The Feature Store in Databricks allows for seamless integration of features and models, ensuring that the required features are correctly used during inference.
References:
✑ Databricks documentation on Feature Store: Feature Store in Databricks

A machine learning engineer wants to parallelize the inference of group-specific models using the Pandas Function API. They have developed theapply_modelfunction that will look up and load the correct model for each group, and they want to apply it to each group of DataFramedf.
They have written the following incomplete code block:

Which piece of code can be used to fill in the above blank to complete the task?

1. A. applyInPandas
2. B. groupedApplyInPandas
3. C. mapInPandas
4. D. predict

Correct Answer: A
To parallelize the inference of group-specific models using the Pandas Function API in PySpark, you can use theapplyInPandasfunction. This function allows you to apply a Python function on each group of a DataFrame and return a DataFrame, leveraging the power of pandas UDFs (user-defined functions) for better performance.
prediction_df = ( df.groupby("device_id") .applyInPandas(apply_model, schema=apply_return_schema) )
In this code:
✑ groupby("device_id"): Groups the DataFrame by the "device_id" column.
✑ applyInPandas(apply_model, schema=apply_return_schema): Applies theapply_modelfunction to each group and specifies the schema of the return DataFrame.
References:
✑ PySpark Pandas UDFs Documentation

A data scientist has been given an incomplete notebook from the data engineering team.
The notebook uses a Spark DataFrame spark_df on which the data scientist needs to perform further feature engineering. Unfortunately, the data scientist has not yet learned the PySpark DataFrame API.
Which of the following blocks of code can the data scientist run to be able to use the pandas API on Spark?

1. A. import pyspark.pandas as ps df = ps.DataFrame(spark_df)
2. B. import pyspark.pandas as ps df = ps.to_pandas(spark_df)
3. C. spark_df.to_pandas()
4. D. import pandas as pddf = pd.DataFrame(spark_df)

Correct Answer: A
To use the pandas API on Spark, the data scientist can run the following code block:
importpyspark.pandasasps df = ps.DataFrame(spark_df)
This code imports the pandas API on Spark and converts the Spark DataFramespark_df into a pandas-on-Spark DataFrame, allowing the data scientist to use familiar pandas functions for further feature engineering.
References:
✑ Databricks documentation on pandas API on Spark: pandas API on Spark