DP-100 : Designing and Implementing a Data Science Solution on Azure : Part 10

Last Updated on November 4, 2022 by InfraExam

DP-100 : Designing and Implementing a Data Science Solution on Azure : Part 10

1. Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.

   After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.

   You train a classification model by using a logistic regression algorithm.

   You must be able to explain the model’s predictions by calculating the importance of each feature, both as an overall global relative importance value and as a measure of local importance for a specific set of predictions.

   You need to create an explainer that you can use to retrieve the required global and local feature importance values.

   Solution: Create a TabularExplainer.

   Does the solution meet the goal?

   • Yes
   • No

   Explanation:

   Instead use Permutation Feature Importance Explainer (PFI).

   Note 1:

   

   Note 2: Permutation Feature Importance Explainer (PFI): Permutation Feature Importance is a technique used to explain classification and regression models. At a high level, the way it works is by randomly shuffling data one feature at a time for the entire dataset and calculating how much the performance metric of interest changes. The larger the change, the more important that feature is. PFI can explain the overall behavior of any underlying model but does not explain individual predictions.

2. Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.

   After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.

   You train a classification model by using a logistic regression algorithm.

   You must be able to explain the model’s predictions by calculating the importance of each feature, both as an overall global relative importance value and as a measure of local importance for a specific set of predictions.

   You need to create an explainer that you can use to retrieve the required global and local feature importance values.

   Solution: Create a PFIExplainer.

   Does the solution meet the goal?

   • Yes
   • No
   Explanation:
   Permutation Feature Importance Explainer (PFI): Permutation Feature Importance is a technique used to explain classification and regression models. At a high level, the way it works is by randomly shuffling data one feature at a time for the entire dataset and calculating how much the performance metric of interest changes. The larger the change, the more important that feature is. PFI can explain the overall behavior of any underlying model but does not explain individual predictions.

3. HOTSPOT

   You are performing feature scaling by using the scikit-learn Python library for x.1 x2, and x3 features.

   Original and scaled data is shown in the following image.

   

   Use the drop-down menus to select the answer choice that answers each question based on the information presented in the graphic.

   NOTE: Each correct selection is worth one point.

   

   

   Explanation:

   Box 1: StandardScaler
   The StandardScaler assumes your data is normally distributed within each feature and will scale them such that the distribution is now centred around 0, with a standard deviation of 1.
   Example:

   

   All features are now on the same scale relative to one another.

   Box 2: Min Max Scaler

   

   Notice that the skewness of the distribution is maintained but the 3 distributions are brought into the same scale so that they overlap.

   Box 3: Normalizer

4. You are determining if two sets of data are significantly different from one another by using Azure Machine Learning Studio.

   Estimated values in one set of data may be more than or less than reference values in the other set of data. You must produce a distribution that has a constant Type I error as a function of the correlation.

   You need to produce the distribution.

   Which type of distribution should you produce?

   • Unpaired t-test with a two-tail option
   • Unpaired t-test with a one-tail option
   • Paired t-test with a one-tail option
   • Paired t-test with a two-tail option
   Explanation:

   Choose a one-tail or two-tail test. The default is a two-tailed test. This is the most common type of test, in which the expected distribution is symmetric around zero.

   Example: Type I error of unpaired and paired two-sample t-tests as a function of the correlation. The simulated random numbers originate from a bivariate normal distribution with a variance of 1.

   

5. DRAG DROP

   You are producing a multiple linear regression model in Azure Machine Learning Studio.

   Several independent variables are highly correlated.

   You need to select appropriate methods for conducting effective feature engineering on all the data.

   Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.

   

   

   Explanation:

   Step 1: Use the Filter Based Feature Selection module
   Filter Based Feature Selection identifies the features in a dataset with the greatest predictive power.
   The module outputs a dataset that contains the best feature columns, as ranked by predictive power. It also outputs the names of the features and their scores from the selected metric.

   Step 2: Build a counting transform
   A counting transform creates a transformation that turns count tables into features, so that you can apply the transformation to multiple datasets.

   Step 3: Test the hypothesis using t-Test

6. You are performing feature engineering on a dataset.

   You must add a feature named CityName and populate the column value with the text London.

   You need to add the new feature to the dataset.

   Which Azure Machine Learning Studio module should you use?

   • Extract N-Gram Features from Text
   • Edit Metadata
   • Preprocess Text
   • Apply SQL Transformation
   Explanation:
   Typical metadata changes might include marking columns as features.

7. Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.

   After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.

   You are creating a model to predict the price of a student’s artwork depending on the following variables: the student’s length of education, degree type, and art form.

   You start by creating a linear regression model.

   You need to evaluate the linear regression model.

   Solution: Use the following metrics: Mean Absolute Error, Root Mean Absolute Error, Relative Absolute Error, Relative Squared Error, and the Coefficient of Determination.

   Does the solution meet the goal?

   • Yes
   • No
   Explanation:

   The following metrics are reported for evaluating regression models. When you compare models, they are ranked by the metric you select for evaluation.

   Mean absolute error (MAE) measures how close the predictions are to the actual outcomes; thus, a lower score is better.

   Root mean squared error (RMSE) creates a single value that summarizes the error in the model. By squaring the difference, the metric disregards the difference between over-prediction and under-prediction.

   Relative absolute error (RAE) is the relative absolute difference between expected and actual values; relative because the mean difference is divided by the arithmetic mean.

   Relative squared error (RSE) similarly normalizes the total squared error of the predicted values by dividing by the total squared error of the actual values.

   Mean Zero One Error (MZOE) indicates whether the prediction was correct or not. In other words: ZeroOneLoss(x,y) = 1 when x!=y; otherwise 0.

   Coefficient of determination, often referred to as R2, represents the predictive power of the model as a value between 0 and 1. Zero means the model is random (explains nothing); 1 means there is a perfect fit. However, caution should be used in interpreting R2 values, as low values can be entirely normal and high values can be suspect.
   AUC.

8. Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.

   After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.

   You are creating a model to predict the price of a student’s artwork depending on the following variables: the student’s length of education, degree type, and art form.

   You start by creating a linear regression model.

   You need to evaluate the linear regression model.

   Solution: Use the following metrics: Accuracy, Precision, Recall, F1 score, and AUC.

   Does the solution meet the goal?

   • Yes
   • No
   Explanation:
   Those are metrics for evaluating classification models, instead use: Mean Absolute Error, Root Mean Absolute Error, Relative Absolute Error, Relative Squared Error, and the Coefficient of Determination.

9. Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.

   After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.

   You are creating a model to predict the price of a student’s artwork depending on the following variables: the student’s length of education, degree type, and art form.

   You start by creating a linear regression model.

   You need to evaluate the linear regression model.

   Solution: Use the following metrics: Relative Squared Error, Coefficient of Determination, Accuracy, Precision, Recall, F1 score, and AUC.

   Does the solution meet the goal?

   • Yes
   • No
   Explanation:
   Relative Squared Error, Coefficient of Determination are good metrics to evaluate the linear regression model, but the others are metrics for classification models.

10. You are a data scientist creating a linear regression model.

    You need to determine how closely the data fits the regression line.

    Which metric should you review?

    • Root Mean Square Error
    • Coefficient of determination
    • Recall
    • Precision
    • Mean absolute error
    Explanation:

    Coefficient of determination, often referred to as R2, represents the predictive power of the model as a value between 0 and 1. Zero means the model is random (explains nothing); 1 means there is a perfect fit. However, caution should be used in interpreting R2 values, as low values can be entirely normal and high values can be suspect.

    Incorrect Answers:
    A: Root mean squared error (RMSE) creates a single value that summarizes the error in the model. By squaring the difference, the metric disregards the difference between over-prediction and under-prediction.

    C: Recall is the fraction of all correct results returned by the model.

    D: Precision is the proportion of true results over all positive results.

    E: Mean absolute error (MAE) measures how close the predictions are to the actual outcomes; thus, a lower score is better.

11. You are creating a binary classification by using a two-class logistic regression model.

    You need to evaluate the model results for imbalance.

    Which evaluation metric should you use?

    • Relative Absolute Error
    • AUC Curve
    • Mean Absolute Error
    • Relative Squared Error
    • Accuracy
    • Root Mean Square Error
    Explanation:
    One can inspect the true positive rate vs. the false positive rate in the Receiver Operating Characteristic (ROC) curve and the corresponding Area Under the Curve (AUC) value. The closer this curve is to the upper left corner; the better the classifier’s performance is (that is maximizing the true positive rate while minimizing the false positive rate). Curves that are close to the diagonal of the plot, result from classifiers that tend to make predictions that are close to random guessing.

12. HOTSPOT

    You are developing a linear regression model in Azure Machine Learning Studio. You run an experiment to compare different algorithms.

    The following image displays the results dataset output:

    

    Use the drop-down menus to select the answer choice that answers each question based on the information presented in the image.

    NOTE: Each correct selection is worth one point.

    

    

    Explanation:

    Box 1: Boosted Decision Tree Regression
    Mean absolute error (MAE) measures how close the predictions are to the actual outcomes; thus, a lower score is better.

    Box 2:
    Online Gradient Descent: If you want the algorithm to find the best parameters for you, set Create trainer mode option to Parameter Range. You can then specify multiple values for the algorithm to try.

13. HOTSPOT

    You are using a decision tree algorithm. You have trained a model that generalizes well at a tree depth equal to 10.

    You need to select the bias and variance properties of the model with varying tree depth values.

    Which properties should you select for each tree depth? To answer, select the appropriate options in the answer area.

    

    

    Explanation:

    In decision trees, the depth of the tree determines the variance. A complicated decision tree (e.g. deep) has low bias and high variance.

    Note: In statistics and machine learning, the bias–variance tradeoff is the property of a set of predictive models whereby models with a lower bias in parameter estimation have a higher variance of the parameter estimates across samples, and vice versa. Increasing the bias will decrease the variance. Increasing the variance will decrease the bias.

14. DRAG DROP

    You have a model with a large difference between the training and validation error values.

    You must create a new model and perform cross-validation.

    You need to identify a parameter set for the new model using Azure Machine Learning Studio.

    Which module you should use for each step? To answer, drag the appropriate modules to the correct steps. Each module may be used once or more than once, or not at all. You may need to drag the split bar between panes or scroll to view content.

    NOTE: Each correct selection is worth one point.

    

    

    Explanation:

    Box 1: Split data

    Box 2: Partition and Sample

    Box 3: Two-Class Boosted Decision Tree

    Box 4: Tune Model Hyperparameters
    Integrated train and tune: You configure a set of parameters to use, and then let the module iterate over multiple combinations, measuring accuracy until it finds a “best” model. With most learner modules, you can choose which parameters should be changed during the training process, and which should remain fixed.

    We recommend that you use Cross-Validate Model to establish the goodness of the model given the specified parameters. Use Tune Model Hyperparameters to identify the optimal parameters.

15. HOTSPOT

    You are analyzing the asymmetry in a statistical distribution.

    The following image contains two density curves that show the probability distribution of two datasets.

    

    Use the drop-down menus to select the answer choice that answers each question based on the information presented in the graphic.

    NOTE: Each correct selection is worth one point.

    

    

    Explanation:

    Box 1: Positive skew
    Positive skew values means the distribution is skewed to the right.

    Box 2: Negative skew
    Negative skewness values mean the distribution is skewed to the left.

16. You are a data scientist building a deep convolutional neural network (CNN) for image classification.

    The CNN model you build shows signs of overfitting.

    You need to reduce overfitting and converge the model to an optimal fit.

    Which two actions should you perform? Each correct answer presents a complete solution.

    NOTE: Each correct selection is worth one point.

    • Add an additional dense layer with 512 input units.
    • Add L1/L2 regularization.
    • Use training data augmentation.
    • Reduce the amount of training data.
    • Add an additional dense layer with 64 input units.
    Explanation:

    B: Weight regularization provides an approach to reduce the overfitting of a deep learning neural network model on the training data and improve the performance of the model on new data, such as the holdout test set.
    Keras provides a weight regularization API that allows you to add a penalty for weight size to the loss function.

    Three different regularizer instances are provided; they are:
    – L1: Sum of the absolute weights.
    – L2: Sum of the squared weights.
    – L1L2: Sum of the absolute and the squared weights.

    D: Because a fully connected layer occupies most of the parameters, it is prone to overfitting. One method to reduce overfitting is dropout. At each training stage, individual nodes are either “dropped out” of the net with probability 1-p or kept with probability p, so that a reduced network is left; incoming and outgoing edges to a dropped-out node are also removed.
    By avoiding training all nodes on all training data, dropout decreases overfitting.

17. Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.

    After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.

    You are creating a model to predict the price of a student’s artwork depending on the following variables: the student’s length of education, degree type, and art form.

    You start by creating a linear regression model.

    You need to evaluate the linear regression model.

    Solution: Use the following metrics: Mean Absolute Error, Root Mean Absolute Error, Relative Absolute Error, Accuracy, Precision, Recall, F1 score, and AUC.

    Does the solution meet the goal?

    • Yes
    • No
    Explanation:

    Accuracy, Precision, Recall, F1 score, and AUC are metrics for evaluating classification models.

    Note: Mean Absolute Error, Root Mean Absolute Error, Relative Absolute Error are OK for the linear regression model.

18. You are building a binary classification model by using a supplied training set.

    The training set is imbalanced between two classes.

    You need to resolve the data imbalance.

    What are three possible ways to achieve this goal? Each correct answer presents a complete solution.

    NOTE: Each correct selection is worth one point.

    • Penalize the classification
    • Resample the dataset using undersampling or oversampling
    • Normalize the training feature set
    • Generate synthetic samples in the minority class
    • Use accuracy as the evaluation metric of the model
    Explanation:

    A: Try Penalized Models
    You can use the same algorithms but give them a different perspective on the problem.
    Penalized classification imposes an additional cost on the model for making classification mistakes on the minority class during training. These penalties can bias the model to pay more attention to the minority class.

    B: You can change the dataset that you use to build your predictive model to have more balanced data.
    This change is called sampling your dataset and there are two main methods that you can use to even-up the classes:
    – Consider testing under-sampling when you have an a lot data (tens- or hundreds of thousands of instances or more)
    – Consider testing over-sampling when you don’t have a lot of data (tens of thousands of records or less)

    D: Try Generate Synthetic Samples
    A simple way to generate synthetic samples is to randomly sample the attributes from instances in the minority class.

19. HOTSPOT

    You train a classification model by using a decision tree algorithm.

    You create an estimator by running the following Python code. The variable feature_names is a list of all feature names, and class_names is a list of all class names.

    from interpret.ext.blackbox import TabularExplainer

    explainer = TabularExplainer(model,
    x_train,
    features=feature_names,
    classes=class_names)

    You need to explain the predictions made by the model for all classes by determining the importance of all features.

    For each of the following statements, select Yes if the statement is true. Otherwise, select No.

    NOTE: Each correct selection is worth one point.

    

    

    Explanation:

    Box 1: Yes
    TabularExplainer calls one of the three SHAP explainers underneath (TreeExplainer, DeepExplainer, or KernelExplainer).

    Box 2: Yes
    To make your explanations and visualizations more informative, you can choose to pass in feature names and output class names if doing classification.

    Box 3: No
    TabularExplainer automatically selects the most appropriate one for your use case, but you can call each of its three underlying explainers underneath (TreeExplainer, DeepExplainer, or KernelExplainer) directly.

20. Case study

    This is a case study. Case studies are not timed separately. You can use as much exam time as you would like to complete each case. However, there may be additional case studies and sections on this exam. You must manage your time to ensure that you are able to complete all questions included on this exam in the time provided.

    To answer the questions included in a case study, you will need to reference information that is provided in the case study. Case studies might contain exhibits and other resources that provide more information about the scenario that is described in the case study. Each question is independent of the other questions in this case study.

    At the end of this case study, a review screen will appear. This screen allows you to review your answers and to make changes before you move to the next section of the exam. After you begin a new section, you cannot return to this section.

    To start the case study
    To display the first question in this case study, click the Next button. Use the buttons in the left pane to explore the content of the case study before you answer the questions. Clicking these buttons displays information such as business requirements, existing environment, and problem statements. If the case study has an All Information tab, note that the information displayed is identical to the information displayed on the subsequent tabs. When you are ready to answer a question, click the Question button to return to the question.

    Overview

    You are a data scientist for Fabrikam Residences, a company specializing in quality private and commercial property in the United States. Fabrikam Residences is considering expanding into Europe and has asked you to investigate prices for private residences in major European cities.
    You use Azure Machine Learning Studio to measure the median value of properties. You produce a regression model to predict property prices by using the Linear Regression and Bayesian Linear Regression modules.

    Datasets

    There are two datasets in CSV format that contain property details for two cities, London and Paris. You add both files to Azure Machine Learning Studio as separate datasets to the starting point for an experiment. Both datasets contain the following columns:

    

    An initial investigation shows that the datasets are identical in structure apart from the MedianValue column. The smaller Paris dataset contains the MedianValue in text format, whereas the larger London dataset contains the MedianValue in numerical format.

    Data issues

    Missing values

    The AccessibilityToHighway column in both datasets contains missing values. The missing data must be replaced with new data so that it is modeled conditionally using the other variables in the data before filling in the missing values.

    Columns in each dataset contain missing and null values. The datasets also contain many outliers. The Age column has a high proportion of outliers. You need to remove the rows that have outliers in the Age column. The MedianValue and AvgRoomsInHouse columns both hold data in numeric format. You need to select a feature selection algorithm to analyze the relationship between the two columns in more detail.

    Model fit

    The model shows signs of overfitting. You need to produce a more refined regression model that reduces the overfitting.

    Experiment requirements

    You must set up the experiment to cross-validate the Linear Regression and Bayesian Linear Regression modules to evaluate performance. In each case, the predictor of the dataset is the column named MedianValue. You must ensure that the datatype of the MedianValue column of the Paris dataset matches the structure of the London dataset.

    You must prioritize the columns of data for predicting the outcome. You must use non-parametric statistics to measure relationships.

    You must use a feature selection algorithm to analyze the relationship between the MedianValue and AvgRoomsInHouse columns.

    Model training

    Permutation Feature Importance

    Given a trained model and a test dataset, you must compute the Permutation Feature Importance scores of feature variables. You must be determined the absolute fit for the model.

    Hyperparameters

    You must configure hyperparameters in the model learning process to speed the learning phase. In addition, this configuration should cancel the lowest performing runs at each evaluation interval, thereby directing effort and resources towards models that are more likely to be successful.

    You are concerned that the model might not efficiently use compute resources in hyperparameter tuning. You also are concerned that the model might prevent an increase in the overall tuning time. Therefore, must implement an early stopping criterion on models that provides savings without terminating promising jobs.

    Testing

    You must produce multiple partitions of a dataset based on sampling using the Partition and Sample module in Azure Machine Learning Studio.

    Cross-validation

    You must create three equal partitions for cross-validation. You must also configure the cross-validation process so that the rows in the test and training datasets are divided evenly by properties that are near each city’s main river. You must complete this task before the data goes through the sampling process.

    Linear regression module

    When you train a Linear Regression module, you must determine the best features to use in a model. You can choose standard metrics provided to measure performance before and after the feature importance process completes. The distribution of features across multiple training models must be consistent.

    Data visualization

    You need to provide the test results to the Fabrikam Residences team. You create data visualizations to aid in presenting the results.

    You must produce a Receiver Operating Characteristic (ROC) curve to conduct a diagnostic test evaluation of the model. You need to select appropriate methods for producing the ROC curve in Azure Machine Learning Studio to compare the Two-Class Decision Forest and the Two-Class Decision Jungle modules with one another.

    1. DRAG DROP

       You need to correct the model fit issue.

       Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.

       

       

       Explanation:

       Step 1: Augment the data
       Scenario: Columns in each dataset contain missing and null values. The datasets also contain many outliers.

       Step 2: Add the Bayesian Linear Regression module.
       Scenario: You produce a regression model to predict property prices by using the Linear Regression and Bayesian Linear Regression modules.

       Step 3: Configure the regularization weight.
       Regularization typically is used to avoid overfitting. For example, in L2 regularization weight, type the value to use as the weight for L2 regularization. We recommend that you use a non-zero value to avoid overfitting.

       Scenario:
       Model fit: The model shows signs of overfitting. You need to produce a more refined regression model that reduces the overfitting.

       Incorrect Answers:
       Multiclass Decision Jungle module:
       Decision jungles are a recent extension to decision forests. A decision jungle consists of an ensemble of decision directed acyclic graphs (DAGs).

       L-BFGS:
       L-BFGS stands for “limited memory Broyden-Fletcher-Goldfarb-Shanno”. It can be found in the wwo-Class Logistic Regression module, which is used to create a logistic regression model that can be used to predict two (and only two) outcomes.