Update PFI Docs (#32900)

* Add PFI

* Update original PFI article

* Update original sample

* Link to AutoML

Author: luisquintanilla

docs/machine-learning/how-to-guides/explain-machine-learning-model-permutation-feature-importance-ml-net.md

@@ -1,7 +1,7 @@
 ---
 title: Interpret ML.NET models with Permutation Feature Importance
 description: Understand the feature importance of models with Permutation Feature Importance in ML.NET
-ms.date: 10/05/2021
+ms.date: 12/06/2022
 author: luisquintanilla
 ms.author: luquinta
 ms.custom: mvc,how-to
@@ -105,38 +105,31 @@ string[] featureColumnNames =
         .Select(column => column.Name)
         .Where(columnName => columnName != "Label").ToArray();
 
-// 2. Define estimator with data pre-processing steps
-IEstimator<ITransformer> dataPrepEstimator =
+// 2. Define training pipeline
+IEstimator<ITransformer> sdcaEstimator =
     mlContext.Transforms.Concatenate("Features", featureColumnNames)
-        .Append(mlContext.Transforms.NormalizeMinMax("Features"));
+        .Append(mlContext.Transforms.NormalizeMinMax("Features"))
+        .Append(mlContext.Regression.Trainers.Sdca());
 
-// 3. Create transformer using the data pre-processing estimator
-ITransformer dataPrepTransformer = dataPrepEstimator.Fit(data);
-
-// 4. Pre-process the training data
-IDataView preprocessedTrainData = dataPrepTransformer.Transform(data);
-
-// 5. Define Stochastic Dual Coordinate Ascent machine learning estimator
-var sdcaEstimator = mlContext.Regression.Trainers.Sdca();
-
-// 6. Train machine learning model
-var sdcaModel = sdcaEstimator.Fit(preprocessedTrainData);
+// 3. Train machine learning model
+var sdcaModel = sdcaEstimator.Fit(data);
 ```
 
 ## Explain the model with Permutation Feature Importance (PFI)
 
 In ML.NET use the [`PermutationFeatureImportance`](xref:Microsoft.ML.PermutationFeatureImportanceExtensions) method for your respective task.
 
 ```csharp
+// Use the model to make predictions
+var transformedData = sdcaModel.Transform(data);
+
+// Calculate feature importance
 ImmutableArray<RegressionMetricsStatistics> permutationFeatureImportance =
     mlContext
         .Regression
-        .PermutationFeatureImportance(sdcaModel, preprocessedTrainData, permutationCount:3);
+        .PermutationFeatureImportance(sdcaModel, transformedData, permutationCount:3);
 ```
 
-> [!NOTE]
-> For pipelines that combine the preprocessing transforms and trainer, assuming that the trainer is at the end of the pipeline, you'll need to extract it using the `LastTransformer` property.
-
 The result of using [`PermutationFeatureImportance`](xref:Microsoft.ML.PermutationFeatureImportanceExtensions) on the training dataset is an [`ImmutableArray`](xref:System.Collections.Immutable.ImmutableArray) of [`RegressionMetricsStatistics`](xref:Microsoft.ML.Data.RegressionMetricsStatistics) objects. [`RegressionMetricsStatistics`](xref:Microsoft.ML.Data.RegressionMetricsStatistics) provides summary statistics like mean and standard deviation for multiple observations of [`RegressionMetrics`](xref:Microsoft.ML.Data.RegressionMetrics) equal to the number of permutations specified by the `permutationCount` parameter.
 
 The metric used to measure feature importance depends on the machine learning task used to solve your problem. For example, regression tasks may use a common evaluation metric such as R-squared to measure importance. For more information on model evaluation metrics, see [evaluate your ML.NET model with metrics](../resources/metrics.md).
@@ -179,6 +172,7 @@ Taking a look at the five most important features for this dataset, the price of
 
 ## Next steps
 
+- [Use Permutation Feature Importance (PFI) with AutoML](how-to-use-the-automl-api.md#determine-feature-importance)
 - [Make predictions with a trained model](machine-learning-model-predictions-ml-net.md)
 - [Retrain a model](retrain-model-ml-net.md)
 - [Deploy a model in an ASP.NET Core Web API](serve-model-web-api-ml-net.md)

docs/machine-learning/how-to-guides/how-to-use-the-automl-api.md

@@ -1,7 +1,7 @@
 ---
 title: How to use the ML.NET Automated ML (AutoML) API
 description: The ML.NET Automated ML (AutoML) API automates the model building process and generates a model ready for deployment. Learn the options that you can use to configure automated machine learning tasks.
-ms.date: 11/10/2022
+ms.date: 12/06/2022
 ms.custom: mvc,how-to
 ms.topic: how-to
 ---
@@ -685,3 +685,41 @@ Checkpoints provide a way for you to save intermediary outputs from the training
 var checkpointPath = Path.Join(Directory.GetCurrentDirectory(), "automl");
 experiment.SetCheckpoint(checkpointPath);
 ```
+
+## Determine feature importance
+
+As machine learning is introduced into more aspects of everyday life such as healthcare, it's of utmost importance to understand why a machine learning model makes the decisions it does. Permutation Feature Importance (PFI) is a technique used to explain classification, ranking, and regression models. At a high level, the way it works is by randomly shuffling data one feature at a time for the entire dataset and calculating how much the performance metric of interest decreases. The larger the change, the more important that feature is. For more information on PFI, see [interpret model predictions using Permutation Feature Importance](explain-machine-learning-model-permutation-feature-importance-ml-net.md).
+
+> [!NOTE]
+> Calculating PFI can be a time consuming operation. How much time it takes to calculate is proportional to the number of feature columns you have. The more features, the longer PFI will take to run.
+
+To determine feature importance using AutoML:
+
+1. Get the best model.
+
+    ```csharp
+    var bestModel = expResult.Model;
+    ```
+
+1. Apply the model to your dataset.
+
+    ```csharp
+    var transformedData = bestModel.Transform(trainValidationData.TrainSet);
+    ```
+
+1. Calculate feature importance using <xref:Microsoft.ML.PermutationFeatureImportanceExtensions.PermutationFeatureImportance%2A>
+
+    In this case, the task is regression but the same concept applies to other tasks like ranking and classification.
+
+    ```csharp
+    var pfiResults = 
+        mlContext.Regression.PermutationFeatureImportance(bestModel, transformedData, permutationCount:3);
+    ```
+
+1. Order feature importance by changes to evaluation metrics.
+
+    ```csharp
+    var featureImportance = 
+        pfi.Select(x => Tuple.Create(x.Key, x.Value.Regression.RSquared))
+            .OrderByDescending(x => x.Item2)    
+    ```