Achieve enterprise-grade monitoring for your Amazon SageMaker models using Fiddler

{"value":"*This is a guest blog post by Danny Brock, Rajeev Govindan and Krishnaram Kenthapadi at Fiddler AI.*\n\nYour [Amazon SageMaker](https://aws.amazon.com/sagemaker/) models are live. They’re handling millions of inferences each day and driving better business outcomes for your company. They’re performing exactly as well as the day they were launched.\n\n*Er, wait. Are they? Maybe. Maybe not.*\n\nWithout enterprise-class [model monitoring](https://www.fiddler.ai/ml-monitoring), your models may be decaying in silence. Your machine learning (ML) teams may never know that these models have actually morphed from miracles of revenue generation to liabilities making incorrect decisions that cost your company time and money.\n\n*Don’t fret. The solution is closer than you think.*\n\n[Fiddler](https://www.fiddler.ai/), an enterprise-class Model Performance Management solution available on the [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-imsoert6oehmm?sr=0-1&ref_=beagle&applicationId=AWSMPContessa), offers model monitoring and explainable AI to help ML teams inspect and address a comprehensive range of model issues. Through model monitoring, model explainability, analytics, and bias detection, Fiddler provides your company with an easy-to-use single pane of glass to ensure your models are behaving as they should. And if they’re not, Fiddler also provides features that allow you to inspect your models to find the underlying root causes of performance decay.\n\nThis post shows how your [MLOps](https://www.fiddler.ai/mlops) team can improve data scientist productivity and reduce time to detect issues for your models deployed in SageMaker by integrating with the Fiddler Model Performance Management Platform in a few simple steps.\n\n### **Solution overview**\n\nThe following reference architecture highlights the primary points of integration. Fiddler exists as a “sidecar” to your existing SageMaker ML workflow.\n\n\n\nThe remainder of this post walks you through the steps to integrate your SageMaker model with Fiddler’s [Model Performance Management Platform](https://www.fiddler.ai/resources/whitepapers/optimizing-mlops-with-ml-mpm-powered-by-explainable-ai):\n\n1. Ensure your model has data capture enabled.\n2. Create a Fiddler trial environment.\n3. Register information about your model in your Fiddler environment.\n4. Create an [AWS Lambda](http://aws.amazon.com/lambda) function to publish SageMaker inferences to Fiddler.\n5. Explore Fiddler’s monitoring capabilities in your Fiddler trial environment.\n\n### **Prerequisites**\n\nThis post assumes that you have set up SageMaker and deployed a model endpoint. To learn how to configure SageMaker for model serving, refer to [Deploy Models for Inference](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html). Some examples are also available on the [GitHub repo](https://github.com/aws/amazon-sagemaker-examples).\n\n### **Ensure your model has data capture enabled**\n\nOn the SageMaker console, navigate to your model’s serving endpoint and ensure you have enabled [data capture](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-data-capture.html) into an [Amazon Simple Storage Service](http://aws.amazon.com/s3) ([Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail)) bucket. This stores the inferences (requests and responses) your model makes each day as [JSON lines files](https://jsonlines.org/) (.jsonl) in [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail).\n\n\n\n\n\n### **Create a Fiddler trial environment**\n\nFrom the [fiddler.ai](http://fiddler.ai/trial) website, you can request a free trial. After filling out a quick form, Fiddler will contact you to understand the specifics of your model performance management needs and will have a trial environment ready for you in a few hours. You can expect a dedicated environment like [https://yourcompany.try.fiddler.ai](https://yourcompany.try.fiddler.ai/).\n\n\n\n### **Register information about your model in your Fiddler environment**\n\nBefore you can begin publishing events from your SageMaker hosted model into Fiddler, you need to create a project within your Fiddler trial environment and provide Fiddler details about your model through a step called model registration. If you want to use a preconfigured notebook from within [Amazon SageMaker Studio](https://aws.amazon.com/sagemaker/studio/) rather than copy and paste the following code snippets, you can reference the Fiddler quickstart notebook on [GitHub](https://github.com/fiddler-labs/fiddler-samples/blob/master/sagemaker/RegisterModelWithFiddler.ipynb). Studio provides a single web-based visual interface where you can perform all ML development steps.\n\nFirst, you must install the [Fiddler Python](https://pypi.org/project/fiddler-client/) client in your SageMaker notebook and instantiate the Fiddler client. You can get the ```AUTH_TOKEN``` from the **Settings** page in your Fiddler trial environment.\n\n\n```\\n\\n# Install the fiddler client\\n!pip install fiddler-client\\n\\n# Connect to the Fiddler Trial Environment\\nimport fiddler as fdl\\nimport pandas as pd\\n\\nfdl.__version__\\n\\nURL = 'https://yourcompany.try.fiddler.ai'\\nORG_ID = 'yourcompany'\\nAUTH_TOKEN = 'UUID-Token-Here-Found-In-Your-Fiddler-Env-Settings-Page'\\n\\nclient = fdl.FiddlerApi(URL, ORG_ID, AUTH_TOKEN)\\n\\n```\n\n\nNext, create a project within your Fiddler trial environment:\n\n```\\n\\n# Create Project\\nPROJECT_ID = 'credit_default' # update this with your project name\\nDATASET_ID = f'{PROJECT_ID}_dataset'\\nMODEL_ID = f'{PROJECT_ID}_model'\\n\\nclient.create_project(PROJECT_ID)\\n\\n```\n\nNow upload your training dataset. The notebook also provides a sample dataset to run Fiddler’s [explainability](https://www.fiddler.ai/explainable-ai) algorithms and as a baseline for monitoring metrics. The dataset is also used to generate the schema for this model in Fiddler.\n\n\n```\\n\\n# Upload Baseline Dataset\\ndf_baseline = pd.read_csv(‘<your-training-file.csv>')\\n\\ndataset_info = fdl.DatasetInfo.from_dataframe(df_baseline, max_inferred_cardinality=1000)\\n\\nupload_result = client.upload_dataset(PROJECT_ID,\\n dataset={'baseline': df_baseline},\\n dataset_id=DATASET_ID,\\n info=dataset_info)\\n\\n```\n\nLastly, before you can start publishing inferences to Fiddler for monitoring, root cause analysis, and explanations, you need to register your model. Let’s first create a ```model_info``` object that contains the metadata about your model:\n\n\n```\\n\\n# Update task from the list below if your model task is not binary classification\\nmodel_task = 'binary' \\n\\nif model_task == 'regression':\\n model_task_fdl = fdl.ModelTask.REGRESSION\\n \\nelif model_task == 'binary':\\n model_task_fdl = fdl.ModelTask.BINARY_CLASSIFICATION\\n\\nelif model_task == 'multiclass':\\n model_task_fdl = fdl.ModelTask.MULTICLASS_CLASSIFICATION\\n\\nelif model_task == 'ranking':\\n model_task_fdl = fdl.ModelTask.RANKING\\n\\n \\n# Specify column types|\\ntarget = 'TARGET'\\noutputs = ['prediction'] # change this to your target variable\\nfeatures = [‘<add your feature list here>’]\\n \\n# Generate ModelInfo\\nmodel_info = fdl.ModelInfo.from_dataset_info(\\n dataset_info=dataset_info,\\n dataset_id=DATASET_ID,\\n model_task=model_task_fdl,\\n target=target,\\n outputs=outputs,\\n features=features,\\n binary_classification_threshold=.125, # update this if your task is not a binary classification\\n description='<model-description>',\\n display_name='<model-display-name>'\\n)\\nmodel_info\\n\\n```\n\nThen you can register the model using your new ```model_info``` object:\n\n\n```\\n\\n# Register Info about your model with Fiddler\\nclient.register_model(\\n project_id=PROJECT_ID,\\n dataset_id=DATASET_ID,\\n model_id=MODEL_ID,\\n model_info=model_info\\n)\\n\\n```\n\nGreat! Now you can publish some events to Fiddler in order to observe the model’s performance.\n\n### **Create a Lambda function to publish SageMaker inferences to Fiddler**\n\nWith the simple-to-deploy serverless architecture of Lambda, you can quickly build the mechanism required to move your inferences from the S3 bucket you set up earlier into your newly provisioned Fiddler trial environment. This Lambda function is responsible for opening any new JSONL event log files in your model’s S3 bucket, parsing and formatting the JSONL content into a dataframe, and then publishing that dataframe of events to your Fiddler trial environment. The following screenshot shows the code details of our function.\n\n\n\nThe Lambda function needs to be configured to trigger off of newly created files in your S3 bucket. The following [tutorial](https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html) guides you through creating an [Amazon EventBridge](https://aws.amazon.com/eventbridge/) trigger that invokes the Lambda function whenever a file is uploaded to [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail). The following screenshot shows our function’s trigger configuration. This makes it simple to ensure that any time your model makes new inferences, those events stored in [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) are loaded into Fiddler to drive the model observability your company needs.\n\nTo simplify this further, the code for this Lambda function is publicly available from [Fiddler’s documentation site](https://docs.fiddler.ai/docs/sagemaker-integration) . This code example currently works for binary classification models with structured inputs. If you have model types with different features or tasks, please contact Fiddler for assistance with minor changes to the code.\n\nThe Lambda function needs to make reference to the Fiddler Python client. Fiddler has created a publicly available Lambda layer that you can reference to ensure that the ```import fiddler as fdl``` step works seamlessly. You can reference this layer via an ARN in the us-west-2 Region: ```arn:aws:lambda:us-west-2:079310353266:layer:fiddler-client-0814:1```, as shown in the following screenshot.\n\n\n\n\nYou also need to specify Lambda environment variables so the Lambda function knows how to connect to your Fiddler trial environment, and what the inputs and outputs are within the .jsonl files being captured by your model. The following screenshot shows a list of the required environment variables, which are also on [Fiddler’s documentation site](https://docs.fiddler.ai/docs/sagemaker-integration). Update the values for the environment variables to match your model and dataset.\n\n\n\n\n### **Explore Fiddler’s monitoring capabilities in your Fiddler trial environment**\n\nYou’ve done it! With your baseline data, model, and traffic connected, you can now explain [data drift](https://www.fiddler.ai/blog/drift-in-machine-learning-how-to-identify-issues-before-you-have-a-problem), outliers, [model bias](https://www.fiddler.ai/blog/ai-explained-understanding-bias-and-fairness-in-ai-systems), data issues, and performance blips, and share dashboards with others. Complete your journey by [watching a demo](https://www.fiddler.ai/watch-demo) about the model performance management capabilities you have introduced to your company.\n\nThe example screenshots below provide a glimpse of model insights like drift, outlier detection, local point explanations, and model analytics that will be found in your Fiddler trial environment.\n\n\n\n### **Conclusion**\n\nThis post highlighted the need for enterprise-class [model monitoring](https://www.fiddler.ai/ml-monitoring) and showed how you can integrate your models deployed in SageMaker with the [Fiddler Model Performance Management Platform](https://www.fiddler.ai/) in just a few steps. Fiddler offers functionality for model monitoring, explainable AI, bias detection, and root cause analysis, and is available on the [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-imsoert6oehmm?sr=0-1&ref_=beagle&applicationId=AWSMPContessa). By providing your [MLOps](https://www.fiddler.ai/mlops) team with an easy-to-use single pane of glass to ensure your models are behaving as expected and to identify the underlying root causes of performance degradation, Fiddler can help improve data scientist productivity and reduce time to detect and resolve issues.\n\nIf you would like to learn more about Fiddler please visit [fiddler.ai](https://www.fiddler.ai/) or if you would prefer to set up a personalized demo and technical discussion email [sales@fiddler.ai](sales@fiddler.ai).\n\n#### **About the Authors**\n\n\n\n[**Danny Brock**](https://www.linkedin.com/in/dannybrock/) is a Sr Solutions Engineer at Fiddler AI. Danny is long tenured in the analytics and ML space, running presales and post-sales teams for startups like Endeca and Incorta. He founded his own big data analytics consulting company, Branchbird, in 2012.","render":"<p><em>This is a guest blog post by Danny Brock, Rajeev Govindan and Krishnaram Kenthapadi at Fiddler AI.</em></p>\\n<p>Your <a href=\\"https://aws.amazon.com/sagemaker/\\" target=\\"_blank\\">Amazon SageMaker</a> models are live. They’re handling millions of inferences each day and driving better business outcomes for your company. They’re performing exactly as well as the day they were launched.</p>\\n<p><em>Er, wait. Are they? Maybe. Maybe not.</em></p>\\n<p>Without enterprise-class <a href=\\"https://www.fiddler.ai/ml-monitoring\\" target=\\"_blank\\">model monitoring</a>, your models may be decaying in silence. Your machine learning (ML) teams may never know that these models have actually morphed from miracles of revenue generation to liabilities making incorrect decisions that cost your company time and money.</p>\\n<p><em>Don’t fret. The solution is closer than you think.</em></p>\\n<p><a href=\\"https://www.fiddler.ai/\\" target=\\"_blank\\">Fiddler</a>, an enterprise-class Model Performance Management solution available on the <a href=\\"https://aws.amazon.com/marketplace/pp/prodview-imsoert6oehmm?sr=0-1&amp;ref_=beagle&amp;applicationId=AWSMPContessa\\" target=\\"_blank\\">AWS Marketplace</a>, offers model monitoring and explainable AI to help ML teams inspect and address a comprehensive range of model issues. Through model monitoring, model explainability, analytics, and bias detection, Fiddler provides your company with an easy-to-use single pane of glass to ensure your models are behaving as they should. And if they’re not, Fiddler also provides features that allow you to inspect your models to find the underlying root causes of performance decay.</p>\\n<p>This post shows how your <a href=\\"https://www.fiddler.ai/mlops\\" target=\\"_blank\\">MLOps</a> team can improve data scientist productivity and reduce time to detect issues for your models deployed in SageMaker by integrating with the Fiddler Model Performance Management Platform in a few simple steps.</p>\\n<h3><a id=\\"Solution_overview_14\\"></a><strong>Solution overview</strong></h3>\\n<p>The following reference architecture highlights the primary points of integration. Fiddler exists as a “sidecar” to your existing SageMaker ML workflow.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/dd01aaf556b841be9a956a6a0799ca7e_image.png\\" alt=\\"image.png\\" /></p>\n<p>The remainder of this post walks you through the steps to integrate your SageMaker model with Fiddler’s <a href=\\"https://www.fiddler.ai/resources/whitepapers/optimizing-mlops-with-ml-mpm-powered-by-explainable-ai\\" target=\\"_blank\\">Model Performance Management Platform</a>:</p>\\n<ol>\\n<li>Ensure your model has data capture enabled.</li>\n<li>Create a Fiddler trial environment.</li>\n<li>Register information about your model in your Fiddler environment.</li>\n<li>Create an <a href=\\"http://aws.amazon.com/lambda\\" target=\\"_blank\\">AWS Lambda</a> function to publish SageMaker inferences to Fiddler.</li>\\n<li>Explore Fiddler’s monitoring capabilities in your Fiddler trial environment.</li>\n</ol>\\n<h3><a id=\\"Prerequisites_28\\"></a><strong>Prerequisites</strong></h3>\\n<p>This post assumes that you have set up SageMaker and deployed a model endpoint. To learn how to configure SageMaker for model serving, refer to <a href=\\"https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model.html\\" target=\\"_blank\\">Deploy Models for Inference</a>. Some examples are also available on the <a href=\\"https://github.com/aws/amazon-sagemaker-examples\\" target=\\"_blank\\">GitHub repo</a>.</p>\\n<h3><a id=\\"Ensure_your_model_has_data_capture_enabled_32\\"></a><strong>Ensure your model has data capture enabled</strong></h3>\\n<p>On the SageMaker console, navigate to your model’s serving endpoint and ensure you have enabled <a href=\\"https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-data-capture.html\\" target=\\"_blank\\">data capture</a> into an <a href=\\"http://aws.amazon.com/s3\\" target=\\"_blank\\">Amazon Simple Storage Service</a> ([Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail)) bucket. This stores the inferences (requests and responses) your model makes each day as <a href=\\"https://jsonlines.org/\\" target=\\"_blank\\">JSON lines files</a> (.jsonl) in [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail).</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/e4e19dce72464e9f923aa6ce1b539b97_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Create_a_Fiddler_trial_environment_40\\"></a><strong>Create a Fiddler trial environment</strong></h3>\\n<p>From the <a href=\\"http://fiddler.ai/trial\\" target=\\"_blank\\">fiddler.ai</a> website, you can request a free trial. After filling out a quick form, Fiddler will contact you to understand the specifics of your model performance management needs and will have a trial environment ready for you in a few hours. You can expect a dedicated environment like <a href=\\"https://yourcompany.try.fiddler.ai/\\" target=\\"_blank\\">https://yourcompany.try.fiddler.ai</a>.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/ba465bf005fc414f917dd9da2e21da00_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Register_information_about_your_model_in_your_Fiddler_environment_46\\"></a><strong>Register information about your model in your Fiddler environment</strong></h3>\\n<p>Before you can begin publishing events from your SageMaker hosted model into Fiddler, you need to create a project within your Fiddler trial environment and provide Fiddler details about your model through a step called model registration. If you want to use a preconfigured notebook from within <a href=\\"https://aws.amazon.com/sagemaker/studio/\\" target=\\"_blank\\">Amazon SageMaker Studio</a> rather than copy and paste the following code snippets, you can reference the Fiddler quickstart notebook on <a href=\\"https://github.com/fiddler-labs/fiddler-samples/blob/master/sagemaker/RegisterModelWithFiddler.ipynb\\" target=\\"_blank\\">GitHub</a>. Studio provides a single web-based visual interface where you can perform all ML development steps.</p>\\n<p>First, you must install the <a href=\\"https://pypi.org/project/fiddler-client/\\" target=\\"_blank\\">Fiddler Python</a> client in your SageMaker notebook and instantiate the Fiddler client. You can get the <code>AUTH_TOKEN</code> from the <strong>Settings</strong> page in your Fiddler trial environment.</p>\\n<pre><code class=\\"lang-\\">\\n# Install the fiddler client\\n!pip install fiddler-client\\n\\n# Connect to the Fiddler Trial Environment\\nimport fiddler as fdl\\nimport pandas as pd\\n\\nfdl.__version__\\n\\nURL = 'https://yourcompany.try.fiddler.ai'\\nORG_ID = 'yourcompany'\\nAUTH_TOKEN = 'UUID-Token-Here-Found-In-Your-Fiddler-Env-Settings-Page'\\n\\nclient = fdl.FiddlerApi(URL, ORG_ID, AUTH_TOKEN)\\n\\n</code></pre>\\n<p>Next, create a project within your Fiddler trial environment:</p>\n<pre><code class=\\"lang-\\">\\n# Create Project\\nPROJECT_ID = 'credit_default' # update this with your project name\\nDATASET_ID = f'{PROJECT_ID}_dataset'\\nMODEL_ID = f'{PROJECT_ID}_model'\\n\\nclient.create_project(PROJECT_ID)\\n\\n</code></pre>\\n<p>Now upload your training dataset. The notebook also provides a sample dataset to run Fiddler’s <a href=\\"https://www.fiddler.ai/explainable-ai\\" target=\\"_blank\\">explainability</a> algorithms and as a baseline for monitoring metrics. The dataset is also used to generate the schema for this model in Fiddler.</p>\\n<pre><code class=\\"lang-\\">\\n# Upload Baseline Dataset\\ndf_baseline = pd.read_csv(‘&lt;your-training-file.csv&gt;')\\n\\ndataset_info = fdl.DatasetInfo.from_dataframe(df_baseline, max_inferred_cardinality=1000)\\n\\nupload_result = client.upload_dataset(PROJECT_ID,\\n dataset={'baseline': df_baseline},\\n dataset_id=DATASET_ID,\\n info=dataset_info)\\n\\n</code></pre>\\n<p>Lastly, before you can start publishing inferences to Fiddler for monitoring, root cause analysis, and explanations, you need to register your model. Let’s first create a <code>model_info</code> object that contains the metadata about your model:</p>\\n<pre><code class=\\"lang-\\">\\n# Update task from the list below if your model task is not binary classification\\nmodel_task = 'binary' \\n\\nif model_task == 'regression':\\n model_task_fdl = fdl.ModelTask.REGRESSION\\n \\nelif model_task == 'binary':\\n model_task_fdl = fdl.ModelTask.BINARY_CLASSIFICATION\\n\\nelif model_task == 'multiclass':\\n model_task_fdl = fdl.ModelTask.MULTICLASS_CLASSIFICATION\\n\\nelif model_task == 'ranking':\\n model_task_fdl = fdl.ModelTask.RANKING\\n\\n \\n# Specify column types|\\ntarget = 'TARGET'\\noutputs = ['prediction'] # change this to your target variable\\nfeatures = [‘&lt;add your feature list here&gt;’]\\n \\n# Generate ModelInfo\\nmodel_info = fdl.ModelInfo.from_dataset_info(\\n dataset_info=dataset_info,\\n dataset_id=DATASET_ID,\\n model_task=model_task_fdl,\\n target=target,\\n outputs=outputs,\\n features=features,\\n binary_classification_threshold=.125, # update this if your task is not a binary classification\\n description='&lt;model-description&gt;',\\n display_name='&lt;model-display-name&gt;'\\n)\\nmodel_info\\n\\n</code></pre>\\n<p>Then you can register the model using your new <code>model_info</code> object:</p>\\n<pre><code class=\\"lang-\\">\\n# Register Info about your model with Fiddler\\nclient.register_model(\\n project_id=PROJECT_ID,\\n dataset_id=DATASET_ID,\\n model_id=MODEL_ID,\\n model_info=model_info\\n)\\n\\n</code></pre>\\n<p>Great! Now you can publish some events to Fiddler in order to observe the model’s performance.</p>\n<h3><a id=\\"Create_a_Lambda_function_to_publish_SageMaker_inferences_to_Fiddler_162\\"></a><strong>Create a Lambda function to publish SageMaker inferences to Fiddler</strong></h3>\\n<p>With the simple-to-deploy serverless architecture of Lambda, you can quickly build the mechanism required to move your inferences from the S3 bucket you set up earlier into your newly provisioned Fiddler trial environment. This Lambda function is responsible for opening any new JSONL event log files in your model’s S3 bucket, parsing and formatting the JSONL content into a dataframe, and then publishing that dataframe of events to your Fiddler trial environment. The following screenshot shows the code details of our function.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/db115d3a0365454dab4a6758474461d8_image.png\\" alt=\\"image.png\\" /></p>\n<p>The Lambda function needs to be configured to trigger off of newly created files in your S3 bucket. The following <a href=\\"https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html\\" target=\\"_blank\\">tutorial</a> guides you through creating an <a href=\\"https://aws.amazon.com/eventbridge/\\" target=\\"_blank\\">Amazon EventBridge</a> trigger that invokes the Lambda function whenever a file is uploaded to [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail). The following screenshot shows our function’s trigger configuration. This makes it simple to ensure that any time your model makes new inferences, those events stored in [Amazon S3](https://aws.amazon.com/cn/s3/?trk=cndc-detail) are loaded into Fiddler to drive the model observability your company needs.</p>\\n<p>To simplify this further, the code for this Lambda function is publicly available from <a href=\\"https://docs.fiddler.ai/docs/sagemaker-integration\\" target=\\"_blank\\">Fiddler’s documentation site</a> . This code example currently works for binary classification models with structured inputs. If you have model types with different features or tasks, please contact Fiddler for assistance with minor changes to the code.</p>\\n<p>The Lambda function needs to make reference to the Fiddler Python client. Fiddler has created a publicly available Lambda layer that you can reference to ensure that the <code>import fiddler as fdl</code> step works seamlessly. You can reference this layer via an ARN in the us-west-2 Region: <code>arn:aws:lambda:us-west-2:079310353266:layer:fiddler-client-0814:1</code>, as shown in the following screenshot.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/b59cb8e67c9f4b569aba4861c996cfed_image.png\\" alt=\\"image.png\\" /></p>\n<p>You also need to specify Lambda environment variables so the Lambda function knows how to connect to your Fiddler trial environment, and what the inputs and outputs are within the .jsonl files being captured by your model. The following screenshot shows a list of the required environment variables, which are also on <a href=\\"https://docs.fiddler.ai/docs/sagemaker-integration\\" target=\\"_blank\\">Fiddler’s documentation site</a>. Update the values for the environment variables to match your model and dataset.</p>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/f9cf4cdaa6994381928b3088f28384b1_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Explore_Fiddlers_monitoring_capabilities_in_your_Fiddler_trial_environment_182\\"></a><strong>Explore Fiddler’s monitoring capabilities in your Fiddler trial environment</strong></h3>\\n<p>You’ve done it! With your baseline data, model, and traffic connected, you can now explain <a href=\\"https://www.fiddler.ai/blog/drift-in-machine-learning-how-to-identify-issues-before-you-have-a-problem\\" target=\\"_blank\\">data drift</a>, outliers, <a href=\\"https://www.fiddler.ai/blog/ai-explained-understanding-bias-and-fairness-in-ai-systems\\" target=\\"_blank\\">model bias</a>, data issues, and performance blips, and share dashboards with others. Complete your journey by <a href=\\"https://www.fiddler.ai/watch-demo\\" target=\\"_blank\\">watching a demo</a> about the model performance management capabilities you have introduced to your company.</p>\\n<p>The example screenshots below provide a glimpse of model insights like drift, outlier detection, local point explanations, and model analytics that will be found in your Fiddler trial environment.</p>\n<p><img src=\\"https://dev-media.amazoncloud.cn/2c2b88afabee4e4786d9abbc96837701_image.png\\" alt=\\"image.png\\" /></p>\n<h3><a id=\\"Conclusion_190\\"></a><strong>Conclusion</strong></h3>\\n<p>This post highlighted the need for enterprise-class <a href=\\"https://www.fiddler.ai/ml-monitoring\\" target=\\"_blank\\">model monitoring</a> and showed how you can integrate your models deployed in SageMaker with the <a href=\\"https://www.fiddler.ai/\\" target=\\"_blank\\">Fiddler Model Performance Management Platform</a> in just a few steps. Fiddler offers functionality for model monitoring, explainable AI, bias detection, and root cause analysis, and is available on the <a href=\\"https://aws.amazon.com/marketplace/pp/prodview-imsoert6oehmm?sr=0-1&amp;ref_=beagle&amp;applicationId=AWSMPContessa\\" target=\\"_blank\\">AWS Marketplace</a>. By providing your <a href=\\"https://www.fiddler.ai/mlops\\" target=\\"_blank\\">MLOps</a> team with an easy-to-use single pane of glass to ensure your models are behaving as expected and to identify the underlying root causes of performance degradation, Fiddler can help improve data scientist productivity and reduce time to detect and resolve issues.</p>\\n<p>If you would like to learn more about Fiddler please visit <a href=\\"https://www.fiddler.ai/\\" target=\\"_blank\\">fiddler.ai</a> or if you would prefer to set up a personalized demo and technical discussion email <a href=\\"sales@fiddler.ai\\" target=\\"_blank\\">sales@fiddler.ai</a>.</p>\\n<h4><a id=\\"About_the_Authors_196\\"></a><strong>About the Authors</strong></h4>\\n<p><img src=\\"https://dev-media.amazoncloud.cn/e5bb1e3caec24de8a688a18923e3be39_image.png\\" alt=\\"image.png\\" /></p>\n<p><a href=\\"https://www.linkedin.com/in/dannybrock/\\" target=\\"_blank\\"><strong>Danny Brock</strong></a> is a Sr Solutions Engineer at Fiddler AI. Danny is long tenured in the analytics and ML space, running presales and post-sales teams for startups like Endeca and Incorta. He founded his own big data analytics consulting company, Branchbird, in 2012.</p>\n"}