Amazon SageMaker scikit-learn Bring Your Own Model
This notebook’s CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.
Hosting a pre-trained scikit-learn Model in Amazon SageMaker scikit-learn Container
Background
Amazon SageMaker includes functionality to support a hosted notebook environment, distributed, serverless training, and real-time hosting. We think it works best when all three of these services are used together, but they can also be used independently. Some use cases may only require hosting. Maybe the model was trained prior to Amazon SageMaker existing, in a different service.
This notebook shows how to use a pre-trained scikit-learn model with the Amazon SageMaker scikit-learn container to quickly create a hosted endpoint for that model. We use the California Housing dataset, present in Scikit-Learn: https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html. The California Housing dataset was originally published in:
Pace, R. Kelley, and Ronald Barry. “Sparse spatial auto-regressions.” Statistics & Probability Letters 33.3 (1997): 291-297.
Setup
Let’s start by specifying:
AWS region.
The IAM role arn used to give learning and hosting access to your data.
The S3 bucket that you want to use for training and model data.
[ ]:
!pip install -U sagemaker
[ ]:
import os
import boto3
import re
import json
import pandas as pd
import numpy as np
import sagemaker
from sagemaker import get_execution_role
from sagemaker.sklearn.model import SKLearnModel
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
region = boto3.Session().region_name
role = get_execution_role()
bucket = sagemaker.Session().default_bucket()
prefix = "sagemaker/DEMO-sklearn-byo-model"
print(f"bucket: {bucket}")
Prepare data for model inference
We load the California housing dataset from sklearn, and will use it to invoke SageMaker Endpoint
[ ]:
data = fetch_california_housing()
[ ]:
X_train, X_test, y_train, y_test = train_test_split(
data.data, data.target, test_size=0.25, random_state=42
)
# we don't train a model, so we will need only the testing data
testX = pd.DataFrame(X_test, columns=data.feature_names)
[ ]:
testX.head(10)
Download a pre-trained model file
Download a pretrained Scikit-Learn Random Forest model.
We used the California Housing dataset, present in Scikit-Learn: https://scikit-learn.org/stable/datasets/real_world.html#california-housing-dataset to train the model.
For more details on how to train the model with Amazon SageMaker, please refer to the Develop, Train, Optimize and Deploy Scikit-Learn Random Forest notebook
[ ]:
!aws s3 cp s3://aws-ml-blog/artifacts/scikit_learn_bring_your_own_model/model.joblib .
Compressed the model file to a GZIP tar archive
Note that the model file name must satisfy the regular expression pattern: ^[a-zA-Z0-9](-*[a-zA-Z0-9])*;. The model file needs to be tar-zipped.
[ ]:
model_file_name = "model.joblib"
[ ]:
!tar czvf model.tar.gz $model_file_name
Upload the pre-trained model model.tar.gz file to S3
[ ]:
fObj = open("model.tar.gz", "rb")
key = os.path.join(prefix, "model.tar.gz")
boto3.Session().resource("s3").Bucket(bucket).Object(key).upload_fileobj(fObj)
Set up hosting for the model
This involves creating a SageMaker model from the model file previously uploaded to S3.
[ ]:
model_data = "s3://{}/{}".format(bucket, key)
print(f"model data: {model_data}")
Write the Inference Script
When using endpoints with the Amazon SageMaker managed Scikit Learn container, we need to provide an entry point script for inference that will at least load the saved model.
After the SageMaker model server has loaded your model by calling model_fn, SageMaker will serve your model. Model serving is the process of responding to inference requests, received by SageMaker InvokeEndpoint API calls.
We will implement also the predict_fn() function that takes the deserialized request object and performs inference against the loaded model.
We will now create this script and call it inference.py and store it at the root of a directory called code.
Note: You would modify the script below to implement your own inferencing logic.
Additional information on model loading and model serving for scikit-learn on SageMaker can be found in the SageMaker Scikit-learn Model Server documentation
There are also several functions for hosting which we won’t define, - input_fn() - Takes request data and deserializes the data into an object for prediction. - output_fn() - Takes the result of prediction and serializes this according to the response content type.
These will take on their default values as described SageMaker Scikit-learn Serve a Model documentation
[ ]:
!pygmentize ./code/inference.py
Installing additional Python dependencies
It also may be necessary to supply a requirements.txt file to ensure any necessary dependencies are installed in the container along with the script. For this script, in addition to the Python standard libraries, we showcase how to install the boto3 requests, and nltk libraries.
[ ]:
!pygmentize ./code/requirements.txt
Deploy with Python SDK
Here we showcase the process of creating a model from s3 artifacts, that could be used to deploy a model that was trained in a different session or even out of SageMaker.
[ ]:
model = SKLearnModel(
role=role,
model_data=model_data,
framework_version="1.2-1",
py_version="py3",
source_dir="code",
entry_point="inference.py",
)
Create endpoint
Lastly, you create the endpoint that serves up the model, through specifying the name and configuration defined above. The end result is an endpoint that can be validated and incorporated into production applications. This takes 5-10 minutes to complete.
[ ]:
%%time
predictor = model.deploy(instance_type="ml.t2.medium", initial_instance_count=1)
Validate the model for use
Now you can obtain the endpoint from the client library using the result from previous operations and generate classifications from the model using that endpoint.
Invoke with the Python SDK
Let’s generate the prediction for a single data point. We’ll pick one from the test data generated earlier.
[ ]:
# the SKLearnPredictor does the serialization from pandas for us
predictions = predictor.predict(testX[data.feature_names])
print(predictions)
Alternative: invoke with boto3
This is useful when invoking the model from external clients, e.g. Lambda Functions, or other micro-services.
[ ]:
runtime = boto3.client("sagemaker-runtime")
Option 1: csv serialization
[ ]:
# csv serialization
response = runtime.invoke_endpoint(
EndpointName=predictor.endpoint,
Body=testX[data.feature_names].to_csv(header=False, index=False).encode("utf-8"),
ContentType="text/csv",
)
print(response["Body"].read())
Option 2: npy serialization
[ ]:
# npy serialization
from io import BytesIO
# Serialise numpy ndarray as bytes
buffer = BytesIO()
# Assuming testX is a data frame
np.save(buffer, testX[data.feature_names].values)
response = runtime.invoke_endpoint(
EndpointName=predictor.endpoint, Body=buffer.getvalue(), ContentType="application/x-npy"
)
print(response["Body"].read())
(Optional) Delete the Endpoint
If you’re ready to be done with this notebook, please run the delete_endpoint line in the cell below. This will remove the hosted endpoint you created and avoid any charges from a stray instance being left on.
[ ]:
predictor.delete_endpoint()
Conclusion
In this notebook you successfully deployed a pre-trained scikit-learn model with the Amazon SageMaker scikit-learn container to quickly create a hosted endpoint for that model. You then used the Python SDK and boto3 to invoke the endpoint with csv payload, and then with npy payload to get predictions from the model.
As next steps you can try to Automatically Scale Amazon SageMaker Models, Register and Deploy Models with Model Registry or Train your Model with Amazon SageMaker.
Notebook CI Test Results
This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.