Deploy For Seller to update: Title_of_your_ML Model Model Package from AWS Marketplace

---

This notebook’s CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

---

For Seller to update: Add overview of the ML Model here

This sample notebook shows you how to deploy For Seller to update:Title_of_your_ML Model using Amazon SageMaker.

Note: This is a reference notebook and it cannot run unless you make changes suggested in the notebook.

Pre-requisites:

1. Note: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.

2. Ensure that IAM role used has AmazonSageMakerFullAccess

3. To deploy this ML model successfully, ensure that:

   1. Either your IAM role has these three permissions and you have authority to make AWS Marketplace subscriptions in the AWS account used:

      1. aws-marketplace:ViewSubscriptions

      2. aws-marketplace:Unsubscribe

      3. aws-marketplace:Subscribe

   2. or your AWS account has a subscription to For Seller to update:Title_of_your_ML Model. If so, skip step: Subscribe to the model package

Contents:

1. Subscribe to the model package

2. Create an endpoint and perform real-time inference

   1. Create an endpoint

   2. Create input payload

   3. Perform real-time inference

   4. Visualize output

   5. Delete the endpoint

3. Perform batch inference

4. Clean-up

   1. Delete the model

   2. Unsubscribe to the listing (optional)

Usage instructions

You can run this notebook one cell at a time (By using Shift+Enter for running a cell).

1. Subscribe to the model package

To subscribe to the model package: 1. Open the model package listing page For Seller to update:Title_of_your_product. 1. On the AWS Marketplace listing, click on the Continue to subscribe button. 1. On the Subscribe to this software page, review and click on “Accept Offer” if you and your organization agrees with EULA, pricing, and support terms. 1. Once you click on Continue to configuration button and then choose a region, you will see a Product Arn displayed. This is the model package ARN that you need to specify while creating a deployable model using Boto3. Copy the ARN corresponding to your region and specify the same in the following cell.

model_package_arn = "<Customer to specify Model package ARN corresponding to their AWS region>"

For Seller to update: Add all necessary imports in following cell, If you need specific packages to be installed, # try to provide them in this section, in a separate cell.

import base64
import json
import uuid
from sagemaker import ModelPackage
import sagemaker as sage
from sagemaker import get_execution_role
from sagemaker import ModelPackage
import boto3
from IPython.display import Image
from PIL import Image as ImageEdit
import numpy as np

role = get_execution_role()

sagemaker_session = sage.Session()

bucket = sagemaker_session.default_bucket()
runtime = boto3.client("runtime.sagemaker")
bucket

2. Create an endpoint and perform real-time inference

If you want to understand how real-time inference with Amazon SageMaker works, see Documentation.

For Seller to update: update values for four variables in following cell. Specify a model/endpoint name using only alphanumeric characters.

model_name = "For Seller to update:<specify-model_or_endpoint-name>"

content_type = "For Seller to update:<specify_content_type_accepted_by_model>"

real_time_inference_instance_type = (
    "For Seller to update:<Update recommended_real-time_inference instance_type>"
)
batch_transform_inference_instance_type = (
    "For Seller to update:<Update recommended_batch_transform_job_inference instance_type>"
)

A. Create an endpoint

# create a deployable model from the model package.
model = ModelPackage(
    role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session
)

# Deploy the model
predictor = model.deploy(1, real_time_inference_instance_type, endpoint_name=model_name)

Once endpoint has been created, you would be able to perform real-time inference.

B. Create input payload

For Seller to update: Add code snippet here that reads the input from ‘data/input/real-time/’ directory and converts it into format expected by the endpoint.

For Seller to update: Ensure that file_name variable points to the payload you created. Ensure that output_file_name variable points to a file-name in which output of real-time inference needs to be stored.

C. Perform real-time inference

For Seller to update: review/update file_name, output_file name, custom attributes in following AWS CLI to perform a real-time inference using the payload file you created from 2.B

!aws sagemaker-runtime invoke-endpoint --endpoint-name $model_name --body fileb://$file_name --content-type $content_type --region $sagemaker_session.boto_region_name $output_file_name

D. Visualize output

For Seller to update: Write code in following cell to display the output generated by real-time inference. This output must match with output available in data/output/real-time folder.

For Seller to update: Get innovative! This is also your opportunity to show-off different capabilities of the model. E.g. if your model does object detection, multi-class classification, or regression, repeat steps 2.B,2.C,2.D to show different inputs using files and outputs for different classes/objects/edge conditions.

E. Delete the endpoint

Now that you have successfully performed a real-time inference, you do not need the endpoint any more. You can terminate the endpoint to avoid being charged.

model.sagemaker_session.delete_endpoint(model_name)
model.sagemaker_session.delete_endpoint_config(model_name)

3. Perform batch inference

In this section, you will perform batch inference using multiple input payloads together. If you are not familiar with batch transform, and want to learn more, see these links: 1. How it works 2. How to run a batch transform job

# upload the batch-transform job input files to S3
transform_input_folder = "data/input/batch"
transform_input = sagemaker_session.upload_data(transform_input_folder, key_prefix=model_name)
print("Transform input uploaded to " + transform_input)

# Run the batch-transform job
transformer = model.transformer(1, batch_transform_inference_instance_type)
transformer.transform(transform_input, content_type=content_type)
transformer.wait()

# output is available on following path
transformer.output_path

For Seller to update: Add code that displays output generated by the batch transform job available in S3. This output must match the output available in data/output/batch folder.

4. Clean-up

A. Delete the model

model.delete_model()

B. Unsubscribe to the listing (optional)

If you would like to unsubscribe to the model package, follow these steps. Before you cancel the subscription, ensure that you do not have any deployable model created from the model package or using the algorithm. Note - You can find this information by looking at the container name associated with the model.

Steps to unsubscribe to product from AWS Marketplace: 1. Navigate to Machine Learning tab on Your Software subscriptions page 2. Locate the listing that you want to cancel the subscription for, and then choose Cancel Subscription to cancel the subscription.

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.