{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy and perform inference on Model Package from AWS Marketplace \n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This notebook provides you instructions on how to deploy and perform inference on model packages from AWS Marketplace image classification model.\n",
    "\n",
    "This notebook is compatible only with those image classification model packages which this notebook is linked to.\n",
    "\n",
    "## Pre-requisites:\n",
    "1. **Note**: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.\n",
    "1. Ensure that IAM role used has **AmazonSageMakerFullAccess**\n",
    "1. To deploy this ML model successfully, ensure that:\n",
    "    1. Either your IAM role has these three permissions and you have authority to make AWS Marketplace subscriptions in the AWS account used: \n",
    "        1. **aws-marketplace:ViewSubscriptions**\n",
    "        1. **aws-marketplace:Unsubscribe**\n",
    "        1. **aws-marketplace:Subscribe**  \n",
    "    2. or your AWS account has a subscription to this image classification model. If so, skip step: [Subscribe to the model package](#1.-Subscribe-to-the-model-package)\n",
    "\n",
    "## Contents\n",
    "1. [Subscribe to the model package](#1.-Subscribe-to-the-model-package)\n",
    "2. [Create an endpoint and perform real-time inference](#2.-Create-an-endpoint-and-perform-real-time-inference)\n",
    "   1. [Create an endpoint](#A.-Create-an-endpoint)\n",
    "   2. [Create input payload](#B.-Create-input-payload)\n",
    "   3. [Perform real-time inference](#C.-Perform-real-time-inference)\n",
    "   4. [Visualize output](#D.-Visualize-output)\n",
    "   5. [Delete the endpoint](#E.-Delete-the-endpoint)\n",
    "3. [Perform batch inference](#3.-Perform-batch-inference) \n",
    "4. [Clean-up](#4.-Clean-up)\n",
    "    1. [Delete the model](#A.-Delete-the-model)\n",
    "    2. [Unsubscribe to the listing (optional)](#B.-Unsubscribe-to-the-listing-(optional))\n",
    "    \n",
    "\n",
    "## Usage instructions\n",
    "You can run this notebook one cell at a time (By using Shift+Enter for running a cell).\n",
    "\n",
    "**Note** - This notebook requires you to follow instructions and specify values for parameters, as instructed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Subscribe to the model package"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To subscribe to the model package:\n",
    "1. Open the model package listing page you opened this notebook for.\n",
    "1. On the AWS Marketplace listing, click on the **Continue to subscribe** button.\n",
    "1. On the **Subscribe to this software** page, review and click on **\"Accept Offer\"** if you and your organization agrees with EULA, pricing, and support terms. \n",
    "1. Once you click on **Continue to configuration button** and then choose a **region**, you will see a **Product Arn** displayed. This is the model package ARN that you need to specify while creating a deployable model using Boto3. Copy the ARN corresponding to your region and specify the same in the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_package_arn = \"<Customer to specify Model package ARN corresponding to their AWS region>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from sagemaker import ModelPackage\n",
    "import sagemaker as sage\n",
    "from sagemaker import get_execution_role\n",
    "from IPython.core.display import Image, display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "role = get_execution_role()\n",
    "sagemaker_session = sage.Session()\n",
    "boto3 = sagemaker_session.boto_session\n",
    "bucket = sagemaker_session.default_bucket()\n",
    "region = sagemaker_session.boto_region_name\n",
    "\n",
    "s3 = boto3.client(\"s3\")\n",
    "runtime = boto3.client(\"runtime.sagemaker\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In next step, you would be deploying the model for real-time inference. For  information on how real-time inference with Amazon SageMaker works, see [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-hosting.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Create an endpoint and perform real-time inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"image-classification-model\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The image classification model packages this notebook notebook is compatible with, support application/x-image as the\n",
    "# content-type.\n",
    "content_type = \"application/x-image\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Review and update the compatible instance type for the model package in the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "real_time_inference_instance_type = \"ml.g4dn.xlarge\"\n",
    "batch_transform_inference_instance_type = \"ml.p2.xlarge\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Create an endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a deployable model from the model package.\n",
    "model = ModelPackage(\n",
    "    role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session\n",
    ")\n",
    "\n",
    "# Deploy the model\n",
    "predictor = model.deploy(1, real_time_inference_instance_type, endpoint_name=model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once endpoint has been created, you would be able to perform real-time inference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Prepare input file for performing real-time inference\n",
    "In this step, we will download class_id_to_label_mapping from S3 bucket. The mapping files has been downloaded from [TensorFlow](https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt). [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_bucket = f\"jumpstart-cache-prod-{region}\"\n",
    "key_prefix = \"inference-notebook-assets\"\n",
    "\n",
    "\n",
    "def download_from_s3(key_filenames):\n",
    "    for key_filename in key_filenames:\n",
    "        s3.download_file(s3_bucket, f\"{key_prefix}/{key_filename}\", key_filename)\n",
    "\n",
    "\n",
    "cat_jpg, dog_jpg, ImageNetLabels = \"cat.jpg\", \"dog.jpg\", \"ImageNetLabels.txt\"\n",
    "\n",
    "# Download images and label-mapping file.\n",
    "download_from_s3(key_filenames=[cat_jpg, dog_jpg, ImageNetLabels])\n",
    "\n",
    "display(Image(filename=\"cat.jpg\"))\n",
    "display(Image(filename=\"dog.jpg\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!head ImageNetLabels.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, open the downloaded images and load them in a variable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(ImageNetLabels, \"r\") as file:\n",
    "    class_id_to_label = file.read().splitlines()[1::]\n",
    "# The label file has 1001 class labels starting with 'background' class\n",
    "# but the model predicts 1000 classes sans the background class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C. Query endpoint that you have created with the opened images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# perform_inference method performs inference on the endpoint and prints predictions.\n",
    "def perform_inference(filename):\n",
    "    with open(filename, \"rb\") as file:\n",
    "        body = file.read()\n",
    "        response = runtime.invoke_endpoint(\n",
    "            EndpointName=model_name, ContentType=content_type, Body=body\n",
    "        )\n",
    "        prediction = json.loads(response[\"Body\"].read())\n",
    "        top5_prediction_ids = sorted(\n",
    "            range(len(prediction)), key=lambda index: prediction[index], reverse=True\n",
    "        )[:5]\n",
    "        top5_class_labels = \", \".join([class_id_to_label[id] for id in top5_prediction_ids])\n",
    "        print(\"Top-5 model predictions are: \" + top5_class_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "perform_inference(cat_jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "perform_inference(dog_jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### D. Delete the endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have successfully performed a real-time inference, you do not need the endpoint any more. You can terminate the endpoint to avoid being charged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.sagemaker_session.delete_endpoint(model_name)\n",
    "model.sagemaker_session.delete_endpoint_config(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Perform batch inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, you will perform batch inference using multiple input payloads together. If you are not familiar with batch transform, and want to learn more, see [How to run a batch transform job](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-batch.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# upload the batch-transform job input files to S3\n",
    "transform_input_key_prefix = \"image-classification-model-transform-input\"\n",
    "transform_input = sagemaker_session.upload_data(cat_jpg, key_prefix=transform_input_key_prefix)\n",
    "print(\"Transform input uploaded to \" + transform_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the batch-transform job\n",
    "transformer = model.transformer(1, batch_transform_inference_instance_type)\n",
    "transformer.transform(transform_input, content_type=content_type)\n",
    "transformer.wait()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# output is available on following path\n",
    "transformer.output_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_bucket_name, output_path = transformer.output_path.replace(\"s3://\", \"\").split(\"/\", 1)\n",
    "obj = s3.get_object(Bucket=output_bucket_name, Key=output_path + \"/cat.jpg.out\")\n",
    "batch_prediction = obj[\"Body\"].read().decode(\"utf-8\")\n",
    "\n",
    "# print out batch-transform job output\n",
    "print(batch_prediction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print labels extracted from batch transform output\n",
    "top5_prediction_ids = sorted(\n",
    "    range(len(batch_prediction)), key=lambda index: batch_prediction[index], reverse=True\n",
    ")[:5]\n",
    "top5_class_labels = \", \".join([class_id_to_label[id] for id in top5_prediction_ids])\n",
    "print(\"Top-5 model predictions are: \" + top5_class_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Clean-up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Delete the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.delete_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Unsubscribe to the listing (optional)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you would like to unsubscribe to the model package, follow these steps. Before you cancel the subscription, ensure that you do not have any [deployable model](https://console.aws.amazon.com/sagemaker/home#/models) created from the model package or using the algorithm. Note - You can find this information by looking at the container name associated with the model. \n",
    "\n",
    "**Steps to unsubscribe to product from AWS Marketplace**:\n",
    "1. Navigate to __Machine Learning__ tab on [__Your Software subscriptions page__](https://aws.amazon.com/marketplace/ai/library?productType=ml&ref_=mlmp_gitdemo_indust)\n",
    "2. Locate the listing that you want to cancel the subscription for, and then choose __Cancel Subscription__  to cancel the subscription.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/aws_marketplace|using-open-source-model-packages|pytorch-ic-model|using-image-classification-models.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "conda_pytorch_latest_p36",
   "language": "python",
   "name": "conda_pytorch_latest_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}