R Serving with Plumber
This notebook’s CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.
Dockerfile
The Dockerfile defines the environment in which our server will be executed.
Below, you can see that the entrypoint for our container will be deploy.R
[ ]:
%pycat Dockerfile
Code: deploy.R
The deploy.R script handles the following steps: * Loads the R libraries used by the server. * Loads a pretrained xgboost
model that has been trained on the classical Iris dataset. * Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science. * Defines an inference function that takes a matrix of iris features and
returns predictions for those iris examples. * Finally, it imports the endpoints.R script and launches the Plumber server app using those endpoint definitions.
[ ]:
%pycat deploy.R
Code: endpoints.R
endpoints.R defines two routes: * /ping
returns a string ‘Alive’ to indicate that the application is healthy * /invocations
applies the previously defined inference function to the input features from the request body
For more information about the requirements for building your own inference container, see: Use Your Own Inference Code with Hosting Services
[ ]:
%pycat endpoints.R
Build the Serving Image
[ ]:
!docker build -t r-plumber .
Launch the Serving Container
[ ]:
!echo "Launching Plumber"
!docker run -d --rm -p 5000:8080 r-plumber
!echo "Waiting for the server to start.." && sleep 10
[ ]:
!docker container list
Define Simple Python Client
[ ]:
import requests
from tqdm import tqdm
import pandas as pd
pd.set_option("display.max_rows", 500)
[ ]:
def get_predictions(examples, instance=requests, port=5000):
payload = {"features": examples}
return instance.post(f"http://127.0.0.1:{port}/invocations", json=payload)
[ ]:
def get_health(instance=requests, port=5000):
instance.get(f"http://127.0.0.1:{port}/ping")
Define Example Inputs
Let’s define example inputs from the Iris dataset.
[ ]:
column_names = ["Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width", "Label"]
iris = pd.read_csv(
"s3://sagemaker-sample-files/datasets/tabular/iris/iris.data", names=column_names
)
[ ]:
iris_features = iris[["Sepal.Length", "Sepal.Width", "Petal.Length", "Petal.Width"]]
[ ]:
example_inputs = iris_features.values.tolist()
Plumber
[ ]:
predicted = get_predictions(example_inputs).json()["output"]
[ ]:
iris["predicted"] = predicted
[ ]:
iris
Stop All Serving Containers
Finally, we will shut down the serving container we launched for the test.
[ ]:
!docker kill $(docker ps -q)
Notebook CI Test Results
This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.