Use SageMaker Batch Transform for PyTorch Batch Inference
This notebook’s CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.
In this notebook, we examine how to do a Batch Transform task with PyTorch in Amazon SageMaker.
First, an image classification model is built on the MNIST dataset. Then, we demonstrate batch transform by using the SageMaker Python SDK PyTorch framework with different configurations: - data_type=S3Prefix: uses all objects that match the specified S3 prefix for batch inference. - data_type=ManifestFile: a manifest file contains a list of object keys to use in batch inference. - instance_count>1: distributes the batch inference dataset to multiple inference instances.
For batch transform in TensorFlow in Amazon SageMaker, you can follow other Jupyter notebooks in the sagemaker_batch_transform directory.
Runtime
This notebook takes approximately 15 minutes to run.
Contents
Setup
We’ll begin with some necessary installs and imports, and get an Amazon SageMaker session to help perform certain tasks, as well as an IAM role with the necessary permissions.
[2]:
!pip install nvidia-ml-py3
!yes | pip uninstall torchvision
!pip install torchvision
Requirement already satisfied: nvidia-ml-py3 in /opt/conda/lib/python3.6/site-packages (7.352.0)
Found existing installation: torchvision 0.5.0+cpu
Uninstalling torchvision-0.5.0+cpu:
Would remove:
/opt/conda/lib/python3.6/site-packages/torchvision-0.5.0+cpu.dist-info/*
/opt/conda/lib/python3.6/site-packages/torchvision/*
Proceed (y/n)? Successfully uninstalled torchvision-0.5.0+cpu
yes: standard output: Broken pipe
Collecting torchvision
Downloading torchvision-0.11.2-cp36-cp36m-manylinux1_x86_64.whl (23.3 MB)
|████████████████████████████████| 23.3 MB 5.9 MB/s eta 0:00:01
Collecting torch==1.10.1
Downloading torch-1.10.1-cp36-cp36m-manylinux1_x86_64.whl (881.9 MB)
|████████████████████████████████| 881.9 MB 4.4 kB/s s eta 0:00:01 |█▊ | 46.2 MB 47.4 MB/s eta 0:00:18
Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from torchvision) (1.19.5)
Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /opt/conda/lib/python3.6/site-packages (from torchvision) (8.1.2)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.6/site-packages (from torch==1.10.1->torchvision) (3.7.4.3)
Requirement already satisfied: dataclasses in /opt/conda/lib/python3.6/site-packages (from torch==1.10.1->torchvision) (0.8)
Installing collected packages: torch, torchvision
Attempting uninstall: torch
Found existing installation: torch 1.4.0
Uninstalling torch-1.4.0:
Successfully uninstalled torch-1.4.0
Successfully installed torch-1.10.1 torchvision-0.11.2
[3]:
%matplotlib inline
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
import numpy as np
import os
from os import listdir
from os.path import isfile, join
from shutil import copyfile
import sagemaker
from sagemaker.pytorch import PyTorchModel
from sagemaker import get_execution_role
sagemaker_session = sagemaker.Session()
role = get_execution_role()
bucket = sagemaker_session.default_bucket()
prefix = "sagemaker/DEMO-pytorch-batch-inference-script"
print("Bucket: {}".format(bucket))
Bucket: sagemaker-us-west-2-521695447989
Model training
Since the main purpose of this notebook is to demonstrate SageMaker PyTorch batch transform, we reuse a SageMaker Python SDK PyTorch MNIST example to train a PyTorch model. It takes around 7 minutes to finish the training.
[4]:
from torchvision.datasets import MNIST
from torchvision import transforms
local_dir = "data"
MNIST.mirrors = ["https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/"]
MNIST(
local_dir,
download=True,
transform=transforms.Compose(
[transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]
),
)
inputs = sagemaker_session.upload_data(path=local_dir, bucket=bucket, key_prefix=prefix)
print("input spec (in this case, just an S3 path): {}".format(inputs))
from sagemaker.pytorch import PyTorch
estimator = PyTorch(
entry_point="model-script/mnist.py",
role=role,
framework_version="1.8.0",
py_version="py3",
instance_count=3,
instance_type="ml.c5.2xlarge",
hyperparameters={
"epochs": 1,
"backend": "gloo",
}, # set epochs to a more realistic number for real training
)
estimator.fit({"training": inputs})
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-images-idx3-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz
Extracting data/MNIST/raw/train-images-idx3-ubyte.gz to data/MNIST/raw
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-labels-idx1-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-labels-idx1-ubyte.gz to data/MNIST/raw/train-labels-idx1-ubyte.gz
Extracting data/MNIST/raw/train-labels-idx1-ubyte.gz to data/MNIST/raw
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-images-idx3-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-images-idx3-ubyte.gz to data/MNIST/raw/t10k-images-idx3-ubyte.gz
Extracting data/MNIST/raw/t10k-images-idx3-ubyte.gz to data/MNIST/raw
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-labels-idx1-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-labels-idx1-ubyte.gz to data/MNIST/raw/t10k-labels-idx1-ubyte.gz
Extracting data/MNIST/raw/t10k-labels-idx1-ubyte.gz to data/MNIST/raw
input spec (in this case, just an S3 path): s3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script
2022-04-18 00:17:13 Starting - Starting the training job...
2022-04-18 00:17:33 Starting - Preparing the instances for trainingProfilerReport-1650241033: InProgress
.........
2022-04-18 00:19:13 Downloading - Downloading input data...
...
INFO:__main__:Test set: Average loss: 0.4341, Accuracy: 8826/10000 (88%)
2022-04-18 00:20:14,348 sagemaker-training-toolkit INFO Reporting training SUCCESS
2022-04-18 00:20:39 Uploading - Uploading generated training model
2022-04-18 00:20:39 Completed - Training job completed
Training seconds: 216
Billable seconds: 216
Prepare batch inference data
Convert the test data into PNG image format.
[5]:
!ls data/MNIST/raw
t10k-images-idx3-ubyte train-images-idx3-ubyte
t10k-images-idx3-ubyte.gz train-images-idx3-ubyte.gz
t10k-labels-idx1-ubyte train-labels-idx1-ubyte
t10k-labels-idx1-ubyte.gz train-labels-idx1-ubyte.gz
[6]:
# untar gz => png
import gzip
import numpy as np
import os
with gzip.open(os.path.join(local_dir, "MNIST/raw", "t10k-images-idx3-ubyte.gz"), "rb") as f:
images = np.frombuffer(f.read(), np.uint8, offset=16).reshape(-1, 28, 28)
[7]:
print(len(images), "test images")
10000 test images
Randomly sample 100 test images and upload them to S3.
[8]:
import random
from PIL import Image as im
ids = random.sample(range(len(images)), 100)
ids = np.array(ids, dtype=np.int)
selected_images = images[ids]
image_dir = "data/images"
if not os.path.exists(image_dir):
os.makedirs(image_dir)
for i, img in enumerate(selected_images):
pngimg = im.fromarray(img)
pngimg.save(os.path.join(image_dir, f"{i}.png"))
[9]:
inference_prefix = "batch_transform"
inference_inputs = sagemaker_session.upload_data(
path=image_dir, bucket=bucket, key_prefix=inference_prefix
)
print("Input S3 path: {}".format(inference_inputs))
Input S3 path: s3://sagemaker-us-west-2-000000000000/batch_transform
Create model transformer
Now, we create a transformer object for creating and interacting with Amazon SageMaker transform jobs. We can create the transformer in two ways: 1. Use a fitted estimator directly. 1. First create a PyTorchModel from a saved model artifact, and then create a transformer from the PyTorchModel object.
Here, we implement the model_fn, input_fn, predict_fn and output_fn function to override the default PyTorch inference handler.
In the input_fn() function, the inferenced images are encoded as a Python ByteArray. That’s why we use the load_from_bytearray() function to load images from io.BytesIO and then use PIL.image to read the images.
def model_fn(model_dir):
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = torch.nn.DataParallel(Net())
with open(os.path.join(model_dir, "model.pth"), "rb") as f:
model.load_state_dict(torch.load(f))
return model.to(device)
def load_from_bytearray(request_body):
image_as_bytes = io.BytesIO(request_body)
image = Image.open(image_as_bytes)
image_tensor = ToTensor()(image).unsqueeze(0)
return image_tensor
def input_fn(request_body, request_content_type):
# if set content_type as "image/jpg" or "application/x-npy",
# the input is also a python bytearray
if request_content_type == "application/x-image":
image_tensor = load_from_bytearray(request_body)
else:
print("not support this type yet")
raise ValueError("not support this type yet")
return image_tensor
# Perform prediction on the deserialized object, with the loaded model
def predict_fn(input_object, model):
output = model.forward(input_object)
pred = output.max(1, keepdim=True)[1]
return {"predictions": pred.item()}
# Serialize the prediction result into the desired response content type
def output_fn(predictions, response_content_type):
return json.dumps(predictions)
[10]:
# Use fitted estimator directly
transformer = estimator.transformer(instance_count=1, instance_type="ml.c5.xlarge")
[11]:
# You can also create a Transformer object from saved model artifact
# Get model artifact location by estimator.model_data, or give an S3 key directly
model_artifact_s3_location = estimator.model_data # "s3://<BUCKET>/<PREFIX>/model.tar.gz"
# Create PyTorchModel from saved model artifact
pytorch_model = PyTorchModel(
model_data=model_artifact_s3_location,
role=role,
framework_version="1.8.0",
py_version="py3",
source_dir="model-script/",
entry_point="mnist.py",
)
# Create transformer from PyTorchModel object
transformer = pytorch_model.transformer(instance_count=1, instance_type="ml.c5.xlarge")
Batch inference
Next, we perform inference on the sampled 100 MNIST images in a batch manner.
Input images directly from S3 location
We set S3DataType=S3Prefix to use all objects that match the specified S3 prefix for batch inference.
[12]:
transformer.transform(
data=inference_inputs,
data_type="S3Prefix",
content_type="application/x-image",
wait=True,
)
......................2022-04-18 00:25:31,379 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.3.0
TS Home: /opt/conda/lib/python3.6/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 948 M
Python executable: /opt/conda/bin/python3.6
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2022-04-18 00:25:31,409 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-04-18 00:25:31,427 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag 5b0df4a34bfd419791760e1c82fe8572
2022-04-18 00:25:31,437 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-04-18 00:25:31,461 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-04-18 00:25:31,623 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:25:31,627 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]47
2022-04-18 00:25:31,627 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,627 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,640 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:25:31,656 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:25:31,663 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:25:31,663 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]48
2022-04-18 00:25:31,663 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]50
2022-04-18 00:25:31,664 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,664 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,664 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,666 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:25:31,664 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:25:31,667 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,668 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:25:31,672 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]49
2022-04-18 00:25:31,673 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,673 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,673 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:25:31,679 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-04-18 00:25:31,679 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2022-04-18 00:25:31,683 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9002.
2022-04-18 00:25:31,683 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9001.
2022-04-18 00:25:31,683 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9003.
2022-04-18 00:25:31,683 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9000.
2022-04-18 00:25:31,687 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
Model server started.
2022-04-18 00:25:32,368 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,370 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:48.106632232666016|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,370 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:7.8092803955078125|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,371 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:14.0|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,371 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:6416.7265625|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,372 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:931.8046875|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,372 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:15.8|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,760 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 978
2022-04-18 00:25:32,785 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 982
2022-04-18 00:25:32,784 [INFO ] W-9002-model_1 TS_METRICS - W-9002-model_1.ms:1339|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,786 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:119|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:32,785 [INFO ] W-9003-model_1 TS_METRICS - W-9003-model_1.ms:1340|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,787 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:116|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:32,810 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 982
2022-04-18 00:25:32,810 [INFO ] W-9000-model_1 TS_METRICS - W-9000-model_1.ms:1368|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,811 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:140|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:32,834 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1052
2022-04-18 00:25:32,834 [INFO ] W-9001-model_1 TS_METRICS - W-9001-model_1.ms:1389|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,834 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:93|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,399 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:34650 "GET /ping HTTP/1.1" 200 13
2022-04-18 00:25:37,400 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,435 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:34658 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:25:37,436 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 19
2022-04-18 00:25:37,399 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:34650 "GET /ping HTTP/1.1" 200 13
2022-04-18 00:25:37,400 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,435 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:34658 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:25:37,436 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 19
2022-04-18T00:25:37.442:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,509 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:10.01|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b991ddd1-11b8-4246-817f-e234377cde44,timestamp:1650241537
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 14
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,615 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:12.89|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e58006b2-3494-4f9d-b427-b2c05a48234a,timestamp:1650241537
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,509 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:10.01|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b991ddd1-11b8-4246-817f-e234377cde44,timestamp:1650241537
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 14
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,615 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:12.89|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e58006b2-3494-4f9d-b427-b2c05a48234a,timestamp:1650241537
2022-04-18 00:25:37,738 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 10
2022-04-18 00:25:37,738 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 10
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:d408fdeb-76de-4ce8-bd12-a1eea1dd98c8,timestamp:1650241537
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,865 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,866 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.9|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:968865ef-ce8f-457c-b5e7-05c15656c9c1,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fe21acba-2b6f-4c7e-8fab-23849016df58,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b3ac9ae1-a77f-486c-98c0-55a1d973d768,timestamp:1650241538
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2428a546-97b7-434d-a57d-039aec65a617,timestamp:1650241538
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:dc5c0414-007c-4371-be4c-f69d96ed0720,timestamp:1650241538
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:35432e1d-1b13-4ac8-8e91-0d1c0167946f,timestamp:1650241538
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:d408fdeb-76de-4ce8-bd12-a1eea1dd98c8,timestamp:1650241537
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,865 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,866 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.9|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:968865ef-ce8f-457c-b5e7-05c15656c9c1,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fe21acba-2b6f-4c7e-8fab-23849016df58,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b3ac9ae1-a77f-486c-98c0-55a1d973d768,timestamp:1650241538
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2428a546-97b7-434d-a57d-039aec65a617,timestamp:1650241538
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:dc5c0414-007c-4371-be4c-f69d96ed0720,timestamp:1650241538
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:35432e1d-1b13-4ac8-8e91-0d1c0167946f,timestamp:1650241538
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,460 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:3.08|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:84f54fd8-bbb2-4ab3-a3ca-610b7242dbf9,timestamp:1650241538
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.93|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:cae123fa-f9f3-40ad-abfd-5bc1040046ac,timestamp:1650241538
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,460 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:3.08|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:84f54fd8-bbb2-4ab3-a3ca-610b7242dbf9,timestamp:1650241538
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.93|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:cae123fa-f9f3-40ad-abfd-5bc1040046ac,timestamp:1650241538
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,670 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:80ac25bc-b2d8-4d7d-b3b6-e329488bd6ea,timestamp:1650241538
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.37|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fb0b9838-d516-441e-9442-976cd42aa26a,timestamp:1650241538
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:54f2e26b-f633-41c1-8176-ea707c86c4fa,timestamp:1650241539
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,198 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1bdb78d-2a97-4035-8f1c-3ddcffabfb32,timestamp:1650241539
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,670 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:80ac25bc-b2d8-4d7d-b3b6-e329488bd6ea,timestamp:1650241538
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.37|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fb0b9838-d516-441e-9442-976cd42aa26a,timestamp:1650241538
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:54f2e26b-f633-41c1-8176-ea707c86c4fa,timestamp:1650241539
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,198 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1bdb78d-2a97-4035-8f1c-3ddcffabfb32,timestamp:1650241539
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 6
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 7
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:026f3936-0c77-4f36-8e0e-56ecb4ae20c9,timestamp:1650241539
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 6
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 7
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:026f3936-0c77-4f36-8e0e-56ecb4ae20c9,timestamp:1650241539
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2a8d0ca9-6f8e-422a-906c-b2f203a2f552,timestamp:1650241539
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,789 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:56f4d62c-2a83-48cb-98e6-6e1cfe075e9b,timestamp:1650241539
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2a8d0ca9-6f8e-422a-906c-b2f203a2f552,timestamp:1650241539
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,789 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:56f4d62c-2a83-48cb-98e6-6e1cfe075e9b,timestamp:1650241539
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:94bf65ca-213c-4b82-a2b1-389731a59dfb,timestamp:1650241539
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,971 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3839c8f1-0fff-4907-b167-6e954c9b6249,timestamp:1650241540
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:94bf65ca-213c-4b82-a2b1-389731a59dfb,timestamp:1650241539
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,971 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3839c8f1-0fff-4907-b167-6e954c9b6249,timestamp:1650241540
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:721d9d18-93a1-4dbc-babe-098a5c83db10,timestamp:1650241540
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.09|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f98f09f-a587-4c18-88c8-1c667a55fa66,timestamp:1650241540
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:51c658e4-bbbf-48ce-9dd4-19025f15637c,timestamp:1650241540
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,409 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ee677be-3ab1-427b-bb39-cc2058eb1d5c,timestamp:1650241540
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e33199cc-ed87-4655-8303-4a1541c90d9a,timestamp:1650241540
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.54|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c0a5cfd2-a0b5-4cd1-a471-0789f663bdc8,timestamp:1650241540
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:721d9d18-93a1-4dbc-babe-098a5c83db10,timestamp:1650241540
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.09|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f98f09f-a587-4c18-88c8-1c667a55fa66,timestamp:1650241540
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:51c658e4-bbbf-48ce-9dd4-19025f15637c,timestamp:1650241540
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,409 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ee677be-3ab1-427b-bb39-cc2058eb1d5c,timestamp:1650241540
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e33199cc-ed87-4655-8303-4a1541c90d9a,timestamp:1650241540
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.54|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c0a5cfd2-a0b5-4cd1-a471-0789f663bdc8,timestamp:1650241540
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,663 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,663 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f064f2b-4452-4892-986a-442e4a1bd239,timestamp:1650241540
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4a8d9a35-2d3f-44c9-a6b9-08754e99a96b,timestamp:1650241540
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f064f2b-4452-4892-986a-442e4a1bd239,timestamp:1650241540
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4a8d9a35-2d3f-44c9-a6b9-08754e99a96b,timestamp:1650241540
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a822bc7-0452-4d01-919f-93820a29ef52,timestamp:1650241540
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,889 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.21|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9e36891e-4df0-4ef8-949b-a195ba717ce8,timestamp:1650241540
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.34|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1ba220c-951d-46de-93d6-ba262e321037,timestamp:1650241541
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:635c2e75-2046-4545-a2b3-ba5bc7494a9d,timestamp:1650241541
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:25e16120-8982-4d72-b6ea-6f836e9b6ec0,timestamp:1650241541
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:afa20143-40ae-4a1a-b2e9-2b10fd63dbf8,timestamp:1650241541
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4d69029d-4109-4ff4-9aae-a2830028fc3a,timestamp:1650241541
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.46|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0d9ab440-6082-4968-ab87-056513209b41,timestamp:1650241541
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a822bc7-0452-4d01-919f-93820a29ef52,timestamp:1650241540
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,889 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.21|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9e36891e-4df0-4ef8-949b-a195ba717ce8,timestamp:1650241540
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.34|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1ba220c-951d-46de-93d6-ba262e321037,timestamp:1650241541
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:635c2e75-2046-4545-a2b3-ba5bc7494a9d,timestamp:1650241541
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:25e16120-8982-4d72-b6ea-6f836e9b6ec0,timestamp:1650241541
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:afa20143-40ae-4a1a-b2e9-2b10fd63dbf8,timestamp:1650241541
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4d69029d-4109-4ff4-9aae-a2830028fc3a,timestamp:1650241541
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.46|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0d9ab440-6082-4968-ab87-056513209b41,timestamp:1650241541
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:37ce426c-151d-4f19-93d3-6312bb589d2a,timestamp:1650241541
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:155552ff-0166-491b-87a1-768065c651eb,timestamp:1650241541
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:37ce426c-151d-4f19-93d3-6312bb589d2a,timestamp:1650241541
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:155552ff-0166-491b-87a1-768065c651eb,timestamp:1650241541
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:efd19fcf-4c61-4b57-b1e3-ee7701885de4,timestamp:1650241541
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,984 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b907dd56-c939-4416-b367-4e97b4013616,timestamp:1650241541
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:92197732-5c4e-440f-953d-9d84731a8658,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f9e9bb4-2a82-4169-af17-d6c45d523bb9,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,158 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ac4dc55c-1762-4cee-9259-60436b74a592,timestamp:1650241542
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a56fabe-4973-40be-9fd7-0484202dfae6,timestamp:1650241542
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.74|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:83aba925-0481-4b31-99ab-0f471176bea3,timestamp:1650241542
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db6d82d7-7539-43af-b399-6b2e68e396c7,timestamp:1650241542
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,594 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:445b172a-d3e4-46ac-a88e-09690b0c18dc,timestamp:1650241542
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:efd19fcf-4c61-4b57-b1e3-ee7701885de4,timestamp:1650241541
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,984 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b907dd56-c939-4416-b367-4e97b4013616,timestamp:1650241541
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:92197732-5c4e-440f-953d-9d84731a8658,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f9e9bb4-2a82-4169-af17-d6c45d523bb9,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,158 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ac4dc55c-1762-4cee-9259-60436b74a592,timestamp:1650241542
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a56fabe-4973-40be-9fd7-0484202dfae6,timestamp:1650241542
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.74|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:83aba925-0481-4b31-99ab-0f471176bea3,timestamp:1650241542
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db6d82d7-7539-43af-b399-6b2e68e396c7,timestamp:1650241542
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,594 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:445b172a-d3e4-46ac-a88e-09690b0c18dc,timestamp:1650241542
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:681542c4-b2f6-4374-8fc1-1548e8649613,timestamp:1650241542
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:654cf5fa-63b1-4876-a975-61cfcdc68e88,timestamp:1650241542
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:681542c4-b2f6-4374-8fc1-1548e8649613,timestamp:1650241542
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:654cf5fa-63b1-4876-a975-61cfcdc68e88,timestamp:1650241542
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:7680909e-dbcf-480f-bb04-9873c45eccd3,timestamp:1650241542
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:7680909e-dbcf-480f-bb04-9873c45eccd3,timestamp:1650241542
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b6db0553-b1ec-414a-beb1-23e061b1454c,timestamp:1650241542
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c9756dad-5c2e-4e3e-b5fd-00d316df8dc4,timestamp:1650241543
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1769f16d-29a3-482c-bd98-efde53e2ffe4,timestamp:1650241543
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:8bc6d547-caf2-4b33-8259-f58b7324eb3a,timestamp:1650241543
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.31|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0a3a5e29-be58-4d46-ba27-38458886f11c,timestamp:1650241543
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2df74a80-1984-4e71-8529-dfe9cf90a8f6,timestamp:1650241543
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b6db0553-b1ec-414a-beb1-23e061b1454c,timestamp:1650241542
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c9756dad-5c2e-4e3e-b5fd-00d316df8dc4,timestamp:1650241543
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1769f16d-29a3-482c-bd98-efde53e2ffe4,timestamp:1650241543
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:8bc6d547-caf2-4b33-8259-f58b7324eb3a,timestamp:1650241543
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.31|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0a3a5e29-be58-4d46-ba27-38458886f11c,timestamp:1650241543
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2df74a80-1984-4e71-8529-dfe9cf90a8f6,timestamp:1650241543
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e3e047b3-424b-42e6-8708-d18083d36bdc,timestamp:1650241543
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e3e047b3-424b-42e6-8708-d18083d36bdc,timestamp:1650241543
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,679 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41139c2c-6e37-44d9-b3fa-ed7a7d79b0a5,timestamp:1650241543
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e4dd2c-1358-4f16-934b-e6a349809496,timestamp:1650241543
2022-04-18 00:25:43,679 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41139c2c-6e37-44d9-b3fa-ed7a7d79b0a5,timestamp:1650241543
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e4dd2c-1358-4f16-934b-e6a349809496,timestamp:1650241543
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a8e94616-c8fb-4b19-bd21-4f255f0bcdff,timestamp:1650241543
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41d23174-17a0-49fe-9db0-9fbd987a6f01,timestamp:1650241543
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5756636f-71fe-4ff3-b01f-8d534f154a50,timestamp:1650241544
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0c9368b2-c75a-4b10-b426-1ea1e485dc96,timestamp:1650241544
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5ee9406b-7a75-4941-980b-51ea5896615f,timestamp:1650241544
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65fb6e50-eb8b-4e14-8db5-9cda18081118,timestamp:1650241544
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,383 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 8
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:5.86|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4afa65a1-2ac5-4472-8949-6234d29fd3e8,timestamp:1650241544
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:6|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aec048c4-6a9e-4636-88c5-cd63fec5813b,timestamp:1650241544
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a8e94616-c8fb-4b19-bd21-4f255f0bcdff,timestamp:1650241543
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41d23174-17a0-49fe-9db0-9fbd987a6f01,timestamp:1650241543
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5756636f-71fe-4ff3-b01f-8d534f154a50,timestamp:1650241544
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0c9368b2-c75a-4b10-b426-1ea1e485dc96,timestamp:1650241544
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5ee9406b-7a75-4941-980b-51ea5896615f,timestamp:1650241544
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65fb6e50-eb8b-4e14-8db5-9cda18081118,timestamp:1650241544
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,383 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 8
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:5.86|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4afa65a1-2ac5-4472-8949-6234d29fd3e8,timestamp:1650241544
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:6|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aec048c4-6a9e-4636-88c5-cd63fec5813b,timestamp:1650241544
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6718ab87-dc9b-4618-9e71-ca5ebfa50101,timestamp:1650241544
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:f67abd31-9d02-4990-9e51-9b6869312e3d,timestamp:1650241544
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6718ab87-dc9b-4618-9e71-ca5ebfa50101,timestamp:1650241544
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:f67abd31-9d02-4990-9e51-9b6869312e3d,timestamp:1650241544
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d5575ea-c1b1-47ed-af77-ef415809b2bd,timestamp:1650241544
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.03|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e650aebf-2df9-4dbf-aac1-1d3d8df4d634,timestamp:1650241544
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 13
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e6fd38ae-8b54-41d8-9ea1-17c8527b2c9a,timestamp:1650241544
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d5575ea-c1b1-47ed-af77-ef415809b2bd,timestamp:1650241544
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.03|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e650aebf-2df9-4dbf-aac1-1d3d8df4d634,timestamp:1650241544
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 13
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e6fd38ae-8b54-41d8-9ea1-17c8527b2c9a,timestamp:1650241544
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3bae0c32-eb6d-4f97-a8d9-da14ec10b964,timestamp:1650241544
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b42e1c6c-fe99-4a2e-b624-676e3117b178,timestamp:1650241545
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:176504c4-16be-4531-9a23-2905adc479ac,timestamp:1650241545
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4506218e-a262-4ac3-864d-df900a919b9f,timestamp:1650241545
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:470bcd94-b192-4e63-a0bf-bd6a6ecb5bf0,timestamp:1650241545
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3bae0c32-eb6d-4f97-a8d9-da14ec10b964,timestamp:1650241544
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b42e1c6c-fe99-4a2e-b624-676e3117b178,timestamp:1650241545
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:176504c4-16be-4531-9a23-2905adc479ac,timestamp:1650241545
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4506218e-a262-4ac3-864d-df900a919b9f,timestamp:1650241545
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:470bcd94-b192-4e63-a0bf-bd6a6ecb5bf0,timestamp:1650241545
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db958c22-670b-4f6e-afe4-5c5a8e70e93d,timestamp:1650241545
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aff03e2e-c9b6-4154-afb7-f8fee4b093ef,timestamp:1650241545
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.73|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ebaea0a-c800-48d6-b3f7-206080f155ea,timestamp:1650241545
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,577 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e5d83b-bc1e-4cb5-8106-41aed195cf96,timestamp:1650241545
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ec1e2a0e-1e71-4050-a7fb-314380c5ed86,timestamp:1650241545
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db958c22-670b-4f6e-afe4-5c5a8e70e93d,timestamp:1650241545
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aff03e2e-c9b6-4154-afb7-f8fee4b093ef,timestamp:1650241545
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.73|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ebaea0a-c800-48d6-b3f7-206080f155ea,timestamp:1650241545
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,577 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e5d83b-bc1e-4cb5-8106-41aed195cf96,timestamp:1650241545
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ec1e2a0e-1e71-4050-a7fb-314380c5ed86,timestamp:1650241545
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.77|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:75937063-aafb-4cde-82e5-ef15c109cfe0,timestamp:1650241545
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:598ceffc-323d-403b-996c-f18fd6163e74,timestamp:1650241545
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,858 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:838b4ff4-466c-477f-9f72-22c67196742b,timestamp:1650241545
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,074 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:27e0174c-e775-4c2f-b198-3a133d49259e,timestamp:1650241546
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6d0f88bc-3934-4555-b8c7-7a5311ffe85e,timestamp:1650241546
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.2|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:927937c1-8998-4686-96f0-56f4094cfeff,timestamp:1650241546
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.77|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:75937063-aafb-4cde-82e5-ef15c109cfe0,timestamp:1650241545
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:598ceffc-323d-403b-996c-f18fd6163e74,timestamp:1650241545
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,858 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:838b4ff4-466c-477f-9f72-22c67196742b,timestamp:1650241545
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,074 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:27e0174c-e775-4c2f-b198-3a133d49259e,timestamp:1650241546
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6d0f88bc-3934-4555-b8c7-7a5311ffe85e,timestamp:1650241546
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.2|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:927937c1-8998-4686-96f0-56f4094cfeff,timestamp:1650241546
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65e09797-8ede-42b7-8fee-9a62fdb8e1a5,timestamp:1650241546
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9015e397-aab8-4259-8638-38703c66ad12,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:19b535b5-7e83-458e-9229-0f322798997a,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9b087c7c-e738-453a-8f85-f6dfe6c535e7,timestamp:1650241546
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65e09797-8ede-42b7-8fee-9a62fdb8e1a5,timestamp:1650241546
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9015e397-aab8-4259-8638-38703c66ad12,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:19b535b5-7e83-458e-9229-0f322798997a,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9b087c7c-e738-453a-8f85-f6dfe6c535e7,timestamp:1650241546
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:af1d81f6-ce62-4106-8959-532467b8dabc,timestamp:1650241546
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a0e9f81d-97c5-4117-8f04-c93584bdcc8d,timestamp:1650241546
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,817 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:af1d81f6-ce62-4106-8959-532467b8dabc,timestamp:1650241546
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a0e9f81d-97c5-4117-8f04-c93584bdcc8d,timestamp:1650241546
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,817 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.79|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f3e36b1-70c1-4840-b9b6-20b1401164b0,timestamp:1650241546
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:bce91358-0a4e-4d85-833a-cb0189043ea9,timestamp:1650241546
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d2c82c1-91d5-48f3-b4f3-09da2d42daa5,timestamp:1650241547
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,095 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c2609e01-7fc7-4817-b804-f7bf071f41fc,timestamp:1650241547
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:97f14871-b6bf-4d79-ada6-9183f92069ed,timestamp:1650241547
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5fda8390-6584-499e-9af2-ea46d5c96353,timestamp:1650241547
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.79|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f3e36b1-70c1-4840-b9b6-20b1401164b0,timestamp:1650241546
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:bce91358-0a4e-4d85-833a-cb0189043ea9,timestamp:1650241546
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d2c82c1-91d5-48f3-b4f3-09da2d42daa5,timestamp:1650241547
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,095 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c2609e01-7fc7-4817-b804-f7bf071f41fc,timestamp:1650241547
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:97f14871-b6bf-4d79-ada6-9183f92069ed,timestamp:1650241547
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5fda8390-6584-499e-9af2-ea46d5c96353,timestamp:1650241547
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
Input images by manifest file
First, we generate a manifest file. Then we use the manifest file containing a list of object keys as inputs to batch inference. Some key points: - content_type = "application/x-image" (here the content_type is for the actual object for inference, not for the manifest file) - data_type = "ManifestFile" - Manifest file format must follow the format as
S3DataSource points out. We create the manifest file by using the jsonlines package.
[
{"prefix": "s3://customer_bucket/some/prefix/"},
"relative/path/to/custdata-1",
"relative/path/custdata-2",
...
"relative/path/custdata-N"
]
[13]:
!pip install -q jsonlines
[14]:
import jsonlines
# Build image list
manifest_prefix = f"s3://{bucket}/{prefix}/images/"
path = image_dir
img_files = [f for f in listdir(path) if isfile(join(path, f))]
print("img_files\n", img_files)
manifest_content = [{"prefix": manifest_prefix}]
manifest_content.extend(img_files)
print("manifest_content\n", manifest_content)
# Write jsonl file
manifest_file = "manifest.json"
with jsonlines.open(manifest_file, mode="w") as writer:
writer.write(manifest_content)
# Upload to S3
manifest_obj = sagemaker_session.upload_data(path=manifest_file, key_prefix=prefix)
print("manifest_obj\n", manifest_obj)
img_files
['2.png', '43.png', '46.png', '45.png', '16.png', '4.png', '92.png', '12.png', '18.png', '75.png', '87.png', '63.png', '81.png', '78.png', '13.png', '34.png', '42.png', '19.png', '73.png', '14.png', '83.png', '3.png', '55.png', '5.png', '38.png', '86.png', '27.png', '7.png', '15.png', '80.png', '89.png', '72.png', '88.png', '99.png', '82.png', '32.png', '70.png', '36.png', '67.png', '26.png', '94.png', '41.png', '30.png', '64.png', '28.png', '59.png', '52.png', '90.png', '69.png', '31.png', '37.png', '47.png', '40.png', '56.png', '58.png', '1.png', '93.png', '61.png', '29.png', '76.png', '23.png', '50.png', '97.png', '79.png', '85.png', '6.png', '57.png', '35.png', '11.png', '22.png', '62.png', '21.png', '95.png', '71.png', '60.png', '17.png', '77.png', '25.png', '49.png', '96.png', '53.png', '54.png', '48.png', '51.png', '74.png', '20.png', '0.png', '91.png', '33.png', '24.png', '65.png', '66.png', '10.png', '68.png', '39.png', '9.png', '98.png', '84.png', '8.png', '44.png']
manifest_content
[{'prefix': 's3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script/images/'}, '2.png', '43.png', '46.png', '45.png', '16.png', '4.png', '92.png', '12.png', '18.png', '75.png', '87.png', '63.png', '81.png', '78.png', '13.png', '34.png', '42.png', '19.png', '73.png', '14.png', '83.png', '3.png', '55.png', '5.png', '38.png', '86.png', '27.png', '7.png', '15.png', '80.png', '89.png', '72.png', '88.png', '99.png', '82.png', '32.png', '70.png', '36.png', '67.png', '26.png', '94.png', '41.png', '30.png', '64.png', '28.png', '59.png', '52.png', '90.png', '69.png', '31.png', '37.png', '47.png', '40.png', '56.png', '58.png', '1.png', '93.png', '61.png', '29.png', '76.png', '23.png', '50.png', '97.png', '79.png', '85.png', '6.png', '57.png', '35.png', '11.png', '22.png', '62.png', '21.png', '95.png', '71.png', '60.png', '17.png', '77.png', '25.png', '49.png', '96.png', '53.png', '54.png', '48.png', '51.png', '74.png', '20.png', '0.png', '91.png', '33.png', '24.png', '65.png', '66.png', '10.png', '68.png', '39.png', '9.png', '98.png', '84.png', '8.png', '44.png']
manifest_obj
s3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script/manifest.json
[15]:
# Batch transform with manifest file
transform_job = transformer.transform(
data=manifest_obj,
data_type="ManifestFile",
content_type="application/x-image",
wait=False,
)
[16]:
print("Latest transform job:", transformer.latest_transform_job.name)
Latest transform job: pytorch-inference-2022-04-18-00-26-08-985
[17]:
# look at the status of the transform job
import pprint as pp
sm_cli = sagemaker_session.sagemaker_client
job_info = sm_cli.describe_transform_job(TransformJobName=transformer.latest_transform_job.name)
pp.pprint(job_info)
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 9, 32000, tzinfo=tzlocal()),
'DataProcessing': {'InputFilter': '$',
'JoinSource': 'None',
'OutputFilter': '$'},
'ModelName': 'pytorch-inference-2022-04-18-00-21-14-564',
'ResponseMetadata': {'HTTPHeaders': {'content-length': '870',
'content-type': 'application/x-amz-json-1.1',
'date': 'Mon, 18 Apr 2022 00:26:09 GMT',
'x-amzn-requestid': '394fa75d-5814-41e3-a604-2a471c52c745'},
'HTTPStatusCode': 200,
'RequestId': '394fa75d-5814-41e3-a604-2a471c52c745',
'RetryAttempts': 0},
'TransformInput': {'CompressionType': 'None',
'ContentType': 'application/x-image',
'DataSource': {'S3DataSource': {'S3DataType': 'ManifestFile',
'S3Uri': 's3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script/manifest.json'}},
'SplitType': 'None'},
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-inference-2022-04-18-00-26-08-985',
'TransformJobName': 'pytorch-inference-2022-04-18-00-26-08-985',
'TransformJobStatus': 'InProgress',
'TransformOutput': {'AssembleWith': 'None',
'KmsKeyId': '',
'S3OutputPath': 's3://sagemaker-us-west-2-000000000000/pytorch-inference-2022-04-18-00-26-08-985'},
'TransformResources': {'InstanceCount': 1, 'InstanceType': 'ml.c5.xlarge'}}
Multiple instance
We use instance_count > 1 to create multiple inference instances. When a batch transform job starts, Amazon SageMaker initializes compute instances and distributes the inference or preprocessing workload between them. Batch Transform partitions the Amazon S3 objects in the input by key and maps Amazon S3 objects to instances. Given multiple files, one instance might process input1.csv, and another instance might process input2.csv. Read more at Use Batch
Transform.
[18]:
dist_transformer = estimator.transformer(instance_count=2, instance_type="ml.c4.xlarge")
dist_transformer.transform(
data=inference_inputs,
data_type="S3Prefix",
content_type="application/x-image",
wait=True,
)
................................2022-04-18 00:31:23,943 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.3.0
TS Home: /opt/conda/lib/python3.6/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 910 M
Python executable: /opt/conda/bin/python3.6
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2022-04-18 00:31:23,983 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-04-18 00:31:24,005 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag c3fb4d26ffa849bbbf79ab8856e86748
2022-04-18 00:31:24,017 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-04-18 00:31:24,049 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-04-18 00:31:24,238 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:24,243 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:24,244 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46
2022-04-18 00:31:24,244 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,244 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,245 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]45
2022-04-18 00:31:24,245 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,245 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,253 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:24,253 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]48
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:24,279 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,282 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]47
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,320 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-04-18 00:31:24,320 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2022-04-18 00:31:24,321 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
2022-04-18 00:31:24,328 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9001.
2022-04-18 00:31:24,335 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9000.
2022-04-18 00:31:24,337 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9003.
2022-04-18 00:31:24,337 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9002.
Model server started.
2022-04-18 00:31:25,106 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,108 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:48.10660934448242|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,109 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:7.809303283691406|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,110 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:14.0|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,111 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:6271.05078125|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,112 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:922.55078125|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,112 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:16.0|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,493 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1042
2022-04-18 00:31:25,494 [INFO ] W-9002-model_1 TS_METRICS - W-9002-model_1.ms:1465|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,495 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:112|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:25,511 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1065
2022-04-18 00:31:25,512 [INFO ] W-9003-model_1 TS_METRICS - W-9003-model_1.ms:1482|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,512 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:107|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:25,624 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1186
2022-04-18 00:31:25,624 [INFO ] W-9001-model_1 TS_METRICS - W-9001-model_1.ms:1596|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,625 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:97|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:25,815 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1337
2022-04-18 00:31:25,815 [INFO ] W-9000-model_1 TS_METRICS - W-9000-model_1.ms:1789|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,815 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:137|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:26,665 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.3.0
TS Home: /opt/conda/lib/python3.6/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 912 M
Python executable: /opt/conda/bin/python3.6
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2022-04-18 00:31:26,709 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-04-18 00:31:26,732 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag 2eb711663c324fc78881634b94b209fd
2022-04-18 00:31:26,744 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-04-18 00:31:26,771 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-04-18 00:31:26,975 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:26,976 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46
2022-04-18 00:31:26,976 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:26,976 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:26,991 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:26,992 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]47
2022-04-18 00:31:26,992 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:26,993 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:26,995 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:26,999 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:27,009 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]48
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:27,028 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-04-18 00:31:27,028 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2022-04-18 00:31:27,029 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9002.
2022-04-18 00:31:27,032 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
2022-04-18 00:31:27,033 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9001.
2022-04-18 00:31:27,034 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9000.
2022-04-18 00:31:27,035 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:27,042 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]49
2022-04-18 00:31:27,043 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:27,043 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:27,044 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:27,052 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9003.
Model server started.
2022-04-18 00:31:27,971 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,973 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:48.106597900390625|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,973 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:7.809314727783203|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,975 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:14.0|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,976 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:6252.3125|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,977 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:947.546875|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,977 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:16.3|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:28,254 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1087
2022-04-18 00:31:28,255 [INFO ] W-9002-model_1 TS_METRICS - W-9002-model_1.ms:1500|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,255 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:122|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:28,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1166
2022-04-18 00:31:28,345 [INFO ] W-9003-model_1 TS_METRICS - W-9003-model_1.ms:1590|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:128|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:28,570 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1391
2022-04-18 00:31:28,571 [INFO ] W-9000-model_1 TS_METRICS - W-9000-model_1.ms:1820|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,572 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:133|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:28,579 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1404
2022-04-18 00:31:28,579 [INFO ] W-9001-model_1 TS_METRICS - W-9001-model_1.ms:1825|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,579 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:127|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:44472 "GET /ping HTTP/1.1" 200 18
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,140 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:44484 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:31:35,141 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,329 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 16
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:44472 "GET /ping HTTP/1.1" 200 18
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,140 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:44484 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:31:35,141 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,329 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 16
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 27
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c799b9aa-9065-4c8c-ae3d-cb85d035b0f1,timestamp:1650241895
2022-04-18 00:31:35,331 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,332 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,413 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 17
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,415 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,417 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,419 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.32|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ce3f0999-05bb-4d18-870c-1f0a9c12da1f,timestamp:1650241895
2022-04-18 00:31:35,583 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 16
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.5|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:40e53ea9-18f8-4664-a76e-8c45221eca14,timestamp:1650241895
2022-04-18 00:31:35,585 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,661 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 14
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:11.35|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4fb87cc8-2720-4456-893b-9351130db55b,timestamp:1650241895
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 27
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c799b9aa-9065-4c8c-ae3d-cb85d035b0f1,timestamp:1650241895
2022-04-18 00:31:35,331 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,332 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,413 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 17
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,415 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,417 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,419 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.32|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ce3f0999-05bb-4d18-870c-1f0a9c12da1f,timestamp:1650241895
2022-04-18 00:31:35,583 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 16
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.5|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:40e53ea9-18f8-4664-a76e-8c45221eca14,timestamp:1650241895
2022-04-18 00:31:35,585 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,661 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 14
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:11.35|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4fb87cc8-2720-4456-893b-9351130db55b,timestamp:1650241895
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.91|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8f2694cc-381e-4ff7-859f-a87eeab21645,timestamp:1650241895
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91728dfd-b512-4ed5-bc56-473d6bb85b95,timestamp:1650241895
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.89|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:86462d03-25d9-4d23-ab00-956a18a4699c,timestamp:1650241895
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.91|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8f2694cc-381e-4ff7-859f-a87eeab21645,timestamp:1650241895
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91728dfd-b512-4ed5-bc56-473d6bb85b95,timestamp:1650241895
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.89|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:86462d03-25d9-4d23-ab00-956a18a4699c,timestamp:1650241895
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.01|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c979e5b8-d351-4866-8eb3-f23e983d5304,timestamp:1650241896
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7e4887c5-77ee-437d-bb09-2c1290a0d6f4,timestamp:1650241896
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.01|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c979e5b8-d351-4866-8eb3-f23e983d5304,timestamp:1650241896
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7e4887c5-77ee-437d-bb09-2c1290a0d6f4,timestamp:1650241896
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e1e978b5-e89b-4184-b870-f8026d93cb34,timestamp:1650241896
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:03bbd271-23a6-4f97-8f74-54d206a11203,timestamp:1650241896
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.68|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:05a39a1f-550f-46d8-a4ab-bb0bad787800,timestamp:1650241896
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.58|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ca413e4c-c8d5-450f-9f5d-51be6e134530,timestamp:1650241896
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.82|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:09736318-febd-4e92-9ccd-fca21691dcb7,timestamp:1650241896
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e1e978b5-e89b-4184-b870-f8026d93cb34,timestamp:1650241896
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:03bbd271-23a6-4f97-8f74-54d206a11203,timestamp:1650241896
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.68|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:05a39a1f-550f-46d8-a4ab-bb0bad787800,timestamp:1650241896
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.58|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ca413e4c-c8d5-450f-9f5d-51be6e134530,timestamp:1650241896
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.82|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:09736318-febd-4e92-9ccd-fca21691dcb7,timestamp:1650241896
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b3247490-10da-4d5c-b4f4-177fb67fdcab,timestamp:1650241896
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,909 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.4|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:71cf38d0-8082-462a-a642-61074509fe05,timestamp:1650241896
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b3247490-10da-4d5c-b4f4-177fb67fdcab,timestamp:1650241896
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,909 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.4|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:71cf38d0-8082-462a-a642-61074509fe05,timestamp:1650241896
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,209 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:33020 "GET /ping HTTP/1.1" 200 14
2022-04-18 00:31:36,209 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,234 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:33028 "GET /execution-parameters HTTP/1.1" 404 2
2022-04-18 00:31:36,235 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,706 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:36,707 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.87|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:227b55e2-a7fa-4598-973b-72af3ff57517,timestamp:1650241896
2022-04-18 00:31:36,707 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 23
2022-04-18 00:31:36,708 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,708 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,708 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,789 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:36,789 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 17
2022-04-18 00:31:36,789 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,790 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,790 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,793 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.16|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:66a7abd0-8acc-4a58-ab8a-4c59227c3a34,timestamp:1650241896
2022-04-18 00:31:36,956 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 14
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 16
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.59|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:89a7c047-6e98-4831-a5ae-4ee298656d11,timestamp:1650241896
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,040 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 6
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 14
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:8|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:11.43|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:53b4927a-4681-4402-92b1-502fdee4c1cc,timestamp:1650241897
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8628f76e-49d3-4ff2-9f9e-1884344a99e2,timestamp:1650241897
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8628f76e-49d3-4ff2-9f9e-1884344a99e2,timestamp:1650241897
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.48|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f0b859d7-9ac3-4961-8e6c-cad21445a0a6,timestamp:1650241897
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,170 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:372f2fa2-9630-4d4b-a336-f0c645acdf86,timestamp:1650241897
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b0ef0f3e-3f59-4f9e-942c-e04d18827555,timestamp:1650241897
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,415 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.31|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a725d82f-b83b-4a36-908c-357f88d93092,timestamp:1650241897
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:feca74ad-59b1-4059-86f4-df1a82621e25,timestamp:1650241897
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:97441582-f139-4090-9210-9da15b779aa3,timestamp:1650241897
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.48|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f0b859d7-9ac3-4961-8e6c-cad21445a0a6,timestamp:1650241897
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,170 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:372f2fa2-9630-4d4b-a336-f0c645acdf86,timestamp:1650241897
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b0ef0f3e-3f59-4f9e-942c-e04d18827555,timestamp:1650241897
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,415 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.31|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a725d82f-b83b-4a36-908c-357f88d93092,timestamp:1650241897
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:feca74ad-59b1-4059-86f4-df1a82621e25,timestamp:1650241897
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:97441582-f139-4090-9210-9da15b779aa3,timestamp:1650241897
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91586bb6-f73d-48e3-82ea-ea21f17cc195,timestamp:1650241897
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,808 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.99|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fb9bb9d8-0b26-4d84-a69e-e617de0fac97,timestamp:1650241897
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,810 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e8c21333-c613-46a8-b6db-f594df078bc8,timestamp:1650241897
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.9|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:785ba516-f3f3-43fb-a1f4-4acedd4b5f98,timestamp:1650241897
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91586bb6-f73d-48e3-82ea-ea21f17cc195,timestamp:1650241897
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,808 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.99|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fb9bb9d8-0b26-4d84-a69e-e617de0fac97,timestamp:1650241897
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,810 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e8c21333-c613-46a8-b6db-f594df078bc8,timestamp:1650241897
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.9|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:785ba516-f3f3-43fb-a1f4-4acedd4b5f98,timestamp:1650241897
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,145 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.82|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:0ca92892-412d-4b4a-8f04-0f24f8449075,timestamp:1650241897
2022-04-18 00:31:37,146 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,146 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:37,147 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,147 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,147 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.04|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:5e808b17-7077-4502-99a4-4c21b7f0ee7e,timestamp:1650241897
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.7|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:728c5e6c-1fe2-43a2-a1b6-ec3c604c1449,timestamp:1650241897
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,482 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.54|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:4438bc7d-213a-4fe4-9091-52a5f5d59367,timestamp:1650241897
2022-04-18 00:31:37,483 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,483 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,483 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,484 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,484 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.62|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:50e762c0-7961-4b73-be5b-f5d8a490139d,timestamp:1650241897
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,577 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,675 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.95|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:c6be9d72-a3bf-485e-93fa-bc33cbbe9366,timestamp:1650241897
2022-04-18 00:31:37,676 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:31:37,676 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,676 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,677 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,677 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,790 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,790 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.88|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:7dd92d30-4f50-4451-ad7a-9a2c0e5bac1f,timestamp:1650241897
2022-04-18 00:31:37,790 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,791 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,791 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,791 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,868 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,868 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.1|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:2a4aa142-240d-43ee-8c48-39203f12c87a,timestamp:1650241897
2022-04-18 00:31:37,869 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,869 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,869 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,870 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.56|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ddd338ae-015d-45f5-afc8-f16a65263bbc,timestamp:1650241897
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,957 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,957 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.99|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:263127c6-6899-4fe7-9afb-864d21461441,timestamp:1650241898
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,029 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,029 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18T00:31:35.166:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18T00:31:35.166:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:43b7c81a-40e5-425e-909a-950eb23214be,timestamp:1650241898
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:edeb3c61-bcc1-4446-80a1-8bad4fa0b859,timestamp:1650241898
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,199 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:43b7c81a-40e5-425e-909a-950eb23214be,timestamp:1650241898
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:edeb3c61-bcc1-4446-80a1-8bad4fa0b859,timestamp:1650241898
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,199 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a0d1e73b-8e62-41b7-8324-715f7ab78bee,timestamp:1650241898
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,278 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.03|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f2ad3be4-bb23-47af-b44d-c5bdb8000fd7,timestamp:1650241898
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,362 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4b339f15-ac1c-493e-81d1-f370f551bf8f,timestamp:1650241898
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a0d1e73b-8e62-41b7-8324-715f7ab78bee,timestamp:1650241898
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,278 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.03|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f2ad3be4-bb23-47af-b44d-c5bdb8000fd7,timestamp:1650241898
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,362 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4b339f15-ac1c-493e-81d1-f370f551bf8f,timestamp:1650241898
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:3fec32ed-2f82-4c16-963f-3024a80c849e,timestamp:1650241898
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,549 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:9fb04cfa-de2d-4e94-ad81-5294fb544348,timestamp:1650241898
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,634 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:56194b87-7837-4b8c-ae2b-29f0bd555db5,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a9de82c8-4e7d-4016-bf09-1cdb32ba362c,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e94c019b-91a6-469b-9037-7862a951b8ef,timestamp:1650241898
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:3fec32ed-2f82-4c16-963f-3024a80c849e,timestamp:1650241898
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,549 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:9fb04cfa-de2d-4e94-ad81-5294fb544348,timestamp:1650241898
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,634 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:56194b87-7837-4b8c-ae2b-29f0bd555db5,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a9de82c8-4e7d-4016-bf09-1cdb32ba362c,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e94c019b-91a6-469b-9037-7862a951b8ef,timestamp:1650241898
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,008 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:dbcd5a31-f816-41c4-9d7d-e863b9c42c41,timestamp:1650241899
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,008 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:dbcd5a31-f816-41c4-9d7d-e863b9c42c41,timestamp:1650241899
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,101 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,101 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.06|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:cfe34dbd-0132-470e-8fbf-510237becf25,timestamp:1650241898
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.61|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:2789dd8b-0aff-40e3-ad39-042e4f24c667,timestamp:1650241898
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,181 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,181 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.22|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:5849d060-58a6-49ca-8aa8-7c3556a73029,timestamp:1650241898
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,344 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.02|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:7d29b540-302f-46f6-9e0b-2d847068f85e,timestamp:1650241898
2022-04-18 00:31:38,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,345 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,346 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,346 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,465 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,465 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.08|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:dbeb7149-b0a4-4a22-b9cf-ed56a5cde68c,timestamp:1650241898
2022-04-18 00:31:38,465 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,466 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,466 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,466 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,533 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,533 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:00839105-e7cf-4333-b753-23cec06c51a3,timestamp:1650241898
2022-04-18 00:31:38,534 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,534 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,534 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,535 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:979783b2-39db-4452-baae-0d8a6197d923,timestamp:1650241898
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,620 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,720 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,720 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:a34bab70-4653-4cad-8309-c90b88ca7abd,timestamp:1650241898
2022-04-18 00:31:38,720 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:38,721 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,721 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,721 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ae9ef618-36d7-45f4-a4b1-6acf15659a05,timestamp:1650241898
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,808 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,808 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,910 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,910 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.0|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:5d44bd8e-fccd-4037-9c1b-5cd7deec6adc,timestamp:1650241898
2022-04-18 00:31:38,910 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,911 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,911 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,911 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.85|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:344973ea-fdc9-4971-bc7b-5e3bb84ac9b4,timestamp:1650241898
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,994 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,065 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e6773617-7ec3-4638-b4d6-cd50946ec2df,timestamp:1650241899
2022-04-18 00:31:39,066 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,066 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,066 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,067 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,067 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.61|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:14a2e880-a272-495b-b826-644e912b9e91,timestamp:1650241899
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:04e4bfe5-44ab-48bf-be75-378c40d7a614,timestamp:1650241899
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c00fa137-4528-42c7-9c44-a6c34c080239,timestamp:1650241899
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.61|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:14a2e880-a272-495b-b826-644e912b9e91,timestamp:1650241899
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:04e4bfe5-44ab-48bf-be75-378c40d7a614,timestamp:1650241899
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c00fa137-4528-42c7-9c44-a6c34c080239,timestamp:1650241899
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.11|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:13561332-7314-476a-ba5a-2dd08cb21559,timestamp:1650241899
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,466 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:bb2810fe-ff27-408c-b755-1851414fad57,timestamp:1650241899
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4581545b-af0b-40e5-9cf4-1f2516283725,timestamp:1650241899
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,849 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7460afef-5259-4f8b-8174-372465fec3ad,timestamp:1650241899
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.83|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:18f7633a-7ea7-42cd-a332-e10ef4e37459,timestamp:1650241899
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.11|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:13561332-7314-476a-ba5a-2dd08cb21559,timestamp:1650241899
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,466 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:bb2810fe-ff27-408c-b755-1851414fad57,timestamp:1650241899
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4581545b-af0b-40e5-9cf4-1f2516283725,timestamp:1650241899
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,849 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7460afef-5259-4f8b-8174-372465fec3ad,timestamp:1650241899
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.83|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:18f7633a-7ea7-42cd-a332-e10ef4e37459,timestamp:1650241899
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,167 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,167 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.02|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:78f258da-cbd4-4257-b8e4-21a52ac6b9e0,timestamp:1650241899
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,239 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,241 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:f6adc969-b682-4898-8629-70f60920769d,timestamp:1650241899
2022-04-18 00:31:39,319 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:6330ed70-1bc4-48fd-9c56-54e0dabf7636,timestamp:1650241899
2022-04-18 00:31:39,319 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.53|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:080bdd6c-0fb0-43f4-b7d5-55d4acb1e3ba,timestamp:1650241899
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.95|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:8156bf18-3b7d-4eff-929e-360271be9cea,timestamp:1650241899
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e4f7270d-c86e-4870-95de-71350e989d2e,timestamp:1650241899
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.92|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:2c8be008-add5-498b-9e38-9be5481cbfbc,timestamp:1650241899
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,688 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,688 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.0|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:3b20f880-2fc4-49ae-91f9-9eec5ca7f62c,timestamp:1650241899
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,896 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:d69cfe35-cfa8-4bd9-b84d-d5e8713da29a,timestamp:1650241899
2022-04-18 00:31:39,896 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,998 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,998 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:0ab7d0c0-6d4a-4453-82d3-b22487820980,timestamp:1650241899
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.92|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fa834bd9-83db-461a-a6bd-aaa21880fde2,timestamp:1650241900
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,042 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:be1cc1e4-7046-4dcc-b30a-93e6352660ee,timestamp:1650241900
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.7|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:d09c61ab-3119-409f-9f2f-2d3219b5c57b,timestamp:1650241900
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:78e83f62-bbbe-43d6-a26b-ad673804d849,timestamp:1650241900
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.92|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fa834bd9-83db-461a-a6bd-aaa21880fde2,timestamp:1650241900
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,042 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:be1cc1e4-7046-4dcc-b30a-93e6352660ee,timestamp:1650241900
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.7|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:d09c61ab-3119-409f-9f2f-2d3219b5c57b,timestamp:1650241900
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:78e83f62-bbbe-43d6-a26b-ad673804d849,timestamp:1650241900
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18T00:31:36.242:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.02|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:6738de45-b281-4d36-b934-b536cdfd6094,timestamp:1650241900
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,201 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:a3ba2aee-7ef6-4875-9e06-b0f66165ca23,timestamp:1650241900
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,283 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e2aabc1b-4c78-4e3c-83f8-69b74ffa0d65,timestamp:1650241900
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.84|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e5ca6928-3980-4d67-bca8-28a7db32e549,timestamp:1650241900
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,601 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,601 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,680 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e7699b7f-84ac-477f-9516-8632d313c6d3,timestamp:1650241900
2022-04-18 00:31:40,680 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,773 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,773 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.46|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ca289c06-8bfb-4b08-8f13-85eb8e419afb,timestamp:1650241900
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:26879460-25da-42cd-9b1f-44d7167777d9,timestamp:1650241900
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,882 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,882 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,959 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,959 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.84|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:1f4658dd-a8df-4b59-a63d-6c8c16d77e71,timestamp:1650241900
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:27625aff-b3fd-40a8-8451-e4361cbc11f3,timestamp:1650241901
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,155 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:4e4901d4-b922-4479-a187-d95f1d7ffbb9,timestamp:1650241901
2022-04-18 00:31:41,156 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:41,156 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,156 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,157 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,157 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,236 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,236 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:72175ab0-8de5-45dd-967a-ec7a232d0253,timestamp:1650241901
2022-04-18 00:31:41,236 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,237 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,237 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,237 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,330 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,330 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:162df6be-8c39-416c-afce-8a67be0f4f58,timestamp:1650241901
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,451 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,451 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.83|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:c864d856-7d5b-46aa-91b1-3d454af757a4,timestamp:1650241901
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,532 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,532 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.38|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ced3a724-9b27-4786-b487-c40c90b89521,timestamp:1650241901
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
Look at all transform jobs
We list and describe the transform jobs to retrieve information about them.
[19]:
transform_jobs = sm_cli.list_transform_jobs()["TransformJobSummaries"]
for job in transform_jobs:
pp.pprint(job)
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 32, 3, 162000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 32, 3, 162000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/automl-churn-sdk-18-00-20-19-dpp0-rpb-1-ab06a7aa54904c088293a6d',
'TransformJobName': 'automl-churn-sdk-18-00-20-19-dpp0-rpb-1-ab06a7aa54904c088293a6d',
'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 32, 0, 946000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 32, 1, 607000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/automl-churn-sdk-18-00-20-19-dpp8-rpb-1-9abdff16bbd34c83872e871',
'TransformJobName': 'automl-churn-sdk-18-00-20-19-dpp8-rpb-1-9abdff16bbd34c83872e871',
'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 31, 58, 909000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 31, 59, 560000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/automl-churn-sdk-18-00-20-19-dpp1-csv-1-1dd140703bb140bdaf51362',
'TransformJobName': 'automl-churn-sdk-18-00-20-19-dpp1-csv-1-1dd140703bb140bdaf51362',
'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 29, 11, 444000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 29, 11, 758000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/sagemaker-scikit-learn-2022-04-18-00-29-11-429',
'TransformJobName': 'sagemaker-scikit-learn-2022-04-18-00-29-11-429',
'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 27, 37, 320000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 31, 17, 282000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/xgboost-2022-04-18-00-27-37-309',
'TransformJobName': 'xgboost-2022-04-18-00-27-37-309',
'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 41, 812000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 30, 29, 572000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/batch-transform-2022-04-18-00-26-41',
'TransformJobName': 'Batch-Transform-2022-04-18-00-26-41',
'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 12, 635000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 31, 42, 971000, tzinfo=tzlocal()),
'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 31, 42, 720000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-training-2022-04-18-00-26-12-621',
'TransformJobName': 'pytorch-training-2022-04-18-00-26-12-621',
'TransformJobStatus': 'Completed'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 9, 32000, tzinfo=tzlocal()),
'FailureReason': 'ClientError: See job logs for more information',
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 30, 13, 568000, tzinfo=tzlocal()),
'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 30, 13, 253000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-inference-2022-04-18-00-26-08-985',
'TransformJobName': 'pytorch-inference-2022-04-18-00-26-08-985',
'TransformJobStatus': 'Failed'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 23, 34, 840000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 28, 0, 728000, tzinfo=tzlocal()),
'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 28, 0, 359000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/r-transformer-2022-04-18-00-23-34-816',
'TransformJobName': 'R-Transformer-2022-04-18-00-23-34-816',
'TransformJobStatus': 'Completed'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 23, 9, 974000, tzinfo=tzlocal()),
'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 26, 38, 885000, tzinfo=tzlocal()),
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/tensorflow-inference-2022-04-18-00-23-09-960',
'TransformJobName': 'tensorflow-inference-2022-04-18-00-23-09-960',
'TransformJobStatus': 'InProgress'}
[20]:
job_info = sm_cli.describe_transform_job(
TransformJobName=dist_transformer.latest_transform_job.name
)
pp.pprint(job_info)
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 12, 635000, tzinfo=tzlocal()),
'DataProcessing': {'InputFilter': '$',
'JoinSource': 'None',
'OutputFilter': '$'},
'Environment': {},
'ModelName': 'pytorch-training-2022-04-18-00-26-11-963',
'ResponseMetadata': {'HTTPHeaders': {'content-length': '907',
'content-type': 'application/x-amz-json-1.1',
'date': 'Mon, 18 Apr 2022 00:32:03 GMT',
'x-amzn-requestid': 'c75d7d27-2982-45b7-b619-f04bffe72fff'},
'HTTPStatusCode': 200,
'RequestId': 'c75d7d27-2982-45b7-b619-f04bffe72fff',
'RetryAttempts': 0},
'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 31, 42, 720000, tzinfo=tzlocal()),
'TransformInput': {'CompressionType': 'None',
'ContentType': 'application/x-image',
'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',
'S3Uri': 's3://sagemaker-us-west-2-000000000000/batch_transform'}},
'SplitType': 'None'},
'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-training-2022-04-18-00-26-12-621',
'TransformJobName': 'pytorch-training-2022-04-18-00-26-12-621',
'TransformJobStatus': 'Completed',
'TransformOutput': {'AssembleWith': 'None',
'KmsKeyId': '',
'S3OutputPath': 's3://sagemaker-us-west-2-000000000000/pytorch-training-2022-04-18-00-26-12-621'},
'TransformResources': {'InstanceCount': 2, 'InstanceType': 'ml.c4.xlarge'},
'TransformStartTime': datetime.datetime(2022, 4, 18, 0, 29, 50, 865000, tzinfo=tzlocal())}
[21]:
import re
def get_bucket_and_prefix(s3_output_path):
trim = re.sub("s3://", "", s3_output_path)
bucket, prefix = trim.split("/")
return bucket, prefix
local_path = "output" # Where to save the output locally
bucket, output_prefix = get_bucket_and_prefix(job_info["TransformOutput"]["S3OutputPath"])
print(bucket, output_prefix)
sagemaker_session.download_data(path=local_path, bucket=bucket, key_prefix=output_prefix)
sagemaker-us-west-2-521695447989 pytorch-training-2022-04-18-00-26-12-621
[22]:
!ls {local_path}
0.png.out 24.png.out 4.png.out 55.png.out 70.png.out 86.png.out
1.png.out 25.png.out 40.png.out 56.png.out 71.png.out 87.png.out
10.png.out 26.png.out 41.png.out 57.png.out 72.png.out 88.png.out
11.png.out 27.png.out 42.png.out 58.png.out 73.png.out 89.png.out
12.png.out 28.png.out 43.png.out 59.png.out 74.png.out 9.png.out
13.png.out 29.png.out 44.png.out 6.png.out 75.png.out 90.png.out
14.png.out 3.png.out 45.png.out 60.png.out 76.png.out 91.png.out
15.png.out 30.png.out 46.png.out 61.png.out 77.png.out 92.png.out
16.png.out 31.png.out 47.png.out 62.png.out 78.png.out 93.png.out
17.png.out 32.png.out 48.png.out 63.png.out 79.png.out 94.png.out
18.png.out 33.png.out 49.png.out 64.png.out 8.png.out 95.png.out
19.png.out 34.png.out 5.png.out 65.png.out 80.png.out 96.png.out
2.png.out 35.png.out 50.png.out 66.png.out 81.png.out 97.png.out
20.png.out 36.png.out 51.png.out 67.png.out 82.png.out 98.png.out
21.png.out 37.png.out 52.png.out 68.png.out 83.png.out 99.png.out
22.png.out 38.png.out 53.png.out 69.png.out 84.png.out
23.png.out 39.png.out 54.png.out 7.png.out 85.png.out
[23]:
# Inspect the output
import json
for f in os.listdir(local_path):
path = os.path.join(local_path, f)
with open(path, "r") as f:
pred = json.load(f)
print(pred)
{'predictions': 9}
{'predictions': 9}
{'predictions': 7}
{'predictions': 3}
{'predictions': 4}
{'predictions': 5}
{'predictions': 7}
{'predictions': 3}
{'predictions': 0}
{'predictions': 2}
{'predictions': 2}
{'predictions': 5}
{'predictions': 8}
{'predictions': 4}
{'predictions': 3}
{'predictions': 8}
{'predictions': 6}
{'predictions': 0}
{'predictions': 0}
{'predictions': 1}
{'predictions': 2}
{'predictions': 4}
{'predictions': 4}
{'predictions': 0}
{'predictions': 7}
{'predictions': 8}
{'predictions': 4}
{'predictions': 7}
{'predictions': 5}
{'predictions': 4}
{'predictions': 6}
{'predictions': 5}
{'predictions': 2}
{'predictions': 1}
{'predictions': 5}
{'predictions': 6}
{'predictions': 4}
{'predictions': 8}
{'predictions': 0}
{'predictions': 2}
{'predictions': 8}
{'predictions': 4}
{'predictions': 7}
{'predictions': 9}
{'predictions': 2}
{'predictions': 0}
{'predictions': 9}
{'predictions': 3}
{'predictions': 5}
{'predictions': 4}
{'predictions': 6}
{'predictions': 7}
{'predictions': 3}
{'predictions': 9}
{'predictions': 6}
{'predictions': 4}
{'predictions': 2}
{'predictions': 9}
{'predictions': 2}
{'predictions': 8}
{'predictions': 0}
{'predictions': 6}
{'predictions': 3}
{'predictions': 5}
{'predictions': 9}
{'predictions': 4}
{'predictions': 6}
{'predictions': 8}
{'predictions': 4}
{'predictions': 0}
{'predictions': 2}
{'predictions': 0}
{'predictions': 6}
{'predictions': 1}
{'predictions': 6}
{'predictions': 5}
{'predictions': 4}
{'predictions': 6}
{'predictions': 0}
{'predictions': 2}
{'predictions': 5}
{'predictions': 2}
{'predictions': 7}
{'predictions': 8}
{'predictions': 1}
{'predictions': 6}
{'predictions': 3}
{'predictions': 5}
{'predictions': 9}
{'predictions': 5}
{'predictions': 4}
{'predictions': 7}
{'predictions': 5}
{'predictions': 1}
{'predictions': 4}
{'predictions': 7}
{'predictions': 7}
{'predictions': 8}
{'predictions': 1}
{'predictions': 8}
Conclusion
In this notebook, we trained a PyTorch model, created a transformer from it, and then performed batch inference using S3 inputs, manifest files, and on multiple instances. This shows a variety of options that are available when running SageMaker Batch Transform jobs for batch inference.
Notebook CI Test Results
This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.