Use SageMaker Batch Transform for PyTorch Batch Inference

In this notebook, we examine how to do a Batch Transform task with PyTorch in Amazon SageMaker.

First, an image classification model is built on the MNIST dataset. Then, we demonstrate batch transform by using the SageMaker Python SDK PyTorch framework with different configurations: - data_type=S3Prefix: uses all objects that match the specified S3 prefix for batch inference. - data_type=ManifestFile: a manifest file contains a list of object keys to use in batch inference. - instance_count>1: distributes the batch inference dataset to multiple inference instances.

For batch transform in TensorFlow in Amazon SageMaker, you can follow other Jupyter notebooks in the sagemaker_batch_transform directory.

Runtime

This notebook takes approximately 15 minutes to run.

Contents

  1. Setup

  2. Model training

  3. Prepare batch inference data

  4. Create model transformer

  5. Batch inference

  6. Look at all transform jobs

  7. Conclusion

Setup

We’ll begin with some necessary installs and imports, and get an Amazon SageMaker session to help perform certain tasks, as well as an IAM role with the necessary permissions.

[2]:
!pip install nvidia-ml-py3
!yes | pip uninstall torchvision
!pip install torchvision
Requirement already satisfied: nvidia-ml-py3 in /opt/conda/lib/python3.6/site-packages (7.352.0)
Found existing installation: torchvision 0.5.0+cpu
Uninstalling torchvision-0.5.0+cpu:
  Would remove:
    /opt/conda/lib/python3.6/site-packages/torchvision-0.5.0+cpu.dist-info/*
    /opt/conda/lib/python3.6/site-packages/torchvision/*
Proceed (y/n)?   Successfully uninstalled torchvision-0.5.0+cpu
yes: standard output: Broken pipe
Collecting torchvision
  Downloading torchvision-0.11.2-cp36-cp36m-manylinux1_x86_64.whl (23.3 MB)
     |████████████████████████████████| 23.3 MB 5.9 MB/s eta 0:00:01
Collecting torch==1.10.1
  Downloading torch-1.10.1-cp36-cp36m-manylinux1_x86_64.whl (881.9 MB)
     |████████████████████████████████| 881.9 MB 4.4 kB/s s eta 0:00:01   |█▊                              | 46.2 MB 47.4 MB/s eta 0:00:18
Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from torchvision) (1.19.5)
Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /opt/conda/lib/python3.6/site-packages (from torchvision) (8.1.2)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.6/site-packages (from torch==1.10.1->torchvision) (3.7.4.3)
Requirement already satisfied: dataclasses in /opt/conda/lib/python3.6/site-packages (from torch==1.10.1->torchvision) (0.8)
Installing collected packages: torch, torchvision
  Attempting uninstall: torch
    Found existing installation: torch 1.4.0
    Uninstalling torch-1.4.0:
      Successfully uninstalled torch-1.4.0
Successfully installed torch-1.10.1 torchvision-0.11.2
[3]:
%matplotlib inline
import matplotlib
import numpy as np
import matplotlib.pyplot as plt
import numpy as np
import os
from os import listdir
from os.path import isfile, join
from shutil import copyfile
import sagemaker
from sagemaker.pytorch import PyTorchModel
from sagemaker import get_execution_role

sagemaker_session = sagemaker.Session()
role = get_execution_role()

bucket = sagemaker_session.default_bucket()
prefix = "sagemaker/DEMO-pytorch-batch-inference-script"
print("Bucket: {}".format(bucket))
Bucket: sagemaker-us-west-2-521695447989

Model training

Since the main purpose of this notebook is to demonstrate SageMaker PyTorch batch transform, we reuse a SageMaker Python SDK PyTorch MNIST example to train a PyTorch model. It takes around 7 minutes to finish the training.

[4]:
from torchvision.datasets import MNIST
from torchvision import transforms

local_dir = "data"
MNIST.mirrors = ["https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/"]
MNIST(
    local_dir,
    download=True,
    transform=transforms.Compose(
        [transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]
    ),
)


inputs = sagemaker_session.upload_data(path=local_dir, bucket=bucket, key_prefix=prefix)
print("input spec (in this case, just an S3 path): {}".format(inputs))

from sagemaker.pytorch import PyTorch

estimator = PyTorch(
    entry_point="model-script/mnist.py",
    role=role,
    framework_version="1.8.0",
    py_version="py3",
    instance_count=3,
    instance_type="ml.c5.2xlarge",
    hyperparameters={
        "epochs": 1,
        "backend": "gloo",
    },  # set epochs to a more realistic number for real training
)

estimator.fit({"training": inputs})
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-images-idx3-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz
Extracting data/MNIST/raw/train-images-idx3-ubyte.gz to data/MNIST/raw

Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-labels-idx1-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/train-labels-idx1-ubyte.gz to data/MNIST/raw/train-labels-idx1-ubyte.gz
Extracting data/MNIST/raw/train-labels-idx1-ubyte.gz to data/MNIST/raw

Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-images-idx3-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-images-idx3-ubyte.gz to data/MNIST/raw/t10k-images-idx3-ubyte.gz
Extracting data/MNIST/raw/t10k-images-idx3-ubyte.gz to data/MNIST/raw

Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-labels-idx1-ubyte.gz
Downloading https://sagemaker-sample-files.s3.amazonaws.com/datasets/image/MNIST/t10k-labels-idx1-ubyte.gz to data/MNIST/raw/t10k-labels-idx1-ubyte.gz
Extracting data/MNIST/raw/t10k-labels-idx1-ubyte.gz to data/MNIST/raw

input spec (in this case, just an S3 path): s3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script
2022-04-18 00:17:13 Starting - Starting the training job...
2022-04-18 00:17:33 Starting - Preparing the instances for trainingProfilerReport-1650241033: InProgress
.........
2022-04-18 00:19:13 Downloading - Downloading input data...
...
INFO:__main__:Test set: Average loss: 0.4341, Accuracy: 8826/10000 (88%)
2022-04-18 00:20:14,348 sagemaker-training-toolkit INFO     Reporting training SUCCESS

2022-04-18 00:20:39 Uploading - Uploading generated training model
2022-04-18 00:20:39 Completed - Training job completed
Training seconds: 216
Billable seconds: 216

Prepare batch inference data

Convert the test data into PNG image format.

[5]:
!ls data/MNIST/raw
t10k-images-idx3-ubyte     train-images-idx3-ubyte
t10k-images-idx3-ubyte.gz  train-images-idx3-ubyte.gz
t10k-labels-idx1-ubyte     train-labels-idx1-ubyte
t10k-labels-idx1-ubyte.gz  train-labels-idx1-ubyte.gz
[6]:
# untar gz => png

import gzip
import numpy as np
import os

with gzip.open(os.path.join(local_dir, "MNIST/raw", "t10k-images-idx3-ubyte.gz"), "rb") as f:
    images = np.frombuffer(f.read(), np.uint8, offset=16).reshape(-1, 28, 28)
[7]:
print(len(images), "test images")
10000 test images

Randomly sample 100 test images and upload them to S3.

[8]:
import random
from PIL import Image as im

ids = random.sample(range(len(images)), 100)
ids = np.array(ids, dtype=np.int)
selected_images = images[ids]

image_dir = "data/images"

if not os.path.exists(image_dir):
    os.makedirs(image_dir)

for i, img in enumerate(selected_images):
    pngimg = im.fromarray(img)
    pngimg.save(os.path.join(image_dir, f"{i}.png"))
[9]:
inference_prefix = "batch_transform"
inference_inputs = sagemaker_session.upload_data(
    path=image_dir, bucket=bucket, key_prefix=inference_prefix
)
print("Input S3 path: {}".format(inference_inputs))
Input S3 path: s3://sagemaker-us-west-2-000000000000/batch_transform

Create model transformer

Now, we create a transformer object for creating and interacting with Amazon SageMaker transform jobs. We can create the transformer in two ways: 1. Use a fitted estimator directly. 1. First create a PyTorchModel from a saved model artifact, and then create a transformer from the PyTorchModel object.

Here, we implement the model_fn, input_fn, predict_fn and output_fn function to override the default PyTorch inference handler.

In the input_fn() function, the inferenced images are encoded as a Python ByteArray. That’s why we use the load_from_bytearray() function to load images from io.BytesIO and then use PIL.image to read the images.

def model_fn(model_dir):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model = torch.nn.DataParallel(Net())
    with open(os.path.join(model_dir, "model.pth"), "rb") as f:
        model.load_state_dict(torch.load(f))
    return model.to(device)


def load_from_bytearray(request_body):
    image_as_bytes = io.BytesIO(request_body)
    image = Image.open(image_as_bytes)
    image_tensor = ToTensor()(image).unsqueeze(0)
    return image_tensor


def input_fn(request_body, request_content_type):
    # if set content_type as "image/jpg" or "application/x-npy",
    # the input is also a python bytearray
    if request_content_type == "application/x-image":
        image_tensor = load_from_bytearray(request_body)
    else:
        print("not support this type yet")
        raise ValueError("not support this type yet")
    return image_tensor


# Perform prediction on the deserialized object, with the loaded model
def predict_fn(input_object, model):
    output = model.forward(input_object)
    pred = output.max(1, keepdim=True)[1]

    return {"predictions": pred.item()}


# Serialize the prediction result into the desired response content type
def output_fn(predictions, response_content_type):
    return json.dumps(predictions)
[10]:
# Use fitted estimator directly
transformer = estimator.transformer(instance_count=1, instance_type="ml.c5.xlarge")
[11]:
# You can also create a Transformer object from saved model artifact

# Get model artifact location by estimator.model_data, or give an S3 key directly
model_artifact_s3_location = estimator.model_data  # "s3://<BUCKET>/<PREFIX>/model.tar.gz"

# Create PyTorchModel from saved model artifact
pytorch_model = PyTorchModel(
    model_data=model_artifact_s3_location,
    role=role,
    framework_version="1.8.0",
    py_version="py3",
    source_dir="model-script/",
    entry_point="mnist.py",
)

# Create transformer from PyTorchModel object
transformer = pytorch_model.transformer(instance_count=1, instance_type="ml.c5.xlarge")

Batch inference

Next, we perform inference on the sampled 100 MNIST images in a batch manner.

Input images directly from S3 location

We set S3DataType=S3Prefix to use all objects that match the specified S3 prefix for batch inference.

[12]:
transformer.transform(
    data=inference_inputs,
    data_type="S3Prefix",
    content_type="application/x-image",
    wait=True,
)
......................2022-04-18 00:25:31,379 [INFO ] main org.pytorch.serve.ModelServer - 
Torchserve version: 0.3.0
TS Home: /opt/conda/lib/python3.6/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 948 M
Python executable: /opt/conda/bin/python3.6
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2022-04-18 00:25:31,409 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-04-18 00:25:31,427 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag 5b0df4a34bfd419791760e1c82fe8572
2022-04-18 00:25:31,437 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-04-18 00:25:31,461 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-04-18 00:25:31,623 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:25:31,627 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]47
2022-04-18 00:25:31,627 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,627 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,640 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:25:31,656 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:25:31,663 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:25:31,663 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]48
2022-04-18 00:25:31,663 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]50
2022-04-18 00:25:31,664 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,664 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,664 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,666 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:25:31,664 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:25:31,667 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,668 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:25:31,672 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]49
2022-04-18 00:25:31,673 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:25:31,673 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:25:31,673 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:25:31,679 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-04-18 00:25:31,679 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2022-04-18 00:25:31,683 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9002.
2022-04-18 00:25:31,683 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9001.
2022-04-18 00:25:31,683 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9003.
2022-04-18 00:25:31,683 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9000.
2022-04-18 00:25:31,687 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
Model server started.
2022-04-18 00:25:32,368 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,370 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:48.106632232666016|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,370 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:7.8092803955078125|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,371 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:14.0|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,371 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:6416.7265625|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,372 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:931.8046875|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,372 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:15.8|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,760 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 978
2022-04-18 00:25:32,785 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 982
2022-04-18 00:25:32,784 [INFO ] W-9002-model_1 TS_METRICS - W-9002-model_1.ms:1339|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,786 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:119|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:32,785 [INFO ] W-9003-model_1 TS_METRICS - W-9003-model_1.ms:1340|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,787 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:116|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:32,810 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 982
2022-04-18 00:25:32,810 [INFO ] W-9000-model_1 TS_METRICS - W-9000-model_1.ms:1368|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,811 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:140|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:32,834 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1052
2022-04-18 00:25:32,834 [INFO ] W-9001-model_1 TS_METRICS - W-9001-model_1.ms:1389|#Level:Host|#hostname:ad582a180490,timestamp:1650241532
2022-04-18 00:25:32,834 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:93|#Level:Host|#hostname:ad582a180490,timestamp:null

2022-04-18 00:25:37,399 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:34650 "GET /ping HTTP/1.1" 200 13
2022-04-18 00:25:37,400 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,435 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:34658 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:25:37,436 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 19
2022-04-18 00:25:37,399 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:34650 "GET /ping HTTP/1.1" 200 13
2022-04-18 00:25:37,400 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,435 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:34658 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:25:37,436 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 19
2022-04-18T00:25:37.442:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,509 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:10.01|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b991ddd1-11b8-4246-817f-e234377cde44,timestamp:1650241537
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 14
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,615 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:12.89|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e58006b2-3494-4f9d-b427-b2c05a48234a,timestamp:1650241537
2022-04-18 00:25:37,507 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,508 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,509 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:10.01|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b991ddd1-11b8-4246-817f-e234377cde44,timestamp:1650241537
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 14
2022-04-18 00:25:37,611 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,612 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,615 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:12.89|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e58006b2-3494-4f9d-b427-b2c05a48234a,timestamp:1650241537
2022-04-18 00:25:37,738 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 10
2022-04-18 00:25:37,738 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 10
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:d408fdeb-76de-4ce8-bd12-a1eea1dd98c8,timestamp:1650241537
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,865 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,866 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.9|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:968865ef-ce8f-457c-b5e7-05c15656c9c1,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fe21acba-2b6f-4c7e-8fab-23849016df58,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b3ac9ae1-a77f-486c-98c0-55a1d973d768,timestamp:1650241538
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2428a546-97b7-434d-a57d-039aec65a617,timestamp:1650241538
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:dc5c0414-007c-4371-be4c-f69d96ed0720,timestamp:1650241538
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:35432e1d-1b13-4ac8-8e91-0d1c0167946f,timestamp:1650241538
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:d408fdeb-76de-4ce8-bd12-a1eea1dd98c8,timestamp:1650241537
2022-04-18 00:25:37,739 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,740 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 11
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 12
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,864 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,865 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,866 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:9.9|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:968865ef-ce8f-457c-b5e7-05c15656c9c1,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fe21acba-2b6f-4c7e-8fab-23849016df58,timestamp:1650241537
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:37,971 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:37,972 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:38,085 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b3ac9ae1-a77f-486c-98c0-55a1d973d768,timestamp:1650241538
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,086 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,169 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2428a546-97b7-434d-a57d-039aec65a617,timestamp:1650241538
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,170 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:dc5c0414-007c-4371-be4c-f69d96ed0720,timestamp:1650241538
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:38,263 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,264 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:35432e1d-1b13-4ac8-8e91-0d1c0167946f,timestamp:1650241538
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,345 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,460 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:3.08|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:84f54fd8-bbb2-4ab3-a3ca-610b7242dbf9,timestamp:1650241538
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.93|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:cae123fa-f9f3-40ad-abfd-5bc1040046ac,timestamp:1650241538
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:25:38,458 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,459 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,460 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:3.08|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:84f54fd8-bbb2-4ab3-a3ca-610b7242dbf9,timestamp:1650241538
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,538 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.93|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:cae123fa-f9f3-40ad-abfd-5bc1040046ac,timestamp:1650241538
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,539 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,670 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:80ac25bc-b2d8-4d7d-b3b6-e329488bd6ea,timestamp:1650241538
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.37|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fb0b9838-d516-441e-9442-976cd42aa26a,timestamp:1650241538
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:54f2e26b-f633-41c1-8176-ea707c86c4fa,timestamp:1650241539
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,198 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1bdb78d-2a97-4035-8f1c-3ddcffabfb32,timestamp:1650241539
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,669 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,670 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:80ac25bc-b2d8-4d7d-b3b6-e329488bd6ea,timestamp:1650241538
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:38,977 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.37|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:fb0b9838-d516-441e-9442-976cd42aa26a,timestamp:1650241538
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:38,978 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,109 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:54f2e26b-f633-41c1-8176-ea707c86c4fa,timestamp:1650241539
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,110 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,198 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1bdb78d-2a97-4035-8f1c-3ddcffabfb32,timestamp:1650241539
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 6
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 7
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:026f3936-0c77-4f36-8e0e-56ecb4ae20c9,timestamp:1650241539
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 6
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 7
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,199 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:39,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:026f3936-0c77-4f36-8e0e-56ecb4ae20c9,timestamp:1650241539
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2a8d0ca9-6f8e-422a-906c-b2f203a2f552,timestamp:1650241539
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,789 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:56f4d62c-2a83-48cb-98e6-6e1cfe075e9b,timestamp:1650241539
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2a8d0ca9-6f8e-422a-906c-b2f203a2f552,timestamp:1650241539
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,788 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,789 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:39,890 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:56f4d62c-2a83-48cb-98e6-6e1cfe075e9b,timestamp:1650241539
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:94bf65ca-213c-4b82-a2b1-389731a59dfb,timestamp:1650241539
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,971 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3839c8f1-0fff-4907-b167-6e954c9b6249,timestamp:1650241540
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,891 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:94bf65ca-213c-4b82-a2b1-389731a59dfb,timestamp:1650241539
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,970 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:39,971 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,138 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3839c8f1-0fff-4907-b167-6e954c9b6249,timestamp:1650241540
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:721d9d18-93a1-4dbc-babe-098a5c83db10,timestamp:1650241540
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.09|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f98f09f-a587-4c18-88c8-1c667a55fa66,timestamp:1650241540
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:51c658e4-bbbf-48ce-9dd4-19025f15637c,timestamp:1650241540
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,409 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ee677be-3ab1-427b-bb39-cc2058eb1d5c,timestamp:1650241540
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e33199cc-ed87-4655-8303-4a1541c90d9a,timestamp:1650241540
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.54|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c0a5cfd2-a0b5-4cd1-a471-0789f663bdc8,timestamp:1650241540
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,139 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:721d9d18-93a1-4dbc-babe-098a5c83db10,timestamp:1650241540
2022-04-18 00:25:40,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.09|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f98f09f-a587-4c18-88c8-1c667a55fa66,timestamp:1650241540
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,321 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:51c658e4-bbbf-48ce-9dd4-19025f15637c,timestamp:1650241540
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,408 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,409 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ee677be-3ab1-427b-bb39-cc2058eb1d5c,timestamp:1650241540
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,487 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,488 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e33199cc-ed87-4655-8303-4a1541c90d9a,timestamp:1650241540
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,571 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.54|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c0a5cfd2-a0b5-4cd1-a471-0789f663bdc8,timestamp:1650241540
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,663 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,662 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,663 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f064f2b-4452-4892-986a-442e4a1bd239,timestamp:1650241540
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4a8d9a35-2d3f-44c9-a6b9-08754e99a96b,timestamp:1650241540
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f064f2b-4452-4892-986a-442e4a1bd239,timestamp:1650241540
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,737 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4a8d9a35-2d3f-44c9-a6b9-08754e99a96b,timestamp:1650241540
2022-04-18 00:25:40,820 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a822bc7-0452-4d01-919f-93820a29ef52,timestamp:1650241540
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,889 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.21|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9e36891e-4df0-4ef8-949b-a195ba717ce8,timestamp:1650241540
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.34|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1ba220c-951d-46de-93d6-ba262e321037,timestamp:1650241541
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:635c2e75-2046-4545-a2b3-ba5bc7494a9d,timestamp:1650241541
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:25e16120-8982-4d72-b6ea-6f836e9b6ec0,timestamp:1650241541
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:afa20143-40ae-4a1a-b2e9-2b10fd63dbf8,timestamp:1650241541
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4d69029d-4109-4ff4-9aae-a2830028fc3a,timestamp:1650241541
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.46|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0d9ab440-6082-4968-ab87-056513209b41,timestamp:1650241541
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,821 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a822bc7-0452-4d01-919f-93820a29ef52,timestamp:1650241540
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,888 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,889 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.21|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9e36891e-4df0-4ef8-949b-a195ba717ce8,timestamp:1650241540
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:40,997 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,075 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.34|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a1ba220c-951d-46de-93d6-ba262e321037,timestamp:1650241541
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,076 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:635c2e75-2046-4545-a2b3-ba5bc7494a9d,timestamp:1650241541
2022-04-18 00:25:41,175 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,176 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:25e16120-8982-4d72-b6ea-6f836e9b6ec0,timestamp:1650241541
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,313 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,314 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:afa20143-40ae-4a1a-b2e9-2b10fd63dbf8,timestamp:1650241541
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,402 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,403 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,503 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4d69029d-4109-4ff4-9aae-a2830028fc3a,timestamp:1650241541
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,504 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.46|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0d9ab440-6082-4968-ab87-056513209b41,timestamp:1650241541
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,609 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,610 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:37ce426c-151d-4f19-93d3-6312bb589d2a,timestamp:1650241541
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:155552ff-0166-491b-87a1-768065c651eb,timestamp:1650241541
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:37ce426c-151d-4f19-93d3-6312bb589d2a,timestamp:1650241541
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,684 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,787 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:155552ff-0166-491b-87a1-768065c651eb,timestamp:1650241541
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:efd19fcf-4c61-4b57-b1e3-ee7701885de4,timestamp:1650241541
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,984 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b907dd56-c939-4416-b367-4e97b4013616,timestamp:1650241541
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:92197732-5c4e-440f-953d-9d84731a8658,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f9e9bb4-2a82-4169-af17-d6c45d523bb9,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,158 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ac4dc55c-1762-4cee-9259-60436b74a592,timestamp:1650241542
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a56fabe-4973-40be-9fd7-0484202dfae6,timestamp:1650241542
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.74|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:83aba925-0481-4b31-99ab-0f471176bea3,timestamp:1650241542
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db6d82d7-7539-43af-b399-6b2e68e396c7,timestamp:1650241542
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,594 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:445b172a-d3e4-46ac-a88e-09690b0c18dc,timestamp:1650241542
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,788 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:efd19fcf-4c61-4b57-b1e3-ee7701885de4,timestamp:1650241541
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,899 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,984 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b907dd56-c939-4416-b367-4e97b4013616,timestamp:1650241541
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:41,985 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,080 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:92197732-5c4e-440f-953d-9d84731a8658,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3f9e9bb4-2a82-4169-af17-d6c45d523bb9,timestamp:1650241542
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,157 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,158 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ac4dc55c-1762-4cee-9259-60436b74a592,timestamp:1650241542
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,230 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6a56fabe-4973-40be-9fd7-0484202dfae6,timestamp:1650241542
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,318 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,319 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.74|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:83aba925-0481-4b31-99ab-0f471176bea3,timestamp:1650241542
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,412 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,413 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db6d82d7-7539-43af-b399-6b2e68e396c7,timestamp:1650241542
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,504 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,505 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,594 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:445b172a-d3e4-46ac-a88e-09690b0c18dc,timestamp:1650241542
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:42,596 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,597 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:681542c4-b2f6-4374-8fc1-1548e8649613,timestamp:1650241542
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:654cf5fa-63b1-4876-a975-61cfcdc68e88,timestamp:1650241542
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:681542c4-b2f6-4374-8fc1-1548e8649613,timestamp:1650241542
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,675 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:654cf5fa-63b1-4876-a975-61cfcdc68e88,timestamp:1650241542
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,757 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:7680909e-dbcf-480f-bb04-9873c45eccd3,timestamp:1650241542
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:7680909e-dbcf-480f-bb04-9873c45eccd3,timestamp:1650241542
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b6db0553-b1ec-414a-beb1-23e061b1454c,timestamp:1650241542
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c9756dad-5c2e-4e3e-b5fd-00d316df8dc4,timestamp:1650241543
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1769f16d-29a3-482c-bd98-efde53e2ffe4,timestamp:1650241543
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:8bc6d547-caf2-4b33-8259-f58b7324eb3a,timestamp:1650241543
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.31|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0a3a5e29-be58-4d46-ba27-38458886f11c,timestamp:1650241543
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2df74a80-1984-4e71-8529-dfe9cf90a8f6,timestamp:1650241543
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,832 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b6db0553-b1ec-414a-beb1-23e061b1454c,timestamp:1650241542
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:42,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c9756dad-5c2e-4e3e-b5fd-00d316df8dc4,timestamp:1650241543
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,083 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,084 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1769f16d-29a3-482c-bd98-efde53e2ffe4,timestamp:1650241543
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,218 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,219 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:8bc6d547-caf2-4b33-8259-f58b7324eb3a,timestamp:1650241543
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,312 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.31|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0a3a5e29-be58-4d46-ba27-38458886f11c,timestamp:1650241543
2022-04-18 00:25:43,399 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,400 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:2df74a80-1984-4e71-8529-dfe9cf90a8f6,timestamp:1650241543
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,504 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,505 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e3e047b3-424b-42e6-8708-d18083d36bdc,timestamp:1650241543
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.32|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e3e047b3-424b-42e6-8708-d18083d36bdc,timestamp:1650241543
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,605 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,679 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41139c2c-6e37-44d9-b3fa-ed7a7d79b0a5,timestamp:1650241543
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e4dd2c-1358-4f16-934b-e6a349809496,timestamp:1650241543
2022-04-18 00:25:43,679 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41139c2c-6e37-44d9-b3fa-ed7a7d79b0a5,timestamp:1650241543
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,680 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:43,764 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e4dd2c-1358-4f16-934b-e6a349809496,timestamp:1650241543
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a8e94616-c8fb-4b19-bd21-4f255f0bcdff,timestamp:1650241543
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41d23174-17a0-49fe-9db0-9fbd987a6f01,timestamp:1650241543
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5756636f-71fe-4ff3-b01f-8d534f154a50,timestamp:1650241544
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0c9368b2-c75a-4b10-b426-1ea1e485dc96,timestamp:1650241544
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5ee9406b-7a75-4941-980b-51ea5896615f,timestamp:1650241544
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65fb6e50-eb8b-4e14-8db5-9cda18081118,timestamp:1650241544
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,383 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 8
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:5.86|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4afa65a1-2ac5-4472-8949-6234d29fd3e8,timestamp:1650241544
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:6|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aec048c4-6a9e-4636-88c5-cd63fec5813b,timestamp:1650241544
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,765 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.72|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a8e94616-c8fb-4b19-bd21-4f255f0bcdff,timestamp:1650241543
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,847 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:43,918 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:41d23174-17a0-49fe-9db0-9fbd987a6f01,timestamp:1650241543
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:43,919 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5756636f-71fe-4ff3-b01f-8d534f154a50,timestamp:1650241544
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,041 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0c9368b2-c75a-4b10-b426-1ea1e485dc96,timestamp:1650241544
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,139 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,140 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.36|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5ee9406b-7a75-4941-980b-51ea5896615f,timestamp:1650241544
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,209 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,210 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.71|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65fb6e50-eb8b-4e14-8db5-9cda18081118,timestamp:1650241544
2022-04-18 00:25:44,281 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,282 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,383 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 8
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:5.86|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4afa65a1-2ac5-4472-8949-6234d29fd3e8,timestamp:1650241544
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,384 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:6|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aec048c4-6a9e-4636-88c5-cd63fec5813b,timestamp:1650241544
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6718ab87-dc9b-4618-9e71-ca5ebfa50101,timestamp:1650241544
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:f67abd31-9d02-4990-9e51-9b6869312e3d,timestamp:1650241544
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,463 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6718ab87-dc9b-4618-9e71-ca5ebfa50101,timestamp:1650241544
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,561 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.33|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:f67abd31-9d02-4990-9e51-9b6869312e3d,timestamp:1650241544
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,632 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d5575ea-c1b1-47ed-af77-ef415809b2bd,timestamp:1650241544
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.03|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e650aebf-2df9-4dbf-aac1-1d3d8df4d634,timestamp:1650241544
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 13
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e6fd38ae-8b54-41d8-9ea1-17c8527b2c9a,timestamp:1650241544
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d5575ea-c1b1-47ed-af77-ef415809b2bd,timestamp:1650241544
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,721 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.03|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e650aebf-2df9-4dbf-aac1-1d3d8df4d634,timestamp:1650241544
2022-04-18 00:25:44,807 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 13
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 15
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,808 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:e6fd38ae-8b54-41d8-9ea1-17c8527b2c9a,timestamp:1650241544
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3bae0c32-eb6d-4f97-a8d9-da14ec10b964,timestamp:1650241544
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b42e1c6c-fe99-4a2e-b624-676e3117b178,timestamp:1650241545
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:176504c4-16be-4531-9a23-2905adc479ac,timestamp:1650241545
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4506218e-a262-4ac3-864d-df900a919b9f,timestamp:1650241545
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:470bcd94-b192-4e63-a0bf-bd6a6ecb5bf0,timestamp:1650241545
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,876 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.22|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:3bae0c32-eb6d-4f97-a8d9-da14ec10b964,timestamp:1650241544
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:44,954 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,041 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:b42e1c6c-fe99-4a2e-b624-676e3117b178,timestamp:1650241545
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,042 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:176504c4-16be-4531-9a23-2905adc479ac,timestamp:1650241545
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,114 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:4506218e-a262-4ac3-864d-df900a919b9f,timestamp:1650241545
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,191 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:470bcd94-b192-4e63-a0bf-bd6a6ecb5bf0,timestamp:1650241545
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db958c22-670b-4f6e-afe4-5c5a8e70e93d,timestamp:1650241545
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aff03e2e-c9b6-4154-afb7-f8fee4b093ef,timestamp:1650241545
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.73|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ebaea0a-c800-48d6-b3f7-206080f155ea,timestamp:1650241545
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,577 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e5d83b-bc1e-4cb5-8106-41aed195cf96,timestamp:1650241545
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ec1e2a0e-1e71-4050-a7fb-314380c5ed86,timestamp:1650241545
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,265 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:db958c22-670b-4f6e-afe4-5c5a8e70e93d,timestamp:1650241545
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,345 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:aff03e2e-c9b6-4154-afb7-f8fee4b093ef,timestamp:1650241545
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,432 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,505 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.73|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1ebaea0a-c800-48d6-b3f7-206080f155ea,timestamp:1650241545
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,506 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,577 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,578 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.3|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:18e5d83b-bc1e-4cb5-8106-41aed195cf96,timestamp:1650241545
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,669 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.51|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:ec1e2a0e-1e71-4050-a7fb-314380c5ed86,timestamp:1650241545
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,670 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.77|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:75937063-aafb-4cde-82e5-ef15c109cfe0,timestamp:1650241545
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:598ceffc-323d-403b-996c-f18fd6163e74,timestamp:1650241545
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,858 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:838b4ff4-466c-477f-9f72-22c67196742b,timestamp:1650241545
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,074 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:27e0174c-e775-4c2f-b198-3a133d49259e,timestamp:1650241546
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6d0f88bc-3934-4555-b8c7-7a5311ffe85e,timestamp:1650241546
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.2|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:927937c1-8998-4686-96f0-56f4094cfeff,timestamp:1650241546
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.77|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:75937063-aafb-4cde-82e5-ef15c109cfe0,timestamp:1650241545
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,780 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:598ceffc-323d-403b-996c-f18fd6163e74,timestamp:1650241545
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,857 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,858 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:838b4ff4-466c-477f-9f72-22c67196742b,timestamp:1650241545
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:45,956 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,074 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.27|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:27e0174c-e775-4c2f-b198-3a133d49259e,timestamp:1650241546
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,075 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:6d0f88bc-3934-4555-b8c7-7a5311ffe85e,timestamp:1650241546
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,211 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,212 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.2|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:927937c1-8998-4686-96f0-56f4094cfeff,timestamp:1650241546
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65e09797-8ede-42b7-8fee-9a62fdb8e1a5,timestamp:1650241546
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9015e397-aab8-4259-8638-38703c66ad12,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:19b535b5-7e83-458e-9229-0f322798997a,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9b087c7c-e738-453a-8f85-f6dfe6c535e7,timestamp:1650241546
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,286 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:65e09797-8ede-42b7-8fee-9a62fdb8e1a5,timestamp:1650241546
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,383 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,384 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,472 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,473 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9015e397-aab8-4259-8638-38703c66ad12,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.25|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:19b535b5-7e83-458e-9229-0f322798997a,timestamp:1650241546
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,569 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.26|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:9b087c7c-e738-453a-8f85-f6dfe6c535e7,timestamp:1650241546
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,662 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:af1d81f6-ce62-4106-8959-532467b8dabc,timestamp:1650241546
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a0e9f81d-97c5-4117-8f04-c93584bdcc8d,timestamp:1650241546
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,817 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:af1d81f6-ce62-4106-8959-532467b8dabc,timestamp:1650241546
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,743 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.53|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:a0e9f81d-97c5-4117-8f04-c93584bdcc8d,timestamp:1650241546
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,816 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,817 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.79|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f3e36b1-70c1-4840-b9b6-20b1401164b0,timestamp:1650241546
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:bce91358-0a4e-4d85-833a-cb0189043ea9,timestamp:1650241546
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d2c82c1-91d5-48f3-b4f3-09da2d42daa5,timestamp:1650241547
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,095 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c2609e01-7fc7-4817-b804-f7bf071f41fc,timestamp:1650241547
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:97f14871-b6bf-4d79-ada6-9183f92069ed,timestamp:1650241547
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5fda8390-6584-499e-9af2-ea46d5c96353,timestamp:1650241547
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.79|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:0f3e36b1-70c1-4840-b9b6-20b1401164b0,timestamp:1650241546
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,898 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:46,994 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.35|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:bce91358-0a4e-4d85-833a-cb0189043ea9,timestamp:1650241546
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:1d2c82c1-91d5-48f3-b4f3-09da2d42daa5,timestamp:1650241547
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,094 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,095 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.28|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:c2609e01-7fc7-4817-b804-f7bf071f41fc,timestamp:1650241547
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,186 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.29|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:97f14871-b6bf-4d79-ada6-9183f92069ed,timestamp:1650241547
2022-04-18 00:25:47,255 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,256 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.23|#ModelName:model,Level:Model|#hostname:ad582a180490,requestID:5fda8390-6584-499e-9af2-ea46d5c96353,timestamp:1650241547
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:34664 "POST /invocations HTTP/1.1" 200 2
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null
2022-04-18 00:25:47,328 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:ad582a180490,timestamp:null

Input images by manifest file

First, we generate a manifest file. Then we use the manifest file containing a list of object keys as inputs to batch inference. Some key points: - content_type = "application/x-image" (here the content_type is for the actual object for inference, not for the manifest file) - data_type = "ManifestFile" - Manifest file format must follow the format as S3DataSource points out. We create the manifest file by using the jsonlines package.

[
    {"prefix": "s3://customer_bucket/some/prefix/"},
    "relative/path/to/custdata-1",
    "relative/path/custdata-2",
    ...
    "relative/path/custdata-N"
]
[13]:
!pip install -q jsonlines
[14]:
import jsonlines

# Build image list
manifest_prefix = f"s3://{bucket}/{prefix}/images/"

path = image_dir
img_files = [f for f in listdir(path) if isfile(join(path, f))]

print("img_files\n", img_files)

manifest_content = [{"prefix": manifest_prefix}]
manifest_content.extend(img_files)

print("manifest_content\n", manifest_content)

# Write jsonl file
manifest_file = "manifest.json"
with jsonlines.open(manifest_file, mode="w") as writer:
    writer.write(manifest_content)

# Upload to S3
manifest_obj = sagemaker_session.upload_data(path=manifest_file, key_prefix=prefix)

print("manifest_obj\n", manifest_obj)
img_files
 ['2.png', '43.png', '46.png', '45.png', '16.png', '4.png', '92.png', '12.png', '18.png', '75.png', '87.png', '63.png', '81.png', '78.png', '13.png', '34.png', '42.png', '19.png', '73.png', '14.png', '83.png', '3.png', '55.png', '5.png', '38.png', '86.png', '27.png', '7.png', '15.png', '80.png', '89.png', '72.png', '88.png', '99.png', '82.png', '32.png', '70.png', '36.png', '67.png', '26.png', '94.png', '41.png', '30.png', '64.png', '28.png', '59.png', '52.png', '90.png', '69.png', '31.png', '37.png', '47.png', '40.png', '56.png', '58.png', '1.png', '93.png', '61.png', '29.png', '76.png', '23.png', '50.png', '97.png', '79.png', '85.png', '6.png', '57.png', '35.png', '11.png', '22.png', '62.png', '21.png', '95.png', '71.png', '60.png', '17.png', '77.png', '25.png', '49.png', '96.png', '53.png', '54.png', '48.png', '51.png', '74.png', '20.png', '0.png', '91.png', '33.png', '24.png', '65.png', '66.png', '10.png', '68.png', '39.png', '9.png', '98.png', '84.png', '8.png', '44.png']
manifest_content
 [{'prefix': 's3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script/images/'}, '2.png', '43.png', '46.png', '45.png', '16.png', '4.png', '92.png', '12.png', '18.png', '75.png', '87.png', '63.png', '81.png', '78.png', '13.png', '34.png', '42.png', '19.png', '73.png', '14.png', '83.png', '3.png', '55.png', '5.png', '38.png', '86.png', '27.png', '7.png', '15.png', '80.png', '89.png', '72.png', '88.png', '99.png', '82.png', '32.png', '70.png', '36.png', '67.png', '26.png', '94.png', '41.png', '30.png', '64.png', '28.png', '59.png', '52.png', '90.png', '69.png', '31.png', '37.png', '47.png', '40.png', '56.png', '58.png', '1.png', '93.png', '61.png', '29.png', '76.png', '23.png', '50.png', '97.png', '79.png', '85.png', '6.png', '57.png', '35.png', '11.png', '22.png', '62.png', '21.png', '95.png', '71.png', '60.png', '17.png', '77.png', '25.png', '49.png', '96.png', '53.png', '54.png', '48.png', '51.png', '74.png', '20.png', '0.png', '91.png', '33.png', '24.png', '65.png', '66.png', '10.png', '68.png', '39.png', '9.png', '98.png', '84.png', '8.png', '44.png']
manifest_obj
 s3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script/manifest.json
[15]:
# Batch transform with manifest file
transform_job = transformer.transform(
    data=manifest_obj,
    data_type="ManifestFile",
    content_type="application/x-image",
    wait=False,
)
[16]:
print("Latest transform job:", transformer.latest_transform_job.name)
Latest transform job: pytorch-inference-2022-04-18-00-26-08-985
[17]:
# look at the status of the transform job
import pprint as pp

sm_cli = sagemaker_session.sagemaker_client

job_info = sm_cli.describe_transform_job(TransformJobName=transformer.latest_transform_job.name)

pp.pprint(job_info)
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 9, 32000, tzinfo=tzlocal()),
 'DataProcessing': {'InputFilter': '$',
                    'JoinSource': 'None',
                    'OutputFilter': '$'},
 'ModelName': 'pytorch-inference-2022-04-18-00-21-14-564',
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '870',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Mon, 18 Apr 2022 00:26:09 GMT',
                                      'x-amzn-requestid': '394fa75d-5814-41e3-a604-2a471c52c745'},
                      'HTTPStatusCode': 200,
                      'RequestId': '394fa75d-5814-41e3-a604-2a471c52c745',
                      'RetryAttempts': 0},
 'TransformInput': {'CompressionType': 'None',
                    'ContentType': 'application/x-image',
                    'DataSource': {'S3DataSource': {'S3DataType': 'ManifestFile',
                                                    'S3Uri': 's3://sagemaker-us-west-2-000000000000/sagemaker/DEMO-pytorch-batch-inference-script/manifest.json'}},
                    'SplitType': 'None'},
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-inference-2022-04-18-00-26-08-985',
 'TransformJobName': 'pytorch-inference-2022-04-18-00-26-08-985',
 'TransformJobStatus': 'InProgress',
 'TransformOutput': {'AssembleWith': 'None',
                     'KmsKeyId': '',
                     'S3OutputPath': 's3://sagemaker-us-west-2-000000000000/pytorch-inference-2022-04-18-00-26-08-985'},
 'TransformResources': {'InstanceCount': 1, 'InstanceType': 'ml.c5.xlarge'}}

Multiple instance

We use instance_count > 1 to create multiple inference instances. When a batch transform job starts, Amazon SageMaker initializes compute instances and distributes the inference or preprocessing workload between them. Batch Transform partitions the Amazon S3 objects in the input by key and maps Amazon S3 objects to instances. Given multiple files, one instance might process input1.csv, and another instance might process input2.csv. Read more at Use Batch Transform.

[18]:
dist_transformer = estimator.transformer(instance_count=2, instance_type="ml.c4.xlarge")

dist_transformer.transform(
    data=inference_inputs,
    data_type="S3Prefix",
    content_type="application/x-image",
    wait=True,
)
................................2022-04-18 00:31:23,943 [INFO ] main org.pytorch.serve.ModelServer - 
Torchserve version: 0.3.0
TS Home: /opt/conda/lib/python3.6/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 910 M
Python executable: /opt/conda/bin/python3.6
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2022-04-18 00:31:23,983 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-04-18 00:31:24,005 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag c3fb4d26ffa849bbbf79ab8856e86748
2022-04-18 00:31:24,017 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-04-18 00:31:24,049 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-04-18 00:31:24,238 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:24,243 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:24,244 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46
2022-04-18 00:31:24,244 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,244 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,245 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]45
2022-04-18 00:31:24,245 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,245 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,253 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:24,253 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]48
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,278 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:24,279 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,282 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]47
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:24,283 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:24,320 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-04-18 00:31:24,320 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2022-04-18 00:31:24,321 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
2022-04-18 00:31:24,328 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9001.
2022-04-18 00:31:24,335 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9000.
2022-04-18 00:31:24,337 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9003.
2022-04-18 00:31:24,337 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9002.
Model server started.
2022-04-18 00:31:25,106 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,108 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:48.10660934448242|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,109 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:7.809303283691406|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,110 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:14.0|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,111 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:6271.05078125|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,112 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:922.55078125|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,112 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:16.0|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,493 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1042
2022-04-18 00:31:25,494 [INFO ] W-9002-model_1 TS_METRICS - W-9002-model_1.ms:1465|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,495 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:112|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:25,511 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1065
2022-04-18 00:31:25,512 [INFO ] W-9003-model_1 TS_METRICS - W-9003-model_1.ms:1482|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,512 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:107|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:25,624 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1186
2022-04-18 00:31:25,624 [INFO ] W-9001-model_1 TS_METRICS - W-9001-model_1.ms:1596|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,625 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:97|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:25,815 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1337
2022-04-18 00:31:25,815 [INFO ] W-9000-model_1 TS_METRICS - W-9000-model_1.ms:1789|#Level:Host|#hostname:6925eb526ceb,timestamp:1650241885
2022-04-18 00:31:25,815 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:137|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:26,665 [INFO ] main org.pytorch.serve.ModelServer - 
Torchserve version: 0.3.0
TS Home: /opt/conda/lib/python3.6/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 4
Max heap size: 912 M
Python executable: /opt/conda/bin/python3.6
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 4
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
2022-04-18 00:31:26,709 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-04-18 00:31:26,732 [INFO ] main org.pytorch.serve.archive.ModelArchive - eTag 2eb711663c324fc78881634b94b209fd
2022-04-18 00:31:26,744 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-04-18 00:31:26,771 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-04-18 00:31:26,975 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:26,976 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]46
2022-04-18 00:31:26,976 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:26,976 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:26,991 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:26,992 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]47
2022-04-18 00:31:26,992 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:26,993 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:26,995 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9000
2022-04-18 00:31:26,999 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9002
2022-04-18 00:31:27,009 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]48
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9001
2022-04-18 00:31:27,010 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:27,028 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-04-18 00:31:27,028 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
2022-04-18 00:31:27,029 [INFO ] W-9002-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9002.
2022-04-18 00:31:27,032 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
2022-04-18 00:31:27,033 [INFO ] W-9001-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9001.
2022-04-18 00:31:27,034 [INFO ] W-9000-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9000.
2022-04-18 00:31:27,035 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Listening on port: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:27,042 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - [PID]49
2022-04-18 00:31:27,043 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Torch worker started.
2022-04-18 00:31:27,043 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9003
2022-04-18 00:31:27,044 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Python runtime: 3.6.13
2022-04-18 00:31:27,052 [INFO ] W-9003-model_1-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Connection accepted: /home/model-server/tmp/.ts.sock.9003.
Model server started.
2022-04-18 00:31:27,971 [INFO ] pool-2-thread-1 TS_METRICS - CPUUtilization.Percent:0.0|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,973 [INFO ] pool-2-thread-1 TS_METRICS - DiskAvailable.Gigabytes:48.106597900390625|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,973 [INFO ] pool-2-thread-1 TS_METRICS - DiskUsage.Gigabytes:7.809314727783203|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,975 [INFO ] pool-2-thread-1 TS_METRICS - DiskUtilization.Percent:14.0|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,976 [INFO ] pool-2-thread-1 TS_METRICS - MemoryAvailable.Megabytes:6252.3125|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,977 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUsed.Megabytes:947.546875|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:27,977 [INFO ] pool-2-thread-1 TS_METRICS - MemoryUtilization.Percent:16.3|#Level:Host|#hostname:aeaba6005455,timestamp:1650241887
2022-04-18 00:31:28,254 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1087
2022-04-18 00:31:28,255 [INFO ] W-9002-model_1 TS_METRICS - W-9002-model_1.ms:1500|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,255 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:122|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:28,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1166
2022-04-18 00:31:28,345 [INFO ] W-9003-model_1 TS_METRICS - W-9003-model_1.ms:1590|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:128|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:28,570 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1391
2022-04-18 00:31:28,571 [INFO ] W-9000-model_1 TS_METRICS - W-9000-model_1.ms:1820|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,572 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:133|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:28,579 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1404
2022-04-18 00:31:28,579 [INFO ] W-9001-model_1 TS_METRICS - W-9001-model_1.ms:1825|#Level:Host|#hostname:aeaba6005455,timestamp:1650241888
2022-04-18 00:31:28,579 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:127|#Level:Host|#hostname:aeaba6005455,timestamp:null

2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:44472 "GET /ping HTTP/1.1" 200 18
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,140 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:44484 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:31:35,141 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,329 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 16
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:44472 "GET /ping HTTP/1.1" 200 18
2022-04-18 00:31:35,112 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,140 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:44484 "GET /execution-parameters HTTP/1.1" 404 1
2022-04-18 00:31:35,141 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,329 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 16
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 27
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c799b9aa-9065-4c8c-ae3d-cb85d035b0f1,timestamp:1650241895
2022-04-18 00:31:35,331 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,332 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,413 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 17
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,415 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,417 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,419 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.32|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ce3f0999-05bb-4d18-870c-1f0a9c12da1f,timestamp:1650241895
2022-04-18 00:31:35,583 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 16
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.5|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:40e53ea9-18f8-4664-a76e-8c45221eca14,timestamp:1650241895
2022-04-18 00:31:35,585 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,661 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 14
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:11.35|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4fb87cc8-2720-4456-893b-9351130db55b,timestamp:1650241895
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 27
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,330 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c799b9aa-9065-4c8c-ae3d-cb85d035b0f1,timestamp:1650241895
2022-04-18 00:31:35,331 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,332 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,413 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 17
2022-04-18 00:31:35,414 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,415 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,417 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,419 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.32|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ce3f0999-05bb-4d18-870c-1f0a9c12da1f,timestamp:1650241895
2022-04-18 00:31:35,583 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 16
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,584 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.5|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:40e53ea9-18f8-4664-a76e-8c45221eca14,timestamp:1650241895
2022-04-18 00:31:35,585 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:5|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,661 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 12
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 14
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:11.35|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4fb87cc8-2720-4456-893b-9351130db55b,timestamp:1650241895
2022-04-18 00:31:35,662 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.91|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8f2694cc-381e-4ff7-859f-a87eeab21645,timestamp:1650241895
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91728dfd-b512-4ed5-bc56-473d6bb85b95,timestamp:1650241895
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.89|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:86462d03-25d9-4d23-ab00-956a18a4699c,timestamp:1650241895
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,663 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,768 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.91|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8f2694cc-381e-4ff7-859f-a87eeab21645,timestamp:1650241895
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,769 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91728dfd-b512-4ed5-bc56-473d6bb85b95,timestamp:1650241895
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:35,867 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,868 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.89|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:86462d03-25d9-4d23-ab00-956a18a4699c,timestamp:1650241895
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,973 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:35,974 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.01|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c979e5b8-d351-4866-8eb3-f23e983d5304,timestamp:1650241896
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7e4887c5-77ee-437d-bb09-2c1290a0d6f4,timestamp:1650241896
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,075 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.01|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c979e5b8-d351-4866-8eb3-f23e983d5304,timestamp:1650241896
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,076 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,199 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7e4887c5-77ee-437d-bb09-2c1290a0d6f4,timestamp:1650241896
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,200 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e1e978b5-e89b-4184-b870-f8026d93cb34,timestamp:1650241896
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:03bbd271-23a6-4f97-8f74-54d206a11203,timestamp:1650241896
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.68|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:05a39a1f-550f-46d8-a4ab-bb0bad787800,timestamp:1650241896
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.58|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ca413e4c-c8d5-450f-9f5d-51be6e134530,timestamp:1650241896
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.82|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:09736318-febd-4e92-9ccd-fca21691dcb7,timestamp:1650241896
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,290 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e1e978b5-e89b-4184-b870-f8026d93cb34,timestamp:1650241896
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,291 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:03bbd271-23a6-4f97-8f74-54d206a11203,timestamp:1650241896
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,388 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,389 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.68|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:05a39a1f-550f-46d8-a4ab-bb0bad787800,timestamp:1650241896
2022-04-18 00:31:36,482 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,483 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.58|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:ca413e4c-c8d5-450f-9f5d-51be6e134530,timestamp:1650241896
2022-04-18 00:31:36,570 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,571 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:36,699 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.82|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:09736318-febd-4e92-9ccd-fca21691dcb7,timestamp:1650241896
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b3247490-10da-4d5c-b4f4-177fb67fdcab,timestamp:1650241896
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,909 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.4|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:71cf38d0-8082-462a-a642-61074509fe05,timestamp:1650241896
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,700 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.59|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b3247490-10da-4d5c-b4f4-177fb67fdcab,timestamp:1650241896
2022-04-18 00:31:36,817 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,818 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,909 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.4|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:71cf38d0-8082-462a-a642-61074509fe05,timestamp:1650241896
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,910 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:36,209 [INFO ] pool-1-thread-5 ACCESS_LOG - /169.254.255.130:33020 "GET /ping HTTP/1.1" 200 14
2022-04-18 00:31:36,209 [INFO ] pool-1-thread-5 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,234 [INFO ] epollEventLoopGroup-3-2 ACCESS_LOG - /169.254.255.130:33028 "GET /execution-parameters HTTP/1.1" 404 2
2022-04-18 00:31:36,235 [INFO ] epollEventLoopGroup-3-2 TS_METRICS - Requests4XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,706 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:36,707 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.87|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:227b55e2-a7fa-4598-973b-72af3ff57517,timestamp:1650241896
2022-04-18 00:31:36,707 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 23
2022-04-18 00:31:36,708 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,708 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,708 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,789 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 15
2022-04-18 00:31:36,789 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 17
2022-04-18 00:31:36,789 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,790 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,790 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,793 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:14.16|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:66a7abd0-8acc-4a58-ab8a-4c59227c3a34,timestamp:1650241896
2022-04-18 00:31:36,956 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 14
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 16
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:13.59|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:89a7c047-6e98-4831-a5ae-4ee298656d11,timestamp:1650241896
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:36,957 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,040 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 6
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 14
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:8|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,041 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:11.43|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:53b4927a-4681-4402-92b1-502fdee4c1cc,timestamp:1650241897
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8628f76e-49d3-4ff2-9f9e-1884344a99e2,timestamp:1650241897
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,012 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:8628f76e-49d3-4ff2-9f9e-1884344a99e2,timestamp:1650241897
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.48|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f0b859d7-9ac3-4961-8e6c-cad21445a0a6,timestamp:1650241897
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,170 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:372f2fa2-9630-4d4b-a336-f0c645acdf86,timestamp:1650241897
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b0ef0f3e-3f59-4f9e-942c-e04d18827555,timestamp:1650241897
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,415 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.31|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a725d82f-b83b-4a36-908c-357f88d93092,timestamp:1650241897
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:feca74ad-59b1-4059-86f4-df1a82621e25,timestamp:1650241897
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:97441582-f139-4090-9210-9da15b779aa3,timestamp:1650241897
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,013 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,088 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.48|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f0b859d7-9ac3-4961-8e6c-cad21445a0a6,timestamp:1650241897
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,089 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,170 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:372f2fa2-9630-4d4b-a336-f0c645acdf86,timestamp:1650241897
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,171 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,172 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:b0ef0f3e-3f59-4f9e-942c-e04d18827555,timestamp:1650241897
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,312 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,313 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,415 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.31|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a725d82f-b83b-4a36-908c-357f88d93092,timestamp:1650241897
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,417 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:feca74ad-59b1-4059-86f4-df1a82621e25,timestamp:1650241897
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,498 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,499 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:97441582-f139-4090-9210-9da15b779aa3,timestamp:1650241897
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,609 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,610 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91586bb6-f73d-48e3-82ea-ea21f17cc195,timestamp:1650241897
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,808 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.99|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fb9bb9d8-0b26-4d84-a69e-e617de0fac97,timestamp:1650241897
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,810 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e8c21333-c613-46a8-b6db-f594df078bc8,timestamp:1650241897
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.9|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:785ba516-f3f3-43fb-a1f4-4acedd4b5f98,timestamp:1650241897
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,721 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:91586bb6-f73d-48e3-82ea-ea21f17cc195,timestamp:1650241897
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,722 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,808 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.99|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fb9bb9d8-0b26-4d84-a69e-e617de0fac97,timestamp:1650241897
2022-04-18 00:31:37,809 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,810 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e8c21333-c613-46a8-b6db-f594df078bc8,timestamp:1650241897
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,885 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,886 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.9|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:785ba516-f3f3-43fb-a1f4-4acedd4b5f98,timestamp:1650241897
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,977 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:37,145 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.82|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:0ca92892-412d-4b4a-8f04-0f24f8449075,timestamp:1650241897
2022-04-18 00:31:37,146 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,146 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:37,147 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,147 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,147 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.04|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:5e808b17-7077-4502-99a4-4c21b7f0ee7e,timestamp:1650241897
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,263 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.7|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:728c5e6c-1fe2-43a2-a1b6-ec3c604c1449,timestamp:1650241897
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,356 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,482 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.54|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:4438bc7d-213a-4fe4-9091-52a5f5d59367,timestamp:1650241897
2022-04-18 00:31:37,483 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,483 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,483 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,484 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,484 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.62|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:50e762c0-7961-4b73-be5b-f5d8a490139d,timestamp:1650241897
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,576 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,577 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,675 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.95|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:c6be9d72-a3bf-485e-93fa-bc33cbbe9366,timestamp:1650241897
2022-04-18 00:31:37,676 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 4
2022-04-18 00:31:37,676 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,676 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,677 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,677 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,790 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:37,790 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.88|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:7dd92d30-4f50-4451-ad7a-9a2c0e5bac1f,timestamp:1650241897
2022-04-18 00:31:37,790 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,791 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,791 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,791 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,868 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,868 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.1|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:2a4aa142-240d-43ee-8c48-39203f12c87a,timestamp:1650241897
2022-04-18 00:31:37,869 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:37,869 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,869 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,870 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.56|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ddd338ae-015d-45f5-afc8-f16a65263bbc,timestamp:1650241897
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:37,956 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,957 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:37,957 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.99|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:263127c6-6899-4fe7-9afb-864d21461441,timestamp:1650241898
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,028 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,029 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,029 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18T00:31:35.166:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18T00:31:35.166:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:43b7c81a-40e5-425e-909a-950eb23214be,timestamp:1650241898
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:edeb3c61-bcc1-4446-80a1-8bad4fa0b859,timestamp:1650241898
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,199 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.44|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:43b7c81a-40e5-425e-909a-950eb23214be,timestamp:1650241898
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,073 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,074 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:edeb3c61-bcc1-4446-80a1-8bad4fa0b859,timestamp:1650241898
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,198 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,199 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a0d1e73b-8e62-41b7-8324-715f7ab78bee,timestamp:1650241898
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,278 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.03|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f2ad3be4-bb23-47af-b44d-c5bdb8000fd7,timestamp:1650241898
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,362 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4b339f15-ac1c-493e-81d1-f370f551bf8f,timestamp:1650241898
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a0d1e73b-8e62-41b7-8324-715f7ab78bee,timestamp:1650241898
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,277 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,278 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.03|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:f2ad3be4-bb23-47af-b44d-c5bdb8000fd7,timestamp:1650241898
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,361 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,362 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,455 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4b339f15-ac1c-493e-81d1-f370f551bf8f,timestamp:1650241898
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,456 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:3fec32ed-2f82-4c16-963f-3024a80c849e,timestamp:1650241898
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,549 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:9fb04cfa-de2d-4e94-ad81-5294fb544348,timestamp:1650241898
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,634 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:56194b87-7837-4b8c-ae2b-29f0bd555db5,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a9de82c8-4e7d-4016-bf09-1cdb32ba362c,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e94c019b-91a6-469b-9037-7862a951b8ef,timestamp:1650241898
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.49|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:3fec32ed-2f82-4c16-963f-3024a80c849e,timestamp:1650241898
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,548 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,549 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.64|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:9fb04cfa-de2d-4e94-ad81-5294fb544348,timestamp:1650241898
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,633 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,634 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,715 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,714 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:56194b87-7837-4b8c-ae2b-29f0bd555db5,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.78|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:a9de82c8-4e7d-4016-bf09-1cdb32ba362c,timestamp:1650241898
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,795 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:e94c019b-91a6-469b-9037-7862a951b8ef,timestamp:1650241898
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,894 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,008 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:dbcd5a31-f816-41c4-9d7d-e863b9c42c41,timestamp:1650241899
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,895 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:4|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,008 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:dbcd5a31-f816-41c4-9d7d-e863b9c42c41,timestamp:1650241899
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,009 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:38,101 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,101 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.06|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:cfe34dbd-0132-470e-8fbf-510237becf25,timestamp:1650241898
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,102 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.61|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:2789dd8b-0aff-40e3-ad39-042e4f24c667,timestamp:1650241898
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,180 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,181 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,181 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.22|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:5849d060-58a6-49ca-8aa8-7c3556a73029,timestamp:1650241898
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,264 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,344 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.02|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:7d29b540-302f-46f6-9e0b-2d847068f85e,timestamp:1650241898
2022-04-18 00:31:38,345 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,345 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,346 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,346 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,346 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,465 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,465 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.08|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:dbeb7149-b0a4-4a22-b9cf-ed56a5cde68c,timestamp:1650241898
2022-04-18 00:31:38,465 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,466 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,466 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,466 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,533 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,533 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:00839105-e7cf-4333-b753-23cec06c51a3,timestamp:1650241898
2022-04-18 00:31:38,534 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,534 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,534 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,535 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:979783b2-39db-4452-baae-0d8a6197d923,timestamp:1650241898
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,619 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,620 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,720 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,720 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.57|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:a34bab70-4653-4cad-8309-c90b88ca7abd,timestamp:1650241898
2022-04-18 00:31:38,720 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 6
2022-04-18 00:31:38,721 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,721 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,721 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.45|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ae9ef618-36d7-45f4-a4b1-6acf15659a05,timestamp:1650241898
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,807 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,808 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,808 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,910 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:38,910 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.0|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:5d44bd8e-fccd-4037-9c1b-5cd7deec6adc,timestamp:1650241898
2022-04-18 00:31:38,910 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:38,911 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,911 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,911 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.85|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:344973ea-fdc9-4971-bc7b-5e3bb84ac9b4,timestamp:1650241898
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,993 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:38,994 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,065 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e6773617-7ec3-4638-b4d6-cd50946ec2df,timestamp:1650241899
2022-04-18 00:31:39,066 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,066 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,066 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,067 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,067 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.61|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:14a2e880-a272-495b-b826-644e912b9e91,timestamp:1650241899
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:04e4bfe5-44ab-48bf-be75-378c40d7a614,timestamp:1650241899
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c00fa137-4528-42c7-9c44-a6c34c080239,timestamp:1650241899
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.61|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:14a2e880-a272-495b-b826-644e912b9e91,timestamp:1650241899
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,101 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:04e4bfe5-44ab-48bf-be75-378c40d7a614,timestamp:1650241899
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,195 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,196 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,321 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:c00fa137-4528-42c7-9c44-a6c34c080239,timestamp:1650241899
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,322 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,323 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.11|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:13561332-7314-476a-ba5a-2dd08cb21559,timestamp:1650241899
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,466 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:bb2810fe-ff27-408c-b755-1851414fad57,timestamp:1650241899
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4581545b-af0b-40e5-9cf4-1f2516283725,timestamp:1650241899
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,849 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7460afef-5259-4f8b-8174-372465fec3ad,timestamp:1650241899
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.83|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:18f7633a-7ea7-42cd-a332-e10ef4e37459,timestamp:1650241899
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.11|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:13561332-7314-476a-ba5a-2dd08cb21559,timestamp:1650241899
2022-04-18 00:31:39,398 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,399 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,466 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,465 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:bb2810fe-ff27-408c-b755-1851414fad57,timestamp:1650241899
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,575 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:4581545b-af0b-40e5-9cf4-1f2516283725,timestamp:1650241899
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,849 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.65|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:7460afef-5259-4f8b-8174-372465fec3ad,timestamp:1650241899
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,850 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.83|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:18f7633a-7ea7-42cd-a332-e10ef4e37459,timestamp:1650241899
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,947 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:39,167 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,167 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.02|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:78f258da-cbd4-4257-b8e4-21a52ac6b9e0,timestamp:1650241899
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 5
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,168 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,239 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,240 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,241 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.97|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:f6adc969-b682-4898-8629-70f60920769d,timestamp:1650241899
2022-04-18 00:31:39,319 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:6330ed70-1bc4-48fd-9c56-54e0dabf7636,timestamp:1650241899
2022-04-18 00:31:39,319 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 0
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,320 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:3|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.53|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:080bdd6c-0fb0-43f4-b7d5-55d4acb1e3ba,timestamp:1650241899
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,425 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,500 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.95|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:8156bf18-3b7d-4eff-929e-360271be9cea,timestamp:1650241899
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,576 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e4f7270d-c86e-4870-95de-71350e989d2e,timestamp:1650241899
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.92|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:2c8be008-add5-498b-9e38-9be5481cbfbc,timestamp:1650241899
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,687 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,688 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,688 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.0|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:3b20f880-2fc4-49ae-91f9-9eec5ca7f62c,timestamp:1650241899
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,783 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,896 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.41|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:d69cfe35-cfa8-4bd9-b84d-d5e8713da29a,timestamp:1650241899
2022-04-18 00:31:39,896 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,897 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,998 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:39,998 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.66|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:0ab7d0c0-6d4a-4453-82d3-b22487820980,timestamp:1650241899
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:39,999 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.92|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fa834bd9-83db-461a-a6bd-aaa21880fde2,timestamp:1650241900
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,042 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:be1cc1e4-7046-4dcc-b30a-93e6352660ee,timestamp:1650241900
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.7|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:d09c61ab-3119-409f-9f2f-2d3219b5c57b,timestamp:1650241900
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:78e83f62-bbbe-43d6-a26b-ad673804d849,timestamp:1650241900
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.92|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:fa834bd9-83db-461a-a6bd-aaa21880fde2,timestamp:1650241900
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,041 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,042 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 1
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:be1cc1e4-7046-4dcc-b30a-93e6352660ee,timestamp:1650241900
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,228 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.7|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:d09c61ab-3119-409f-9f2f-2d3219b5c57b,timestamp:1650241900
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,326 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,407 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.63|#ModelName:model,Level:Model|#hostname:aeaba6005455,requestID:78e83f62-bbbe-43d6-a26b-ad673804d849,timestamp:1650241900
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:44492 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18 00:31:40,408 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:2|#Level:Host|#hostname:aeaba6005455,timestamp:null
2022-04-18T00:31:36.242:[sagemaker logs]: MaxConcurrentTransforms=1, MaxPayloadInMB=6, BatchStrategy=MULTI_RECORD
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:2.02|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:6738de45-b281-4d36-b934-b536cdfd6094,timestamp:1650241900
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,200 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,201 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:a3ba2aee-7ef6-4875-9e06-b0f66165ca23,timestamp:1650241900
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,282 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,283 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.43|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e2aabc1b-4c78-4e3c-83f8-69b74ffa0d65,timestamp:1650241900
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,422 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.84|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e5ca6928-3980-4d67-bca8-28a7db32e549,timestamp:1650241900
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,600 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,601 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,601 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,680 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.6|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:e7699b7f-84ac-477f-9516-8632d313c6d3,timestamp:1650241900
2022-04-18 00:31:40,680 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,681 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,773 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,773 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.46|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ca289c06-8bfb-4b08-8f13-85eb8e419afb,timestamp:1650241900
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,774 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.47|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:26879460-25da-42cd-9b1f-44d7167777d9,timestamp:1650241900
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:40,881 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,882 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,882 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,959 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:40,959 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.84|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:1f4658dd-a8df-4b59-a63d-6c8c16d77e71,timestamp:1650241900
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:40,960 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.67|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:27625aff-b3fd-40a8-8451-e4361cbc11f3,timestamp:1650241901
2022-04-18 00:31:41,057 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,155 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.39|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:4e4901d4-b922-4479-a187-d95f1d7ffbb9,timestamp:1650241901
2022-04-18 00:31:41,156 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 3
2022-04-18 00:31:41,156 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,156 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,157 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,157 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,236 [INFO ] W-9001-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,236 [INFO ] W-9001-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.5|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:72175ab0-8de5-45dd-967a-ec7a232d0253,timestamp:1650241901
2022-04-18 00:31:41,236 [INFO ] W-9001-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,237 [INFO ] W-9001-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,237 [INFO ] W-9001-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,237 [INFO ] W-9001-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,330 [INFO ] W-9000-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,330 [INFO ] W-9000-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.76|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:162df6be-8c39-416c-afce-8a67be0f4f58,timestamp:1650241901
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,331 [INFO ] W-9000-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,451 [INFO ] W-9002-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,451 [INFO ] W-9002-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.83|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:c864d856-7d5b-46aa-91b1-3d454af757a4,timestamp:1650241901
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 4
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,452 [INFO ] W-9002-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,532 [INFO ] W-9003-model_1 org.pytorch.serve.wlm.WorkerThread - Backend response time: 2
2022-04-18 00:31:41,532 [INFO ] W-9003-model_1-stdout MODEL_METRICS - PredictionTime.Milliseconds:1.38|#ModelName:model,Level:Model|#hostname:6925eb526ceb,requestID:ced3a724-9b27-4786-b487-c40c90b89521,timestamp:1650241901
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 ACCESS_LOG - /169.254.255.130:33038 "POST /invocations HTTP/1.1" 200 3
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 TS_METRICS - Requests2XX.Count:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 TS_METRICS - QueueTime.ms:0|#Level:Host|#hostname:6925eb526ceb,timestamp:null
2022-04-18 00:31:41,533 [INFO ] W-9003-model_1 TS_METRICS - WorkerThreadTime.ms:1|#Level:Host|#hostname:6925eb526ceb,timestamp:null

Look at all transform jobs

We list and describe the transform jobs to retrieve information about them.

[19]:
transform_jobs = sm_cli.list_transform_jobs()["TransformJobSummaries"]
for job in transform_jobs:
    pp.pprint(job)
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 32, 3, 162000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 32, 3, 162000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/automl-churn-sdk-18-00-20-19-dpp0-rpb-1-ab06a7aa54904c088293a6d',
 'TransformJobName': 'automl-churn-sdk-18-00-20-19-dpp0-rpb-1-ab06a7aa54904c088293a6d',
 'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 32, 0, 946000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 32, 1, 607000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/automl-churn-sdk-18-00-20-19-dpp8-rpb-1-9abdff16bbd34c83872e871',
 'TransformJobName': 'automl-churn-sdk-18-00-20-19-dpp8-rpb-1-9abdff16bbd34c83872e871',
 'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 31, 58, 909000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 31, 59, 560000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/automl-churn-sdk-18-00-20-19-dpp1-csv-1-1dd140703bb140bdaf51362',
 'TransformJobName': 'automl-churn-sdk-18-00-20-19-dpp1-csv-1-1dd140703bb140bdaf51362',
 'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 29, 11, 444000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 29, 11, 758000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/sagemaker-scikit-learn-2022-04-18-00-29-11-429',
 'TransformJobName': 'sagemaker-scikit-learn-2022-04-18-00-29-11-429',
 'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 27, 37, 320000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 31, 17, 282000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/xgboost-2022-04-18-00-27-37-309',
 'TransformJobName': 'xgboost-2022-04-18-00-27-37-309',
 'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 41, 812000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 30, 29, 572000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/batch-transform-2022-04-18-00-26-41',
 'TransformJobName': 'Batch-Transform-2022-04-18-00-26-41',
 'TransformJobStatus': 'InProgress'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 12, 635000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 31, 42, 971000, tzinfo=tzlocal()),
 'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 31, 42, 720000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-training-2022-04-18-00-26-12-621',
 'TransformJobName': 'pytorch-training-2022-04-18-00-26-12-621',
 'TransformJobStatus': 'Completed'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 9, 32000, tzinfo=tzlocal()),
 'FailureReason': 'ClientError: See job logs for more information',
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 30, 13, 568000, tzinfo=tzlocal()),
 'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 30, 13, 253000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-inference-2022-04-18-00-26-08-985',
 'TransformJobName': 'pytorch-inference-2022-04-18-00-26-08-985',
 'TransformJobStatus': 'Failed'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 23, 34, 840000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 28, 0, 728000, tzinfo=tzlocal()),
 'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 28, 0, 359000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/r-transformer-2022-04-18-00-23-34-816',
 'TransformJobName': 'R-Transformer-2022-04-18-00-23-34-816',
 'TransformJobStatus': 'Completed'}
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 23, 9, 974000, tzinfo=tzlocal()),
 'LastModifiedTime': datetime.datetime(2022, 4, 18, 0, 26, 38, 885000, tzinfo=tzlocal()),
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/tensorflow-inference-2022-04-18-00-23-09-960',
 'TransformJobName': 'tensorflow-inference-2022-04-18-00-23-09-960',
 'TransformJobStatus': 'InProgress'}
[20]:
job_info = sm_cli.describe_transform_job(
    TransformJobName=dist_transformer.latest_transform_job.name
)

pp.pprint(job_info)
{'CreationTime': datetime.datetime(2022, 4, 18, 0, 26, 12, 635000, tzinfo=tzlocal()),
 'DataProcessing': {'InputFilter': '$',
                    'JoinSource': 'None',
                    'OutputFilter': '$'},
 'Environment': {},
 'ModelName': 'pytorch-training-2022-04-18-00-26-11-963',
 'ResponseMetadata': {'HTTPHeaders': {'content-length': '907',
                                      'content-type': 'application/x-amz-json-1.1',
                                      'date': 'Mon, 18 Apr 2022 00:32:03 GMT',
                                      'x-amzn-requestid': 'c75d7d27-2982-45b7-b619-f04bffe72fff'},
                      'HTTPStatusCode': 200,
                      'RequestId': 'c75d7d27-2982-45b7-b619-f04bffe72fff',
                      'RetryAttempts': 0},
 'TransformEndTime': datetime.datetime(2022, 4, 18, 0, 31, 42, 720000, tzinfo=tzlocal()),
 'TransformInput': {'CompressionType': 'None',
                    'ContentType': 'application/x-image',
                    'DataSource': {'S3DataSource': {'S3DataType': 'S3Prefix',
                                                    'S3Uri': 's3://sagemaker-us-west-2-000000000000/batch_transform'}},
                    'SplitType': 'None'},
 'TransformJobArn': 'arn:aws:sagemaker:us-west-2:000000000000:transform-job/pytorch-training-2022-04-18-00-26-12-621',
 'TransformJobName': 'pytorch-training-2022-04-18-00-26-12-621',
 'TransformJobStatus': 'Completed',
 'TransformOutput': {'AssembleWith': 'None',
                     'KmsKeyId': '',
                     'S3OutputPath': 's3://sagemaker-us-west-2-000000000000/pytorch-training-2022-04-18-00-26-12-621'},
 'TransformResources': {'InstanceCount': 2, 'InstanceType': 'ml.c4.xlarge'},
 'TransformStartTime': datetime.datetime(2022, 4, 18, 0, 29, 50, 865000, tzinfo=tzlocal())}
[21]:
import re


def get_bucket_and_prefix(s3_output_path):
    trim = re.sub("s3://", "", s3_output_path)
    bucket, prefix = trim.split("/")
    return bucket, prefix


local_path = "output"  # Where to save the output locally

bucket, output_prefix = get_bucket_and_prefix(job_info["TransformOutput"]["S3OutputPath"])
print(bucket, output_prefix)

sagemaker_session.download_data(path=local_path, bucket=bucket, key_prefix=output_prefix)
sagemaker-us-west-2-521695447989 pytorch-training-2022-04-18-00-26-12-621
[22]:
!ls {local_path}
0.png.out   24.png.out  4.png.out   55.png.out  70.png.out  86.png.out
1.png.out   25.png.out  40.png.out  56.png.out  71.png.out  87.png.out
10.png.out  26.png.out  41.png.out  57.png.out  72.png.out  88.png.out
11.png.out  27.png.out  42.png.out  58.png.out  73.png.out  89.png.out
12.png.out  28.png.out  43.png.out  59.png.out  74.png.out  9.png.out
13.png.out  29.png.out  44.png.out  6.png.out   75.png.out  90.png.out
14.png.out  3.png.out   45.png.out  60.png.out  76.png.out  91.png.out
15.png.out  30.png.out  46.png.out  61.png.out  77.png.out  92.png.out
16.png.out  31.png.out  47.png.out  62.png.out  78.png.out  93.png.out
17.png.out  32.png.out  48.png.out  63.png.out  79.png.out  94.png.out
18.png.out  33.png.out  49.png.out  64.png.out  8.png.out   95.png.out
19.png.out  34.png.out  5.png.out   65.png.out  80.png.out  96.png.out
2.png.out   35.png.out  50.png.out  66.png.out  81.png.out  97.png.out
20.png.out  36.png.out  51.png.out  67.png.out  82.png.out  98.png.out
21.png.out  37.png.out  52.png.out  68.png.out  83.png.out  99.png.out
22.png.out  38.png.out  53.png.out  69.png.out  84.png.out
23.png.out  39.png.out  54.png.out  7.png.out   85.png.out
[23]:
# Inspect the output

import json

for f in os.listdir(local_path):
    path = os.path.join(local_path, f)
    with open(path, "r") as f:
        pred = json.load(f)
        print(pred)
{'predictions': 9}
{'predictions': 9}
{'predictions': 7}
{'predictions': 3}
{'predictions': 4}
{'predictions': 5}
{'predictions': 7}
{'predictions': 3}
{'predictions': 0}
{'predictions': 2}
{'predictions': 2}
{'predictions': 5}
{'predictions': 8}
{'predictions': 4}
{'predictions': 3}
{'predictions': 8}
{'predictions': 6}
{'predictions': 0}
{'predictions': 0}
{'predictions': 1}
{'predictions': 2}
{'predictions': 4}
{'predictions': 4}
{'predictions': 0}
{'predictions': 7}
{'predictions': 8}
{'predictions': 4}
{'predictions': 7}
{'predictions': 5}
{'predictions': 4}
{'predictions': 6}
{'predictions': 5}
{'predictions': 2}
{'predictions': 1}
{'predictions': 5}
{'predictions': 6}
{'predictions': 4}
{'predictions': 8}
{'predictions': 0}
{'predictions': 2}
{'predictions': 8}
{'predictions': 4}
{'predictions': 7}
{'predictions': 9}
{'predictions': 2}
{'predictions': 0}
{'predictions': 9}
{'predictions': 3}
{'predictions': 5}
{'predictions': 4}
{'predictions': 6}
{'predictions': 7}
{'predictions': 3}
{'predictions': 9}
{'predictions': 6}
{'predictions': 4}
{'predictions': 2}
{'predictions': 9}
{'predictions': 2}
{'predictions': 8}
{'predictions': 0}
{'predictions': 6}
{'predictions': 3}
{'predictions': 5}
{'predictions': 9}
{'predictions': 4}
{'predictions': 6}
{'predictions': 8}
{'predictions': 4}
{'predictions': 0}
{'predictions': 2}
{'predictions': 0}
{'predictions': 6}
{'predictions': 1}
{'predictions': 6}
{'predictions': 5}
{'predictions': 4}
{'predictions': 6}
{'predictions': 0}
{'predictions': 2}
{'predictions': 5}
{'predictions': 2}
{'predictions': 7}
{'predictions': 8}
{'predictions': 1}
{'predictions': 6}
{'predictions': 3}
{'predictions': 5}
{'predictions': 9}
{'predictions': 5}
{'predictions': 4}
{'predictions': 7}
{'predictions': 5}
{'predictions': 1}
{'predictions': 4}
{'predictions': 7}
{'predictions': 7}
{'predictions': 8}
{'predictions': 1}
{'predictions': 8}

Conclusion

In this notebook, we trained a PyTorch model, created a transformer from it, and then performed batch inference using S3 inputs, manifest files, and on multiple instances. This shows a variety of options that are available when running SageMaker Batch Transform jobs for batch inference.