目录
1.启动docker
2.模型由来
3.部署模型
1） 拉取镜像
2）启动服务
3）查看模型状态
4）查看模型的输入输出
4.模型接口预测：REST
1.启动docker
安装详细介绍可参考：https://blog.csdn.net/weixin_42530066/article/details/107850996
2.模型由来
此处使用已训练好的model，需注意TensorFlow Serving是使用SavedModel格式来保存模型的。由于SavedModel包含了完整的TensorFlow程序，包括权重和计算，它不需要运行原始的模型构建代码，因此对共享或部署使用TFLite，TensorFlow.js，TensorFLow Serving或者TensorFlow Hub很有用
注：关于SavedModel可参考：https://www.tensorflow.org/guide/saved_model，此处不详细介绍
文件目录树如下：
└─exported_model
└─saved_model
└─1
└─savedmodel.pb
3.部署模型
获取官网的tfserving，参考链接，链接中有详细的docker pull命令，链接如下：https://hub.docker.com/r/tensorflow/serving/tags
1） 拉取镜像docker pull tensorflow/serving
λ docker pull tensorflow/serving
Using default tag: latest
latest: Pulling from tensorflow/serving
Digest: sha256:4a45f9bcaf96e71ef1b6d824e490562cc962ea66e1213ab06015a7da8054a53b
Status: Image is up to date for tensorflow/serving:latest
docker.io/tensorflow/serving:latest
2）启动服务
docker run -p 8500:8500 -p 8501:8501 --name tfserving --mount type=bind,source=/exported_model/saved_model,target=/models/saved_model -e MODEL_NAME=saved_model -t tensorflow/serving
参数：
端口8500：TensorFlow Serving提供的gRPC端口
端口8501：REST API服务端口
source=模型存放的绝对路径
-e MODEL_NAME=saved_model指TensorFlow Serving需要加载的模型名称
输出结果如下：
2020-08-06 10:25:18.022501: I tensorflow_serving/model_servers/server.cc:86] Building single TensorFlow model file config: model_name: saved_model model_base_path: /models/saved_model
2020-08-06 10:25:18.058268: I tensorflow_serving/model_servers/server_core.cc:464] Adding/updating models.
2020-08-06 10:25:18.058323: I tensorflow_serving/model_servers/server_core.cc:575] (Re-)adding model: saved_model
2020-08-06 10:25:18.594777: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: saved_model version: 1}
2020-08-06 10:25:18.594894: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: saved_model version: 1}
2020-08-06 10:25:18.594938: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: saved_model version: 1}
2020-08-06 10:25:18.596078: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/saved_model/1
2020-08-06 10:25:19.515086: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2020-08-06 10:25:19.515310: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:295] Reading SavedModel debug info (if present) from: /models/saved_model/1
2020-08-06 10:25:19.780411: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:234] Restoring SavedModel bundle.
2020-08-06 10:25:19.781742: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:244] The specified SavedModel has no variables; no checkpoints were restored. File does not exist: /models/saved_model/1/variables/variables.index
2020-08-06 10:25:19.781852: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:364] SavedModel load for tags { serve }; Status: success: OK. Took 1185786 microseconds.
2020-08-06 10:25:19.801197: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:105] No warmup data file found at /models/saved_model/1/assets.extra/tf_serving_warmup_requests
2020-08-06 10:25:19.808024: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: saved_model version: 1}
2020-08-06 10:25:19.829410: I tensorflow_serving/model_servers/server.cc:355] Running gRPC ModelServer at 0.0.0.0:8500 ...
[warn] getaddrinfo: address family for nodename not supported
[evhttp_server.cc : 238] NET_LOG: Entering the event loop ...
2020-08-06 10:25:19.849060: I tensorflow_serving/model_servers/server.cc:375] Exporting HTTP/REST API at:localhost:8501 ...
3）查看模型状态
使用curl命令查看启动服务状态及模型状态，TensorFlow Serving使用1作为模型的版本号
λ curl http://localhost:8501/v1/models/saved_model
{
"model_version_status": [
{
"version": "1",
"state": "AVAILABLE",
"status": {
"error_code": "OK",
"error_message": ""
}
}
]
}
4）查看模型的输入输出
saved_model_cli show --dir /exported_model/saved_model/1/ --all
注：1为version，version不同填写不一样
输出如下：
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_UINT8
shape: (-1, -1, -1, 3)
name: image_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 100, 4)
name: detection_boxes:0
outputs['detection_classes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 100)
name: detection_classes:0
outputs['detection_multiclass_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 100, 2)
name: detection_multiclass_scores:0
outputs['detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 100)
name: detection_scores:0
outputs['num_detections'] tensor_info:
dtype: DT_FLOAT
shape: (-1)
name: num_detections:0
outputs['raw_detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, -1, 4)
name: raw_detection_boxes:0
outputs['raw_detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, -1, 2)
name: raw_detection_scores:0
Method name is: tensorflow/serving/predict
4.模型接口预测：REST
安装：pip install tensorflow-serving-api
TensorFlow Serving提供REST API和gRPC两种请求方式，接下来具体介绍REST
脚本如下：
import numpy as np
import json
from PIL import Image
import requests
import time
MODEL_NAME = "saved_model"
URL_TFSERVING = f"http://127.0.0.1:8501/v1/models/{MODEL_NAME}:predict"
PATH_IMAGE = "test.jpg"
PATH_LABEL_MAP = "./exported_model/label_map.json"
THRESHOLD = 0.3
def predict(label_map, path_image):
img = Image.open(path_image)
img = img.resize((320, 320))
print(img.size)
data_to_send = {
"instances": np.expand_dims(img, axis=0).tolist(),
}
start = time.time()
ret = requests.post(URL_TFSERVING, timeout=200, json=data_to_send)
detect_ret = ret.json()
width, height = img.size
predictions = detect_ret['predictions'][0]
cnt = 0
for score in predictions['detection_scores']:
if score > THRESHOLD:
cnt += 1
else:
break
detection_boxes = predictions['detection_boxes'][:cnt]
detection_classes = predictions['detection_classes'][:cnt]
boxes = [
{'xmin': int(width * b[1]), 'ymin': int(height * b[0]), 'xmax': int(width * b[3]), 'ymax': int(height * b[2])}
for b in detection_boxes
]
labels = [label_map.get(str(int(i + 1e-6)), "UNKONWN") for i in detection_classes]
print(time.time() - start)
return boxes, labels
if __name__ == '__main__':
with open(PATH_LABEL_MAP, 'r') as f:
label_map = json.load(f)
boxes, labels = predict(label_map, PATH_IMAGE)
print(boxes)
print(labels)
输出如下：
λ python3 pb_tfserving_local_with_label_map.py
(320, 320)
2.8160481452941895
[{'xmin': 199, 'ymin': 69, 'xmax': 266, 'ymax': 222}]
['water_reflect']
查看进程：
λ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
18fd96918d0a tensorflow/serving "/usr/bin/tf_serving…" 30 minutes ago Up 30 minutes 0.0.0.0:8500-8501->8500-8501/tcp tfserving
停止服务进程：
docker stop id #id=18fd96918d0a
docker stop 18fd96918d0a
以上仅供参考
参考：
https://medium.com/tensorflow/serving-ml-quickly-with-tensorflow-serving-and-docker-7df7094aa008
https://www.tensorflow.org/tfx/tutorials/serving/rest_simple
