KF Serving

https://github.com/kubeflow/kfserving

kubeflow/kfserving

 

Install KFServing

 

$ git clone https://github.com/kubeflow/kfserving.git

 

$ TAG=v0.3.0

$ kubectl apply -f ./install/$TAG/kfserving.yaml

namespace/kfserving-system created customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kubeflow.org created clusterrole.rbac.authorization.k8s.io/kfserving-proxy-role created clusterrole.rbac.authorization.k8s.io/manager-role created clusterrolebinding.rbac.authorization.k8s.io/kfserving-proxy-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/manager-rolebinding created configmap/inferenceservice-config created secret/kfserving-webhook-server-secret created service/kfserving-controller-manager-metrics-service created service/kfserving-controller-manager-service created service/kfserving-webhook-server-service created statefulset.apps/kfserving-controller-manager created mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kubeflow.org created validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kubeflow.org created unable to recognize "./install/v0.3.0/kfserving.yaml": no matches for kind "Certificate" in version "cert-manager.io/v1alpha2" unable to recognize "./install/v0.3.0/kfserving.yaml": no matches for kind "Issuer" in version "cert-manager.io/v1alpha2"

오류조치

Cert Manager 설치가 안되서 발생.

=> self signed ca 쉘 수행.

https://github.com/kubeflow/kfserving/blob/master/hack/self-signed-ca.sh

 

 

$ kubectl patch mutatingwebhookconfiguration inferenceservice.serving.kubeflow.org --patch '{"webhooks":[{"name": "inferenceservice.kfserving-webhook-server.pod-mutator","objectSelector":{"matchExpressions":[{"key":"serving.kubeflow.org/inferenceservice", "operator": "Exists"}]}}]}'

The MutatingWebhookConfiguration "inferenceservice.serving.kubeflow.org" is invalid: * webhooks[0].sideEffects: Required value: must specify one of None, NoneOnDryRun, Some, Unknown * webhooks[0].clientConfig: Required value: exactly one of url or service is required * webhooks[0].admissionReviewVersions: Required value: must specify one of v1, v1beta1

 

$ kubectl get po -n kfserving-system

NAME                             READY   STATUS    RESTARTS   AGE kfserving-controller-manager-0   2/2     Running   0          3m35s

 

sklearn 샘플 Inferencing 수행

$ cd kfserving

 

$ kubectl apply -f docs/samples/sklearn/sklearn.yaml

inferenceservice.serving.kubeflow.org/sklearn-iris created

 

$ kubectl get inferenceservices sklearn-iris

NAME           URL                                                                 READY   DEFAULT TRAFFIC   CANARY TRAFFIC   AGE sklearn-iris   http://sklearn-iris.default.terrycloud.com/v1/models/sklearn-iris   True    100                                2m36s

 

$ vi /etc/hosts

192.168.19.128 sklearn-iris.default.terrycloud.com

 

$ curl -v http://sklearn-iris.default.terrycloud.com/v1/models/sklearn-iris:predict -d @./docs/samples/sklearn/iris-input.json

* About to connect() to sklearn-iris.default.terrycloud.com port 80 (#0) *   Trying 192.168.19.128... * Connected to sklearn-iris.default.terrycloud.com (192.168.19.128) port 80 (#0) > POST /v1/models/sklearn-iris:predict HTTP/1.1 > User-Agent: curl/7.29.0 > Host: sklearn-iris.default.terrycloud.com > Accept: */* > Content-Length: 76 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 76 out of 76 bytes < HTTP/1.1 503 Service Unavailable < date: Sun, 15 Mar 2020 23:01:46 GMT < server: istio-envoy < connection: close < content-length: 0 < * Closing connection 0

업로드는 완료되었으나, 모델 접근이 안되는 것 같다.

 

github을 찾아보니, 유사한 내용이 이슈로 올라와 있다.

https://github.com/kubeflow/kfserving/issues/740

the following request failed. $ curl -v -H "Host: sklearn-iris.default.example.com" http://${INGRESS_IP}/v1/models/sklearn-iris:predict -d @./docs/samples/sklearn/iris-input.json   However, the following request succeed. $ curl -v -H "Host: sklearn-iris-predictor-default.default.example.com" http://${INGRESS_IP}/v1/models/sklearn-iris:predict -d @./docs/samples/sklearn/iris-input.json

 

$ vi /etc/hosts

192.168.19.128 sklearn-iris-predictor-default.default.terrycloud.com

 

$ curl -v http://sklearn-iris-predictor-default.default.terrycloud.com/v1/models/sklearn-iris:predict -d @./docs/samples/sklearn/iris-input.json

* About to connect() to sklearn-iris-predictor-default.default.terrycloud.com port 80 (#0) *   Trying 192.168.19.128... * Connected to sklearn-iris-predictor-default.default.terrycloud.com (192.168.19.128) port 80 (#0) > POST /v1/models/sklearn-iris:predict HTTP/1.1 > User-Agent: curl/7.29.0 > Host: sklearn-iris-predictor-default.default.terrycloud.com > Accept: */* > Content-Length: 76 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 76 out of 76 bytes < HTTP/1.1 200 OK < content-length: 23 < content-type: application/json; charset=UTF-8 < date: Mon, 16 Mar 2020 09:23:28 GMT < server: istio-envoy < x-envoy-upstream-service-time: 21 < * Connection #0 to host sklearn-iris-predictor-default.default.terrycloud.com left intact {"predictions": [1, 1]}

 

정상적으로 sklearn-iris 수행후 predictions 이 떨어진다.

아마도, 아직 KFserving 버전이 0.3 이다보니 bug가 많은것 같다. 2021년도에 v1.0 릴리즈 예정이라고 한다.