Skip to main content
Version: Next

Run RayService

Note

This example demonstrates how to set up KubeRay and run a RayService with the YuniKorn scheduler. It relies on an admission controller to configure the default applicationId and queue name. If you want more details, please refer to Yunikorn supported labels and Yunikorn queue setting.

Modify YuniKorn settings​

Follow YuniKorn install guide and modify YuniKorn configmap "yunikorn-defaults" to allow ray operator based on k8s service account.

kubectl patch configmap yunikorn-defaults -n yunikorn --patch '{"data":{"admissionController.accessControl.systemUsers": "^system:serviceaccount:kube-system:,^system:serviceaccount:default:"}}'

Setup a KubeRay operator​

helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
helm install kuberay-operator kuberay/kuberay-operator --version 1.2.2 --set batchScheduler.name=yunikorn
  • The result should be as shown below ray_cluster_operator

Configure your Ray CRD(optional)​

If you disable admission controller, you need to add the schedulerName: yunikorn in raycluster labels.

#example
metadata:
  labels:
    applicaionId: ray-<service_type>-0001
    queue: root.ray.<specific_queue_name>
spec:
  schedulerName: yunikorn # k8s will inform yunikorn based on this

Run a RayService​

kubectl apply -f https://raw.githubusercontent.com/ray-project/kuberay/v1.1.1/ray-operator/config/samples/ray-service.sample.yaml
  • RayService result ray_service_pod
  • YuniKorn UI ray_service_on_ui

Verify the status of the Ray Serve applications​

Services in Kubernetes aren't directly accessible by default. However, you can use port-forwarding to connect to them locally.

kubectl port-forward svc/rayservice-sample-head-svc 8265:8265

After port-forward set up, you can access serve page in Ray dashboard by going to http://localhost:8265/#/serve in your web browser.

kuberay-dashboard-ui

Send requests to the RayService for testing​

kubectl run curl --image=radial/busyboxplus:curl -i --tty

curl -X POST -H 'Content-Type: application/json' rayservice-sample-serve-svc:8000/fruit/ -d '["MANGO", 2]'

curl -X POST -H 'Content-Type: application/json' rayservice-sample-serve-svc:8000/calc/ -d '["MUL", 3]'

Example: kuberay-service-api-testing