Release Announcement v0.10.0

It gives us great pleasure to announce that the Apache YuniKorn (Incubating) community has voted to release 0.10.0. Apache YuniKorn (Incubating) is a standalone resource scheduler, designed for managing, and scheduling Big Data workloads on the container orchestration framework for on-prem and on-cloud use cases.

The notable feature added in this release is the Gang Scheduling, YuniKorn now can provide gang scheduling capabilities while running applications on Kubernetes. Along with that, various bug fixes and improvements are included in this release.

Overview

The Apache YuniKorn (Incubating) community has fixed over 187 JIRAs in this release: YuniKorn jiras resolved in 0.10.0

Release manager: Tao Yang

Release date: 2021-04-09

Highlights

Supported Kubernetes Versions

From this release on, supported Kubernetes versions have been updated to 1.16.x, 1.17.x and 1.18.x. (Earlier versions support 1.13.x, 1.14.x and 1.15.x.) YuniKorn support matrix primarily supports 3 major Kubernetes versions.

Gang Scheduling

In this release, YuniKorn starts to support the Gang Scheduling. Users can apply Gang Scheduling for the applications requiring gang scheduling semantics, such as Spark, Tensorflow, Pytorch, etc. YuniKorn proactively reserves resources for gang scheduling applications, which can work more efficiently with cluster-autoscaler. The initial support has been well tested with Spark, and it can be used with the native Spark on K8s or the Spark K8s operator. For more information how to enable and use Gang Scheduling, please read the doc here.

Simplify the logic in the Scheduler Core

The community has done a major code refactoring for the scheduler core, tracked by (YUNKORN-317)[https://issues.apache.org/jira/browse/YUNIKORN-317]. This refactoring combines the cache and scheduler module into one that removes a lot of complexity in the code, improves the efficiency of the core scheduling logic. And as a result, it is much simpler to read and mitigates the maintenance effort.

Application Tracking API and CRD Phase One

This release introduces an application tracking API and K8s custom resource definition (CRD) to further improve the user experience. The CRD will be used by the app operator/job server to interact with YuniKorn, to provide a better app lifecycle management. The first phase has defined the common protocol messages and CRD object formats.

Web UI Refurbishment

The community has made some usability improvements for the web UI, including a redesigned web UI layout, tweaked look and feel to provide better user experience, bug fixes, etc.

Community

The Apache YuniKorn community is pleased to welcome new committers Julia Kinga Marton and Tingyao Huang. We would love to see more committers joining the community and help to shape the project. In the past few months, we continue to have bi-weekly community meetings in 2 different time zones, ad-hoc meetings, offline channel discussions. The community will continue to be open.