Skip to main content

Release Announcement v1.6.0

We are pleased to announce that the Apache YuniKorn community has voted to release 1.6.0. Apache YuniKorn is a standalone resource scheduler, designed for managing and scheduling Batch and Data workloads on container orchestration frameworks like Kubernetes for on-prem and on-cloud use cases.

Overview​

The Apache YuniKorn community has resolved 316 JIRAs in this release.

Release manager: Peter Bacsko

Release date: 2024-09-17

Highlights​

Kubernetes version support​

YuniKorn 1.6.0 supports running on Kubernetes clusters from version 1.24 through 1.31. See YUNIKORN-2830 for details.

Unification of Ask and Allocation​

YuniKorn 1.6.0 has been simplified on a code level - we no longer have "Ask" objects in the scheduler core. A resource ask is simply an allocation without a node assigned to it. See YUNIKORN-2457 for details.

Preemption hardening​

After a more thorough analysis of the preemption feature, we identified several use cases where it works either inconsistently or produces unexpected, non-intuitive results. See YUNIKORN-2493 for details.

Establish consistent usage guidelines for labels and annotations​

Before YuniKorn 1.6.0, labels and annotations were not used in a consistent way. Certain metadata existed either as a label or annotation. If it existed for both, the precedence was not clear. If both were defined with different values, no error was reported. This was fixed in YUNIKORN-2501.

Support for OIDC credentials​

In YUNIKORN-2281 we added support for OIDC user names. Previously, these were rejected in the admission controller because certain characters were treated as illegal.

Improved code quality​

In YUNIKORN-182 we improved the code quality of Yunikorn so that the Go linters no longer complain about potential coding problems.

Coverage of various parts of the code was significantly enhanced. See the JIRA filter for the list of items.

Deadlock detection, fixed potential locking issues​

The tool go-deadlock has been integrated to the codebase, which makes it possible to detect potential deadlocks at runtime. Since this slows down the scheduler and increases memory usage, it's disabled by default. However, it's enabled during the unit test phase with "make test". See YUNIKORN-2539 for details.

With go-deadlock, several potential deadlocks have been identified and fixed in YUNIKORN-2544.

Reproducible binaries​

Before YuniKorn 1.6.0, the binaries we build for YuniKorn differ from one build to the next. We standardized our build output so that independently built binaries from the same source code can be validated. See YUNIKORN-2419.

Incompatible changes​

Removed StateAware scheduling​

State-aware scheduling has been removed in YuniKorn 1.6.0.

Plugin mode is deprecated​

From YuniKorn 1.6.0, the plugin mode is considered deprecated and no new plugin-related code will be added.

Fair queue sorting has changed​

The previous version of the fair share sorting algorithm was broken and did not return the ratio correctly. This is now fixed in YUNIKORN-2678. This change fundamentally affects scheduling decisions, because in each cycle the scheduler might end up choosing a different pending request than it did before.

Default queue is no longer set on the pod​

From YuniKorn 1.6.0, the admission controller no longer updates the queue label on the pod if it's unset. See YUNIKORN-2711 for details.

Credits​

The YuniKorn 1.6.0 release would not have been possible without the hard work of our community and we would like to thank the following contributors to this release:

ChenChen Lai, amikai, YuTeng Chen, Arthur Wang, Kuan-Po Tseng, Craig Condit, Hsien-Cheng (Ryan) Huang, Jacob Salway, Manikandan R, Michael Akinyemi, mean-world, Mit Desai, Nick Chao, Paul Santa Clara, Peter Bacsko, 400Ping, Qi Zhu, rich7420, Rich Scott, ryankert, Ryan Lo, SHIAO SZU CHEN, SP12893678, steinsgateted, targetoee, Tseng Hsi-Huang, Tzu-Hua Lan, Wilfred Spiegelenburg, Yongjun Zhang, Yu-Lin Chen, YUN SUN