Release Announcement v1.0.0
We are pleased to announce that the Apache YuniKorn community has voted to release 1.0.0. This is the first release of Apache YuniKorn after graduation from the incubator.
Apache YuniKorn is a standalone resource scheduler, designed for managing, and scheduling Big Data workloads on container orchestration frameworks like Kubernetes for on-prem and on-cloud use cases.
Overview
The Apache YuniKorn community has fixed 173 JIRAs in this release.
Release manager: Wilfred Spiegelenburg
Release date: 3 May 2022
Incompatible changes
As part of this release a number of incompatible changes have been introduced.
helm upgrade
With the changes made to the helm charts running a helm upgrade from an older version to 1.0.0 might not work. The workaround is to first uninstall the old version and then perform a clean installation of the current release.
Configuration changes
Size modifiers are now supported for the queue configuration. This changes the way quota values are interpreted in the configuration. Before 1.0.0 all memory values were interpreted as megabytes and vcore (cpu) values as milli cores. From 1.0.0 onwards memory values are specified in bytes and vcores are specified in cores YUNIKORN-629.
To maintain the same interpretation of values in the configuration make the following changes to the pre-1.0.0 configuration:
| type | change | old value | new value |
|---|---|---|---|
| memory | add M | memory: 2048 | memory: 2048M |
| vcore | add m | vcore: 250 | vcore: 250m |
| other | none | license: 1 | license: 1 |
Resource Display and Communication
The conversion of memory resources of pods and other objects by the k8shim into megabytes could cause a discrepancy between the size accounted for by the Kubernetes tools and YuniKorn. This could lead to issues around node fits and autoscaling of nodes. In 1.0.0 all memory resources are converted to bytes before being used in calculation and communications YUNIKORN-1105.
Resources in all REST response objects are no longer a single string but are JSON structures using a key-value pair for each resource type. An example of the new resource JSON response object:
"allocatedResource": {
"memory": 54000000,
"vcore": 80
}
The REST representation does not include or support size modifiers. The change for using bytes in all communication for memory flows through into the REST response objects. CPU values, i.e. vcore, communication has not changed and still uses milli cores. The REST documentation has been updated to show the new resource structures YUNIKORN-165.
Timestamps in REST responses
All timestamps in the rest API are now returned as the number of nanoseconds elapsed since 1 January 1970 UTC.
The time range in a int64 that can be displayed ranges from the year 1678 or to the year 2262, see Go Time UnixNano for details.
Timestamps in earlier releases were converted to a string as part of the REST response creation with inconsistent layouts YUNIKORN-1170.
Removal of old REST API endpoints
The REST API has been updated with endpoints that support specifying partitions. The old versions which do not take a partition argument have been removed:
/ws/v1/queuesreplaced with/ws/v1/partition/{partitionName}/queues/ws/v1/appsreplaced withws/v1/partition/{partitionName}/queues/{queueName}/application/ws/v1/nodesreplaced with/ws/v1/partition/{partitionName}/nodes
Highlights
Plugin mode technical Preview
As part of this release a new deployment mode is added which is built on top of the Scheduling Framework. This is a major change in the way the scheduler k8shim works. As the implementation is not considered mature yet the deployment mode is considered a tech preview. The design document can be found on the website as the K8s Scheduler Plugin design document. Both standard and plugin deployment images are build by the default build process. The choice between deployment types is based on a helm variable.
Memory and cpu usage fixes
As part of the release the communication between the core and the k8shim has been simplified YUNIKORN-462. Using those changes the caches in the k8shim have been simplified and numerous memory leaks have been fixed. More detail in the jiras YUNIKORN-876 and YUNIKORN-1100.
An idle loop in the core has been removed which drops CPU usage of the scheduler if no scheduling action are performed YUNIKORN-1106.
Security improvements
A number of changes have been made to improve security.
The scheduler no longer uses cluster-admin privileges.
The privileges used by the scheduler are the same as the default scheduler uses.
Some additional privileges required for placeholder pods have been added YUNIKORN-997.
The processes within the docker images no longer run as the root user but use the yunikorn user YUNIKORN-1111.
Certificates used by the admission controller are automatically rotated before they expire without downtime YUNIKORN-1119.
Placeholders run a simple pause image. This pause image by default is loaded from an external image repository. YuniKorn now allows setting a custom image for placeholder. One image can be specified for the deployment. It accepts an image name, including a hostname, that can use a local repository for air-gapped deployments YUNIKORN-638.
The placeholder image should be a lightweight image, preferable a pause or sleep image.
Validation tool
As part of the release a new script is distributed that allows the creation of a simple kind based Kubernetes cluster.
Unpack the archive and follow the instructions in the README.md file to build the images.
Run the script for more instructions and to list the tools required for validating the release:
./validate_cluster.sh
The kind cluster created is a small, but fully functional Kubernetes cluster, with Apache YuniKorn deployed YUNIKORN-764
Community
Since the last release Apache YuniKorn has graduated from the incubator.