Kubernetes release 1.28.0

Breaking change in EnqueueExtension for custom scheduler plugin developers. EventsToRegister now returns ClusterEventWithHint.

• CephFS volume plugin deprecated; use CephFS CSI driver.
• Deprecated support for CSI migration of Ceph RBD volumes.
• RBD volume plugin deprecated; use RBD CSI driver.

• Changed kubectl version default output; removed --short flag.
• Deprecated flags in kube-controller-manager and kubelet.
• Removed tracking annotation from validation and defaulting.
• Removed withdrawn feature NetworkPolicyStatus.
• KMSv1 deprecated; use KMSv2.

• Added ServedVersions field to StorageVersion API.
• Indexed Job pods now have pod completion index set as a pod label.
• Added IP mode field to loadbalancer status ingress.
• Added new fields podReplacementPolicy and terminating to job API.
• New feature gate SidecarContainers introduced.

• Added '--concurrent-cron-job-syncs' flag for kube-controller-manager.
• New feature gate SchedulerQueueingHints (enabled by default).
• Full cgroup v2 swap support for both Limited and Unlimited swap.
• Enhanced dynamic resource allocation handling.
• Enabled pods with volumes and user namespaces.

• Added new event FailedToRetrieveImagePullSecret for missing ImagePullSecret.
• Fixed data race in TopologyCache.
• Fixed kubelet startup with NewVolumeManagerReconstruction feature enabled.
• Resolved issue with static pods taking extra time to restart after update.
• Fixed performance regression in scheduler.