Hyperdisk ML: Integration To Speed Up Loading AI/ML Data
Hyperdisk ML can speed up the loading of AI/ML data. This tutorial explains how to use it to streamline and speed up the loading of AI/ML model weights on GKE
Using an existing Google Cloud Hyperdisk, create a Hyperdisk ML volume: Make a Kubernetes volume that points to the data-loaded Hyperdisk ML volume
GKE does not verify that the disk content is consistent across zones when utilizing pre-existing disks with a multi-zone Hyperdisk ML volume
Make sure your program considers the possibility of inconsistencies between zones if any of the disks have divergent material
On new Autopilot and Standard clusters, the Compute Engine Persistent Disc driver is on by default and cannot be turned off or changed while Autopilot is in use
See Enabling the Compute Engine Persistent Disk CSI Driver on an Existing Cluster if you need to enable the Cluster’s Compute Engine Persistent Disk CSI driver
This source offers a thorough tutorial on how to use Hyperdisk ML to speed up AI/ML data loading on Google Kubernetes Engine (GKE)