#Install Alauda AI

#Procedure

In Administrator view:

1. Click Marketplace / OperatorHub.

2. At the top of the console, from the Cluster dropdown list, select the destination cluster where you want to install Alauda AI.

3. Select Alauda AI, then click Install.

   Install Alauda AI window will pop up.

4. Then in the Install Alauda AI window.

5. Leave Channel unchanged.

6. Check whether the Version matches the Alauda AI version you want to install.

7. Leave Installation Location unchanged, it should be aml-operator by default.

8. Select Manual for Upgrade Strategy.

9. Click Install.

#Verification

Confirm that the Alauda AI tile shows one of the following states:

• Installing: installation is in progress; wait for this to change to Installed.
• Installed: installation is complete.

#Deploy Alauda AI

In Administrator view:

1. Click Marketplace / OperatorHub.

2. At the top of the console, from the Cluster dropdown list, select the destination cluster where you want to install the Alauda AI Operator.

3. Select Alauda AI, then click it.

4. In the Alauda AI page, click All Instances from the tab.

5. Wait for the default AmlCluster instance to appear, then edit it.

6. Select Deploy Flavor from dropdown:

   1. single-node for non HA deployments.
   2. ha-cluster for HA cluster deployments (Recommended for production).

7. Input a valid domain for Domain field.

   INFO

   This domain is used by ingress gateway for exposing model serving services. Most likely, you will want to use a wildcard name, like *.example.com.

   You can specify the following certificate types by updating the Domain Certificate Type field:

   • Provided
   • SelfSigned
   • ACPDefaultIngress

   By default, the configuration uses SelfSigned certificate type for securing ingress traffic to your cluster, the certificate is stored in the knative-serving-cert secret that is specified in the Domain Certificate Secret field.

#Component management states

Alauda AI 2.8 automatically installs and manages the components selected by the AmlCluster configuration. The following values show the default component settings. Merge the entries you need into the existing default AmlCluster; do not create a second instance.

spec:
  components:
    authorino:
      managementState: Managed
    envoyAIGateway:
      managementState: Unmanaged
    envoyGateway:
      managementState: SharedManaged
    feast:
      managementState: Unmanaged
    knativeServing:
      managementState: Unmanaged
      providerType: Operator
    llamaStack:
      managementState: Unmanaged
    lws:
      managementState: Managed
    mlflow:
      managementState: Unmanaged
    npuOperator:
      managementState: Unmanaged
      values:
        clusterd:
          enabled: false
        devicePlugin:
          enabled: true
        driver:
          enabled: true
          recoveryPolicy:
            autoRecover: false
          upgradePolicy:
            autoUpgrade: false
          version: 25.5.0
        exporter:
          enabled: true
        mindioacp:
          enabled: false
        mindiotft:
          enabled: false
        nodeD:
          enabled: false
        ociRuntime:
          enabled: true
        rscontroller:
          enabled: false
        trainer:
          enabled: false
    postgres:
      managementState: SharedManaged
    redis:
      managementState: SharedManaged
    servingRuntimeOperator:
      managementState: Unmanaged
    trustyAI:
      managementState: Unmanaged
    workbench:
      managementState: Unmanaged
      values:
        global:
          istio:
            enabled: false

The spec.components field uses configuration keys that correspond to the following products and Operators:

The management states have the following meanings:

For common 2.8 deployments, set the following component switches to Managed when the corresponding capabilities are required. The servingRuntimeOperator setting depends on the accelerator and serving-runtime requirements of your environment.

spec:
  components:
    envoyAIGateway:
      managementState: Managed
    servingRuntimeOperator:
      managementState: Managed
    workbench:
      managementState: Managed

spec:
  components:
    knativeServing:
      managementState: SharedManaged

#Configure KServe

Alauda Build of KServe is installed and managed by Alauda AI. The kserve component in the default AmlCluster instance defaults to Managed, so Alauda AI installs the KServe Operator and automatically creates the KServe instance. You do not need to upload the KServe Operator package, install the Operator from OperatorHub, or create a KServe custom resource manually.

The KServe parameters are exposed through spec.components.kserve.values. Merge the settings you need into the existing default AmlCluster:

spec:
  components:
    kserve:
      managementState: Managed
      values:
        defaultDeploymentMode: RawDeployment
        domain: example.com
        ingressGateway: knative-serving/knative-ingress-gateway
        kserve:
          storage:
            uidModelcar: 1000
        preset:
          envoyAIGateway:
            port: 1063
            service: ai-gateway-controller
          envoyGateway:
            createInstance: true
            deployType: ControllerNamespace
            instanceName: aieg
            saNamespace: envoy-gateway-system
            serviceAccount: envoy-gateway
          gie:
            enabled: true
          kserveGateway:
            enabled: true
            name: kserve-ingress-gateway
            namespace: kserve
            port: 80
            service_type: NodePort

The main kserve component settings:

The gateway presets configure the AI gateway stack that fronts inference traffic:

Verify that the AmlCluster reconciles and the KServe instance is created by Alauda AI:

kubectl get amlcluster default

kubectl get kserve default-kserve -n kserve-operator

The AmlCluster should report Phase=Ready and Reason=Reconciled, and the KServe instance is ready when the status shows DEPLOYED: True.

#Configure a custom OIDC provider (Optional)

By default, Alauda AI uses ACP Dex as the OIDC provider. In this default setup, no additional spec.oidc configuration is required in the AmlCluster instance.

If you want Alauda AI to use another OIDC provider, register an OAuth2/OIDC client in that provider, allow the Alauda AI callback URL, and then update spec.oidc in the AmlCluster YAML. The callback URL is:

https://<platform-address>/clusters/<cluster-name>/aml/oauth2/callback

Alauda AI reads the OIDC client secret from a Kubernetes Secret in the kubeflow namespace of the Alauda AI installation cluster. The default Secret name is aml-oidc-secret, and the Secret key must be client-secret. Update this Secret with the client secret from your OIDC provider:

kubectl create secret generic aml-oidc-secret \
  -n kubeflow \
  --from-literal=client-secret='<oidc-client-secret>' \
  --dry-run=client -o yaml | kubectl apply -f -

Then configure spec.oidc:

spec:
  oidc:
    # OIDC issuer URL. This must match the issuer value advertised by the
    # provider, for example a Keycloak realm URL.
    issuerURL: https://<oidc-provider-issuer>
    # OAuth2/OIDC client ID registered in the external provider.
    clientID: <oidc-client-id>
    # Kubernetes Secret name in the kubeflow namespace. The Secret must
    # contain a client-secret key. Default: aml-oidc-secret.
    clientSecretName: aml-oidc-secret
    # OAuth2 scopes requested during login. Keep this minimal to avoid
    # oversized ID/access tokens and oversized login cookies.
    # Default: openid profile email.
    scope: openid profile email
    # Whether to use the preferred_username claim as the email value.
    # Default: true.
    preferredUsernameAsEmail: true
    main:
      # OIDC authorization endpoint. If you use discovery, this maps from
      # authorization_endpoint.
      loginURL: https://<oidc-provider-authorization-endpoint>
      # OAuth2 callback URL registered in the external provider.
      redirectURL: https://<platform-address>/clusters/<cluster-name>/aml/oauth2/callback

If the provider exposes a standard OIDC discovery document at <issuerURL>/.well-known/openid-configuration, Alauda AI automatically fills redeemURL, jwksURL, and profileURL from discovery when these fields are not set. If discovery is unavailable, configure these fields explicitly:

Use the mapped loginURL value for main.loginURL, or for secondary.loginURL if you configure a secondary endpoint.

spec:
  oidc:
    redeemURL: https://<oidc-provider-token-endpoint>
    jwksURL: https://<oidc-provider-jwks-endpoint>
    profileURL: https://<oidc-provider-userinfo-endpoint>

Example Keycloak configuration:

• In the target realm, create an OpenID Connect client.
• Set Client ID to the value used in spec.oidc.clientID, for example aml.
• Turn on Client authentication.
• Under Authentication flow, select Standard flow.
• Turn on Require PKCE and set PKCE Method to S256.
• Set Valid redirect URIs to https://<platform-address>/clusters/<cluster-name>/aml/*.
• Copy the client secret from the Keycloak client Credentials tab and update the aml-oidc-secret Secret shown above.
• In the client Client scopes settings, set basic, email, and profile to Default, and set other scopes to Optional unless your environment explicitly needs them. Avoid adding large claim mappers such as groups, realm roles, client roles, address, phone, offline access, and other application-specific claims unless required. Large tokens can make the oauth2-proxy cookie exceed browser or ingress header size limits and cause login loops or HTTP 431/400 errors.

#Configure the console languages (Optional)

Alauda AI reads the available console languages from spec.i18n.languages in the AmlCluster instance. The default value is en.

Supported language codes are:

en and zh are built-in languages. Other languages only add the language as a selectable entry. To switch to those languages successfully, you must also provide the corresponding translation file.

You can use the built-in English and Chinese translation files as references when preparing translation files for other languages:

/clusters/<cluster-name>/aml/console-aml/assets/i18n/en/lich-single.json
/clusters/<cluster-name>/aml/console-aml/assets/i18n/zh/lich-single.json

Generate the target-language translation from these files, or contact Alauda to obtain the latest translation content for the required language.

For example, to make English, Chinese, and French selectable in the console, update the AmlCluster YAML:

spec:
  i18n:
    languages:
      - en
      - zh
      - fr

Then create a translation file ConfigMap for French in the cpaas-system namespace:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    image-load-config: "true"
  name: aml-i18n-fr
  namespace: cpaas-system
data:
  overrides: |
    [{
      "dest": "/aml-i18n/fr/lich-single.json",
      "configMap": {
         "name": "aml-i18n-fr",
         "key": "lich-single.json"
      }
    }]
  lich-single.json: |
    {
      "nav_model_repo": "Referentiel de modeles",
      "nav_infer_svc": "Service d'inference",
      "workbench": "Workbench",
      "tool": "Outils"
    }

Use the same pattern for other non-built-in languages by replacing fr with the target language code and providing the translated lich-single.json content.

#Configure Model Catalog

• Model OCI Registry Address: Registry address hosting model OCI artifacts for Model Catalog. This field has no default value and must be configured for your environment.

  This registry stores the model OCI images used by Model Catalog. Use Harbor or another production-mode OCI registry with HTTPS access enabled. Model Catalog does not support configuring imagePullSecret for pulling model OCI images, so the Harbor project or repository used for Model Catalog must allow anonymous pull access from inference cluster nodes. In Harbor, set the project that stores Model Catalog images to Public.

  If you cannot deploy a registry with HTTPS in the target environment, you can use an HTTP registry as a fallback. Configure the container runtime on every node in the inference cluster before deploying models. For containerd, add an insecure registry mirror for the registry address, for example by creating /etc/containerd/certs.d/<registry-host:port>/hosts.toml:

  server = "http://<registry-host:port>"
  
  [host."http://<registry-host:port>"]
    capabilities = ["pull", "resolve"]

  Then restart containerd or apply the equivalent node-runtime configuration through your cluster management system. This configuration must exist on the nodes where inference service pods are scheduled; otherwise the pod image pull will fail even if Model Catalog can list the model. The exact containerd configuration path can vary by Kubernetes distribution; after applying the configuration, verify that the node can pull a Model Catalog image, for example with crictl pull <registry-host:port>/<repository>:<tag>.

• Source of PVC: Choose whether to reuse an existing PVC or create a new one. Use CreateNew to let the installation create the PVC.

• StorageClass Name: StorageClass used when creating a new PVC.

#Verification

Review the configuration and save the default AmlCluster instance, then check its status:

kubectl get amlcluster default

The resource should be Ready:

NAME      PHASE   READY   REASON
default   Ready   True    Reconciled

#Obtaining the OCI image tarballs

Built-in model images are delivered as OCI archive tarballs (.tar files compliant with the OCI Image Layout Specification). Each tarball contains a multi-architecture image (linux/amd64 + linux/arm64) for one model.

Download the tarballs from the Customer Portal Marketplace, or contact your Alauda support representative to obtain the package matching your Alauda AI version.

#Pushing to Harbor

The recommended target is Harbor. The example below uses an HTTP Harbor registry. If your Harbor registry uses HTTPS, omit --plain-http and change the API URLs from http:// to https://.

Run the commands on a node that has ctr, curl, and jq installed and can reach Harbor.

First, set the environment variables:

export REG=<harbor-address>:<port>
export REPO=mlops/modelcar-qwen3.5-0.8b       
export TAG=v0.1.0                             
export TAR=./Qwen3.5-0.8B.oci.tar             
export AUTH='user:password'

The tarball usually carries its own tag (e.g. v0.1.0) inside the OCI image layout. If needed, extract it from the tarball:

export TAG=$(tar -xOf "$TAR" index.json \
  | jq -r '.manifests[0].annotations["org.opencontainers.image.ref.name"]')
echo "$TAG"   # should print something like v0.1.0

Check whether the image tag already exists in Harbor:

URL="http://$REG/api/v2.0/projects/${REPO%%/*}/repositories/$(printf '%s' "${REPO#*/}" | sed 's|/|%2F|g')/artifacts/$TAG"

HTTP=$(curl -s -u "$AUTH" -o /tmp/harbor-artifact.json -w '%{http_code}' "$URL")

echo "HTTP=$HTTP  URL=$URL"

[ "$HTTP" = 200 ] && jq '{digest, size, push_time, arch: .extra_attrs.architecture, tags: [.tags[].name], platforms: [.references[]?.platform]}' /tmp/harbor-artifact.json \
  || jq -r '.errors[]?.message' /tmp/harbor-artifact.json

If the Harbor project does not exist yet, create it before pushing:

PROJECT="${REPO%%/*}"

curl -s -u "$AUTH" -X POST "http://$REG/api/v2.0/projects" \
  -H 'Content-Type: application/json' \
  -d "{\"project_name\":\"$PROJECT\",\"public\":true}" \
  -w '\nHTTP %{http_code}\n'

If the project already exists, Harbor returns a non-2xx status code. After confirming the project exists, make sure it is configured as a public project, then continue with the import and push. Model Catalog does not support configuring imagePullSecret when deploying model OCI images, so inference cluster nodes must be able to pull these images anonymously.

Then run the import and push procedure:

# 1. Import into the node's containerd content store.
#    --base-name prepends $REG/$REPO to the tag carried inside the tarball,
#    producing a fully-qualified reference $REG/$REPO:$TAG.
ctr -n k8s.io images import \
  --all-platforms \
  --base-name "$REG/$REPO" \
  "$TAR"

# 2. Verify the import. You should see "$REG/$REPO:$TAG".
ctr -n k8s.io images ls -q | grep "$REPO"

# 3. Push to Harbor. Use --plain-http only for HTTP Harbor.
ctr -n k8s.io images push \
  --plain-http \
  --user "$AUTH" \
  "$REG/$REPO:$TAG"

# 4. Clean up the local reference on the node. Blob data is reclaimed by
#    containerd's garbage collector, leaving no persistent state on the node.
ctr -n k8s.io images rm "$REG/$REPO:$TAG"

Repeat this procedure for each built-in model tarball, varying $REPO, $TAG, and $TAR per model.

#Verifying the Harbor import

Confirm that Harbor now serves the image:

URL="http://$REG/api/v2.0/projects/${REPO%%/*}/repositories/$(printf '%s' "${REPO#*/}" | sed 's|/|%2F|g')/artifacts/$TAG"

HTTP=$(curl -s -u "$AUTH" -o /tmp/harbor-artifact.json -w '%{http_code}' "$URL")

echo "HTTP=$HTTP  URL=$URL"

[ "$HTTP" = 200 ] && jq '{digest, size, push_time, arch: .extra_attrs.architecture, tags: [.tags[].name], platforms: [.references[]?.platform]}' /tmp/harbor-artifact.json \
  || jq -r '.errors[]?.message' /tmp/harbor-artifact.json

HTTP=200 means the image was successfully imported into Harbor. Expected output includes the digest, size, push time, tag, and platform information:

{
  "digest": "sha256:...",
  "size": 123456789,
  "push_time": "2026-05-06T00:00:00.000Z",
  "arch": "amd64",
  "tags": ["v0.1.0"],
  "platforms": [
    {"architecture": "amd64", "os": "linux"},
    {"architecture": "arm64", "os": "linux"}
  ]
}