Skip to content

Additional Documentation on Which Endpoint use Vertex vs GKE #43

Description

@igooch

We got feedback from user that it is not clear which endpoints call Vertex vs. which endpoints call a LLM on the GKE cluster, and how to switch between the two.

We should:

  1. Update documentation to make clear which endpoints call Vertex (/genai, /genai/chat, /genai/code, /genai/image, /genai/text, etc.)
  2. Update documentation for instructions on how to switch between running a LLM on GKE (current default) and running on Vertex in the NPC chat I
    # GenAI provider - GKEGenAI or VertexAI. Note that switching GenAI implementations switches the
    # embedding model requiring a data regeneration using the /reset_world_data endpoint.
    genai = "GKEGenAI"
    # genai = "VertexAI"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions