Skip to content

Model

A model represents a generative AI model configuration that can be used for inference tasks such as text generation, chat, or embeddings. Models define how to connect to and configure specific AI providers like OpenAI, Anthropic, AWS Bedrock, or others.

Each model must have a unique id and specify a provider. The model acts as a reusable configuration that can be referenced by steps like LLMInference, Agent, or vector search operations.

Rules and Behaviors

  • Unique IDs: Each model must have a unique id within the application. Duplicate model IDs will result in a validation error.
  • Model ID Resolution: If model_id is not specified, the model's id field is used as the model identifier for the provider.
  • Provider Requirement: The provider field is required and specifies which AI service to use (e.g., "openai", "anthropic", "aws-bedrock").
  • Authentication: Models can reference an AuthorizationProvider for API authentication, either by ID reference or as an embedded object.
  • Inference Parameters: Optional inference_params allow customization of model behavior (temperature, max_tokens, etc.) that are passed to the provider's API.

Model Types

Model

Describes a generative model configuration, including provider and model ID.

  • id (str): Unique ID for the model.
  • auth (AuthorizationProvider | str | None): AuthorizationProvider used for model access.
  • inference_params (dict[str, Any] | None): Optional inference parameters like temperature or max_tokens.
  • model_id (str | None): The specific model name or ID for the provider. If None, id is used
  • provider (str): Name of the provider, e.g., openai or anthropic.

EmbeddingModel

Describes an embedding model configuration, extending the base Model class.

  • dimensions (int): Dimensionality of the embedding vectors produced by this model.

Models can reference AuthorizationProvider for secure API access.

Example Usage