Model
A model represents a generative AI model configuration that can be used for inference tasks such as text generation, chat, or embeddings. Models define how to connect to and configure specific AI providers like OpenAI, Anthropic, AWS Bedrock, or others.
Each model must have a unique id
and specify a provider
. The model acts as a reusable configuration that can be referenced by steps like LLMInference
, Agent
, or vector search operations.
Rules and Behaviors
- Unique IDs: Each model must have a unique
id
within the application. Duplicate model IDs will result in a validation error. - Model ID Resolution: If
model_id
is not specified, the model'sid
field is used as the model identifier for the provider. - Provider Requirement: The
provider
field is required and specifies which AI service to use (e.g., "openai", "anthropic", "aws-bedrock"). - Authentication: Models can reference an
AuthorizationProvider
for API authentication, either by ID reference or as an embedded object. - Inference Parameters: Optional
inference_params
allow customization of model behavior (temperature, max_tokens, etc.) that are passed to the provider's API.
Model Types
Model
Describes a generative model configuration, including provider and model ID.
- id (
str
): Unique ID for the model. - auth (
AuthorizationProvider | str | None
): AuthorizationProvider used for model access. - inference_params (
dict[str, Any] | None
): Optional inference parameters like temperature or max_tokens. - model_id (
str | None
): The specific model name or ID for the provider. If None, id is used - provider (
str
): Name of the provider, e.g., openai or anthropic.
EmbeddingModel
Describes an embedding model configuration, extending the base Model class.
- dimensions (
int
): Dimensionality of the embedding vectors produced by this model.
Related Concepts
Models can reference AuthorizationProvider for secure API access.