AI Systems

Cloud Remote AI System Creation

Note: For remote cloud providers, first create auth-data entries (API keys) using the User Auth methods. Then provide auth_data_ids and a primary_auth_data_id to AI system creation. Do not pass plaintext API keys directly to model creation; the server resolves keys from your auth-data.

Create OpenAI AI System

`create_openai_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)`

Creates a new remote openai model object. The model object can then be used for running evaluations and tests.

Method Parameters

name | required string

Model name.

api_instance | required string

Model identifier name from OpenAI.
Example: "gpt3.5-turbo-0125"

auth_data_ids | required List[int]

List of auth-data ids to attach to this model.

primary_auth_data_id | required int

Auth-data id to use as primary for this model.

key | optional string

Unique model identifier key. Will be autogenerated if not provided.

Returns

RemoteModel object.

Example

model = dfl.create_openai_model(
    name="GPT 3.5 Model",
    api_instance="gpt-3.5-turbo-0125",
    auth_data_ids=[101, 102],
    primary_auth_data_id=101,
)

Create Azure AI System

`create_azure_model(name, api_instance, model_endpoint, auth_data_ids, primary_auth_data_id, api_version?, key?)`

Creates a new remote Azure OpenAI model object. The model object can then be used for running evaluations and tests.

Method Parameters

name | required string

Model name.

api_instance | required string

Model identifier name from Azure OpenAI. Example: "gpt35-turbo"

model_endpoint | required string

URL string representing model endpoint. Example: "https://abc-azure-openai.openai.azure.com/"

auth_data_ids | required List[int]

List of auth-data ids to attach to this model.

primary_auth_data_id | required int

Auth-data id to use as primary for this model.

api_version | optional string

Azure OpenAI version to use. Example: "2023-07-01-preview"

key | optional string

Unique model identifier key. Will be autogenerated if not provided.

Returns

RemoteModel object.

Example

model = dfl.create_azure_model(
    name="GPT 3.5 Model",
    api_instance="gpt35-turbo",
    model_endpoint="https://abc-azure-openai.openai.azure.com/",
    auth_data_ids=[201, 202],
    primary_auth_data_id=201,
    api_version="2023-07-01-preview",
)

Create Databricks AI System

`create_databricks_model(name, model_endpoint, auth_data_ids, primary_auth_data_id, key?)`

Creates a new remote Azure OpenAI model object. The model object can then be used for running evaluations and tests.

Method Parameters

name | required string

Model name.

model_endpoint | required string

Databricks model endpoint. Example: "<host>/serving-endpoints/<some-model-name>/invocations"

auth_data_ids | required List[int]

List of auth-data ids to attach to this model.

primary_auth_data_id | required int

Auth-data id to use as primary for this model.

key | optional string

Unique model identifier key. Will be autogenerated if not provided.

Returns

RemoteModel object.

Example

model = dfl.create_databricks_model(
    name="Databricks Mixtral",
    model_endpoint="<host>/serving-endpoints/<mixtral-model-name>/invocations",
    auth_data_ids=[301],
    primary_auth_data_id=301,
)

Create Anthropic AI System

`create_anthropic_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)`

Creates a new remote Anthropic model object. The model object can then be used for running evaluations and tests.

Method Parameters

name | required string

Model name.

api_instance | required string

Model identifier name from Anthropic. Example: "claude-3-sonnet-20240229"

auth_data_ids | required List[int]

List of auth-data ids to attach to this model.

primary_auth_data_id | required int

Auth-data id to use as primary for this model.

key | optional string

Unique model identifier key. Will be autogenerated if not provided.

Returns

RemoteModel object.

Example

model = dfl.create_anthropic_model(
    name="Claude 3 Sonnet",
    api_instance="claude-3-sonnet-20240229",
    auth_data_ids=[401, 402],
    primary_auth_data_id=401,
)

Create Mistral AI System

`create_mistral_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)`

Creates a new remote Mistral model object. The model object can then be used for running evaluations and tests.

Method Parameters

name | required string

Model name.

api_instance | required string

Model identifier name from Mistral. Example: "mistral-large-latest"

auth_data_ids | required List[int]

List of auth-data ids to attach to this model.

primary_auth_data_id | required int

Auth-data id to use as primary for this model.

key | optional string

Unique model identifier key. Will be autogenerated if not provided.

Returns

RemoteModel object.

Example

model = dfl.create_mistral_model(
    name="Mistral Large",
    api_instance="mistral-large-latest",
    auth_data_ids=[501],
    primary_auth_data_id=501,
)

Custom Remote AI System Creation

Create Custom AI System

`create_custom_model(name, remote_model_endpoint, remote_api_auth_config, request_transformation_expression?, response_transformation_expression?, response_type?, batch_size?, multi_turn_support?, enable_retry?, key?)`

Creates a new custom remote model object. The model object can then be used for running evaluations and tests.

Method Parameters

name | required string

Model name.

remote_model_endpoint | required string

Endpoint URL for the remote model.

remote_api_auth_config | required dict

Authentication configuration for the remote API.

request_transformation_expression | optional string

Expression to transform the outbound request payload to the remote model.

response_transformation_expression | optional string

Expression to transform the inbound response payload from the remote model.

response_type | optional string

Response type returned after transformation. Defaults to "string".

batch_size | optional int

Batch size for requests. Defaults to 1.

multi_turn_support | optional boolean

Indicates if multi‑turn interactions are supported. Defaults to True.

enable_retry | optional boolean

Enable client‑side retry for failed requests. Defaults to False.

key | optional string

Unique model identifier key. Will be autogenerated if not provided.

Returns

RemoteModel object.

Example

model = dfl.create_custom_model(
    name="My Custom Model",
    remote_model_endpoint="https://my-remote-llm/api/infer",
    remote_api_auth_config={"type": "bearer", "token": "***"},
    request_transformation_expression="payload = {'inputs': input_text}",
    response_transformation_expression="return response_json['output']",
    response_type="string",
    batch_size=1,
    multi_turn_support=True,
    enable_retry=False,
)

AI Systems - Helpers

Get Model

`get_model(key)`

Returns model object based on identifier key.

Method Parameters

key | required string

Unique model identifier key.

Returns

LocalModel or RemoteModel object.

Example

model = dfl.get_model('unique_identifier_key')

Cloud Remote AI System Creation​

Create OpenAI AI System​

create_openai_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)​

Method Parameters​

Returns​

Example​

Create Azure AI System​

create_azure_model(name, api_instance, model_endpoint, auth_data_ids, primary_auth_data_id, api_version?, key?)​

Method Parameters​

Returns​

Example​

Create Databricks AI System​

create_databricks_model(name, model_endpoint, auth_data_ids, primary_auth_data_id, key?)​

Method Parameters​

Returns​

Example​

Create Anthropic AI System​

create_anthropic_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)​

Method Parameters​

Returns​

Example​

Create Mistral AI System​

create_mistral_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)​

Method Parameters​

Returns​

Example​

Custom Remote AI System Creation​

Create Custom AI System​

create_custom_model(name, remote_model_endpoint, remote_api_auth_config, request_transformation_expression?, response_transformation_expression?, response_type?, batch_size?, multi_turn_support?, enable_retry?, key?)​

Method Parameters​

Returns​

Example​

AI Systems - Helpers​

Get Model​

get_model(key)​

Method Parameters​

Returns​

Example​

Cloud Remote AI System Creation

Create OpenAI AI System

`create_openai_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)`

Method Parameters

Returns

Example

Create Azure AI System

`create_azure_model(name, api_instance, model_endpoint, auth_data_ids, primary_auth_data_id, api_version?, key?)`

Method Parameters

Returns

Example

Create Databricks AI System

`create_databricks_model(name, model_endpoint, auth_data_ids, primary_auth_data_id, key?)`

Method Parameters

Returns

Example

Create Anthropic AI System

`create_anthropic_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)`

Method Parameters

Returns

Example

Create Mistral AI System

`create_mistral_model(name, api_instance, auth_data_ids, primary_auth_data_id, key?)`

Method Parameters

Returns

Example

Custom Remote AI System Creation

Create Custom AI System

`create_custom_model(name, remote_model_endpoint, remote_api_auth_config, request_transformation_expression?, response_transformation_expression?, response_type?, batch_size?, multi_turn_support?, enable_retry?, key?)`

Method Parameters

Returns

Example

AI Systems - Helpers

Get Model

`get_model(key)`

Method Parameters

Returns

Example