Send a prompt to a LLM provider

This function is responsible for sending prompts to a LLM provider for evaluation. The function will interact with the LLM provider until a successful response is received or the maximum number of interactions is reached. The function will apply extraction and validation functions to the LLM response, as specified in the prompt wraps (see prompt_wrap()). If the maximum number of interactions

Usage

send_prompt(
  prompt,
  llm_provider = llm_provider_ollama(),
  max_interactions = 10,
  clean_chat_history = FALSE,
  verbose = NULL,
  stream = NULL,
  return_mode = c("only_response", "full"),
  max_requests = NULL
)

Arguments

prompt: A string or a tidyprompt object
llm_provider: llm_provider object (default is llm_provider_ollama()). This object and its settings will be used to evaluate the prompt. You may also pass an ellmer::chat() object directly; in that case, send_prompt() will clone it, clear any existing turns, and wrap the clean clone with llm_provider_ellmer() before evaluation. When using llm_provider_ellmer() or a raw ellmer::chat(), the working ellmer turns are rebuilt from tidyprompt's own chat history for each call. Note that the 'verbose' and 'stream' settings in the LLM provider will be overruled by the 'verbose' and 'stream' arguments in this function when those are not NULL. Furthermore, advanced tidyprompt objects may carry '$parameter_fn' functions which can set parameters in the llm_provider object (see prompt_wrap() and llm_provider for more ).
max_interactions: Maximum number of interactions allowed with the LLM provider. Default is 10. If the maximum number of interactions is reached without a successful response, 'NULL' is returned as the response (see return value). The first interaction is the initial chat completion. Every response is extracted and validated, including the last allowed response. This controls the outer extraction, validation and feedback loop; it does not count individual model requests within provider tool loops. Use max_requests to also bound those requests.
clean_chat_history: If the chat history should be cleaned after each interaction. Cleaning the chat history means that only the first and last message from the user, the last message from the assistant, all messages from the system, and all tool results are kept in a 'clean' chat history. This clean chat history is used when requesting a new chat completion. With 'ellmer', native tool requests, results and associated turn content are preserved together, including in the returned chat_history_clean. Rows marked as non-replayable are excluded from new requests regardless of this setting, so the returned transcript may contain more rows than the model actually sees on a retry or follow-up call. (i.e., if a LLM repeatedly fails to provide a correct response, only its last failed response will included in the context window). This may increase the LLM performance on the next interaction
verbose: If the interaction with the LLM provider should be printed to the console. This will overrule the 'verbose' setting in the LLM provider
stream: If the interaction with the LLM provider should be streamed. This setting will only be used if the LLM provider already has a 'stream' parameter (which indicates there is support for streaming). Note that when 'verbose' is set to FALSE, the 'stream' setting will be ignored
return_mode: One of 'full' or 'only_response'. See return value
max_requests: Optional positive whole number of model requests allowed during this evaluation, equivalent to applying limit_requests() to prompt. The default, NULL, adds no request limit and preserves any limit already attached to the prompt. If both are supplied, the smaller limit applies. Counts the initial request, tool follow-ups, feedback requests and nested llm_verify() requests, excluding streaming chunks and transport-level retries. Attempting another request after the limit raises a tidyprompt_request_limit error. See limit_requests() for provider requirements and counting details.

Value

If return mode 'only_response', the function will return only the LLM response after extraction and validation functions have been applied (NULL is returned when unsuccessful after the maximum number of interactions).
If return mode 'full', the function will return a list with the following elements:
- 'response' (the LLM response after extraction and validation functions have been applied; NULL is returned when unsuccessful after the maximum number of interactions),
- 'interactions' (the number of interactions with the LLM provider),
- 'chat_history' (a dataframe with the full chat history which led to the final response. This may include rows that are retained for inspection but not re-sent to the model; such rows are marked with column hidden_from_llm = TRUE),
- 'chat_history_clean' (a dataframe with the cleaned chat history which led to the final response; here, only the first and last message from the user, the last message from the assistant, and all messages from the system are kept, after excluding any non-replayable rows),
- 'start_time' (the time when the function was called),
- 'end_time' (the time when the function ended),
- 'duration_seconds' (the duration of the function in seconds),
- 'http' (a list with all HTTP requests and responses made during the interactions; as returned by llm_provider$complete_chat()),
- 'ellmer_chat' (if llm_provider_ellmer() or a raw ellmer::chat() object was used, this will be the updated 'ellmer' chat object, containing for instance the turns and possible tool calls. (As this function uses a clone of the provided LLM provider, the 'ellmer' chat object in the LLM provider will not be updated; use this return value when you need the updated native ellmer turns from the current evaluation. Note that turns in the 'ellmer' chat object may not contain the full chat history when clean_chat_history = TRUE was used.)

Request errors

HTTP request failures from the built-in OpenAI-compatible and Ollama providers signal a condition of class tidyprompt_request_error, for both streaming connection setup and non-streaming requests. Its status_code and request_id fields contain HTTP metadata when available, and are NULL otherwise (for example, a connection failure may have no HTTP response). The parent field preserves the original condition, including its class, call and any attached 'httr2' response. A provider's explicit JSON error message is included when available; the full response body is not appended. Applications can use these fields in their tryCatch(error = ...) handlers. The parent may contain request data and credentials, so applications should select diagnostic fields instead of logging the entire condition object. Errors from custom or 'ellmer' providers retain those providers' behavior.

Examples

if (FALSE) { # \dontrun{
  "Hi!" |>
    send_prompt(llm_provider_ollama())
  # --- Sending request to LLM provider (llama3.1:8b): ---
  #   Hi!
  # --- Receiving response from LLM provider: ---
  #   It's nice to meet you. Is there something I can help you with, or would you like to chat?
  # [1] "It's nice to meet you. Is there something I can help you with, or would you like to chat?"

  "Hi!" |>
    send_prompt(llm_provider_ollama(), return_mode = "full")
  # --- Sending request to LLM provider (llama3.1:8b): ---
  #   Hi!
  # --- Receiving response from LLM provider: ---
  #   It's nice to meet you. Is there something I can help you with, or would you like to chat?
  # $response
  # [1] "It's nice to meet you. Is there something I can help you with, or would you like to chat?"
  #
  # $chat_history
  # ...
  #
  # $chat_history_clean
  # ...
  #
  # $start_time
  # [1] "2024-11-18 15:43:12 CET"
  #
  # $end_time
  # [1] "2024-11-18 15:43:13 CET"
  #
  # $duration_seconds
  # [1] 1.13276
  #
  # $http
  # $http$responses[[1]]
  # Response [http://localhost:11434/api/chat]
  #   Date: 2024-11-18 14:43
  #   Status: 200
  #   Content-Type: application/x-ndjson
  # <EMPTY BODY>

  "Hi!" |>
    add_text("What is 5 + 5?") |>
    answer_as_integer() |>
    send_prompt(llm_provider_ollama(), verbose = FALSE)
  # [1] 10
} # }

Usage

Arguments

Value

Request errors

See also

Examples