A Livy session keeps Spark running while you submit several pieces of code
Create one with fabric_livy_session(), call $wait() once it starts, use
$run() to execute code, and call $close() when finished. Most users do
not need to call this 'R6' class directly. These lifecycle methods do not
have separate free-function wrappers
Format
An R6::R6Class generator
Public fields
idFabric session or high-concurrency acquisition ID
urlSession lifecycle URL
stateLatest service state
responseLatest raw service response
closedWhether
$close()completedhigh_concurrencyWhether this is a high-concurrency session
session_idUnderlying Livy session ID for HC sessions
repl_idIsolated REPL ID for HC sessions
verboseWhether lifecycle messages are enabled
Methods
FabricLivySession$new()
Internal constructor used by fabric_livy_session()
Usage
FabricLivySession$new(
livy_url,
credential,
payload,
response = NULL,
high_concurrency = FALSE,
verbose = TRUE
)FabricLivySession$status()
Return the latest session response
FabricLivySession$submit()
Submit code without waiting for completion
Usage
FabricLivySession$submit(
code,
kind = c("spark", "pyspark", "sparkr", "sql"),
source_id = NULL
)Arguments
codeOne string of Spark code
kindStatement language
source_idOptional caller-defined source identifier
FabricLivySession$run()
Submit code, wait, and return its parsed result
Usage
FabricLivySession$run(
code,
kind = c("spark", "pyspark", "sparkr", "sql"),
source_id = NULL,
timeout = 600,
poll_interval = 2
)Examples
if (FALSE) { # \dontrun{
# fabric_livy_session() creates this class for a discovered Lakehouse
workspace <- fabric_workspaces()[[1L]]
lakehouse <- fabric_lakehouses(workspace)[[1L]]
session <- fabric_livy_session(lakehouse)
inherits(session, "FabricLivySession")
# Wait before running code, and close the Spark session when finished
session$wait()
session$run("print(1 + 1)", kind = "pyspark")
session$close()
} # }