Read and write R or Arrow objects in OneLake Files

These object-aware helpers sit above fabric_onelake_download() and fabric_onelake_upload(). They serialize data frames, tibbles, and lazy Arrow inputs without collecting the complete object in R memory, and decode supported OneLake files directly to a tibble or Arrow stream.

Usage

fabric_onelake_read_file(
  workspace,
  item = NULL,
  path = "",
  format = c("auto", "parquet", "csv", "arrow"),
  result = c("tibble", "arrow_stream"),
  item_type = NULL,
  tenant_id = Sys.getenv("FABRICQUERYR_TENANT_ID"),
  client_id = Sys.getenv("FABRICQUERYR_CLIENT_ID", unset =
    "04b07795-8ddb-461a-bbee-02f9e1bf7b46"),
  token = NULL,
  auth_args = list(),
  dfs_base = "https://onelake.dfs.fabric.microsoft.com",
  col_names = TRUE,
  na = "NA",
  col_types = NULL,
  csv_numeric = c("exact", "infer")
)

fabric_onelake_write_file(
  workspace,
  item = NULL,
  path = "",
  data,
  format = c("auto", "parquet", "csv", "arrow"),
  overwrite = FALSE,
  if_match = NULL,
  compression = "snappy",
  include_header = TRUE,
  na = "NA",
  create_parents = TRUE,
  item_type = NULL,
  tenant_id = Sys.getenv("FABRICQUERYR_TENANT_ID"),
  client_id = Sys.getenv("FABRICQUERYR_CLIENT_ID", unset =
    "04b07795-8ddb-461a-bbee-02f9e1bf7b46"),
  token = NULL,
  auth_args = list(),
  dfs_base = "https://onelake.dfs.fabric.microsoft.com",
  allow_managed_tables = FALSE,
  chunk_size = getOption("fabricqueryr.onelake.chunk_size", 8 * 1024^2)
)

Arguments

workspace

Workspace name, ID, object from fabric_workspaces(), or a complete OneLake HTTPS/ABFSS path.

item

Item name, GUID, or discovered Fabric item. Use NULL when workspace is a complete OneLake path.

path

Item-relative path, normally below Files/.

format

File format. "auto" infers "parquet", "csv", or "arrow" from the path extension.

result

Return a "tibble" or a disk-backed, single-use "arrow_stream". Tibbles preserve decimals as character strings. Signed 64-bit integers use bit64::integer64, or character if the column contains the minimum signed value (reserved for missing values by bit64). Int32 columns containing -2147483648 use exact R doubles. Nested lists retain character 64-bit integers and decimals, and double 32-bit integers. Structs with null parents require result = "arrow_stream": tibble collection cannot distinguish them from valid structs with all-null fields and raises fabric_arrow_null_struct_error before discarding that distinction.

item_type

Optional Fabric item type used to resolve a named item.

tenant_id

Entra tenant ID. Defaults to FABRICQUERYR_TENANT_ID.

client_id

Entra application ID. Defaults to FABRICQUERYR_CLIENT_ID, then the Azure CLI application ID.

token

Optional access token or audience-aware token-provider function.

auth_args

Additional sign-in options passed to AzureAuth::get_azure_token().

dfs_base

OneLake DFS service address. A private or regional endpoint on a discovered object is preferred when this argument is omitted.

col_names

Whether a CSV has a header, or a character vector of column names. Use FALSE for files written with include_header = FALSE.

na

Text used for missing values in a written CSV, or character values interpreted as missing when reading CSV. The default, "NA", preserves empty strings separately from missing values and keeps missing-only rows visible in CSV files. Blank records are retained when reading. For legacy files whose empty fields mean missing, read with na = c("", "NA"). Any literal text matching a chosen missing marker is also read as missing; choose the same custom marker for writing and reading when needed.

col_types

Optional Arrow Schema specifying CSV column types, such as arrow::schema(amount = arrow::decimal128(38, 15), id = arrow::uint64()). Columns omitted from the schema follow csv_numeric. Use arrow::utf8() to preserve all spelling, including leading zeros in identifiers. Arrow infers omitted column types from an initial block, not the entire file. Later values incompatible with that type raise an error, including oversized integers after small integers, or text after an all-null block. Supply explicit types for such columns, for example arrow::schema(id = arrow::utf8(), label = arrow::utf8()), to read all blocks consistently without losing the original text.

csv_numeric

CSV numeric inference policy. The default, "exact", retains inferred floating-point columns as character strings to avoid rounding decimals or oversized integers. Inferred integer columns retain their exact values but canonicalize spellings such as leading zeros. "infer" opts into Arrow's ordinary inference, including approximate floating-point values. Explicit col_types entries override this policy. These options apply equally to tibbles and Arrow streams. CSV does not store type metadata; use col_types to recover known types after writing, or Parquet/Arrow IPC to retain numeric types automatically.

data

A data frame, tibble, Arrow Table/RecordBatch, lazy Arrow Dataset/Scanner/query, RecordBatchReader, or Arrow-compatible array stream.

overwrite

Whether an existing OneLake file may be replaced.

if_match

Optional destination ETag for conditional replacement.

compression

Parquet compression codec passed to Arrow.

include_header

Whether a written CSV includes column names.

create_parents

Whether missing parent directories are created.

allow_managed_tables

Whether direct writes below Tables/ are permitted. Keep the safe default, FALSE, for managed Delta tables. This guard checks only the supplied path and does not resolve shortcuts. A Files/ shortcut can lead to a managed table, where writes change the target's data even with allow_managed_tables = FALSE.

chunk_size

Upload chunk size in bytes.

Value

fabric_onelake_read_file() returns a tibble or a disk-backed nanoarrow_array_stream. fabric_onelake_write_file() returns OneLake metadata with additional format, rows, and columns fields.

References

Connect to OneLake with ADLS APIs

Get data into OneLake

Examples

if (FALSE) { # \dontrun{
# Discover the Lakehouse that will store the Parquet file
workspace <- fabric_workspaces()[[1L]]
lakehouse <- fabric_lakehouses(workspace)[[1L]]

# Serialize the R data frame directly to OneLake as Parquet
fabric_onelake_write_file(
  workspace,
  lakehouse,
  "Files/exports/orders.parquet",
  data.frame(id = 1:3, amount = c(10.5, NA, 30))
)

# Read the same file back as a tibble
orders <- fabric_onelake_read_file(
  workspace,
  lakehouse,
  "Files/exports/orders.parquet"
)
} # }