Skip to contents

Introduction

The future.callr package is a backend for the future framework, also known as the Futureverse. The Futureverse makes it easy to parallelize existing R code - often with only a minor change of code. Most users interact with it via higher-level packages such as futurize, future.apply, furrr, and doFuture, and you choose where and how the code runs by setting a future backend with plan(). This package adds the callr backend, which runs each future in a fresh background R process on the local machine using the callr package.

library(futurize)
plan(future.callr::callr)  # parallelize on the current machine

# A slow function
slow_fcn <- function(x) {
  Sys.sleep(1.0)
  x^2
}

# Example data
xs <- 1:10

# Sequential and parallel base R apply
ys <- lapply(xs, slow_fcn)
ys <- lapply(xs, slow_fcn) |> futurize()

# Sequential and parallel purrr map
library(purrr)
ys <- xs |>  map(slow_fcn)
ys <- xs |>  map(slow_fcn) |> futurize()

# Sequential and parallel foreach calls
library(foreach)
ys <- foreach(x = xs) %do% slow_fcn(x)
ys <- foreach(x = xs) %do% slow_fcn(x) |> futurize()

The only thing that differs from using any other future backend is the plan() call. The code does not have to change, it gives identical results, and it behaves exactly the same.

The callr backend

Backend Description Alternative in future package
callr parallel evaluation in a separate R process (on current machine) plan(multisession)

Each callr future is resolved in a fresh background R process, which ends as soon as the value of the future has been collected. In contrast, multisession futures are resolved by R workers that stay alive and serve many futures over their life spans.

Advantages

  • Each future starts from a clean R session. Futures cannot affect each other via memory allocations, finalizers, modified options, or loaded and attached packages.

  • More than 125 futures can run in parallel. The multisession and cluster backends use one R connection per worker, and R has a default limit of 125 connections per session. callr does not use R connections, so this limit does not apply.

  • No ports are used, which means there are no port clashes with other processes, and no firewall alerts on Windows.

Limitations

  • Launching a new R process for each future adds overhead. This makes the callr backend less suited for many small futures, where plan(multisession) might be faster.

  • The R processes always run on the local machine. For workers on other machines, or on a high-performance compute (HPC) cluster, use another future backend, e.g. plan(cluster) or future.batchtools.

Installation

R package future.callr is available on CRAN and can be installed in R as:

install.packages("future.callr")

Pre-release version

To install the pre-release version, use:

install.packages("future.callr", repos = c("https://futureverse-devel.r-universe.dev", "https://cloud.r-project.org"))