Package {deepImp}

Type: Package
Title: Imputation with Deep Learning Methods
Version: 1.2.0
Description: Imputation of mixed-type and compositional data with neural networks. Each network head is a generalised linear model plus a neural-network correction. Missing values are imputed by point predictions or, for multiple imputation, by predictive draws on bootstrap fits, with pooling through 'mice'. The architecture (number and size of hidden layers, dropout, activation, optimiser) and the training settings are user-configurable. See Templ (2021) <doi:10.1007/978-3-030-71175-7>.
License: GPL-2
URL: https://github.com/matthias-da/deepImp
BugReports: https://github.com/matthias-da/deepImp/issues
Encoding: UTF-8
LazyData: TRUE
ByteCompile: TRUE
Depends: R (≥ 4.1)
Imports: torch, nnet, VIM, stats, utils, graphics
Suggests: future, future.apply, keras3, knitr, mice, rmarkdown, robCompositions, testthat (≥ 3.0.0)
RoxygenNote: 7.3.3
Config/testthat/edition: 3
VignetteBuilder: knitr
NeedsCompilation: no
Packaged: 2026-11-14 11:40:00 UTC; matthias
Author: Matthias Templ ORCID iD [aut, cre]
Maintainer: Matthias Templ <matthias.templ@gmail.com>
Repository: CRAN
Date/Publication: 2026-11-14 13:50:02 UTC

deepImp: Imputation with Deep Learning Methods

Description

Imputes missing values with configurable neural networks, for both mixed-type data (impNNet()) and compositional data with rounded zeros (impNNetCoDa()). The network architecture (depth, width, dropout, activation, optimiser) is described by a deepimp_arch() object. Networks run on a native torch backend by default (no Python required) or, optionally, on keras3. Results are returned as a "deepimp" object; read the completed data with getImputed().

Details

See vignette("deepImp") for a walkthrough.

Author(s)

Maintainer: Matthias Templ matthias.templ@gmail.com (ORCID)

References

Templ, M. (2021). Imputation of rounded zeros for compositional data using neural networks. In: Advances in Compositional Data Analysis (Festschrift). Springer. doi:10.1007/978-3-030-71175-7

See Also

Useful links:

Convert a deepimp object to a mice mids object

Description

Hands the completed data sets to the mice analysis and pooling machinery, e.g. with(as_mids(imp), lm(y ~ x)) followed by mice::pool(). Rubin's rules are only meaningful for multiple imputations, i.e. for objects from impNNet() with draw = "pmm" or "normal" and m > 1. Point-mode replicates (draw = "none") differ only by retraining; pooling them understates the between-imputation variance, so the conversion warns.

Usage

as_mids(object)

Arguments

object

a "deepimp" object.

Value

a mids object (requires the mice package).

See Also

impNNet(), getImputed()

Examples

if (requireNamespace("torch", quietly = TRUE) && torch::torch_is_installed() &&
    requireNamespace("mice", quietly = TRUE)) {

  data(sleep, package = "VIM")
  mi <- impNNet(sleep, arch = deepimp_arch_small(dropout = 0), m = 3,
                draw = "pmm", iterations = 2, epochs = 5, seed = 1)
  fit <- with(as_mids(mi), lm(Sleep ~ log(BodyWgt) + Danger))
  summary(mice::pool(fit))

}

keras3 backend for deepImp

Description

Internal backend implementing the deepImp model seam (build/fit/predict) with keras3 and its Keras/TensorFlow backend. Optional alternative to the default torch backend; not called directly by users.

torch backend for deepImp

Description

Internal backend implementing the deepImp model seam (build/fit/predict) with torch and a plain training loop on preloaded tensors. Not called directly by users.

Beer ageing volatile-compound data

Description

Concentrations of 16 volatile compounds measured in beer samples, together with an indicator distinguishing fresh from aged beer. The volatile compounds are recognised markers of beer flavour and ageing (e.g. furfural, 5-hydroxymethylfurfural, hexanal, methional-related aldehydes). The data are used to illustrate imputation of mixed continuous measurements.

Usage

data(beer)

Format

A data frame with 86 rows and 17 variables:

v3MeBual

3-methylbutanal concentration.

v3MeBuon

3-methylbutanone concentration.

v2MeBual

2-methylbutanal concentration.

vHexanal

hexanal concentration.

v2FurMeol

2-furanmethanol (furfuryl alcohol) concentration.

vHeptanal

heptanal concentration.

v2AcFur

2-acetylfuran concentration.

v5Me2Fur

5-methyl-2-furaldehyde concentration.

vEssFuEst

furanoic acid ethyl ester concentration.

v2Ac5MeFu

2-acetyl-5-methylfuran concentration.

v2PhEtal

2-phenylethanal (phenylacetaldehyde) concentration.

vNicEtEst

nicotinic acid ethyl ester concentration.

v2PhEssEt

2-phenylethyl acetate concentration.

vgNonalac

gamma-nonalactone concentration.

vFurfural

furfural concentration.

vHMF

5-hydroxymethylfurfural (HMF) concentration.

vnewold2

beer condition: 1 = fresh, 2 = aged.

Source

Derived from a raw beer-ageing table without the brewery identifiers; the derivation is documented in data-raw/beer.R in the package sources.

The deepimp object

Description

impNNet() and impNNetCoDa() return an S3 object of class "deepimp", a list with the elements

data

the incomplete input data.

imputed

a list of m completed data frames.

arch

the deepimp_arch() object used.

info

a list of fitting metadata: backend, vartypes, draw, bootstrap, pmm_k, iterations, eps, initialize, control (the deepimp_control() object), convergence (per replicate, the pooled standardised change per sweep), converged (per replicate; NA in draw mode, which runs a fixed number of sweeps), trace (arrays mean and sd of the imputed cells, variable x sweep x replicate), history (one row per fit: replicate, sweep, variable, epochs used, best epoch, best validation loss, safeguard status), skipped (variables caught by a safeguard), seeds, n_missing, and models (with keep_models = TRUE: per replicate and variable the exported model used by predict.deepimp()). impNNetCoDa() adds coda, dl and correction.

Details

Methods: getImputed() and as.data.frame() (completed data), print(), summary.deepimp(), plot.deepimp() (convergence and chain trace), as_mids() (pooling with mice), predict.deepimp() (imputing new data). Objects created by deepImp 1.1.0 lack some info fields; all methods still work on them.

See Also

impNNet(), impNNetCoDa(), new_deepimp()

Architecture configuration for neural-network imputation

Description

A backend-neutral description of the multilayer perceptron used by impNNet(). The same object is translated into a torch (and, later, keras) model.

Usage

deepimp_arch(
  hidden = c(256, 128, 64),
  dropout = 0.1,
  activation = "relu",
  batchnorm = TRUE,
  optimizer = "adam",
  learning_rate = 0.001,
  glm_offset = TRUE,
  glm_ridge = 0.001,
  weight_decay = 0
)

deepimp_arch_small(
  dropout = 0.1,
  activation = "relu",
  batchnorm = TRUE,
  optimizer = "adam",
  learning_rate = 0.001,
  glm_offset = TRUE,
  glm_ridge = 0.001,
  weight_decay = 0
)

Arguments

hidden

integer vector of hidden-layer widths; its length is the depth.

dropout

dropout rate applied after every hidden layer; a scalar in ⁠[0, 1)⁠.

activation

hidden-layer activation: one of "relu", "tanh", "sigmoid", "elu", "leaky_relu".

batchnorm

logical; apply batch normalisation after each hidden linear layer.

optimizer

one of "adam", "sgd", "rmsprop".

learning_rate

positive learning rate for the optimiser.

glm_offset

logical; if TRUE (default), every network head is a generalised linear model plus an MLP correction: a linear model under the regression heads and a multinomial logit (via nnet::multinom()) under the classification heads, both fitted on the training rows of each fit. The last MLP layer starts at zero and the untrained state competes in the restore-best step, so the head nests the GLM and the network only has to learn what the GLM misses (nonlinearity, interactions). FALSE gives the plain MLP of deepImp 1.1.0.

glm_ridge

non-negative ridge penalty of the GLM offset (ridge on the least-squares fit, intercept unpenalised; decay of nnet::multinom()). A small guard against separation, rare levels and nearly as many predictors as rows; it is negligible otherwise.

weight_decay

non-negative weight decay of the MLP weights (the GLM offset has its own glm_ridge). torch: decoupled weight decay for "adam" (AdamW), an L2 penalty for "sgd" and "rmsprop"; keras: the optimiser's weight_decay. Useful when there are few observed rows per predictor.

Value

an object of class "deepimp_arch".

See Also

impNNet(), impNNetCoDa()

Examples

deepimp_arch(hidden = c(128, 64), dropout = 0.2)

Training controls for neural-network imputation

Description

A validated set of training settings for impNNet() and impNNetCoDa(), the counterpart of deepimp_arch() for the fitting procedure. The top-level arguments epochs, patience, batch_size and validation_split of impNNet() remain as shortcuts; supply either those or control, not both.

Usage

deepimp_control(
  epochs = 400L,
  patience = 10L,
  batch_size = "auto",
  validation_split = 0.2,
  restore_best = TRUE,
  lr_schedule = c("none", "plateau"),
  device = c("cpu", "cuda", "auto"),
  num_threads = NULL,
  warm_start = NULL,
  parallel = FALSE,
  min_obs = 20L
)

Arguments

epochs

maximum number of training epochs per model (0 keeps the untrained head, i.e. the GLM offset alone).

patience

early stopping: number of epochs without improvement of the validation loss before training stops. The defaults (patience = 10, batch_size = "auto", warm starts in point mode) were about ten times faster than those of deepImp 1.1.0 (patience = 40, batch_size = 32) at n = 1000-3000 in a simulation, with no measurable loss of accuracy.

batch_size

minibatch size, or "auto" for about 32 minibatches per epoch, rounded to a power of two between 32 and 512.

validation_split

share of the observed rows held out for early stopping (not used with bootstrap fits, which validate on the out-of-bag rows).

restore_best

logical; restore the weights of the epoch with the lowest validation loss after training (default). With FALSE the weights of the last epoch are kept.

lr_schedule

"none" (constant learning rate) or "plateau" (halve the learning rate whenever the validation loss has not improved for max(2, patience %/% 4) epochs, down to 1/1000 of the initial rate).

device

torch backend only: "cpu" (default), "cuda", or "auto" (CUDA when available, else CPU). Apple's MPS device is not offered: its operator coverage is incomplete and it gives no gain for the small networks used here. The keras backend uses TensorFlow's device placement.

num_threads

torch backend only: number of CPU threads for libtorch, or NULL to leave the setting alone. libtorch accepts a new value only once per R session, before any parallel work. One or two threads are usually fastest for the small networks used here.

warm_start

logical; from the second sweep on, continue training each column model from its weights of the previous sweep (with patience max(2, patience %/% 4)) instead of from new random weights. The previous weights compete in restore-best, so a sweep never makes a model worse on its validation rows. Saves most of the training time after the first sweep. NULL (default) means TRUE in point mode and FALSE in draw mode, where every sweep refits from scratch.

parallel

logical; run the m replicates in parallel with future.apply on the current future::plan() (torch backend only). Every worker uses num_threads threads (default 1). With a seed, the results equal those of a sequential run with the same number of threads (the thread count changes the order of floating-point sums).

min_obs

minimum number of observed values a variable needs for a network fit; variables with fewer keep their starting values, with a warning.

Value

an object of class "deepimp_control".

See Also

impNNet(), impNNetCoDa(), deepimp_arch()

Examples

deepimp_control(epochs = 100, patience = 10, batch_size = "auto")

Extract imputed data from a deepimp object

Description

Extract imputed data from a deepimp object

Usage

getImputed(object, m = 1L, ...)

Arguments

object

a "deepimp" object from impNNet().

m

which completed dataset to return: an integer index, "all" for the list of all imputations, or "long" for all of them stacked in one data.frame in the mice long format (columns .imp and .id; .imp = 0 holds the incomplete data). Only draw-mode replicates (impNNet(draw = "pmm") or "normal") are multiple imputations; see impNNet().

...

unused.

Value

a data.frame, or a list of data.frames when m = "all".

See Also

impNNet(), as_mids()

Guess the measurement type of each variable

Description

Classifies each column as "numeric", "mixed" (semi-continuous, i.e. a spike of repeated values such as zeros plus a continuous part), "binary", "nominal", or "count". Used by impNNet() to choose the model head, and by summary() of a "deepimp" object.

Usage

guessType(x)

Arguments

x

a data.frame, data.table, or tibble.

Value

a list with indices (a type-by-variable logical matrix) and type (a character vector, one entry per column).

Author(s)

Matthias Templ

Examples

data(sleep, package = "VIM")
guessType(sleep)

Neural-network imputation for mixed-type data

Description

Iterative, chained imputation of numeric, count, semi-continuous, binary and nominal variables with a configurable multilayer perceptron (torch backend).

Usage

impNNet(
  data,
  arch = NULL,
  m = 1L,
  backend = c("torch", "keras"),
  vartypes = "guess",
  initialize = "knn",
  iterations = NULL,
  eps = 0.01,
  normalize = TRUE,
  epochs = 400L,
  patience = 10L,
  validation_split = 0.2,
  batch_size = "auto",
  seed = NULL,
  verbose = FALSE,
  draw = c("none", "pmm", "normal"),
  bootstrap = NULL,
  pmm_k = 5L,
  control = NULL,
  keep_models = FALSE,
  ...
)

Arguments

data

a data.frame (tibbles/data.tables are coerced).

arch

a deepimp_arch() object, or NULL to build one from .... Supplying both arch and architecture scalars via ... is an error.

m

number of completed data sets (replicates). See "Point mode and draw mode": only draw-mode replicates are multiple imputations.

backend

"torch" (default) or "keras" (optional; requires the keras3 package and a working Keras/TensorFlow backend).

vartypes

"guess" (use guessType()) or a length-ncol(data) vector.

initialize

starting values for NAs: "knn", "mean", or "hotdeck".

iterations

number of chained sweeps: the maximum in point mode (default 3), the fixed number in draw mode (default 5). NULL picks the default of the mode.

eps

convergence tolerance on the standardised mean change (point mode only).

normalize

standardise numeric predictors.

epochs, patience, validation_split, batch_size

shortcuts for the corresponding deepimp_control() settings. validation_split is not used when bootstrap = TRUE.

seed

optional integer seed (sets the R and backend RNGs). Exact reproducibility is guaranteed when dropout = 0; with dropout > 0 the backend's training-time mask sampling is not fully pinned by the seed in the current torch build, so repeated runs are close but may not be bit-identical.

verbose

0 (or FALSE, default) silent; 1 (or TRUE) a progress bar over replicates, sweeps and variables; 2 a message per variable fit and per training run.

draw

"none" (point imputation, default), "pmm" (predictive mean matching) or "normal" (prediction plus normal noise). Any draw mode samples the classification heads from their softmax.

bootstrap

logical; fit every column model on a bootstrap resample of the observed rows, with the out-of-bag rows as the validation set. Defaults to TRUE in draw mode and FALSE in point mode.

pmm_k

number of donors for draw = "pmm".

control

a deepimp_control() object with the training settings, or NULL to build one from the shortcuts epochs, patience, batch_size and validation_split. Supply either control or the shortcuts.

keep_models

logical; keep the fitted models of the last sweep (as plain R arrays, so the object can be saved with saveRDS()) for imputing new data with predict.deepimp(). Variables without missing values get a model too, fitted once after the last sweep. Memory grows with the number of variables, replicates and network weights.

...

architecture scalar overrides forwarded to deepimp_arch().

Value

a "deepimp" object (see deepimp); read the data with getImputed().

Point mode and draw mode

With draw = "none" (point mode) every missing cell gets the network's conditional mean (regression heads) or its most probable class (classification heads), and the chain stops once the pooled change falls below eps. Point imputations suit prediction-type use, but replicates (m > 1) then differ only by retraining, so they understate the imputation uncertainty and must not be pooled with Rubin's rules.

With draw = "pmm" or "normal" (draw mode) every missing cell gets a draw from the predictive distribution, and by default (bootstrap = TRUE) every fit uses a bootstrap resample of the observed rows. The classification heads then sample from their softmax probabilities. Each replicate runs all iterations sweeps. The m completed data sets are multiple imputations that can be analysed and pooled with Rubin's rules, e.g. with mice::pool() via as_mids(). PMM copies observed values, so it keeps count and semi-continuous variables on their observed support; "normal" adds Gaussian noise with the residual SD of the held-out (out-of-bag) rows and suits continuous variables. For exactly reproducible draws use dropout = 0.

Author(s)

Matthias Templ

See Also

deepimp_arch(), deepimp_control(), getImputed(), as_mids(), predict.deepimp(), guessType()

Examples

if (requireNamespace("torch", quietly = TRUE) && torch::torch_is_installed()) {

  data(sleep, package = "VIM")
  imp <- impNNet(sleep, arch = deepimp_arch_small(), epochs = 5, seed = 1)
  head(getImputed(imp))

  # multiple imputation: predictive draws on bootstrap fits
  mi <- impNNet(sleep, arch = deepimp_arch_small(dropout = 0), m = 3,
                draw = "pmm", iterations = 2, epochs = 5, seed = 1)
  head(getImputed(mi, m = "long"))

}

Neural-network imputation of rounded zeros in compositional data

Description

Imputes rounded zeros (values below a detection limit) in compositional data with a configurable neural network, following Templ (2021). Each part with rounded zeros is pivoted to the front, transformed to pivot log-ratio coordinates, regressed on the remaining coordinates, censored at the detection limit, and back-transformed with preservation of the observed absolute values.

Usage

impNNetCoDa(
  x,
  dl = NULL,
  label = 0,
  coda = TRUE,
  correction = c("truncate", "expectation", "none"),
  initialize = "kNNa",
  arch = NULL,
  m = 1L,
  backend = c("torch", "keras"),
  iterations = 2L,
  eps = 0.01,
  normalize = TRUE,
  epochs = 400L,
  patience = 10L,
  validation_split = 0.2,
  batch_size = "auto",
  seed = NULL,
  verbose = FALSE,
  control = NULL,
  ...
)

Arguments

x

a data.frame of compositional parts (positive values; rounded zeros marked by label).

dl

detection limits, a numeric vector of length ncol(x). Required unless correction = "none".

label

the value marking a rounded zero in x (converted to NA internally). Default 0.

coda

if TRUE (default), work in pivot log-ratio coordinates; if FALSE, impute on the raw scale (ablation).

correction

censoring of imputed values: "truncate" (default; set values above the detection limit to the limit, per Templ 2021), "expectation" (truncated-normal mean; opt-in, beyond Templ 2021), or "none" (no censoring).

initialize

starting values for the rounded zeros: "kNNa" (default, compositional kNN), "knn", "hotdeck", or "mean".

arch

a deepimp_arch() object, or NULL to build one from ....

m

number of stochastic replicate completions (not valid multiple imputation; see impNNet()).

backend

"torch" (default) or "keras" (optional; requires keras3).

iterations

maximum number of chained sweeps.

eps

convergence tolerance on the standardised mean change.

normalize

standardise numeric predictors.

epochs, patience, validation_split, batch_size

shortcuts for the corresponding deepimp_control() settings.

seed

optional integer seed (sets the R and backend RNGs). Exact reproducibility is guaranteed when dropout = 0; with dropout > 0 the backend's training-time mask sampling is not fully pinned by the seed in the current torch build, so repeated runs are close but may not be bit-identical.

verbose

0/FALSE silent, 1/TRUE a progress bar, 2 a message per fit.

control

a deepimp_control() object, or NULL to build one from the shortcuts. Supply either control or the shortcuts.

...

architecture scalar overrides forwarded to deepimp_arch().

Details

Needs the robCompositions package (pivot coordinates and the compositional kNN start), which deepImp suggests but does not import.

Value

a "deepimp" object; read the data with getImputed().

Author(s)

Matthias Templ

References

Templ, M. (2021) Imputation of rounded zeros for high-dimensional compositional data.

See Also

impNNet(), deepimp_arch(), getImputed()

Examples

if (requireNamespace("torch", quietly = TRUE) && torch::torch_is_installed() &&
    requireNamespace("robCompositions", quietly = TRUE)) {

  set.seed(1)
  x <- data.frame(a = runif(50, 5, 10), b = runif(50, 5, 10), c = runif(50, 5, 10))
  x$a[1:5] <- 0
  imp <- impNNetCoDa(x, dl = c(1, 1, 1), label = 0,
                     arch = deepimp_arch_small(), epochs = 5, seed = 1)
  head(getImputed(imp))

}

Construct a deepimp object

Description

Low-level constructor for the object returned by impNNet(). Most users do not call this directly; use getImputed() to read the completed data.

Usage

new_deepimp(data, imputed, arch, info = list())

Arguments

data

data.frame with missing values (original input).

imputed

list of completed data.frames (one per imputation).

arch

a deepimp_arch() object describing the network.

info

list with training metadata (backend, vartypes, convergence, ...).

Value

an object of class "deepimp".

See Also

getImputed(), impNNet()

Plot diagnostics of a deepimp object

Description

type = "convergence" plots the pooled standardised change of the imputed cells per sweep (first replicate). type = "trace" plots, per incomplete variable, the mean and SD of the imputed cells by sweep with one line per replicate (as plot() of a mice::mids object); factors enter by their integer codes. Well-mixed draw-mode chains show intertwined lines without a trend.

Usage

## S3 method for class 'deepimp'
plot(x, type = c("convergence", "trace"), vars = NULL, ...)

Arguments

x

a "deepimp" object.

type

"convergence" (default) or "trace".

vars

for type = "trace": names of the variables to show (default: all incomplete variables).

...

further arguments passed to graphics::matplot() (trace) or graphics::plot() (convergence).

Value

x, invisibly.

Impute new data with the models of a deepimp object

Description

Imputes the missing values of new observations with the models fitted by impNNet() with keep_models = TRUE, without retraining. Starting values for the new rows come from the method of the fit (initialize) with the first completed training data set as donors; then the chained sweeps run with the stored models of each replicate, in the mode of the fit (point imputation or draws). Each replicate of the fit gives one completed version of newdata.

Usage

## S3 method for class 'deepimp'
predict(object, newdata, iterations = NULL, seed = NULL, ...)

Arguments

object

a "deepimp" object from impNNet() with keep_models = TRUE.

newdata

a data.frame with the columns of the training data (by name). Factor levels unknown to the fit are set to NA and imputed, with a warning.

iterations

number of sweeps over the new rows; default: the number used in the fit.

seed

optional integer seed for the draws.

...

unused.

Value

a "deepimp" object for newdata, with info$predicted = TRUE.

See Also

impNNet(), deepimp

Examples

if (requireNamespace("torch", quietly = TRUE) && torch::torch_is_installed()) {

  data(sleep, package = "VIM")
  fit <- impNNet(sleep[1:50, ], arch = deepimp_arch_small(), epochs = 5,
                 keep_models = TRUE, seed = 1)
  new <- predict(fit, sleep[51:62, ])
  getImputed(new)

}

Summarise a deepimp object

Description

Summarise a deepimp object

Usage

## S3 method for class 'deepimp'
summary(object, ...)

## S3 method for class 'summary.deepimp'
print(x, ...)

Arguments

object

a "deepimp" object.

...

unused.

x

a "summary.deepimp" object.

Value

summary() returns an object of class "summary.deepimp": a list with the dimensions, the mode ("point", "pmm", "normal" or "coda"), the missing values per variable, the sweeps run and the convergence flag per replicate, the epochs used per fit (summary statistics) and the variables caught by a safeguard. print() shows it.

See Also

deepimp, impNNet()