Data frames containing factor, character or logical columns can be
passed directly to midas_fit() and midas().
Previously any such frame was rejected by the server with an HTTP 422
“float parsing” error, because the client serialised data through
as.matrix(), which coerces a mixed data frame to a
character matrix.
Categorical data are now sent using the Parquet transport, which preserves column types, so the MIDAS engine fits a softmax head over the levels of a multi-level factor and a binary head over a two-level one – rather than requiring the user to one-hot encode and thereby modelling the resulting dummies as independent Bernoulli variables.
arrow package (in
Suggests). If arrow is not installed,
categorical data now raise a clear, actionable error naming the package
to install, rather than an opaque HTTP error from the server.midas_transform() and midas() now return
data frames whose columns have the classes passed to
midas_fit(), including factor levels and ordering.
Previously every returned column was character, because the response was
rebuilt with do.call(rbind, ...), which coerces to a
character matrix as soon as one column is non-numeric.
The imputation engine ingests data as float32 and returns observed cells from that downcast copy, so an observed numeric value came back as the binary32 round-trip of the input – a relative change of about 6e-8. That is invisible on standardised covariates but amounts to roughly 1e-4 on data of order 1000. Nothing was being re-estimated; the loss was purely the representation change on ingest.
midas_transform() gains a data argument.
Pass the frame given to midas_fit() and every cell that was
observed in it is restored to its original value, so only missing cells
differ from the input.midas() does this automatically, since it is handed the
data.combine() returned every column of the Rubin’s rules
table – including estimate, std.error,
statistic, df and p.value – as
character. These are now numeric, so the result can be used directly
without manual coercion. This affected all models, not only those with
categorical data.imp_mean() now fails with an explanatory message when
the model contains categorical columns, instead of surfacing an HTTP 500
from the server. An element-wise mean over factor levels is not defined;
use midas_transform() and pool in the manner appropriate to
the analysis.to_nested_list() now validates its input and reports a
clear internal error rather than silently coercing non-numeric
columns..Rbuildignore now excludes .Rhistory files
in subdirectories, not only at the package root.?midas_fit gains a “Column types” section describing
how each R class is modelled, and a “Choosing dropout_prob”
section. The default of 0.5 suits wide data; on narrow frames (fewer
than roughly 15 columns) a smaller value such as 0.2 can substantially
improve reconstruction accuracy. The default is unchanged.start_server(): Launch the MIDAS2 Python API
server.stop_server(): Stop the running server.ensure_server(): Start the server if it is not already
running.midas_fit(): Fit a MIDAS denoising autoencoder
model.midas_transform(): Generate multiple imputations from a
fitted model.midas(): All-in-one fit + transform convenience
function.imp_mean(): Compute element-wise mean across
imputations.combine(): Run Rubin’s rules regression across
imputations.overimpute(): Overimputation diagnostic
(mask-and-reimpute RMSE).install_backend(): Install the Python backend into an
isolated environment.