# NEWS for SVEMnet

[development]
- Gaussian SVEMnet() and svem_forward() fits now retain an aligned per-member diagnostic table with validation SSE, selected objective value, support size, effective df, raw/bounded Kish validation size, support-based residual df/scale, fallback status, and admissibility warnings. SVEMnet() now computes the selected member's existing gamma-blended effective-df diagnostic even when the default support-count selector is used; this does not change which path point is selected.
- New opt-in store_member_weights = TRUE retains exact FRW uniforms plus mean-one training and validation weights. The large weight matrices are absent by default, and enabling them does not change the random-number stream. New svem_ij_variance() uses those weights to expose an experimental FRW infinitesimal-jackknife analogue for the raw Gaussian ensemble mean, with an ordinary-bootstrap-inspired finite-B correction. It returns no confidence limits and is explicitly not calibrated: prediction bias, selection nonregularity, and new-observation noise are not handled.
- Existing defaults, point predictions, member-percentile intervals, wAIC/wBIC/wSSE selection, and binomial fitting behavior are unchanged.

[3.5.0]
- New function svem_forward(): fits an SVEM ensemble whose per-bootstrap base learner is greedy forward selection on training-weighted least squares instead of the glmnet elastic-net path. The forward path over whole model terms plays the lambda-path role; every path point is refit on the training-weighted rows and scored on the fractional-random-weight validation copy with the same wAIC (default), wBIC, or wSSE criteria as SVEMnet(), including the Kish effective-sample-size admissibility guardrail (path growth stops at the wAIC/wBIC ceiling; wSSE paths are never truncated) and the weighted intercept-only fallback. Multi-column terms (factor contrast blocks) enter and leave as a unit. Gaussian responses only. Accepts the same formula or bigexp_terms() spec front end as SVEMnet() (response swap, unseen-level handling, locked levels/contrasts, blocking, sampling schema) and returns an svem_model-classed object, so predict() (including se.fit/interval and debias), coef(), plot(), svem_nonzero(), svem_random_table_multi(), svem_score_random(), and svem_thompson_batch() work unchanged. Adds per-term bootstrap selection frequencies, path-length, and admissibility-truncation diagnostics.
- New function forward_aicc(): deterministic greedy forward selection over whole model terms for Gaussian responses, minimizing least-squares AICc (default), AIC, or BIC, as a single-model benchmark companion to SVEMnet() in the same spirit as glmnet_with_cv(). Uses the same formula/bigexp front end and returns an svem_model-compatible object (single-row coef_matrix; bootstrap uncertainty options unavailable) plus the selection path (term, model size, RSS, criterion, improvement per accepted step). Candidate models that are rank deficient given the current path (for example the last mixture component) or whose criterion is undefined (AICc requires n - K - 1 > 0) are skipped; selection stops when no candidate improves the criterion.
- No changes to existing fitting, prediction, scoring, or selection behavior; the new fitters live in a separate file with a private copy of the design front end.

[3.4.0]
- Debugging and CRAN-readiness release (2026-08-02). The published SVEM fitting mathematics are unchanged: fractional random weights, validation-weighted wSSE/wAIC/wBIC selection, support counting, response-scale ensemble aggregation, and the Karl (2024) whole-model statistic retain their documented conventions.
- svem_significance_test_parallel() now uses R's modern, uniform `sample.kind = "Rejection"` by default. Set `rng_sample_kind = "Rounding"` to recover the legacy random-number stream. The WMT learner and simulation-budget defaults remain the 3.3.1 SVEM-Lasso settings (`glmnet_alpha = 1`, `relaxed = FALSE`, `nBoot = 100`, `nPoint = 2000`, `nSVEM = 10`, `nPerm = 150`, `percent = 90`).
- WMT execution is serial by default (`nCore = 1`) and no longer replaces the caller's foreach backend. Explicit multi-core runs use a private PSOCK cluster. Count, seed, percentage, and core controls are validated before work begins; in particular the four-parameter SHASH tail fit requires `nPerm >= 5` and ensemble standard errors require `nBoot >= 2`. The earlier NEWS claim that `nPerm = 1` was supported was incorrect.
- The WMT evaluation grid now reproduces the raw classes of categorical predictors: ordered factors are sampled as ordered factors (also in svem_random_table_multi()) and logical predictors as logicals. Previously both were rebuilt as plain factors, so the new-data class validation added in this release rejected every original-data fit slot ("Predictor ... was fitted as ordered/logical but `newdata` supplies class factor").
- WMT failures no longer silently reduce the requested reference sample. Invalid inner fits receive bounded deterministic retries, degenerate standard errors are surfaced, zero-variance grid columns are removed explicitly, the paper's strict variance-retention threshold is used, and SHASH right-tail probabilities are evaluated without subtractive cancellation. Returned diagnostics record the RNG mode, retries, and removed columns.
- SVEMnet() and glmnet_with_cv() correctly fit one-column designs by using an excluded internal sentinel column for glmnet compatibility. They reject multivariate responses and explicit no-intercept specifications up front, matching the intercept convention in Karl (2024, 2026). Bootstrap fallbacks are summarized rather than silently hiding complete fitting failure.
- Prediction now preserves supported formula transformations in a compact evaluation environment, validates new-data variable classes before rebuilding design matrices, handles zero-row inputs, and rejects uncertainty requests when fewer than two ensemble members are available. Relaxed-CV coefficient extraction now uses the selected full-data glmnet path rather than an unrelated extra CV run.
- glmnet_with_cv(relaxed = TRUE) now aggregates the complete relaxed cross-validation surfaces across repeats and jointly selects alpha, lambda, and gamma, matching cv.glmnet's relaxed tuning convention. Previously it aggregated only the top-level gamma = 1 curves to choose lambda and then combined that lambda with a gamma chosen in a separate final CV fit. The selected gamma is now returned as best_gamma and recorded in diagnostics and metadata. The SVEMnet() relaxation search was already joint and is unchanged.
- svem_score_random() and svem_thompson_batch() support transformed response labels consistently. Goal/control validation and finite-value candidate ranking now fail early on malformed inputs instead of allowing vector-length errors, silent truncation, or non-finite winners.
- Binomial ROC AUC and average precision now group tied scores by threshold. Results are invariant to row order; a constant score has AUC 0.5 and average precision equal to prevalence.
- Documentation examples were reduced to CRAN-safe toy workloads or marked as non-running production workflows. New regression tests are skipped on CRAN and exercised in the full local suite.
- inst/CITATION now derives the package version from package metadata so it cannot drift across releases.

[3.3.1]
- Bug-fix release following a full-source review (2026-07-31). No changes to the default SVEMnet() fitting path, wAIC/wBIC selection, prediction aggregation, or scoring mathematics; the fixes address off-default paths, silent output defects, and misleading errors.
- bigexp_terms(): spec$formula now carries the "bigexp_spec" attribute, so the documented workflow SVEMnet(spec$formula, data, ...) is equivalent to SVEMnet(bigexp_formula(spec, "y"), data, ...). Previously the bare stored formula silently took the plain-formula path, losing blocking, discrete-numeric supports, and locked levels/contrasts. bigexp_formula(spec) (no response) therefore also returns the attributed formula now.
- bigexp_terms()/bigexp_formula()/SVEMnet(): expansion formulas are now created with env = baseenv(). Previously they captured the builder's evaluation frame -- including a hidden copy of the training data and the temporary expanded model matrix -- so every saved spec or model embedded the data (a 20k-row spec serialized to ~2.6 MB) and name collisions could resolve variables from the closure instead of erroring.
- glmnet_with_cv(): the per-factor contrasts are now captured before the intercept column is dropped (matrix subsetting discards attributes), so the fitted object's schema$contrasts is populated as documented. Previously it was always NULL when factors were present, and predicting in a session with different global contrasts options silently zero-filled the factor columns. (Same fix that SVEMnet() already had.)
- plot.svem_model(type = "calibration"): the binned reliability curve is drawn again. A matrix-column aggregate() result was mis-indexed, so the calibration curve and bin-size points were silently absent from every calibration plot (only the raw points and diagonal rendered). Arguments passed via '...' (e.g. alpha, size) now override the raw-point layer defaults instead of erroring with "matched by multiple actual arguments".
- Transformed responses such as log(y) ~ ... no longer crash SVEMnet(), svem_significance_test_parallel(), svem_random_table_multi(), or svem_wmt_multi() (all four used as.character(formula[[2]]), which is length 2 for a call LHS and errors in scalar contexts on R >= 4.2). The significance test now permutes the raw response column and refits the transformed formula (previously a transformed LHS would have written modeled-scale values back into the raw column); it requires exactly one response variable on the LHS and says so. svem_random_table_multi() names prediction columns by the deparsed LHS (e.g. "log(y)_pred").
- svem_significance_test_parallel(): blocking variables recorded in spec$settings$blocking are now held fixed at a single reference value in the evaluation grid (numeric blocks at the range midpoint snapped to the discrete support; categorical blocks at the most frequent training level), matching svem_random_table_multi(), and are rejected as mixture variables; documented. Previously blocking factors were swept across their full range/levels, so between-block variation could inflate the observed distances relative to the permutation null.
- svem_significance_test_parallel(): validates nSVEM >= 1 and nPerm >= 1 up front (previously nPerm = 0 crashed mid-run with an RNG error after all original fits completed); warns on the master when original or permutation fits fail and are dropped (worker messages are invisible on PSOCK clusters, so failures previously vanished silently); guards the permutation column SDs against NA (a single surviving permutation row previously crashed svd() with an obscure error); ordered-factor predictors are classified as categorical in the non-spec path (previously sampled as numeric codes, yielding all-NA predictions and a misleading "All SVEM fits ... failed" error); and the foreach backend is de-registered on exit (registerDoSEQ()) so later user %dopar% calls do not hit a dead cluster.
- svem_select_candidates()/svem_select_from_score_table(): requesting k >= the top-set size now returns all top rows instead of crashing in cluster::pam() (which requires k <= n-1); a 1-row top set no longer crashes daisy(). svem_select_from_score_table() stops with a clear error when explicitly supplied predictor_cols are not columns of the table (previously a misspelled vector silently fell through to the heuristic column set).
- svem_score_random(): the svem_predictor_cols attribute on original_data_scored now contains only the model's design factors (previously all columns of 'data', including responses and ID columns, which silently contaminated Gower diversity clustering when selecting candidates from scored original data with k > 0). Warns when 'data' already contains scored columns from a previous round (stale duplicates), when wmt multiplier names do not align with responses (previously a misspelled name silently left that response at full weight), and when spec-limit augmentation fails (previously a verbose-only message). Explicit degenerate desirability anchors (upper_acceptable <= lower_acceptable) now stop with an informative error in svem_score_random() and svem_thompson_batch() instead of silently neutralizing the response.
- Spec-limit columns: when specs are supplied but no response has an active bound, the joint columns (p_joint_mean, joint_in_spec_point, all-NA) are now appended instead of crashing cbind(); empty (0-row) tables keep the same appended-column contract; non-numeric bounds (e.g. "abc") produce the intended error message instead of R's generic NA-condition error; and all models are validated against the first model's predictor set with a clear error.
- svem_wmt_multi(): wmt_objects keeps a NULL element for failed tests instead of silently shrinking (positional alignment with p_values/multipliers is preserved); duplicate response names are made unique with make.unique() (the previous "_2" suffix scheme could collide with user-supplied names and silently skip a formula; note duplicates are now suffixed "_1", "_2", ... instead of starting at "_2").
- bigexp_terms(): factorial_order = 1 now yields main effects (plus pure polynomial powers) only -- partial-cubic cross terms require factorial_order >= 2 (two-way) or >= 3 (three-way), matching the documentation. intercept/include_pc_2way/include_pc_3way are validated as logical scalars (previously intercept = 1 behaved inconsistently between branches and include_pc_2way = 1 disabled the terms while print() reported them enabled). The y ~ . detection no longer false-positives on predictors whose names begin with a dot. Non-syntactic column names (e.g. "Flow Rate (mL/min)") get a clear "rename your columns" error instead of a misleading transformed-terms message.
- SVEMnet(): a predictor literally named "Intercept" is no longer silently deleted from the design (only the model.matrix "(Intercept)" column is dropped), matching glmnet_with_cv().
- glmnet_with_cv(): meta$cv_object is now retained only when keep = TRUE is explicitly supplied, matching the documentation (previously it was retained whenever 'keep' was not supplied, inflating serialized relaxed fits).
- Algorithm-control arguments that only glmnet.control() understands (e.g. epsnr, mxitnr) are now routed through the per-call control list when the installed glmnet supports it, or ignored with a warning otherwise (previously they were accepted by the misspelling guard and then silently did nothing).
- Documentation: package overview no longer advertises a nonexistent plot grouping argument or a fitted-null/p-value overlay in plot.svem_significance_test(); lipid_screen composition columns documented as summing to exactly 1 in every row.

[3.3.0]
- New function svem_thompson_batch(): proposes a batch of follow-up runs by parallel Thompson sampling from the SVEM bootstrap ensembles (Thompson 1933; Kandasamy et al. 2018). For each batch slot, one bootstrap member is drawn per response, member predictions are mapped to Derringer-Suich desirabilities under the same 'goals' specification as svem_score_random(), and the candidate maximizing the combined score is selected (previously selected candidates are excluded). Candidates may be user-supplied or drawn from the models' sampling schema (with mixture constraints) via svem_random_table_multi(). Supports Gaussian and binomial responses; returns drawn member indices, per-slot draws/desirabilities, and the resolved desirability anchors so they can be held fixed across sequential iterations.
- Motivation: in sequential-optimization benchmarks on small-sample mixture-process problems, Thompson-sampling proposals concentrated exploration among candidates with high posterior probability of being best, were more replication-stable than score-only or widest-CI selection, and were robust to uninformative (pure-noise) responses, which can dominate CI-width-based exploration.
- No changes to existing fitting, prediction, scoring, or selection behavior.

[3.2.3]
- Maintenance release: edge-case hardening and documentation cleanup. No changes to default SVEMnet() fitting, prediction, whole-model testing, or scoring behavior.
- User-supplied 'offset' is no longer accepted: SVEMnet() and glmnet_with_cv() ignore it with a warning (predictions and downstream scoring never carried offsets, so fitting with one produced inconsistent results). User-supplied 'weights' are likewise ignored with a warning in glmnet_with_cv(), matching SVEMnet(): the package controls its own weighting.
- Misspelling protection: argument names in '...' that the installed glmnet does not recognize are now ignored with a warning in SVEMnet(), glmnet_with_cv(), and svem_significance_test_parallel() (glmnet itself silently swallows unknown arguments).
- svem_significance_test_parallel() now validates mixture_groups against the model predictors (mirroring svem_random_table_multi()), so a misspelled mixture variable stops with an error instead of producing an off-simplex evaluation grid.
- bigexp_terms() stops with a clear message when a categorical predictor has fewer than 2 levels in the training data.
- svem_select_from_score_table() records the evaluated 'label' in its return value, and svem_export_candidates_csv() uses it for selection_label; top-set size reporting is now consistent with the PAM step.
- New experimental option SVEMnet(complexity = "edf") for further study: uses a gamma-blended effective-degrees-of-freedom complexity in the wAIC/wBIC penalty (ridge-type trace for elastic-net path points, blended with the active-set rank by the relaxed gamma; Zou, Hastie, and Tibshirani 2007). Identical to the default "support" count for lasso (alpha = 1) paths; Gaussian only; not used by the whole-model test. The default complexity = "support" is unchanged and remains the published behavior.
- DESCRIPTION and CITATION now reference the published package paper: Karl (2026), "SVEMnet: An R package for self-validated elastic-net ensembles and multi-response optimization in small-sample mixture-process experiments", Chemometrics and Intelligent Laboratory Systems, 271, 105660, doi:10.1016/j.chemolab.2026.105660.
- glmnet_with_cv(): the non-default choose_rule = "1se" now follows the standard glmnet convention (largest lambda within one combined SE of the minimum). For relaxed fits, the relaxed mixing parameter gamma is now selected by the rule matching choose_rule ("gamma.min" for "min", "gamma.1se" for "1se"), and the relaxed lambda/gamma tuning flow is documented. Predictions in the final refit and ridge-fallback paths now pass newoffset when an offset is supplied.
- Discrete-numeric sampling in svem_random_table_multi() and svem_significance_test_parallel() uses index-based sampling so single-value supports are preserved exactly.
- bigexp_terms(): dot exclusions such as y ~ . - X1 are honored, and transformed right-hand-side terms (for example log(X1)) now stop upfront with a clear message.
- svem_random_table_multi(): stops with an informative error when factor levels cannot be recovered from the model schema, instead of substituting placeholder levels.
- svem_significance_test_parallel(): runs with nSVEM = 1 (or nPerm = 1); documented that weight_scheme is fixed to "SVEM" to match the whole-model test in Karl (2024).
- svem_score_random(): the CI-width normalization anchors are computed from the sampled score_table and reused when scoring the original data, so uncertainty_measure is on a comparable scale in score_table and original_data_scored.
- SVEMnet(): glmnet algorithm-control values such as maxit supplied directly are now routed in a version-compatible way (previously only control = list(...) was routed); reserved arguments (nlambda, lambda, lambda.min.ratio) are ignored with a warning.

[3.2.2]
- Documentation and example update clarifying candidate selection targets.
- Clarify that `score`, per-response desirabilities, per-response weighted CI-width summaries, and `uncertainty_measure` are robust bounded summaries and may tie at 0 or 1 because of percentile clipping.
- Update the example candidate-selection workflow to use raw prediction columns such as `<response>_pred` for single maximum-response selection, and a user-created raw CI-width column such as `<response>_ciw = <response>_upr - <response>_lwr` for widest-interval exploration.
- No changes to SVEM fitting, scoring, prediction, or selection function behavior.

[3.2.1]
- Maintenance release for compatibility with glmnet 5.0 while preserving compatibility with glmnet 4.1-10.
- Route glmnet algorithm-control values through per-call control lists when supported by the installed glmnet version, with legacy argument handling for older glmnet versions.
- No intentional changes to SVEM weighting, model selection, prediction, or fitted object structure.

[3.1.9]
- No longer use doRNG package in svem_significance_test_parallel.
- Added discrete numeric argument to bigexp_terms()
- Added column-type checking to bigexp_terms() to help prevent unexpected behavior (e.g. user provides a column of numbers that is cast as character)

[3.1.4]
- Add blocking argument to bigexp_terms()

[3.1.2]
- Replaced svem_optimize_random() with svem_score_random and svem_select_score_from_table. Changes to "auto" objective rule.

[2.5.4]
- Added support for family="binomial". Improvements to svem_random_table_multi() and svem_optimize_random().

[2.3.1]
- Improvements to svem_random_table_multi() and svem_optimize_random()

[2.2.4]
- Added new functions to automatically build interaction and curvature effects from list of main effects. Added a Monte Carlo optimization function. Removed wGIC objective.

[2.1.2]
- Added relaxed option to SVEMnet, default to TRUE. Default objective "auto" uses weighted BIC when the number of rows of the model matrix is <= 1.3 times the number of columns, and weighted AIC otherwise.

[1.4.0]
- Add two experimental functions allowing for whole-model testing with mixture factors.

[1.3.1]
- Code changes to handle edge cases.

[1.3.0]
- Added functions glmnet_with_cv() and predict_cv() to act as a wrapper for cv.glmnet().
- Added examples to vignette

[1.2.1]
- Corrected bug in predict() introduced in 1.1.1
- Added svem_significance_test_parallel
- Added svem_significance_test_parallel to vignette

[1.1.1]
- Changed default for `predict()` to `debias=FALSE`.
- Changed default to `objective="wAIC"` for `weight_scheme="SVEM"`.
- Added SVEMnet Vignette

[1.0.3]
- First public release.
