Online classifiers: non-stationary state estimation and drift-adaptive architectures¶

These tools analyze a robot's real-time joint signals to determine its operational state—such as moving freely, making intentional contact, or experiencing a collision. Rather than failing when external conditions change, these models continuously adapt their internal statistics or deploy background copies to keep the state estimate usable.

A robot has to tell, sample by sample, which state it is in: free motion, contact, collision. It has to do this from its own joint signals, and it has to keep doing it right when the robot's world changes — a different payload, a worn gripper, a slightly different surface.

OnlineGaussianNB and OnlineSoftmaxRegression learn one sample at a time. DriftAdaptiveClassifier wraps either of them with a drift detector on its own prediction loss and swaps in a fresh copy when the old one goes stale.

1. The data¶

6,000 samples, 3 states from a Markov chain (stay probability 0.95, mean dwell 20 samples), 4 joint features with unit noise:

free: (0, 0, 0, 0)
contact: (2, 2, 0, 0)
collision: (4, 4, 4, 4)

From sample 3000 the contact load moves to the other two joints: contact becomes (0, 0, 2, 2). The classifier was trained on the old signature.

2. Gaussian naive Bayes, one sample at a time¶

For each class and each feature, the classifier keeps a running mean and variance, updated by Welford's algorithm:

n ← n + 1
δ = v − m
m ← m + δ / n
M2 ← M2 + δ (v − m)
variance = M2 / n

Hand case. Three values 2, 4, 6:

v = 2: n = 1, m = 2, M2 = 0
v = 4: n = 2, m = 3, M2 = 2
v = 6: n = 3, m = 4, M2 = 8

Mean 4, variance 8 / 3 = 2.667.

The classifier picks the class with the highest

log prior + Σ log N(x_i; m_i, var_i)

With alpha = 0, the statistics equal the batch MLE of a Gaussian NB on the observed samples; the wrapper matches a batch Gaussian NB to numerical precision (max difference 1.8e-15 on 500 samples). With alpha > 0, the statistics forget exponentially: old counts are multiplied by (1 − α) before the new sample is added, so the classifier adapts when the class-conditional distributions change.

import numpy as np
from dense_armor.utility.learn.online_classifiers import OnlineGaussianNB

rng = np.random.default_rng(42)
s = np.zeros(6000, int)
for t in range(1, 6000):
    s[t] = s[t - 1] if rng.random() < 0.95 else (s[t - 1] + rng.integers(1, 3)) % 3
mu = np.array([[0, 0, 0, 0], [2, 2, 0, 0], [4, 4, 4, 4]], float)
X = mu[s] + rng.standard_normal((6000, 4))
X[3000:][s[3000:] == 1] += [-2, -2, 2, 2]
clf, hit = OnlineGaussianNB(), []
for x, y in zip(X, s):
    f = dict(zip("abcd", x)); hit.append(clf.predict_one(f) == y); clf.learn_one(f, y)
print(np.mean(hit[500:3000]).round(3), np.mean(hit[3000:3200]).round(3), np.mean(hit[5000:5900]).round(3))

0.952 0.48 0.879

Prequential accuracy (predict first, then learn) before the change, right after it, and late.

3. What happens when the signature changes¶

The contact state moves from joints 0–1 to joints 2–3 at sample 3000. A classifier without forgetting keeps the old contact mean and gets every post-change contact sample wrong. Prequential accuracy over [500, 3000) (pre), [3000, 3200) (post), and [5000, 5900) (late):

Classifier	pre	post	late	recovery
`OnlineGaussianNB(alpha=0)`	0.952	0.480	0.879	617
`OnlineGaussianNB(alpha=0.01)`	0.952	0.790	0.941	59
`OnlineGaussianNB(alpha=0.1)`	0.950	0.925	0.913	0
`OnlineSoftmaxRegression(eta=0.1, l2=1e-4)`	0.934	0.565	0.919	213
`DriftAdaptiveClassifier(OnlineGaussianNB(alpha=0))`	0.952	0.480	0.931	145
`DriftAdaptiveClassifier(OnlineSoftmaxRegression)`	0.912	0.775	0.904	38

"Recovery" is the number of samples after 3000 until the 200-sample accuracy is back within 0.05 of pre.

Without forgetting, Gaussian NB goes from 0.952 to 0.480 at the change and needs 617 samples to recover. A fixed forgetting factor reacts faster if you already know how fast the world changes — alpha = 0.1 never loses accuracy, at the cost of a noisier late phase (0.913 instead of 0.941).

The drift-adaptive wrapper is the third option: it does not need a forgetting factor chosen in advance.

4. The drift-adaptive wrapper¶

DriftAdaptiveClassifier wraps any classifier with a drift detector on the classifier's own prediction loss. Per sample:

Predict, then compute the log-loss −log p(y).
Feed the 50-sample rolling mean of the log-loss to the detector.
Learn.
If the detector fires, start a background copy trained only on new samples.
When the background's accuracy over the last 50 samples beats the main classifier's over the same window, promote the background to main.

The 50-sample rolling mean is not optional in practice: the raw log-loss is near zero on most samples, so its robust scale (MAD) is tiny and a CUSUM fed with the raw value fires on almost every error — hundreds of alarms in 6,000 samples. The rolling mean gives the detector a stable scale.

import numpy as np
from dense_armor.utility.learn.online_classifiers import OnlineGaussianNB, DriftAdaptiveClassifier
from dense_armor.utility.drift.detector import CUSUMDriftDetector

rng = np.random.default_rng(42)
s = np.zeros(6000, int)
for t in range(1, 6000):
    s[t] = s[t - 1] if rng.random() < 0.95 else (s[t - 1] + rng.integers(1, 3)) % 3
mu = np.array([[0, 0, 0, 0], [2, 2, 0, 0], [4, 4, 4, 4]], float)
X = mu[s] + rng.standard_normal((6000, 4))
X[3000:][s[3000:] == 1] += [-2, -2, 2, 2]
det = CUSUMDriftDetector(reference="fixed", radius=20, ref_mult=5, two_sided=False)
clf, hit = DriftAdaptiveClassifier(OnlineGaussianNB(), det, window=50, smooth_window=50), []
for x, y in zip(X, s):
    f = dict(zip("abcd", x)); hit.append(clf.predict_one(f) == y); clf.learn_one(f, y)
print(np.mean(hit[500:3000]).round(3), np.mean(hit[3000:3200]).round(3), np.mean(hit[5000:5900]).round(3))

0.952 0.48 0.931

On the running data the wrapper leaves its pre-drift accuracy untouched (0.952), fires about 55 samples after the change, and recovers in 145 samples. It reaches 0.931 in the late phase — better than the no-forgetting baseline (0.879) and close to the tuned alpha = 0.1 (0.913).

5. The softmax regression alternative¶

OnlineSoftmaxRegression is a multinomial logistic regression with AdaGrad. It uses the same interface and is often better when the class boundaries are not Gaussian.

The formula, symbol by symbol¶

Probabilities are softmax(W z) where z = [features..., 1]. The gradient for class c on sample (z, y) is

(p_c − [c = y]) · z

with p_c the softmax output for class c and [c = y] equal to 1 when c is the true label and 0 otherwise. The AdaGrad step is η / √G where G is the running sum of squared gradients, updated as G ← G + g² before the parameter step. η is the base learning rate (default 0.1); l2 adds a small penalty.

Classes are added on the fly when a new label arrives.

6. Cost¶

Per sample, on one CPU core:

classifier	µs per sample
`OnlineGaussianNB`	10.5
`OnlineSoftmaxRegression`	36.8
`DriftAdaptiveClassifier(OnlineGaussianNB)`	75.8

At 100 Hz the loop has 10 ms per cycle: all three fit with three orders of magnitude to spare. The wrapper is 7× the bare NB because it runs a background copy in parallel after each drift alarm.

API reference¶

online_classifiers ¶

Online classifiers for robot states with drift adaptation.

Three classifiers for one-sample-at-a-time learning on robot joint signals: an online Gaussian naive Bayes, an online softmax regression, and a DriftAdaptiveClassifier wrapper that swaps in a background copy when a drift detector fires. All return calibrated-ready probabilities, so OnlinePlattScaling (binary case) can wrap them.

OnlineGaussianNB ¶

OnlineGaussianNB(
    alpha: float = 0.0,
    var_smoothing: float = 0.0,
    eps: float = 1e-12,
)

Bases: Classifier

Gaussian naive Bayes updated one sample at a time.

Running mean and variance per class and feature, updated by Welford's algorithm. With alpha = 0 the statistics equal the batch MLE of a Gaussian NB on the observed samples, so predict_proba_one matches a batch Gaussian NB trained on the same data (to numerical precision). With alpha > 0 the statistics forget exponentially at rate alpha, so the classifier adapts when the class-conditional distributions drift.

Parameters¶

alpha Forgetting factor in [0, 1). 0 is batch behaviour, larger values weight recent samples more. var_smoothing Additive smoothing on the variance. Default 0 so alpha = 0 matches the batch MLE exactly. eps Floor on the variance to avoid log(0) when a class has a single sample.

Examples¶

from dense_armor.utility.learn.online_classifiers import OnlineGaussianNB nb = OnlineGaussianNB() for x, y in [({"a": 0.0, "b": 0.0}, 0), ... ({"a": 0.1, "b": 0.1}, 0), ... ({"a": 1.0, "b": 1.0}, 1), ... ({"a": 1.1, "b": 1.1}, 1)]: ... _ = nb.learn_one(x, y) nb.predict_one({"a": 0.05, "b": 0.05}) 0 nb.predict_one({"a": 1.05, "b": 1.05}) 1

References¶

Welford, B. P. (1962). Note on a method for calculating corrected sums of squares and products. Technometrics 4(3), 419-420.

Source code in dense_armor/utility/learn/online_classifiers.py

def __init__(self, alpha: float = 0.0, var_smoothing: float = 0.0,
             eps: float = 1e-12):
    self.alpha = alpha
    self.var_smoothing = var_smoothing
    self.eps = eps
    self._keys: Optional[list] = None
    self._n: dict = {}
    self._mu: dict = {}
    self._m2: dict = {}

OnlineSoftmaxRegression ¶

OnlineSoftmaxRegression(
    eta: float = 0.1,
    l2: float = 0.0,
    adaptive: bool = True,
    eps: float = 1e-08,
)

Bases: Classifier

Multinomial logistic regression with AdaGrad.

Online softmax over W^T z where z = [features..., 1]. Classes are added on the fly when a new label arrives. Optional L2 penalty, optional AdaGrad preconditioning of the learning rate.

Parameters¶

eta Learning rate. The adaptive step is eta / (sqrt(G) + eps) where G is the running sum of squared gradients. Default 0.1. l2 L2 penalty added to the loss gradient. Default 0. adaptive If True, AdaGrad preconditioning. If False, plain SGD with step eta. eps Numerical floor in the AdaGrad denominator.

Examples¶

from dense_armor.utility.learn.online_classifiers import OnlineSoftmaxRegression sr = OnlineSoftmaxRegression(eta=0.5) for _ in range(20): ... _ = sr.learn_one({"a": 0.0, "b": 0.0}, 0) ... _ = sr.learn_one({"a": 1.0, "b": 1.0}, 1) sr.predict_one({"a": 0.0, "b": 0.0}) 0 sr.predict_one({"a": 1.0, "b": 1.0}) 1

References¶

Duchi, J., Hazan, E., Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12, 2121-2159.

Source code in dense_armor/utility/learn/online_classifiers.py

def __init__(self, eta: float = 0.1, l2: float = 0.0,
             adaptive: bool = True, eps: float = 1e-8):
    self.eta = eta
    self.l2 = l2
    self.adaptive = adaptive
    self.eps = eps
    self._keys: Optional[list] = None
    self._W: dict = {}
    self._G: dict = {}
    self._classes: list = []

DriftAdaptiveClassifier ¶

DriftAdaptiveClassifier(
    classifier,
    detector,
    window: int = 50,
    smooth_window: int = 50,
)

Bases: Classifier

Classifier with a background copy swapped in on drift detection.

Wraps any Classifier with a drift detector. Per sample: predict, compute the log-loss -log(p_correct), feed the (smoothed) log-loss to the detector, learn, and if the detector fires start a background copy trained only on new samples. Main and background are both scored prequentially (each predicts a sample before learning it); when the background's accuracy over the last window samples beats the main classifier's over the same samples, the background is promoted to main.

Parameters¶

classifier The wrapped classifier; a fresh copy is trained as the background. detector A drift detector with update(x) and drift_detected, e.g. CUSUMDriftDetector(reference="fixed", radius=20, ref_mult=5, two_sided=False): one-sided, because only a rise of the loss means the classifier went stale. window Size of the accuracy sliding window used to compare main and background. smooth_window Size of the rolling mean applied to the log-loss before feeding it to the detector. The raw log-loss is near zero on most samples, so its robust scale is tiny and every single error looks like a shift; a rolling mean over ~50 samples gives the detector a stable scale.

Examples¶

from dense_armor.utility.learn.online_classifiers import ( ... OnlineGaussianNB, DriftAdaptiveClassifier, ... ) from dense_armor.utility.drift.detector import CUSUMDriftDetector clf = DriftAdaptiveClassifier( ... OnlineGaussianNB(alpha=0.01), ... CUSUMDriftDetector(reference="adaptive", radius=10, ref_mult=3), ... smooth_window=20, ... ) for _ in range(200): ... _ = clf.learn_one({"a": 0.0}, 0) ... _ = clf.learn_one({"a": 1.0}, 1) clf.predict_one({"a": 0.05}) 0 clf.predict_one({"a": 0.95}) 1

References¶

Gama, J., Medas, P., Castillo, G., Rodrigues, P. (2004). Learning with drift detection. In SBIA.

Source code in dense_armor/utility/learn/online_classifiers.py

def __init__(self, classifier, detector, window: int = 50,
             smooth_window: int = 50):
    self.classifier = classifier
    self.detector = detector
    self.window = window
    self.smooth_window = smooth_window
    self._main = deepcopy(classifier)
    self._bg = None
    self._main_hits = deque(maxlen=window)
    self._bg_hits = deque(maxlen=window)
    self._err_buf = deque(maxlen=max(1, smooth_window))

Details¶

The Gaussian NB variance uses the exponentially weighted Welford update (Welford 1962); OnlineSoftmaxRegression uses AdaGrad steps (Duchi, Hazan and Singer, JMLR 2011); the background-learner swap follows Gama et al., Learning with drift detection (SBIA 2004).

All three classifiers return probabilities, so calibration (OnlinePlattScaling) can wrap them in the binary case. The two together — adaptive learning and honest probabilities — are what a control policy needs to be able to trust the output.