Stock Trading with Machine Learning - Ensembles

0. What is an "Ensemble"?

Imagine you’re trying to guess how many jelly beans are in a huge jar.
If one friend makes a guess, they might be close… or way off. "Ensemble" is simply the fancy term used to describe the phenomenon Aristotle first popularized, known as Wisdom of the Crowd (Wisdom of the crowd - Wikipedia)

Picture a team of five friends:

1. One friend is really good at counting the bottom layer.

2. Another has sharp eyes for the middle layers.

3. A third friend remembers how big similar jars were last year.

4. The fourth notices the shapes and spaces between beans.

5. The last friend just trusts their “gut feeling.”

Each friend whispers their number on a sticky note. You look at all five notes, then average (or “vote”) their answers together. That combined guess is almost always better than any single friend’s guess.

In machine-learning land, every “friend” is a different computer model.
Working together as an ensemble means:

• They each spot patterns the others miss.

• Their mistakes cancel out, so the group answer is steadier.

• The final decision is smarter—like a superhero team instead of a lone hero.

So an ensemble is simply a group of models teaming up to solve the same problem, just like friends teaming up to crack the jelly-bean jar.

1. Why an Ensemble?

Markets switch regimes fast: some weeks reward momentum, others mean-reversion, and sometimes macro factors overwhelm both. No single model class can track every pattern, so we train a diverse model zoo and let validation data decide the mix. The code wires up five families:

All five are trained separately, then we optimize convex weights so the validation AUC of the weighted blend is maximized (a simple but effective stacker) raw.githubusercontent.com.

2. Deep-Learning Micro-details

The GPU block sets mixed-precision (torch.cuda.amp) to squeeze larger batches into memory, uses gradient-accumulation so even 4 GB cards can train, and clips gradients to stop explosive updates. Early-stopping on validation AUC prevents over-fit and saves electricity. When sequences are too short (e.g., freshly-listed stocks) the trainer gracefully skips the deep nets and relies on tree models instead raw.githubusercontent.com.

3. Ensemble Governance

• Feature alignment – before inference we create missing columns and robust-scale everything so trees & nets see identical distributions.

• Per-model health checks – any model that returns NaNs is zero-weighted.

• Persistence – tree boosters are saved as JSON/TXT; neural nets as .pth; plus a single ensemble_config.pkl keeps weightings and scalers. Loading is one command, so live-trading nodes never retrain unintentionally raw.githubusercontent.com.

Execution & Costs

Turning paper alpha into net P&L

4. The Four-Part Cost Stack

1. Commission – per-share fee with min/max guardrails.

2. Slippage – a base 5 bp spread, penalised further at the open (×1.5) or close (×1.3).

3. Market-impact – square-root model: ΔP = k·price·√(size / ADV) with a 10 % ADV cap.

4. Limit-order behaviour – optional: 85 % fill probability that improves when your limit is closer to the day’s low (buys) or high (sells) raw.githubusercontent.com.

These are configurable in a @dataclass so research and live trading share the exact cost engine.

5. Vectorized & Real-Time Modes

For nightly backtests, we run numpy-vectorized entry/exit simulations across hundreds of symbols in milliseconds. During live hours the same class handles a single order and spits out execution price, slippage dollars, and commission so the strategy can decide whether the edge still beats the cost raw.githubusercontent.com.

6. Order-Book & Spread Model (Advanced)

When realism matters (e.g., micro-caps) we drop into AdvancedExecutionSimulator:

• Dynamic spread widens with intraday volatility and shrinks with higher tape volume.

• Bid-ask generator lets you back-test queue jumpers vs. passive posting.

All in pure Python—no vendor lock-in, and easy to extend to dark-pool or VWAP algos later