Direct answer: before training or buying a pipeline model, reconstruct CRM state exactly as it could have been known at each forecast cutoff. For every opportunity and feature, retain when the fact became effective, when the company observed it, and when it became available to scoring. Distinguish late arrival, correction, and restatement. Then run rolling, time-ordered backtests against a simple baseline. If the model sees a stage, amount, close date, product, or outcome updated after the cutoff, it is not forecasting revenue. It is reading the future.
This failure is easy to miss because a CRM is optimized to operate the current record. A rep moves a close date in November, corrects amount in December, and closes the deal in January. A February extract brings today’s values into every training row. The table contains old opportunities but not old knowledge. Offline accuracy and expected ROI rise; live usefulness collapses.
Old records are not historical state
Created and closed timestamps do not reconstruct what a forecast knew. Stage, probability, owner, amount, products, expected date, and forecast category may have been overwritten. To answer “what was knowable on June 30?” each field must be resolved from changes valid and available by that instant—not from the most recent value now in the application.
Removing closed-won is not enough. Loss reasons entered later, finalized line items copied backward, activities logged late, a renewal-owner change, post-signature amount reconciliation, and segments recalculated with later data can all leak. Apply an operational test: did this information exist, was it permitted, and would it have reached the live scoring path at decision time?
Write the Pipeline As-Of State Contract
- Decision and cutoff — forecast target, entity, horizon, timezone, cadence, and exact scoring instant.
- Identity — stable opportunity_id and links to account, contact, product, and commercial versions.
- Effective time — when the fact became true in the business.
- Observed time — when a person or system recorded it.
- Available time — when the validated feature reached the scoring process.
- Value and provenance — stage, amount, close date, owner, product, activity, source, and transformations.
- Change type — new evidence, late arrival, error correction, reclassification, restatement, or deletion.
- Outcome maturity — when the final result is stable enough to label and what reopens history.
- Reconstruction — ordering, tie-break, version, timezone, retention, backfill, unknowns, and forbidden fields.
The three times prevent a subtle shortcut. A buyer meeting can happen Monday, be logged Wednesday, and reach the warehouse Thursday. A Tuesday forecast did not have it. Effective time can matter later, but available time controls the historical prediction. A single updated_at collapses the distinction and gives the backtest data production never had.
What CRM platforms provide—and what remains your responsibility
Salesforce Stage History records changes to Amount, Probability, Stage, and Close Date, while field history can cover selected fields. HubSpot can export historical property values, update timestamps, and the changing user or tool. Dataverse can audit record changes, previous values, users, and timestamps when environment, table, and column auditing is configured. These are useful inputs; coverage, start date, retention, delay, permission, and eligible fields differ.
None automatically produces a point-in-time-correct training set. BigQuery documentation describes retrieving feature values as they existed at a cutoff to avoid leakage. Your team still owns entity, cutoff, late-event policy, snapshot frequency, transformations, and mature label. If tracking began recently, disclose the usable window and narrow the first use case instead of manufacturing a backfill.
Example: the $1 million June pipeline
On June 30, five opportunities totaled $1 million. By August, two had closed won; reps corrected products, increased amounts, and moved dates. A September export attaches those final values to June. A model learns that product mix, amount, and close date predicted the wins because it received versions created after the outcome.
The June snapshot should use the amounts and products knowable then, preserve missing fields as unknown, and make later corrections available only from August. The label may be the outcome matured 90 days later, but it never flows back into features. For product and period forecasting, first use the grain in https://makinai.co/insights/en/opportunity-product-market-period-revenue-grain-ai-forecast; temporal integrity at the wrong grain still answers the wrong question.
Separate rules, AI, and accountable judgment
- Rules enforce cutoff, ordering, keys, active version, availability, maturity, forbidden fields, and reconciliation.
- Data teams retain raw events, snapshots, versioned transformations, tests, and replay.
- AI/ML estimates probability, amount, date, or priority only from released features and can flag anomalies.
- RevOps defines field meaning, legitimate changes, exceptions, and use in the revenue cadence.
- Sales confirms commercial state without rewriting history to improve the model.
- Marketing defines which investment decision depends on pipeline and which cohorts must be compared.
- Finance validates realized value and maturity; engineering runs connectors; leadership owns the release gate.
Do not ask an LLM to silently reconstruct missing history. It may summarize a source note with a citation, classify a change type, or assist review. Unknown must remain unknown. Amount, stage, date, outcome, eligibility, and availability time need reproducible logic and provenance. Every delivered forecast should retain its model version, cutoff, and feature snapshot.
A six-week shadow pilot
- Week 1 — choose one decision, one motion, and six to ten features; fix entity, cutoff, horizon, and baseline.
- Week 2 — audit 50–100 opportunities for overwritten fields, tracking start, retention, late events, and real coverage.
- Week 3 — implement the Contract, an event table, and daily or weekly snapshots; reconstruct one reliable period.
- Week 4 — run rolling-origin evaluation: train on past periods, score the next period, and advance without shuffling time.
- Week 5 — compare baseline, model, cohort error, and decisions that would have changed; run current data in shadow mode.
- Week 6 — review false signals with Sales, RevOps, Marketing, Data, and Finance; release reversible alerts or keep the baseline.
The release gate requires zero post-cutoff features in an audited sample, known coverage and latency, deterministic replay, mature outcomes, a documented baseline, acceptable calibration by segment, versioning, exception ownership, and rollback. Google recommends evaluating on later data and logging serving-time features; these practices bring offline tests closer to production, but they do not choose the business trade-off.
Measure integrity before celebrating model accuracy
- Reproducibility — the same cutoff, code, and sources reproduce the same snapshot.
- Leakage — post-cutoff features or values found by sampling and automated tests.
- Availability — share of features actually present at scoring time by source and segment.
- Latency — gaps among effective, observed, and available time; late events by cause.
- Calibration — among opportunities scored at 30%, does the mature rate align by cohort?
- Business error — amount and date error, excess or missing capacity, and avoidable decision cost.
- Incremental value — improvement over the baseline and current process, net of review and operation.
- Countermetrics — a smoother but less stable forecast, underweighted valid opportunities, and incentives to game fields.
Cost rises with the number of CRMs, fields, objects, history start and retention, snapshot cadence, late events, backfill, line items, currencies, timezones, schema changes, volume, maturity, and decision criticality. A versioned weekly export can be the right first implementation. Streaming earns its cost only when faster data changes a decision often enough to create material value.
Use https://makinai.co/insights/en/how-much-historical-data-first-ai-marketing-workflow to size minimum evidence, https://makinai.co/insights/en/ai-opportunity-stage-evidence-crm-pipeline to define valid progression, and https://makinai.co/insights/en/ai-marketing-rolling-reforecast-demand-budget to connect forecasts to scenarios and decision rights. If the backtest looks excellent but no one can reproduce what the CRM knew on forecast day, bring MAKINAI at https://makinai.co/services/en/data-content-intelligence-systems an anonymized history sample, the current report, and three cutoffs. We can scope reconstruction and the pilot before you invest in a model that predicts a corrected past.
The sources document ML principles and history or audit capabilities; they do not prove that your CRM captured every state. The Contract, pilot, and gates are editorial recommendations. Better temporal integrity can make error more honest; it does not guarantee revenue, establish causality, or replace financial validation.