A Sensor Data Quality and Validation Framework For Large-Scale Industrial Monitoring: Detecting Faulty, Drifting, And Missing Telemetry Before It Reaches Analytics

Abstract

Bad sensor data rarely fails loudly. A drifting transmitter or a silently dropped signal keeps producing plausible values, so dashboards, alarms, and models built on that data degrade without triggering any obvious fault. Analytics teams continue to trust numbers that no longer reflect physical reality, and the delay is discovered only when an operational event forces a retrospective look at the trend. This paper proposes a sensor data quality and validation framework that screens industrial telemetry for faults, drift, and missingness before it reaches downstream analytics, rather than cleaning it afterwards. The framework is specified as an ordered pipeline: structural validation at ingestion, physical plausibility bounds derived from instrument class, drift detection against each tag's own rolling baseline rather than a fixed limit, staleness and value-repetition detection against expected reporting frequency, and a quality status assigned per reading that gates what is allowed to inform a dashboard, model, or alert. Two discriminations carry the design. Sensor fault is separated from genuine process change by testing whether an anomaly is isolated to one tag or corroborated across physically related tags. Gradual drift is separated from abrupt fault by persistence and pattern shape rather than instantaneous magnitude. The framework is a proposed design. It has not been implemented or evaluated against live or historical data, and no detection rate, false-alarm rate, or detection latency is reported. Section 7 specifies the evaluation that would be required to establish them.