Groundwater Profiler Data Quality Control For Reliable Measurements
Groundwater profilers reveal how temperature, conductivity, turbidity, dissolved constituents, and suspended material change with depth. That vertical detail is valuable for tracing contaminant transport, identifying sediment disturbance, locating groundwater–surface-water exchange, and evaluating remediation. It also creates a demanding data-quality problem: a single bad observation can look like a narrow plume, a sharp interface, or a genuine hydrogeological event.
Spike detection and filtering should therefore be treated as measurement-control procedures rather than cosmetic data cleanup. The objective is to remove readings caused by bubbles, fouling, cable motion, optical interference, electrical noise, or settling particles while preserving real gradients and short-lived features.
A sound workflow combines sensor knowledge, deployment records, neighboring observations, and field context. Automated rules can flag suspicious points quickly, but final acceptance should account for profiler speed, sampling interval, sensor response time, borehole conditions, and the behavior of related variables.
Why Profile Data Need Quality Control
A groundwater profiler often collects a vertical series while moving continuously or in fixed increments. This means the data are spatially ordered, even when the instrument records time as its primary coordinate. A sudden turbidity increase over one sample may represent an actual sediment layer, but it may also result from a loose connector, a trapped air bubble, or a brief change in lowering speed.
Optical sensors are particularly sensitive to the physical environment around the measurement path. Bubbles scatter light, wipers can redistribute deposits, and high concentrations of dark or irregular particles can alter the relationship between optical response and suspended solids. Conductivity and temperature probes have their own issues, including equilibration lag, thermal stratification, and contamination from stagnant water.
Quality control protects the interpretation of the complete profile. A false spike can distort averages, trigger an incorrect regression, exaggerate a contaminant boundary, or lead an operator to infer sediment resuspension where none occurred. Conversely, an aggressive filter can erase a thin but important layer. Every correction should therefore leave an audit trail showing what was flagged, changed, retained, or rejected.
Define A Spike Before Removing It
A spike is usually an observation that departs sharply from the local pattern and then returns toward that pattern within a short distance or time. A simple residual rule compares each reading with a local median or moving baseline. For observation (x_i), a robust score can be calculated from the median absolute deviation:
[ z_i = \frac{x_i-\operatorname{median}(x)}{1.4826 \times \operatorname{MAD}(x)} ]
A large absolute score identifies a candidate outlier without allowing a few extreme values to control the baseline. The window should be selected in physical units where possible. For example, a filter might evaluate the nearest 0.1 m of profile rather than an arbitrary number of samples, because sampling density can vary with profiling speed.
A spike should not be defined by amplitude alone. A 5 NTU change may be extraordinary in clear groundwater but insignificant in a highly turbid recovery zone. Use relative change, instrument range, expected noise, and the surrounding hydrogeological setting together. A point that differs from both its preceding and following neighbors is more suspicious than a step change that persists through many samples.
The behavior of companion channels adds useful evidence. If turbidity jumps while temperature, conductivity, and pressure remain stable, an optical artifact becomes more likely, though not certain. If several channels change together at the same depth, the feature may be real. The profiler’s descent and ascent traces can also be compared: a feature repeated at the same elevation deserves more confidence than a feature seen only once.
Detect Suspicious Readings In Context
Begin with basic validation before applying statistical tests. Check timestamps, depth direction, coordinates, sensor status flags, impossible values, duplicated records, missing intervals, and abrupt changes in profiling speed. Confirm that units and calibration coefficients are correct. A conversion error can produce a dataset that appears to contain spikes even though every raw measurement is internally consistent.
Next, inspect the signal visually at several scales. A full-profile plot shows broad zones and interfaces, while a local plot exposes isolated deviations. Plot raw and quality-controlled values separately rather than replacing the original series. Mark pauses, reversals, pump activity, purging, sampler contact, and known disturbances so that apparent anomalies can be linked to field events.
Useful detection tests include a rolling median residual, a Hampel filter, a rate-of-change limit, and a comparison with replicate passes. A rate-of-change test is especially useful when sensor response and profiler velocity are known. It should be based on change per meter, not simply change per record. A slow profile and a fast profile can produce very different sample-to-sample differences for the same physical boundary.
Interpretation also benefits from understanding turbidity as an optical proxy rather than a direct mass measurement. When suspended-solids concentration matters, use a site-specific relationship and document its limits; guidance on correlating turbidity and TSS helps explain why a sensor response should not be treated as a universal solids concentration.
Compare Detection And Filtering Methods
Detection identifies questionable points; filtering changes or excludes them. These are separate decisions. A flagged observation can be retained for review, assigned a quality code, or removed from a derived dataset. Interpolation is appropriate only when the missing or rejected interval is short and the surrounding signal is sufficiently smooth.
| Method | Best use | Main strength | Main risk |
|---|---|---|---|
| Rolling median | Isolated spikes in ordered profiles | Robust against extreme values | Can flatten narrow real features |
| Hampel filter | Local outlier screening | Uses a robust deviation estimate | Sensitive to window size |
| Rate-of-change limit | Motion-related or electrical jumps | Connects QC to profiling speed | May reject genuine sharp interfaces |
| Savitzky–Golay smoothing | Preserving broad shape while reducing noise | Maintains trend and approximate slope | Can create edge artifacts and retain outliers |
| Low-pass filtering | High-frequency noise in stable signals | Useful for continuous records | Blurs thin layers and rapid transitions |
| Replicate-pass comparison | Confirming physical features | Strong field-based validation | Requires repeat measurements |
| Gap flagging and interpolation | Short rejected intervals | Preserves a usable sequence | Can invent values across real boundaries |
A rolling median is often a good first-pass screen because it preserves a central value even when a window contains an extreme observation. However, the window must be smaller than the thinnest feature that the project needs to resolve. If a layer is 0.15 m thick, a 0.5 m smoothing window can remove the very evidence being investigated.
Smoothing should generally follow flagging, not replace it. A filter applied to raw data can spread the influence of a spike into adjacent samples, making the artifact harder to recognize. For reporting, retain at least the raw value, the QC status, the filtered value where applicable, and the method or parameter responsible for the change.
Select Filters That Match Sensor Behavior
The ideal filter depends on the instrument, target variable, and movement through the water column. Turbidity data may contain short optical disturbances, while conductivity can show slower equilibration after the sensor enters a new water mass. Applying the same window and threshold to every channel is convenient but scientifically weak.
For continuous profiling, calculate distance-based windows using depth and movement information. If the profiler advances 0.02 m per sample, a ten-sample median represents 0.2 m. If it advances 0.08 m per sample during another pass, the same setting represents 0.8 m and can erase meaningful structure. A quality-control script should therefore record sample spacing and flag sections where speed exceeds the validated operating range.
Filtering should also account for sensor response time. A probe with a slow response may produce a gradual transition even where the water chemistry changes abruptly. Deconvolution or aggressive edge sharpening is rarely justified without a validated dynamic model. Instead, document the response limitation and interpret the transition at the instrument’s effective resolution.
For turbidity interpretation, site conditions are essential. Fine clay, organic particles, bubbles, and mineral grains can produce different optical responses. Recreational-water applications also demonstrate why clarity measurements require context: turbidity and water clarity are related concepts, but clarity assessments depend on the measurement method, environmental setting, and decision threshold.
Validate Results With Field Evidence
A reliable QC process compares automated flags with deployment notes and independent observations. Review whether suspect points occur during probe entry, a direction change, contact with the bottom, cable vibration, pump operation, or a pause. In a monitoring well, examine whether the screen interval, stagnant water, or development history could explain the pattern.
Replicate profiles are among the strongest validation tools. Run the same path in both directions when practical, or repeat the profile after a brief stabilization period. A real layer should generally appear at a comparable elevation, allowing for movement, mixing, and instrument lag. A one-pass spike that disappears on the return pass deserves a lower confidence rating.
Physical samples can support sensor interpretation. Collect discrete water samples across representative turbidity ranges and preserve the chain of custody. Laboratory total suspended solids, particle-size information, or microscopic inspection can help distinguish a true sediment signal from bubbles or optical fouling. These samples should span the expected concentration range rather than focus only on clean conditions.
Use quality codes that communicate confidence clearly. A practical system may distinguish valid, valid with caution, suspect, rejected, interpolated, and missing. Store the reason for each flag, such as “isolated optical spike,” “rapid movement,” “out-of-range,” or “unverified feature.” This makes the dataset defensible for environmental research, dredging-related investigations, and regulatory review.
Recommended Workflow For Profiler Deployments
The most effective data-quality control begins before fieldwork. Record sensor serial numbers, calibration dates, firmware, deployment depth, profiler speed, sampling interval, cleaning procedures, and environmental conditions. Establish expected ranges and acceptable rates of change using previous surveys, bench tests, or a short pilot deployment.
During acquisition, monitor live traces where possible. A real-time display can reveal bubbles, fouling, cable snagging, or unstable readings before the profile is complete. Keep raw files untouched and create a separate processed file. If a sensor requires a warm-up, flushing period, or settling interval, mark those records rather than silently deleting them.
A consistent workflow can be organized around these practices:
- Apply range, timestamp, depth-order, and missing-data checks before statistical spike detection.
- Use robust local tests with distance-based windows and thresholds tied to sensor precision and field conditions.
- Compare ascent and descent profiles, replicate passes, and related sensor channels before rejecting a feature.
- Preserve raw observations, QC flags, filtered values, parameters, and operator comments in the project archive.
- Review flagged intervals manually when they influence a plume boundary, compliance decision, calibration model, or scientific conclusion.
Automation is valuable when it produces transparent results. A script should report how many values were flagged by each rule, where flags cluster, how much data were interpolated, and whether the final profile differs materially from the raw record. Thresholds should be version-controlled so that a later analyst can reproduce the same processing.
Turn Quality-Controlled Profiles Into Decisions
The purpose of filtering is not to make every profile look smooth. It is to produce measurements that are sufficiently trustworthy for the decision at hand while preserving uncertainty. A remediation study may prioritize contaminant boundaries, a dredging project may focus on suspended-material excursions, and an OEM system may require stable machine-readable outputs. Each use case warrants different tolerances and reporting detail.
When a feature remains after robust screening, replicate validation, and review of field metadata, it should be treated as evidence rather than noise. When uncertainty remains, retain the feature with an appropriate quality code instead of forcing a binary keep-or-delete choice. This approach protects the scientific value of unusual observations and prevents polished graphics from concealing unresolved measurement limitations.
D & A Instruments’ optical sensing and hydrology experience provides useful context for interpreting profiler and water-quality measurements, while Campbell Scientific now provides product-management and support information for the product line. Use that technical foundation alongside documented calibration, disciplined spike detection, and traceable filtering to turn vertical groundwater measurements into defensible evidence. Build the QC workflow into every deployment, review the flagged data before reporting, and preserve the complete processing record with the final profile.