Example: MHKiT-MATLAB Quality Control Module
The following example runs a simple quality control analysis on wave elevation data using the MHKiT QC module. The data file used in this example is stored in the \MHKiT\examples\data directory.
Import Data
Load the data into a table. The data includes several issues, including timestamps that are out of order, corrupt data with values of -999, data outside expected range, and stagant data.
data=readtable('../examples/data/qc/wave_elevation_data.csv');
disp(data)
Time probe1 probe2 probe3
______ ______ ______ ______
10 24.48 28.27 1.3
10.002 34.48 40.27 -8.7
10.004 30.48 38.27 -13.7
10.006 12.48 24.27 -32.7
10.008 13.48 22.27 -21.7
10.01 7.48 15.27 -13.7
10.012 4.48 13.27 -19.7
10.014 -9.52 -0.73 -11.7
10.016 8.48 19.27 -16.7
10.018 -4.52 8.27 -34.7
10.02 16.48 27.27 5.3
10.022 -13.52 -0.73 -36.7
10.024 -13.52 0.27 -39.7
10.026 -15.52 -3.73 -32.7
10.028 -22.53 -17.73 -23.7
10.03 -22.52 -16.73 16.3
10.032 -30.52 -23.73 1.3
10.034 -30.52 -16.73 11.3
10.036 -36.52 -24.73 13.3
10.038 -26.52 -13.73 23.3
10.04 -28.52 -14.73 25.3
10.046 -14.52 -11.73 33.3
10.048 -10.52 -1.73 36.3
10.05 -19.52 -10.73 28.3
10.052 -3.52 7.27 39.3
10.054 7.48 20.27 26.3
10.056 -2.52 14.27 11.3
10.058 4.48 19.27 13.3
10.06 2.48 17.27 16.3
10.062 9.48 22.27 2.3
10.064 17.48 23.27 19.3
10.066 16.48 27.27 26.3
10.068 33.48 42.27 5.3
10.07 28.48 36.27 -6.7
10.072 20.48 36.27 2.3
10.074 17.48 30.27 -7.7
10.076 11.48 23.27 -14.7
10.078 16.48 24.27 2.3
10.08 4.48 12.27 -8.7
10.082 15.48 22.27 -9.7
10.084 9.48 21.27 -15.7
10.086 -2.52 7.27 -14.7
10.088 -7.52 9.27 -31.7
10.09 3.48 17.27 -40.7
10.092 -2.52 8.27 -46.7
10.094 -13.52 -2.73 -19.7
10.096 -4.52 3.27 -13.7
10.098 -14.52 -7.73 NaN
10.1 -18.52 -11.73 NaN
10.102 -40.52 -22.73 NaN
10.104 -33.52 -18.73 NaN
10.106 -40.52 -23.73 NaN
10.108 -34.52 -18.73 NaN
10.11 -999 -27.73 NaN
10.112 -999 -19.73 NaN
10.114 -999 -2.73 NaN
10.116 -999 -21.73 46.3
10.118 -999 -26.73 21.3
10.12 -999 -14.73 31.3
10.122 -999 -13.73 30.3
10.124 -999 -16.73 15.3
10.126 -999 -15.73 34.3
10.128 -999 -18.73 47.3
10.13 -999 -0.73 31.3
10.132 -999 -1.73 48.3
10.134 -999 4.27 44.3
10.136 4.48 13.27 22.3
10.138 15.48 20.27 10.3
10.14 4.48 20.27 13.3
10.142 18.48 27.27 21.3
10.144 23.48 34.27 -1.7
10.146 20.48 26.27 -7.7
10.148 13.48 21.27 17.3
10.15 27.48 36.27 -1.7
10.152 20.48 31.27 -1.7
10.154 16.48 28.27 -17.7
10.156 12.48 27.27 -27.7
10.158 24.48 31.27 -35.7
10.16 10.48 20.27 -15.7
10.162 10.48 17.27 -25.7
10.164 11.48 16.27 -3.7
10.166 3.48 7.27 -25.7
10.168 -14.52 -7.73 -24.7
10.17 -5.52 6.27 -24.7
10.172 -7.52 1.27 -24.7
10.174 -13.52 -2.73 -42.7
10.176 -13.52 -3.73 -19.7
10.178 -20.52 -11.73 -8.7
10.18 -11.52 -10.73 -21.7
10.182 -27.52 -22.73 2.3
10.184 -28.52 -21.73 17.3
10.186 -29.52 -17.73 11.3
10.188 -28.52 -17.73 0.3
10.19 -41.52 -30.73 30.3
10.192 -35.52 -26.73 18.3
10.194 -33.52 -26.73 10.3
10.196 -20.52 -17.73 33.3
10.198 -17.52 -11.73 29.3
10.2 -10.52 -8.73 49.3
10.202 4.48 13.27 28.3
10.204 2.48 12.27 25.3
10.206 0.48 14.27 21.3
10.208 -7.52 3.27 NaN
10.21 -6.52 1.27 NaN
10.212 16.48 19.27 18.3
10.214 9.48 16.27 19.3
10.216 12.48 19.27 26.3
10.218 24.48 31.27 14.3
10.22 20.48 31.27 -3.7
10.222 18.48 31.27 -0.7
10.224 15.48 27.27 13.3
10.226 30.48 34.27 -11.7
10.228 16.48 23.27 -15.7
10.288 28.48 23.27 5.3
10.23 26.48 23.27 -0.7
10.232 23.48 28.27 8.3
10.234 13.48 16.27 -36.7
10.236 1.48 8.27 -17.7
10.238 16.48 23.27 -24.7
10.24 -7.52 4.27 -50.7
10.242 -12.52 -2.73 -21.7
10.244 -2.52 1.27 -4.7
10.246 -6.52 -9.73 -19.7
10.248 -17.52 -16.73 -21.7
10.25 -12.52 -13.73 1.3
10.252 -22.52 -17.73 0.3
10.254 -24.52 -21.73 -20.7
10.256 -38.52 -30.73 21.3
10.258 -34.52 -24.73 10.3
10.26 -22.52 -18.73 30.3
10.262 -24.52 -24.73 21.3
10.264 -25.52 -23.73 42.3
10.266 -18.52 -16.73 15.3
10.268 -20.52 -19.73 17.3
10.27 -29.52 -26.73 23.3
10.272 -21.52 -18.73 26.3
10.274 -3.52 5.27 30.3
10.276 4.48 7.27 34.3
10.278 5.48 8.27 45.3
10.28 7.48 7.27 44.3
10.282 26.48 24.27 22.3
10.284 17.48 16.27 9.3
10.286 23.48 24.27 5.3
10.29 33.48 36.27 -9.7
10.292 28.48 35.27 -0.7
10.294 31.48 30.27 1.3
10.296 19.48 32.27 -4.7
10.298 18.48 20.27 -12.7
10.3 7.48 15.27 -14.7
10.302 8.48 9.27 -25.7
10.304 10.48 8.27 -24.7
10.306 12.48 13.27 -25.7
10.308 8.48 16.27 -39.7
10.31 4.48 14.27 -34.7
10.312 17.48 11.27 -23.7
10.314 1.48 16.27 -23.7
10.316 -2.52 -6.73 -28.7
10.318 -9.52 -1.73 -19.7
10.32 -13.52 -3.73 -28.7
10.322 -17.52 -14.73 -11.7
10.324 -19.52 -11.73 -11.7
10.326 -19.52 -16.73 -14.7
10.328 -13.52 -15.73 -4.7
10.33 -11.52 -13.73 17.3
10.332 -19.52 -19.73 13.3
10.334 -14.52 -15.73 18.3
10.336 -20.52 -18.73 23.3
10.338 -25.52 -22.73 10.3
10.34 -24.52 -18.73 10.3
10.34 -24.52 -18.73 10.3
10.342 -21.52 -19.73 -3.7
10.344 -34.52 -29.73 16.3
10.346 -21.52 -22.73 33.3
10.348 0.48 -3.73 39.3
10.35 -1.52 -1.73 30.3
10.352 4.48 3.27 19.3
10.354 8.48 12.27 32.3
10.356 5.48 13.27 17.3
10.358 15.48 17.27 -0.7
10.36 13.48 16.27 2.3
10.362 18.48 14.27 23.3
10.364 24.48 21.27 -0.7
10.366 28.48 31.27 -4.7
10.368 20.48 23.27 5.3
10.37 36.48 30.27 -13.7
10.372 20.48 29.27 -6.7
10.374 19.48 22.27 -34.7
10.376 11.48 12.27 -14.7
10.378 23.48 22.27 -17.7
10.38 13.48 9.27 -22.7
10.382 8.48 5.27 -34.7
10.384 15.48 9.27 -23.7
10.386 5.48 1.27 -40.7
10.388 -3.52 -3.73 -38.7
10.39 -13.52 -13.73 -31.7
10.392 0.48 -0.73 -21.7
10.394 -2.52 -3.73 -24.7
10.396 -14.52 -17.73 -29.7
10.398 -12.52 -18.73 -6.7
10.4 -10.52 -15.73 -14.7
10.402 -30.52 -15.73 -8.7
10.404 -28.52 -15.73 5.3
10.406 -24.52 -15.73 11.3
10.408 -29.52 -15.73 0.3
10.41 -26.52 -15.73 21.3
10.412 -28.52 -15.73 40.3
10.414 -25.52 -15.73 36.3
10.416 -12.52 -15.73 24.3
10.418 -20.52 -15.73 1.3
10.42 -25.52 -15.73 9.3
10.422 4.48 -15.73 8.3
10.424 -1.52 -15.73 -4.7
10.426 -6.52 -15.73 21.3
10.428 21.48 -15.73 9.3
10.43 24.48 -15.73 22.3
10.432 20.48 -15.73 26.3
10.434 12.48 -15.73 5.3
10.436 20.48 -15.73 3.3
10.438 28.48 -15.73 -0.7
10.44 32.48 -15.73 -4.7
10.442 27.48 -15.73 -0.7
10.444 36.48 -15.73 -9.7
10.446 43.48 -15.73 -23.7
10.448 26.48 -15.73 1.3
10.45 24.48 -15.73 -21.7
10.452 17.48 -15.73 -13.7
10.454 16.48 -15.73 -39.7
10.456 1.48 -15.73 -15.7
10.458 11.48 -15.73 -23.7
10.46 19.48 -15.73 -19.7
10.462 7.48 -15.73 -36.7
10.464 10.48 -15.73 -42.7
10.466 2.48 -15.73 -22.7
10.468 -3.52 -15.73 -58.7
10.47 -12.52 -15.73 -42.7
10.472 -10.52 -15.73 -29.7
10.474 -21.52 -15.73 -16.7
10.476 -25.52 -15.73 -11.7
10.478 -32.52 -15.73 9.3
10.48 -21.52 -15.73 11.3
10.482 -17.52 -15.73 2.3
10.484 -21.52 -15.73 13.3
10.486 -22.52 -15.73 6.3
10.488 -11.52 -15.73 19.3
10.49 -9.52 -15.73 14.3
10.492 -29.52 -15.73 10.3
10.494 -17.52 -15.73 15.3
10.496 -4.52 -15.73 30.3
10.498 -6.52 -15.73 18.3
10.5 -6.52 -15.73 22.3
10.502 17.48 -15.73 10.3
10.504 18.48 -15.73 19.3
10.506 9.48 -15.73 -9.7
10.508 18.48 -15.73 11.3
10.51 16.48 -15.73 -1.7
10.512 28.48 -15.73 -23.7
10.514 17.48 -15.73 -8.7
10.516 25.48 -15.73 -11.7
10.518 25.48 -15.73 -15.7
10.52 31.48 -15.73 -13.7
10.522 15.48 -15.73 -15.7
10.524 11.48 -15.73 -22.7
10.526 21.48 -15.73 -15.7
10.528 17.48 -15.73 -22.7
10.53 10.48 -15.73 -23.7
10.532 9.48 -15.73 -29.7
10.534 17.48 -15.73 -30.7
10.536 0.48 -15.73 -39.7
10.538 -4.52 -15.73 -38.7
10.54 0.48 -15.73 -24.7
10.542 -10.52 -15.73 -34.7
10.544 -7.52 -15.73 -31.7
10.546 -12.52 -18.73 9.3
10.548 -10.52 -18.73 5.3
10.55 -21.52 -27.73 -16.7
10.552 -30.52 -33.73 1.3
10.554 -34.52 -34.73 -6.7
10.556 -24.52 -23.73 -20.7
10.558 -28.52 -31.73 9.3
10.56 -34.52 -36.73 8.3
10.562 -25.52 -30.73 23.3
10.564 -12.52 -22.73 42.3
10.566 -22.52 -29.73 38.3
10.568 -29.52 -31.73 28.3
10.57 -2.52 -5.73 36.3
10.572 -13.52 -11.73 25.3
10.574 -9.52 -10.73 16.3
10.576 -9.52 -11.73 1.3
10.578 -2.52 -5.73 13.3
10.58 28.48 17.27 10.3
10.582 17.48 7.27 26.3
10.584 0.48 -6.73 14.3
10.586 8.48 6.27 1.3
10.588 15.48 14.27 -5.7
10.59 9.48 12.27 5.3
10.592 7.48 7.27 6.3
10.594 32.48 28.27 -3.7
10.596 25.48 20.27 -27.7
10.598 23.48 17.27 -12.7
10.6 24.48 20.27 -21.7
10.602 25.48 20.27 -39.7
10.604 2.48 3.27 -19.7
10.606 10.48 8.27 -31.7
10.608 2.48 3.27 -17.7
10.61 -3.52 -13.73 -24.7
10.612 -6.52 -10.73 -7.7
10.614 3.48 -1.73 -6.7
10.616 -2.52 -8.73 -23.7
10.618 11.48 6.27 -36.7
10.62 -1.52 -3.73 -29.7
10.622 -9.52 -7.73 -35.7
10.624 -5.52 -3.73 -21.7
10.626 -27.52 -26.73 -27.7
10.628 -29.52 -33.73 -17.7
10.63 -24.52 -27.73 -13.7
10.632 -21.52 -26.73 8.3
10.634 -32.52 -33.73 1.3
10.636 -41.52 -38.73 25.3
10.638 -28.52 -27.73 9.3
10.64 -29.52 -25.73 11.3
10.642 -28.52 -24.73 23.3
10.644 -22.52 -22.73 23.3
10.646 1.48 -6.73 16.3
10.648 -6.52 -13.73 48.3
10.65 11.48 6.27 28.3
10.652 15.48 9.27 16.3
10.654 15.48 12.27 5.3
10.656 12.48 17.27 18.3
10.658 21.48 23.27 7.3
10.66 15.48 12.27 8.3
10.662 18.48 15.27 8.3
10.664 18.48 12.27 -7.7
10.666 20.48 15.27 -15.7
10.668 16.48 12.27 -20.7
10.67 9.48 12.27 -22.7
10.672 10.48 13.27 -34.7
10.674 8.48 9.27 -22.7
10.676 9.48 6.27 -35.7
10.678 -1.52 -5.73 -23.7
10.68 3.48 0.27 -7.7
10.682 3.48 -1.73 -23.7
10.684 -9.52 -13.73 -39.7
10.686 -11.52 -10.73 -24.7
10.688 -9.52 -8.73 -24.7
10.69 -4.52 0.27 -28.7
10.692 -26.52 -23.73 -20.7
10.694 -20.52 -21.73 10.3
10.696 -24.52 -29.73 9.3
10.698 -26.52 -34.73 18.3
10.7 -41.52 -44.73 22.3
10.702 -38.52 -38.73 17.3
10.704 -37.52 -34.73 -0.7
10.706 -38.52 -39.73 5.3
10.708 -38.52 -33.73 13.3
10.71 -19.52 -24.73 26.3
10.712 -15.52 -24.73 33.3
10.714 -20.52 -25.73 48.3
10.716 -20.52 -31.73 68.3
10.718 -15.52 -18.73 46.3
10.72 -9.52 -13.73 22.3
10.722 -10.52 -9.73 38.3
10.724 9.48 11.27 39.3
10.726 16.48 15.27 22.3
10.728 24.48 16.27 15.3
10.73 27.48 23.27 21.3
10.732 27.48 21.27 6.3
10.734 27.48 23.27 -5.7
10.736 13.48 16.27 -8.7
10.738 18.48 17.27 -3.7
10.74 16.48 16.27 -11.7
10.742 16.48 14.27 -24.7
10.744 4.48 5.27 -7.7
10.746 18.48 14.27 -19.7
10.748 25.48 20.27 -19.7
10.75 13.48 11.27 -16.7
10.752 1.48 -0.73 -29.7
10.754 9.48 9.27 -29.7
10.756 11.48 9.27 -45.7
10.758 -15.52 -11.73 -30.7
10.76 1.48 1.27 -13.7
10.762 0.48 -3.73 -8.7
10.764 -9.52 -13.73 -29.7
10.766 -18.52 -22.73 -21.7
10.768 -17.52 -14.73 5.3
10.77 -12.52 -13.73 -8.7
10.772 -20.52 -16.73 -5.7
10.774 -22.52 -19.73 2.3
10.776 -34.52 -30.73 -3.7
10.778 -19.52 -21.73 8.3
10.78 -27.52 -32.73 13.3
10.782 -32.52 -34.73 9.3
10.784 -32.52 -29.73 38.3
10.786 -21.52 -21.73 33.3
10.788 -32.52 -26.73 10.3
10.79 -37.52 -33.73 41.3
10.792 -21.52 -18.73 22.3
10.794 -18.52 -21.73 23.3
10.796 -5.52 -6.73 19.3
10.798 -2.52 -3.73 23.3
10.8 11.48 12.27 9.3
10.802 18.48 17.27 28.3
10.804 9.48 14.27 25.3
10.806 23.48 26.27 10.3
10.808 15.48 22.27 2.3
10.81 18.48 17.27 7.3
10.812 11.48 13.27 11.3
10.814 31.48 27.27 5.3
10.816 26.48 27.27 8.3
10.818 21.48 21.27 -9.7
10.82 11.48 15.27 10.3
10.822 26.48 29.27 -31.7
10.824 10.48 16.27 -36.7
10.826 0.48 1.27 -13.7
10.828 8.48 5.27 -13.7
10.83 12.48 5.27 -3.7
10.832 1.48 -2.73 5.3
10.834 1.48 -0.73 -999
10.836 -4.52 -5.73 -999
10.838 -20.52 -13.73 -999
10.84 -13.52 -8.73 -999
10.842 -27.52 -22.73 -999
10.844 -22.52 -21.73 -999
10.846 -18.52 -23.73 -999
10.848 -26.52 -26.73 -999
10.85 -24.52 -29.73 21.3
10.852 -33.52 -32.73 18.3
10.854 -30.52 -33.73 25.3
10.856 -35.52 -32.73 28.3
10.858 -33.52 -26.73 36.3
10.86 -21.52 -22.73 38.3
10.862 -22.52 -24.73 34.3
10.864 -11.52 -14.73 39.3
10.866 -18.52 -19.73 39.3
10.868 -15.52 -15.73 14.3
10.87 -4.52 -1.73 19.3
10.872 -2.52 5.27 21.3
10.874 2.48 7.27 23.3
10.876 12.48 11.27 17.3
10.878 24.48 24.27 46.3
10.88 9.48 8.27 39.3
10.882 31.48 30.27 15.3
10.884 26.48 24.27 13.3
10.886 20.48 21.27 6.3
10.888 21.48 26.27 2.3
10.89 33.48 40.27 -20.7
10.892 20.48 22.27 0.3
10.894 19.48 19.27 -16.7
10.896 24.48 23.27 0.3
10.898 25.48 21.27 42.3
10.9 16.48 13.27 -13.7
10.902 9.48 14.27 -8.7
10.904 9.48 9.27 -20.7
10.906 10.48 13.27 -17.7
10.908 11.48 9.27 -13.7
10.91 4.48 6.27 -29.7
10.912 9.48 6.27 -19.7
10.914 12.48 9.27 -24.7
10.916 13.48 8.27 -25.7
10.918 -3.52 -8.73 -22.7
10.92 -5.52 -3.73 -11.7
10.922 -5.52 -2.73 -14.7
10.924 -13.52 -11.73 -19.7
10.926 -12.52 -11.73 -8.7
10.928 -20.52 -21.73 -14.7
10.93 -19.52 -21.73 1.3
10.932 -17.52 -21.73 3.3
10.934 -21.52 -22.73 30.3
10.936 -11.52 -11.73 32.3
10.938 -12.52 -11.73 25.3
10.94 -27.52 -18.73 31.3
10.942 -24.52 -18.73 30.3
10.944 -21.52 -18.73 19.3
10.946 -15.52 -15.73 24.3
10.948 3.48 0.27 39.3
10.95 13.48 14.27 34.3
10.952 21.48 21.27 9.3
10.954 20.48 24.27 14.3
10.956 34.48 37.27 15.3
10.958 28.48 30.27 -6.7
10.96 19.48 22.27 -8.7
10.962 27.48 28.27 8.3
10.964 33.48 27.27 9.3
10.966 18.48 17.27 2.3
10.968 20.48 20.27 8.3
10.97 15.48 16.27 -13.7
10.972 19.48 21.27 -21.7
10.974 8.48 13.27 -13.7
10.976 11.48 14.27 -29.7
10.978 18.48 16.27 -11.7
10.98 2.48 -5.73 -9.7
10.982 4.48 1.27 -17.7
10.984 -3.52 -8.73 -13.7
10.986 -3.52 -5.73 -28.7
10.988 -10.52 -10.73 -20.7
10.99 -26.52 -22.73 -28.7
10.992 -12.52 -11.73 -25.7
10.994 -11.52 -10.73 2.3
10.996 -12.52 -16.73 -4.7
10.998 -19.52 -23.73 19.3
11 -14.52 -19.73 18.3
set(gcf, 'Position', [100, 100, 2000, 600]); % setting the size of the plots
plot(data.Time,[data.probe1,data.probe2,data.probe3]);
ylim([-60 60]);
xlabel('Time')
legend('probe 1', 'probe 2', 'probe 3');
The MHKiT-Matlab QC module utilizes structures. WE need to convert the table of data into a structure.
datast.time=data.Time;
datast.values = table2array(data(:,2:4));
datast
datast = struct with fields:
time: [500×1 double]
values: [500×3 double]
Quality control tests
The following quality control tests are used to identify timestamp issues, corrupt data, data outside expected range, and stagnant data.
Each quality control tests results in the following information in a results structure:
- Cleaned data that has NaN in place of data that did not pass the quality control test
- Boolean mask with True/False that indicates if each data point passed the quality control test
Check timestamp
Quality control analysis generally starts by checking the timestamp index of the data.
The follow test checks to see if 1) the data contains duplicate timestamps, 2) timestamps are not monotonically increasing, and 3) timestamps occur at irregular intervals (an interval of 0.002s is expected for this data).
If duplicate timestamps are found, the resulting structure keeps the first occurrence. If timestamps are not monotonic, the timestamps in the resulting structure are reordered.
% define expected frequency of the data, in seconds
frequency = 0.002;
% run the timestamp quality control test
results = check_timestamp(datast,frequency);
The cleaned data and boolean mask are shown below.
disp(results)
values: [501×3 double]
mask: [501×3 int64]
time: [1×501 datetime]
plot(results.time,results.values);
ylim([-60 60]);
xlabel('Time')
legend('probe 1', 'probe 2', 'probe 3');
Check for corrupt data
In the following qualtiy control tests, the cleaned data from the previous test are used as input to the subsequent test. For each quality control test, a plot of the cleaned data is shown.
The quality control test below checks for corrupt data, indicated by a value of -999.
%define corrupt values
corrupt_values = {-999};
% run the corrupt data QC test
results = check_corrupt(results,corrupt_values);
% Plot the cleaned data
plot(results.time,results.values);
ylim([-60 60]);
xlabel('Time')
legend('probe 1', 'probe 2', 'probe 3');
Check for data ourside expected range
The next quality control test checks for data that is greater than 50 or less than -50. Note that expected range tests can also be used to compare measured values to a model, or analyze the expected relationships between data columns.
% define expected lower and upper bound
expected_bounds = [-50,50];
% run expected range QC test
results = check_range(results,expected_bounds);
% Plot the cleaned data
plot(results.time,results.values);
ylim([-60 60]);
xlabel('Time')
legend('probe 1', 'probe 2', 'probe 3');
Check for stagnant data
The final quality control test checks for stagnant data by looking for data that changes by less than 0.001 within a 0.02 second moving window.
% define the lower bound (no upper bound is specified in this example)
expected_bound = {0.001,py.None};
% Define the moving window, in seconds
window = 0.02;
% input the data, boundaries, py.None indicates apply to all columns, and window
results = check_delta(results,expected_bound,window);
% plot cleaned data
plot(results.time,results.values);
ylim([-60 60]);
xlabel('Time')
legend('probe 1', 'probe 2', 'probe 3');
Cleaned Data
The cleaned data can be used directly in MHKiT analysis, or the missing values can be replaced using various methods before analysis is run. Data replacement strategies are generally defined on a case by case basis. Pandas includes methods to interpolate, replace, and fill missing values.
% Extract final cleaned data for MHKiT analysis
results
results = struct with fields:
values: [501×3 double]
mask: [501×3 int64]
time: [1×501 datetime]
disp(results.values)
24.4800 28.2700 1.3000
34.4800 40.2700 -8.7000
30.4800 38.2700 -13.7000
12.4800 24.2700 -32.7000
13.4800 22.2700 -21.7000
7.4800 15.2700 -13.7000
4.4800 13.2700 -19.7000
-9.5200 -0.7300 -11.7000
8.4800 19.2700 -16.7000
-4.5200 8.2700 -34.7000
16.4800 27.2700 5.3000
-13.5200 -0.7300 -36.7000
-13.5200 0.2700 -39.7000
-15.5200 -3.7300 -32.7000
-22.5300 -17.7300 -23.7000
-22.5200 -16.7300 16.3000
-30.5200 -23.7300 1.3000
-30.5200 -16.7300 11.3000
-36.5200 -24.7300 13.3000
-26.5200 -13.7300 23.3000
-28.5200 -14.7300 25.3000
NaN NaN NaN
NaN NaN NaN
-14.5200 -11.7300 33.3000
-10.5200 -1.7300 36.3000
-19.5200 -10.7300 28.3000
-3.5200 7.2700 39.3000
7.4800 20.2700 26.3000
-2.5200 14.2700 11.3000
4.4800 19.2700 13.3000
2.4800 17.2700 16.3000
9.4800 22.2700 2.3000
17.4800 23.2700 19.3000
16.4800 27.2700 26.3000
33.4800 42.2700 5.3000
28.4800 36.2700 -6.7000
20.4800 36.2700 2.3000
17.4800 30.2700 -7.7000
11.4800 23.2700 -14.7000
16.4800 24.2700 2.3000
4.4800 12.2700 -8.7000
15.4800 22.2700 -9.7000
9.4800 21.2700 -15.7000
-2.5200 7.2700 -14.7000
-7.5200 9.2700 -31.7000
3.4800 17.2700 -40.7000
-2.5200 8.2700 -46.7000
-13.5200 -2.7300 -19.7000
-4.5200 3.2700 -13.7000
-14.5200 -7.7300 NaN
-18.5200 -11.7300 NaN
-40.5200 -22.7300 NaN
-33.5200 -18.7300 NaN
-40.5200 -23.7300 NaN
-34.5200 -18.7300 NaN
NaN -27.7300 NaN
NaN -19.7300 NaN
NaN -2.7300 NaN
NaN -21.7300 46.3000
NaN -26.7300 21.3000
NaN -14.7300 31.3000
NaN -13.7300 30.3000
NaN -16.7300 15.3000
NaN -15.7300 34.3000
NaN -18.7300 47.3000
NaN -0.7300 31.3000
NaN -1.7300 48.3000
NaN 4.2700 44.3000
4.4800 13.2700 22.3000
15.4800 20.2700 10.3000
4.4800 20.2700 13.3000
18.4800 27.2700 21.3000
23.4800 34.2700 -1.7000
20.4800 26.2700 -7.7000
13.4800 21.2700 17.3000
27.4800 36.2700 -1.7000
20.4800 31.2700 -1.7000
16.4800 28.2700 -17.7000
12.4800 27.2700 -27.7000
24.4800 31.2700 -35.7000
10.4800 20.2700 -15.7000
10.4800 17.2700 -25.7000
11.4800 16.2700 -3.7000
3.4800 7.2700 -25.7000
-14.5200 -7.7300 -24.7000
-5.5200 6.2700 -24.7000
-7.5200 1.2700 -24.7000
-13.5200 -2.7300 -42.7000
-13.5200 -3.7300 -19.7000
-20.5200 -11.7300 -8.7000
-11.5200 -10.7300 -21.7000
-27.5200 -22.7300 2.3000
-28.5200 -21.7300 17.3000
-29.5200 -17.7300 11.3000
-28.5200 -17.7300 0.3000
-41.5200 -30.7300 30.3000
-35.5200 -26.7300 18.3000
-33.5200 -26.7300 10.3000
-20.5200 -17.7300 33.3000
-17.5200 -11.7300 29.3000
-10.5200 -8.7300 49.3000
4.4800 13.2700 28.3000
2.4800 12.2700 25.3000
0.4800 14.2700 21.3000
-7.5200 3.2700 NaN
-6.5200 1.2700 NaN
16.4800 19.2700 18.3000
9.4800 16.2700 19.3000
12.4800 19.2700 26.3000
24.4800 31.2700 14.3000
20.4800 31.2700 -3.7000
18.4800 31.2700 -0.7000
15.4800 27.2700 13.3000
30.4800 34.2700 -11.7000
16.4800 23.2700 -15.7000
26.4800 23.2700 -0.7000
23.4800 28.2700 8.3000
13.4800 16.2700 -36.7000
1.4800 8.2700 -17.7000
16.4800 23.2700 -24.7000
-7.5200 4.2700 NaN
-12.5200 -2.7300 -21.7000
-2.5200 1.2700 -4.7000
-6.5200 -9.7300 -19.7000
-17.5200 -16.7300 -21.7000
-12.5200 -13.7300 1.3000
-22.5200 -17.7300 0.3000
-24.5200 -21.7300 -20.7000
-38.5200 -30.7300 21.3000
-34.5200 -24.7300 10.3000
-22.5200 -18.7300 30.3000
-24.5200 -24.7300 21.3000
-25.5200 -23.7300 42.3000
-18.5200 -16.7300 15.3000
-20.5200 -19.7300 17.3000
-29.5200 -26.7300 23.3000
-21.5200 -18.7300 26.3000
-3.5200 5.2700 30.3000
4.4800 7.2700 34.3000
5.4800 8.2700 45.3000
7.4800 7.2700 44.3000
26.4800 24.2700 22.3000
17.4800 16.2700 9.3000
23.4800 24.2700 5.3000
28.4800 23.2700 5.3000
33.4800 36.2700 -9.7000
28.4800 35.2700 -0.7000
31.4800 30.2700 1.3000
19.4800 32.2700 -4.7000
18.4800 20.2700 -12.7000
7.4800 15.2700 -14.7000
8.4800 9.2700 -25.7000
10.4800 8.2700 -24.7000
12.4800 13.2700 -25.7000
8.4800 16.2700 -39.7000
4.4800 14.2700 -34.7000
17.4800 11.2700 -23.7000
1.4800 16.2700 -23.7000
-2.5200 -6.7300 -28.7000
-9.5200 -1.7300 -19.7000
-13.5200 -3.7300 -28.7000
-17.5200 -14.7300 -11.7000
-19.5200 -11.7300 -11.7000
-19.5200 -16.7300 -14.7000
-13.5200 -15.7300 -4.7000
-11.5200 -13.7300 17.3000
-19.5200 -19.7300 13.3000
-14.5200 -15.7300 18.3000
-20.5200 -18.7300 23.3000
-25.5200 -22.7300 10.3000
-24.5200 -18.7300 10.3000
-21.5200 -19.7300 -3.7000
-34.5200 -29.7300 16.3000
-21.5200 -22.7300 33.3000
0.4800 -3.7300 39.3000
-1.5200 -1.7300 30.3000
4.4800 3.2700 19.3000
8.4800 12.2700 32.3000
5.4800 13.2700 17.3000
15.4800 17.2700 -0.7000
13.4800 16.2700 2.3000
18.4800 14.2700 23.3000
24.4800 21.2700 -0.7000
28.4800 31.2700 -4.7000
20.4800 23.2700 5.3000
36.4800 30.2700 -13.7000
20.4800 29.2700 -6.7000
19.4800 22.2700 -34.7000
11.4800 12.2700 -14.7000
23.4800 22.2700 -17.7000
13.4800 9.2700 -22.7000
8.4800 5.2700 -34.7000
15.4800 9.2700 -23.7000
5.4800 1.2700 -40.7000
-3.5200 -3.7300 -38.7000
-13.5200 -13.7300 -31.7000
0.4800 -0.7300 -21.7000
-2.5200 -3.7300 -24.7000
-14.5200 -17.7300 -29.7000
-12.5200 -18.7300 -6.7000
-10.5200 NaN -14.7000
-30.5200 NaN -8.7000
-28.5200 NaN 5.3000
-24.5200 NaN 11.3000
-29.5200 NaN 0.3000
-26.5200 NaN 21.3000
-28.5200 NaN 40.3000
-25.5200 NaN 36.3000
-12.5200 NaN 24.3000
-20.5200 NaN 1.3000
-25.5200 NaN 9.3000
4.4800 NaN 8.3000
-1.5200 NaN -4.7000
-6.5200 NaN 21.3000
21.4800 NaN 9.3000
24.4800 NaN 22.3000
20.4800 NaN 26.3000
12.4800 NaN 5.3000
20.4800 NaN 3.3000
28.4800 NaN -0.7000
32.4800 NaN -4.7000
27.4800 NaN -0.7000
36.4800 NaN -9.7000
43.4800 NaN -23.7000
26.4800 NaN 1.3000
24.4800 NaN -21.7000
17.4800 NaN -13.7000
16.4800 NaN -39.7000
1.4800 NaN -15.7000
11.4800 NaN -23.7000
19.4800 NaN -19.7000
7.4800 NaN -36.7000
10.4800 NaN -42.7000
2.4800 NaN -22.7000
-3.5200 NaN NaN
-12.5200 NaN -42.7000
-10.5200 NaN -29.7000
-21.5200 NaN -16.7000
-25.5200 NaN -11.7000
-32.5200 NaN 9.3000
-21.5200 NaN 11.3000
-17.5200 NaN 2.3000
-21.5200 NaN 13.3000
-22.5200 NaN 6.3000
-11.5200 NaN 19.3000
-9.5200 NaN 14.3000
-29.5200 NaN 10.3000
-17.5200 NaN 15.3000
-4.5200 NaN 30.3000
-6.5200 NaN 18.3000
-6.5200 NaN 22.3000
17.4800 NaN 10.3000
18.4800 NaN 19.3000
9.4800 NaN -9.7000
18.4800 NaN 11.3000
16.4800 NaN -1.7000
28.4800 NaN -23.7000
17.4800 NaN -8.7000
25.4800 NaN -11.7000
25.4800 NaN -15.7000
31.4800 NaN -13.7000
15.4800 NaN -15.7000
11.4800 NaN -22.7000
21.4800 NaN -15.7000
17.4800 NaN -22.7000
10.4800 NaN -23.7000
9.4800 NaN -29.7000
17.4800 NaN -30.7000
0.4800 NaN -39.7000
-4.5200 NaN -38.7000
0.4800 NaN -24.7000
-10.5200 NaN -34.7000
-7.5200 NaN -31.7000
-12.5200 -18.7300 9.3000
-10.5200 -18.7300 5.3000
-21.5200 -27.7300 -16.7000
-30.5200 -33.7300 1.3000
-34.5200 -34.7300 -6.7000
-24.5200 -23.7300 -20.7000
-28.5200 -31.7300 9.3000
-34.5200 -36.7300 8.3000
-25.5200 -30.7300 23.3000
-12.5200 -22.7300 42.3000
-22.5200 -29.7300 38.3000
-29.5200 -31.7300 28.3000
-2.5200 -5.7300 36.3000
-13.5200 -11.7300 25.3000
-9.5200 -10.7300 16.3000
-9.5200 -11.7300 1.3000
-2.5200 -5.7300 13.3000
28.4800 17.2700 10.3000
17.4800 7.2700 26.3000
0.4800 -6.7300 14.3000
8.4800 6.2700 1.3000
15.4800 14.2700 -5.7000
9.4800 12.2700 5.3000
7.4800 7.2700 6.3000
32.4800 28.2700 -3.7000
25.4800 20.2700 -27.7000
23.4800 17.2700 -12.7000
24.4800 20.2700 -21.7000
25.4800 20.2700 -39.7000
2.4800 3.2700 -19.7000
10.4800 8.2700 -31.7000
2.4800 3.2700 -17.7000
-3.5200 -13.7300 -24.7000
-6.5200 -10.7300 -7.7000
3.4800 -1.7300 -6.7000
-2.5200 -8.7300 -23.7000
11.4800 6.2700 -36.7000
-1.5200 -3.7300 -29.7000
-9.5200 -7.7300 -35.7000
-5.5200 -3.7300 -21.7000
-27.5200 -26.7300 -27.7000
-29.5200 -33.7300 -17.7000
-24.5200 -27.7300 -13.7000
-21.5200 -26.7300 8.3000
-32.5200 -33.7300 1.3000
-41.5200 -38.7300 25.3000
-28.5200 -27.7300 9.3000
-29.5200 -25.7300 11.3000
-28.5200 -24.7300 23.3000
-22.5200 -22.7300 23.3000
1.4800 -6.7300 16.3000
-6.5200 -13.7300 48.3000
11.4800 6.2700 28.3000
15.4800 9.2700 16.3000
15.4800 12.2700 5.3000
12.4800 17.2700 18.3000
21.4800 23.2700 7.3000
15.4800 12.2700 8.3000
18.4800 15.2700 8.3000
18.4800 12.2700 -7.7000
20.4800 15.2700 -15.7000
16.4800 12.2700 -20.7000
9.4800 12.2700 -22.7000
10.4800 13.2700 -34.7000
8.4800 9.2700 -22.7000
9.4800 6.2700 -35.7000
-1.5200 -5.7300 -23.7000
3.4800 0.2700 -7.7000
3.4800 -1.7300 -23.7000
-9.5200 -13.7300 -39.7000
-11.5200 -10.7300 -24.7000
-9.5200 -8.7300 -24.7000
-4.5200 0.2700 -28.7000
-26.5200 -23.7300 -20.7000
-20.5200 -21.7300 10.3000
-24.5200 -29.7300 9.3000
-26.5200 -34.7300 18.3000
-41.5200 -44.7300 22.3000
-38.5200 -38.7300 17.3000
-37.5200 -34.7300 -0.7000
-38.5200 -39.7300 5.3000
-38.5200 -33.7300 13.3000
-19.5200 -24.7300 26.3000
-15.5200 -24.7300 33.3000
-20.5200 -25.7300 48.3000
-20.5200 -31.7300 NaN
-15.5200 -18.7300 46.3000
-9.5200 -13.7300 22.3000
-10.5200 -9.7300 38.3000
9.4800 11.2700 39.3000
16.4800 15.2700 22.3000
24.4800 16.2700 15.3000
27.4800 23.2700 21.3000
27.4800 21.2700 6.3000
27.4800 23.2700 -5.7000
13.4800 16.2700 -8.7000
18.4800 17.2700 -3.7000
16.4800 16.2700 -11.7000
16.4800 14.2700 -24.7000
4.4800 5.2700 -7.7000
18.4800 14.2700 -19.7000
25.4800 20.2700 -19.7000
13.4800 11.2700 -16.7000
1.4800 -0.7300 -29.7000
9.4800 9.2700 -29.7000
11.4800 9.2700 -45.7000
-15.5200 -11.7300 -30.7000
1.4800 1.2700 -13.7000
0.4800 -3.7300 -8.7000
-9.5200 -13.7300 -29.7000
-18.5200 -22.7300 -21.7000
-17.5200 -14.7300 5.3000
-12.5200 -13.7300 -8.7000
-20.5200 -16.7300 -5.7000
-22.5200 -19.7300 2.3000
-34.5200 -30.7300 -3.7000
-19.5200 -21.7300 8.3000
-27.5200 -32.7300 13.3000
-32.5200 -34.7300 9.3000
-32.5200 -29.7300 38.3000
-21.5200 -21.7300 33.3000
-32.5200 -26.7300 10.3000
-37.5200 -33.7300 41.3000
-21.5200 -18.7300 22.3000
-18.5200 -21.7300 23.3000
-5.5200 -6.7300 19.3000
-2.5200 -3.7300 23.3000
11.4800 12.2700 9.3000
18.4800 17.2700 28.3000
9.4800 14.2700 25.3000
23.4800 26.2700 10.3000
15.4800 22.2700 2.3000
18.4800 17.2700 7.3000
11.4800 13.2700 11.3000
31.4800 27.2700 5.3000
26.4800 27.2700 8.3000
21.4800 21.2700 -9.7000
11.4800 15.2700 10.3000
26.4800 29.2700 -31.7000
10.4800 16.2700 -36.7000
0.4800 1.2700 -13.7000
8.4800 5.2700 -13.7000
12.4800 5.2700 -3.7000
1.4800 -2.7300 5.3000
1.4800 -0.7300 NaN
-4.5200 -5.7300 NaN
-20.5200 -13.7300 NaN
-13.5200 -8.7300 NaN
-27.5200 -22.7300 NaN
-22.5200 -21.7300 NaN
-18.5200 -23.7300 NaN
-26.5200 -26.7300 NaN
-24.5200 -29.7300 21.3000
-33.5200 -32.7300 18.3000
-30.5200 -33.7300 25.3000
-35.5200 -32.7300 28.3000
-33.5200 -26.7300 36.3000
-21.5200 -22.7300 38.3000
-22.5200 -24.7300 34.3000
-11.5200 -14.7300 39.3000
-18.5200 -19.7300 39.3000
-15.5200 -15.7300 14.3000
-4.5200 -1.7300 19.3000
-2.5200 5.2700 21.3000
2.4800 7.2700 23.3000
12.4800 11.2700 17.3000
24.4800 24.2700 46.3000
9.4800 8.2700 39.3000
31.4800 30.2700 15.3000
26.4800 24.2700 13.3000
20.4800 21.2700 6.3000
21.4800 26.2700 2.3000
33.4800 40.2700 -20.7000
20.4800 22.2700 0.3000
19.4800 19.2700 -16.7000
24.4800 23.2700 0.3000
25.4800 21.2700 42.3000
16.4800 13.2700 -13.7000
9.4800 14.2700 -8.7000
9.4800 9.2700 -20.7000
10.4800 13.2700 -17.7000
11.4800 9.2700 -13.7000
4.4800 6.2700 -29.7000
9.4800 6.2700 -19.7000
12.4800 9.2700 -24.7000
13.4800 8.2700 -25.7000
-3.5200 -8.7300 -22.7000
-5.5200 -3.7300 -11.7000
-5.5200 -2.7300 -14.7000
-13.5200 -11.7300 -19.7000
-12.5200 -11.7300 -8.7000
-20.5200 -21.7300 -14.7000
-19.5200 -21.7300 1.3000
-17.5200 -21.7300 3.3000
-21.5200 -22.7300 30.3000
-11.5200 -11.7300 32.3000
-12.5200 -11.7300 25.3000
-27.5200 -18.7300 31.3000
-24.5200 -18.7300 30.3000
-21.5200 -18.7300 19.3000
-15.5200 -15.7300 24.3000
3.4800 0.2700 39.3000
13.4800 14.2700 34.3000
21.4800 21.2700 9.3000
20.4800 24.2700 14.3000
34.4800 37.2700 15.3000
28.4800 30.2700 -6.7000
19.4800 22.2700 -8.7000
27.4800 28.2700 8.3000
33.4800 27.2700 9.3000
18.4800 17.2700 2.3000
20.4800 20.2700 8.3000
15.4800 16.2700 -13.7000
19.4800 21.2700 -21.7000
8.4800 13.2700 -13.7000
11.4800 14.2700 -29.7000
18.4800 16.2700 -11.7000
2.4800 -5.7300 -9.7000
4.4800 1.2700 -17.7000
-3.5200 -8.7300 -13.7000
-3.5200 -5.7300 -28.7000
-10.5200 -10.7300 -20.7000
-26.5200 -22.7300 -28.7000
-12.5200 -11.7300 -25.7000
-11.5200 -10.7300 2.3000
-12.5200 -16.7300 -4.7000
-19.5200 -23.7300 19.3000
-14.5200 -19.7300 18.3000