Utils Module

The utils module contains the essential utility functions for data conversion, statistical analysis, caching, and event detection for the MHKiT library.

Note

The names of the functions below are of the convention path.path.function. Only the function name is used when calling the function in MATLAB. For example, to call on mhkit.wave.io.read_NDBC_file simply use read_NDBC_file.

Functions

Description

get_statistics

Calculate mean, max, min and stdev statistics of continuous data for a given statistical window.

excel_to_datetime

Convert Excel datenum format to Python datetime.

magnitude_phase

Calculates magnitude and phase in two or three dimensions of the supplied vector.

bplot

This function will create a nice boxplot from a set of data.

cached_webread

Perform webread with caching to avoid redundant downloads.

check_name

Check if a given string is a valid field name for MATLAB struct.

convert_numeric_dataframe_to_struct

Converts a numeric pandas DataFrame to a MATLAB struct.

convert_numeric_struct_to_dataframe

Converts a MATLAB struct to a numeric pandas DataFrame.

nc_file_precheck

Check NetCDF filename before working on it

nc_get_var_info

Append NetCDF variable data and info to an existing structure (res).

read_nc_file

Read NetCDF data into a MATLAB struct().

read_nc_file_group

Read NetCDF data into a MATLAB struct().

read_nc_file_var

Read NetCDF data structure.

reloadPy

Reloads Python for use after Python code change.

toggleToolbox

Utility to enable/disable MATLAB toolboxes.

uninstall_all_toolboxes

Uninstall all currently installed toolboxes

mhkit.utils.excel_to_datetime(excel_num)

Convert excel datenum format to datetime

Parameters:

excel_num (vector) – vector of excel datenums to be converted

Returns:

time (DateTimeIndex) – vector of corresponding python datetime values

mhkit.utils.blues_colormap(m)

Colormap similar to matplotlib blues

mhkit.utils.convert_numeric_dataframe_to_struct(df)

Converts a numeric pandas DataFrame to a MATLAB struct.

Parameters:

df (DataFrame) – A numeric pandas DataFrame.

Returns:

result_struct (struct) – A MATLAB struct containing the data from the DataFrame.

mhkit.utils.read_nc_file_group(fname, varargin)

Read NetCDF data into a MATLAB struct().

Parameters:
  • filename (string) – Filename of NetCDF file to read.

  • vnms (cell array of characters (optional)) –

    Variable names to read. Read all variables if vnms not given. Otherwise, read specified variables by calling read_nc_file_var(fname,vnms,opt_out=0), which returns a structure (ds) with fields ds.(vname) as described below. If the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,

    ’GROUP_NAME2/SUBGROUP_NAME2/VAR_NAME2’,…}

  • extracted (Note 1. Variables in root group ('/') can be) – directly by ‘VARNAME’.

  • variable (Note 2. To find the group info of a) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • reading (other than) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • be (An example ncdisp(filename.nc) output can)

  • ...

    /GrpName/SubGrpName/ Variables:

    … VAR_I_Need

    Size: … Dimensions: … …

  • filename.nc (To get VAR_I_Need from) –

    ds = read_nc_file_group(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • as (one can run this function) –

    ds = read_nc_file_group(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • Group (Note 3. If needs to read all variables within a) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file_group(‘filename.nc’,vnms);

  • ncinfo() (one can use) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file_group(‘filename.nc’,vnms);

Returns:

ds (struct)

  1. If vnms not specified, ds has fields:

    groups: struct that contains names of child groups as fields LongName: path to current group Attributes: Attributes of current group Variables: struct that contains each variable as a struct and

    a list of all variables within this group. e.g., Variables.(varname1), Variables.(varname2), …

    Variables.(varname) includes fields:

    Name: full name of the variable Data: metadata Dims: dimension names FillValue: V.FillValue or V.Attributes{‘_FillValue’} if

    the former is not given.

    Attrs: V.Attributes except for V.Attributes{‘_FillValue’}

  2. If vnms specified, ds has fields: ds.(varname).

    Each ds.(varname) contains fields same as Variables.(varname) described above.

precheck:

mhkit.utils.cmocean(ColormapName, varargin)

cmocean returns perceptually-uniform colormaps created by Kristen Thyng.

Syntax

cmocean cmap = cmocean(‘ColormapName’) cmap = cmocean(‘-ColormapName’) cmap = cmocean(…,NLevels) cmap = cmocean(…,’pivot’,PivotValue) cmap = cmocean(…,’negative’) cmocean(…)

Description

cmocean without any inputs displays the options for colormaps.

cmap = cmocean(‘ColormapName’) returns a 256x3 colormap. ColormapName can be any of of the following:

SEQUENTIAL: DIVERGING: ‘thermal’ ‘balance’ ‘haline’ ‘delta’ ‘solar’ ‘curl’ ‘ice’ ‘diff’ ‘gray’ ‘tarn’ ‘oxy’ ‘deep’ CONSTANT LIGHTNESS: ‘dense’ ‘phase’ ‘algae’ ‘matter’ OTHER: ‘turbid’ ‘topo’ ‘speed’ ‘amp’ ‘tempo’ ‘rain’

cmap = cmocean(‘-ColormapName’) a minus sign preceeding any ColormapName flips the order of the colormap.

cmap = cmocean(…,NLevels) specifies a number of levels in the colormap. Default value is 256.

cmap = cmocean(…,’pivot’,PivotValue) centers a diverging colormap such that white corresponds to a given value and maximum extents are set using current caxis limits. If no PivotValue is set, 0 is assumed. Early versions of this function used ‘zero’ as the syntax for ‘pivot’,0 and the old syntax is still supported.

cmap = cmocean(…,’negative’) inverts the lightness profile of the colormap. This can be useful particularly for divergent colormaps if the default white point of divergence gets lost in a white background.

cmocean(…) without any outputs sets the current colormap to the current axes.

Examples Using this sample plot:

imagesc(peaks(1000)+1) colorbar

Set the colormap to ‘algae’:

cmocean(‘algae’)

Same as above, but with an inverted algae colormap:

cmocean(‘-algae’)

Set the colormap to a 12-level ‘solar’:

cmocean(‘solar’,12)

Get the RGB values of a 5-level thermal colormap:

RGB = cmocean(‘thermal’,5)

Some of those values are below zero and others are above. If this dataset represents anomalies, perhaps a diverging colormap is more appropriate:

cmocean(‘balance’)

It’s unlikely that 1.7776 is an interesting value about which the data values diverge. If you want to center the colormap on zero using the current color axis limits, simply include the ‘pivot’ option:

cmocean(‘balance’,’pivot’,0)

Author Info This function was written by Chad A. Greene of the Institute for Geophysics at the University of Texas at Austin (UTIG), June 2016, using colormaps created by Kristen Thyng of Texas A&M University, Department of Oceanography. More information on the cmocean project can be found at http://matplotlib.org/cmocean/.

Citing this colormap: If you find an occasion to cite these colormaps for any reason, or if you just want some nice beach reading, check out the following paper from the journal Oceanography:

Thyng, K.M., C.A. Greene, R.D. Hetland, H.M. Zimmerle, and S.F. DiMarco. 2016. True colors of oceanography: Guidelines for effective and accurate colormap selection. Oceanography 29(3):9?13, http://dx.doi.org/10.5670/oceanog.2016.66.

See also colormap and caxis.

mhkit.utils.sanitize_fieldname(input_str)

Convert a string into a valid MATLAB structure fieldname

Parameters:

input_str (string or char) – String to be converted into a valid fieldname

Returns:

valid_fieldname (char) – Valid MATLAB structure fieldname

Notes

Function performs the following conversions: 1. Removes invalid characters 2. Ensures it starts with a letter 3. Ensures it only contains letters, numbers, and underscores 4. Converts camelCase to snake_case 5. Handles MATLAB keywords by prefixing with ‘x_

Examples

sanitize_fieldname(‘My Field (2)’) returns ‘my_field_2’ sanitize_fieldname(‘123field’) returns ‘f123field’ sanitize_fieldname(‘$special@chars#’) returns ‘special_chars’ sanitize_fieldname(‘myCamelCase’) returns ‘my_camel_case’

mhkit.utils.uninstall_all_toolboxes()

Uninstall all currently installed toolboxes

Parameters:

none

Returns:

result (boolean) – 0 = success, 1 = failure, not all toolboxes uninstalled

mhkit.utils.mhkit_nanmean(x, dim)

Compute mean ignoring NaN values

Computes the mean of input array, ignoring NaN values. This function provides equivalent functionality to Python numpy.nanmean()%

Parameters:
  • x (numeric array) – Input array containing numeric data with potential NaN values

  • dim (positive integer, optional) – Dimension along which to compute mean. If not specified, operates along first non-singleton dimension [dimensionless]

Returns:

m (numeric array) – Mean values with NaN values excluded. Output dimensions depend on input array and specified dimension [same units as input]

Examples

% Vector example x = [1, 2, NaN, 4, 5]; m = mhkit_nanmean(x); % Returns 3 (mean of [1,2,4,5])

% Matrix example - column-wise mean x = [1 2 3; NaN 5 6; 7 8 NaN]; m = mhkit_nanmean(x, 1); % Returns [4, 5, 4.5]

mhkit.utils.cached_webread(url, options)

Perform webread with caching to avoid redundant downloads.

Parameters:
  • url (char) – The URL from which to retrieve the data.

  • options – (Optional) Additional options for the webread function.

mhkit.utils.twilight(N)

Perceptually uniform cyclic colormap from MatPlotLib.

Copyright (c) 2015 Bastian Bechtold Copyright (c) 2017-2024 Stephen Cobeldick

Syntax:

map = twilight map = twilight(N)

[map,num] = twilight(..)

Colormap designed by Bastian Bechtold.

Cyclic colormaps are designed for plotting data that repeats, therefore the colormap loops around (the first and last colors tend to the same color).

Examples %%

Plot the scheme’s RGB values: rgbplot(twilight(256))

New colors for the COLORMAP example: load spine imagesc(X) colormap(twilight)

New colors for the SURF example: [X,Y,Z] = peaks(30); surfc(X,Y,Z) colormap(twilight) axis([-3,3,-3,3,-10,5])

Input and Output Arguments %%

Inputs (**=default): N = NumericScalar, N>=0, an integer to define the colormap length.

= []**, same length as MATLAB’s inbuilt colormap functions.

Outputs: map = NumericMatrix, size Nx3, a colormap of RGB values between 0 and 1. num = NumericScalar, the length of the defining colormap (number of nodes).

See Also CIVIDIS INFERNO MAGMA PLASMA TWILIGHT_SHIFTED VIRIDIS

TAB10 TAB20 TAB20B TAB20C BREWERMAP CUBEHELIX CMOCEAN LBMAP PARULA COLORMAP SET FIGURE

mhkit.utils.read_nc_file(fname, varargin)

Read NetCDF data into a MATLAB struct().

Parameters:
  • filename (string) – Filename of NetCDF file to read.

  • vnms (cell array of characters (optional)) –

    Variable names to read. Read all variables if vnms not given. Otherwise, read specified variables. If the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,

    ’GROUP_NAME2/SUBGROUP_NAME2/VAR_NAME2’,…}

  • extracted (Note 1. Variables in root group ('/') can be) – directly by ‘VARNAME’.

  • variable (Note 2. To find the group info of a) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • reading (other than) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • be (An example ncdisp(filename.nc) output can)

  • ...

    /GrpName/SubGrpName/ Variables:

    … VAR_I_Need

    Size: … Dimensions: … …

  • filename.nc (To get VAR_I_Need from) –

    ds = read_nc_file(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • as (one can run this function) –

    ds = read_nc_file(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • Group (Note 3. If needs to read all variables within a) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file(‘filename.nc’,vnms);

  • ncinfo() (one can use) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file(‘filename.nc’,vnms);

Returns:

ds (struct)

  1. If vnms not specified, ds has fields:

    groups: struct that contains names of child groups as fields LongName: path to current group Attributes: Attributes of current group Variables: struct that contains each variable as a struct and

    a list of all variables within this group. e.g., Variables.(varname1), Variables.(varname2), …

    Variables.(varname) includes fields:

    Name: full name of the variable Data: metadata Dims: dimension names FillValue: V.FillValue or V.Attributes{‘_FillValue’} if

    the former is not given.

    Attrs: V.Attributes except for V.Attributes{‘_FillValue’}

  2. If vnms specified, ds has fields: ds.(varname).

    Each ds.(varname) contains fields same as Variables.(varname) described above.

mhkit.utils.check_name(input_name)
Check if a given string is a valid

field name for MATLAB struct.

If not, convert it to a valid field name

“Field names can contain ASCII letters (A–Z, a–z), digits (0–9), and underscores, and must begin with a letter. The maximum length of a field name is $namelengthmax$.” – MATLAB ref for struct

Parameters:

input_name (string) – Name to check.

Returns:

  • output_name (the same as input_name if it is valid,) – otherwise, it is the modified name.

  • 1. All special characters are replaced by ‘_SC’

  • 2. Names not beigining with letters now begins with ‘A’.

  • 3. Name length exceeding $namelengthmax$ (=63) are truncated – to name with a length of $namelengthmax$.

mhkit.utils.magnitude_phase(vector)

Calculates magnitude and phase in two or three dimensions call -> [mag, theta] = magnitude_phase({x; y}) call -> [mag, theta, phi] = magnitude_phase({x; y; z})

Parameters:

vector (cell array) –

cell array consisting of x, y, and optionally the z component

of vector

x: array_like x component y: array like y component z: array like z component defined positive up (Optional)

Returns:

varargout (array) – depending on number of inputs it will either output an array for magnitude and theta or one for magnitude, theta, and phi

magnitude: array - magnitude of vector theta: array - radians from the x axis phi: array - radians from the z-axis defined as positive up

mhkit.utils.nc_get_var_info(filename, vinfo, varpath, res)

Append NetCDF variable data and info to an existring structure (res).

Parameters:
  • filename (string) – Filename of NetCDF file to read.

  • vinfo – NetCDF variable schema, a structure.

  • varpath – Variable path in the NetCDF file, E.G., ‘GROUP_NAME/SUBGROUP_NAME/VAR_NAME’

  • res – Structure to hold the variable info

Returns:

res (structure that holds the variable info) – fields will include variables from nc file e.g., res.(varname1), res.(varname2), … res.(varname) includes fields:

res.(varname).Name: full name of the variable res.(varname).Data: metadata res.(varname).Dims: dimension names res.(varname).FillValue: V.FillValue or

V.Attributes{‘_FillValue’} if the former is not given.

res.(varname).Attrs: V.Attributes except for

V.Attributes{‘_FillValue’}

mhkit.utils.read_nc_file_var(filename, vnms, opt_out)

Read NetCDF data structure.

Parameters:
  • filename (string) – Filename of NetCDF file to read.

  • vnms (cell array of characters) – variable names to read. if the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,’GROUP_NAME2/VAR_NAME2’,…}

  • opt_out – 1 will output as structure array, ds(N).

Returns:

ds (structure or structure array) –

  1. structure that has fields: 1.1 variable names from nc file

e.g., res.(varname1), res.(varname2), … res.(varname) includes fields:

res.(varname).Name: full name of the variable res.(varname).FillValue: V.FillValue or

V.Attributes{‘_FillValue’} if the former is not given.

res.(varname).Data: metadata res.(varname).Dims: dimension names res.(varname).Attrs: V.Attributes except for

V.Attributes{‘_FillValue’}

1.2 global attributes from nc file

2. struct array, and each struct has the same fields as res.(varname) stated above.

mhkit.utils.reloadPy()

RELOADPY Reloads Python for use after Python code changes

mhkit.utils.bluewhitered_colormap(m)

Creates a diverging colormap from blue to white to red The colormap is centered on white and handles the transitions separately to avoid hue wrapping issues

mhkit.utils.bplot(x, varargin)

save the initial hold state of the figure.

mhkit.utils.convert_numeric_struct_to_dataframe(input_struct)

Converts a MATLAB struct to a numeric pandas DataFrame.

Parameters:

input_struct (struct) – A MATLAB struct containing numeric arrays for each field.

Returns:

df (DataFrame) – A numeric pandas DataFrame containing the data from the struct.

mhkit.utils.viridis_colormap(m)

From: https://www.mathworks.com/matlabcentral/fileexchange/51986-perceptually-uniform-colormaps

mhkit.utils.get_statistics(data, freq, options)

Calculate mean, max, min and stdev statistics of continuous data for a given statistical window. Default length of statistical window (period) is based on IEC TS 62600-3:2020 ED1. Also allows calculation of statistics for multiple statistical windows of continuous data.

Parameters:
  • data (strucutre) – structure of variables to get statistics for with field called time.

  • freq (double or int) – Sample rate of data [Hz]

  • period (double/int (optional)) – Statistical window of interest [sec], default = 600 to call: get_statistics(data,freq,”period”,period)

  • vector_channels (cell array of strings (optional)) – List of channel names that are to be vector averaged to call: get_statistics(data,freq,”vector_channels”,vector_channels)

Returns:

stats (structure) – Structure with mean, max, min, and stdev of each variable

mhkit.utils.nc_file_precheck(filename)

Check NetCDF filename before working on it

Parameters:

filename (string) – Filename of NetCDF file to read.

Returns:

No return.

check to see if the filename input is a string

mhkit.utils.toggleToolbox(varargin)

Initialize