Utils Module

The utils module contains the essential utility functions for data conversion, statistical analysis, caching, and event detection for the MHKiT library.

Note

The names of the functions below are of the convention path.path.function. Only the function name is used when calling the function in MATLAB. For example, to call on mhkit.wave.io.read_NDBC_file simply use read_NDBC_file.

Functions

Description

get_statistics

Calculate mean, max, min and stdev statistics of continuous data for a given statistical window.

excel_to_datetime

Convert Excel datenum format to Python datetime.

magnitude_phase

Calculates magnitude and phase in two or three dimensions of the supplied vector.

bplot

This function will create a nice boxplot from a set of data.

cached_webread

Perform webread with caching to avoid redundant downloads.

check_name

Check if a given string is a valid field name for MATLAB struct.

convert_numeric_dataframe_to_struct

Converts a numeric pandas DataFrame to a MATLAB struct.

convert_numeric_struct_to_dataframe

Converts a MATLAB struct to a numeric pandas DataFrame.

nc_file_precheck

Check NetCDF filename before working on it

nc_get_var_info

Append NetCDF variable data and info to an existing structure (res).

read_nc_file

Read NetCDF data into a MATLAB struct().

read_nc_file_group

Read NetCDF data into a MATLAB struct().

read_nc_file_var

Read NetCDF data structure.

reloadPy

Reloads Python for use after Python code change.

toggleToolbox

Utility to enable/disable MATLAB toolboxes.

uninstall_all_toolboxes

Uninstall all currently installed toolboxes
mhkit.utils.excel_to_datetime(excel_num)

Convert excel datenum format to datetime

Parameters:

excel_num (vector) – vector of excel datenums to be converted

Returns:

time (DateTimeIndex) – vector of corresponding python datetime values

mhkit.utils.cached_webread(url, options)

Perform webread with caching to avoid redundant downloads.

urlchar

The URL from which to retrieve the data.

optionsweboptions

(Optional) Additional options for the webread function.

data<Url Dependent (json, XML, plain text, etc)>

The data retrieved from the specified URL.

Define the number of retries if the URL request fails

mhkit.utils.bplot(x, varargin)

save the initial hold state of the figure.

mhkit.utils.convert_numeric_struct_to_dataframe(input_struct)

Converts a MATLAB struct to a numeric pandas DataFrame.

Parameters:

input_struct (struct) – A MATLAB struct containing numeric arrays for each field.

Returns:

df (DataFrame) – A numeric pandas DataFrame containing the data from the struct.

mhkit.utils.get_statistics(data, freq, options)

Calculate mean, max, min and stdev statistics of continuous data for a given statistical window. Default length of statistical window (period) is based on IEC TS 62600-3:2020 ED1. Also allows calculation of statistics for multiple statistical windows of continuous data.

Parameters:

  • data (strucutre) – structure of variables to get statistics for with field called time.

  • freq (double or int) – Sample rate of data [Hz]

  • period (double/int (optional)) – Statistical window of interest [sec], default = 600 to call: get_statistics(data,freq,”period”,period)

  • vector_channels (cell array of strings (optional)) – List of channel names that are to be vector averaged to call: get_statistics(data,freq,”vector_channels”,vector_channels)

Returns:

stats (structure) – Structure with mean, max, min, and stdev of each variable

mhkit.utils.read_nc_file_group(fname, varargin)

Read NetCDF data into a MATLAB struct().

Parameters:

  • filename (string) – Filename of NetCDF file to read.

  • vnms (cell array of characters (optional)) –

    Variable names to read. Read all variables if vnms not given. Otherwise, read specified variables by calling read_nc_file_var(fname,vnms,opt_out=0), which returns a structure (ds) with fields ds.(vname) as described below. If the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,

    ’GROUP_NAME2/SUBGROUP_NAME2/VAR_NAME2’,…}

  • extracted (Note 1. Variables in root group ('/') can be) – directly by ‘VARNAME’.

  • variable (Note 2. To find the group info of a) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • reading (other than) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • be (An example ncdisp(filename.nc) output can)

  • ... –

    /GrpName/SubGrpName/ Variables:

    … VAR_I_Need

    Size: … Dimensions: … …

  • filename.nc (To get VAR_I_Need from) –

    ds = read_nc_file_group(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • as (one can run this function) –

    ds = read_nc_file_group(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • Group (Note 3. If needs to read all variables within a) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file_group(‘filename.nc’,vnms);

  • ncinfo() (one can use) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file_group(‘filename.nc’,vnms);

Returns:

ds (struct)

  1. If vnms not specified, ds has fields:

    groups: struct that contains names of child groups as fields LongName: path to current group Attributes: Attributes of current group Variables: struct that contains each variable as a struct and

    a list of all variables within this group. e.g., Variables.(varname1), Variables.(varname2), …

    Variables.(varname) includes fields:

    Name: full name of the variable Data: metadata Dims: dimension names FillValue: V.FillValue or V.Attributes{‘_FillValue’} if

    the former is not given.

    Attrs: V.Attributes except for V.Attributes{‘_FillValue’}

  2. If vnms specified, ds has fields: ds.(varname).

    Each ds.(varname) contains fields same as Variables.(varname) described above.

precheck:

mhkit.utils.toggleToolbox(varargin)

Initialize

mhkit.utils.read_nc_file_var(filename, vnms, opt_out)

Read NetCDF data structure.

Parameters:

  • filename (string) – Filename of NetCDF file to read.

  • vnms (cell array of characters) – variable names to read. if the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,’GROUP_NAME2/VAR_NAME2’,…}

  • opt_out – 1 will output as structure array, ds(N).

Returns:

ds (structure or structure array) –

  1. structure that has fields: 1.1 variable names from nc file

e.g., res.(varname1), res.(varname2), … res.(varname) includes fields:

res.(varname).Name: full name of the variable res.(varname).FillValue: V.FillValue or

V.Attributes{‘_FillValue’} if the former is not given.

res.(varname).Data: metadata res.(varname).Dims: dimension names res.(varname).Attrs: V.Attributes except for

V.Attributes{‘_FillValue’}

1.2 global attributes from nc file

2. struct array, and each struct has the same fields as res.(varname) stated above.

mhkit.utils.nc_file_precheck(filename)

Check NetCDF filename before working on it

Parameters:

filename (string) – Filename of NetCDF file to read.

Returns:

No return.

check to see if the filename input is a string

mhkit.utils.viridis_colormap(m)

From: https://www.mathworks.com/matlabcentral/fileexchange/51986-perceptually-uniform-colormaps

mhkit.utils.reloadPy()

RELOADPY Reloads Python for use after Python code changes

mhkit.utils.uninstall_all_toolboxes()

Uninstall all currently installed toolboxes

Parameters:

none

Returns:

result (boolean) – 0 = success, 1 = failure, not all toolboxes uninstalled

mhkit.utils.convert_numeric_dataframe_to_struct(df)

Converts a numeric pandas DataFrame to a MATLAB struct.

Parameters:

df (DataFrame) – A numeric pandas DataFrame.

Returns:

result_struct (struct) – A MATLAB struct containing the data from the DataFrame.

mhkit.utils.check_name(input_name)
Check if a given string is a valid

field name for MATLAB struct.

If not, convert it to a valid field name

“Field names can contain ASCII letters (A–Z, a–z), digits (0–9), and underscores, and must begin with a letter. The maximum length of a field name is $namelengthmax$.” – MATLAB ref for struct

Parameters:

input_name (string) – Name to check.

Returns:

  • output_name (the same as input_name if it is valid,) – otherwise, it is the modified name.

  • 1. All special characters are replaced by ‘_SC’

  • 2. Names not beigining with letters now begins with ‘A’.

  • 3. Name length exceeding $namelengthmax$ (=63) are truncated – to name with a length of $namelengthmax$.

mhkit.utils.nc_get_var_info(filename, vinfo, varpath, res)

Append NetCDF variable data and info to an existring structure (res).

Parameters:

  • filename (string) – Filename of NetCDF file to read.

  • vinfo – NetCDF variable schema, a structure.

  • varpath – Variable path in the NetCDF file, E.G., ‘GROUP_NAME/SUBGROUP_NAME/VAR_NAME’

  • res – Structure to hold the variable info

Returns:

res (structure that holds the variable info) – fields will include variables from nc file e.g., res.(varname1), res.(varname2), … res.(varname) includes fields:

res.(varname).Name: full name of the variable res.(varname).Data: metadata res.(varname).Dims: dimension names res.(varname).FillValue: V.FillValue or

V.Attributes{‘_FillValue’} if the former is not given.

res.(varname).Attrs: V.Attributes except for

V.Attributes{‘_FillValue’}

mhkit.utils.magnitude_phase(vector)

Calculates magnitude and phase in two or three dimensions call -> [mag, theta] = magnitude_phase({x; y}) call -> [mag, theta, phi] = magnitude_phase({x; y; z})

Parameters:

vector (cell array) –

cell array consisting of x, y, and optionally the z component

of vector

x: array_like x component y: array like y component z: array like z component defined positive up (Optional)

Returns:

varargout (array) – depending on number of inputs it will either output an array for magnitude and theta or one for magnitude, theta, and phi

magnitude: array - magnitude of vector theta: array - radians from the x axis phi: array - radians from the z-axis defined as positive up

mhkit.utils.read_nc_file(fname, varargin)

Read NetCDF data into a MATLAB struct().

Parameters:

  • filename (string) – Filename of NetCDF file to read.

  • vnms (cell array of characters (optional)) –

    Variable names to read. Read all variables if vnms not given. Otherwise, read specified variables. If the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,

    ’GROUP_NAME2/SUBGROUP_NAME2/VAR_NAME2’,…}

  • extracted (Note 1. Variables in root group ('/') can be) – directly by ‘VARNAME’.

  • variable (Note 2. To find the group info of a) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • reading (other than) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.

  • be (An example ncdisp(filename.nc) output can)

  • ... –

    /GrpName/SubGrpName/ Variables:

    … VAR_I_Need

    Size: … Dimensions: … …

  • filename.nc (To get VAR_I_Need from) –

    ds = read_nc_file(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • as (one can run this function) –

    ds = read_nc_file(‘filename.nc’,…

    {‘/GrpName/SubGrpName/VAR_I_Need’});

  • Group (Note 3. If needs to read all variables within a) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file(‘filename.nc’,vnms);

  • ncinfo() (one can use) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file(‘filename.nc’,vnms);

Returns:

ds (struct)

  1. If vnms not specified, ds has fields:

    groups: struct that contains names of child groups as fields LongName: path to current group Attributes: Attributes of current group Variables: struct that contains each variable as a struct and

    a list of all variables within this group. e.g., Variables.(varname1), Variables.(varname2), …

    Variables.(varname) includes fields:

    Name: full name of the variable Data: metadata Dims: dimension names FillValue: V.FillValue or V.Attributes{‘_FillValue’} if

    the former is not given.

    Attrs: V.Attributes except for V.Attributes{‘_FillValue’}

  2. If vnms specified, ds has fields: ds.(varname).

    Each ds.(varname) contains fields same as Variables.(varname) described above.