Utils Module
The utils module contains the essential utility functions for data conversion, statistical analysis, caching, and event detection for the MHKiT library.
Note
The names of the functions below are of the convention path.path.function. Only the function name is used when calling the function in MATLAB. For example, to call on mhkit.wave.io.read_NDBC_file simply use read_NDBC_file.
Functions |
Description |
|---|---|
|
Calculate mean, max, min and stdev statistics of continuous data for a given statistical window. |
|
Convert Excel datenum format to Python datetime. |
|
Calculates magnitude and phase in two or three dimensions of the supplied vector. |
|
This function will create a nice boxplot from a set of data. |
|
Perform webread with caching to avoid redundant downloads. |
|
Check if a given string is a valid field name for MATLAB struct. |
|
Converts a numeric pandas DataFrame to a MATLAB struct. |
|
Converts a MATLAB struct to a numeric pandas DataFrame. |
|
Check NetCDF filename before working on it |
|
Append NetCDF variable data and info to an existing structure (res). |
|
Read NetCDF data into a MATLAB struct(). |
|
Read NetCDF data into a MATLAB struct(). |
|
Read NetCDF data structure. |
|
Reloads Python for use after Python code change. |
|
Utility to enable/disable MATLAB toolboxes. |
|
Uninstall all currently installed toolboxes |
- mhkit.utils.excel_to_datetime(excel_num)
Convert excel datenum format to datetime
- Parameters:
excel_num (
vector) – vector of excel datenums to be converted- Returns:
time (DateTimeIndex) – vector of corresponding python datetime values
- mhkit.utils.blues_colormap(m)
Colormap similar to matplotlib blues
- mhkit.utils.convert_numeric_dataframe_to_struct(df)
Converts a numeric pandas DataFrame to a MATLAB struct.
- Parameters:
df (
DataFrame) – A numeric pandas DataFrame.- Returns:
result_struct (struct) – A MATLAB struct containing the data from the DataFrame.
- mhkit.utils.read_nc_file_group(fname, varargin)
Read NetCDF data into a MATLAB struct().
- Parameters:
filename (
string) – Filename of NetCDF file to read.vnms (
cell array of characters (optional)) –Variable names to read. Read all variables if vnms not given. Otherwise, read specified variables by calling read_nc_file_var(fname,vnms,opt_out=0), which returns a structure (ds) with fields ds.(vname) as described below. If the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,
’GROUP_NAME2/SUBGROUP_NAME2/VAR_NAME2’,…}
extracted (
Note 1. Variables in root group ('/') can be) – directly by ‘VARNAME’.variable (
Note 2. To find the group info of a) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.reading (
other than) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.be (
An example ncdisp(filename.nc) output can)... –
/GrpName/SubGrpName/ Variables:
… VAR_I_Need
Size: … Dimensions: … …
filename.nc (
To get VAR_I_Need from) –- ds = read_nc_file_group(‘filename.nc’,…
{‘/GrpName/SubGrpName/VAR_I_Need’});
as (
one can run this function) –- ds = read_nc_file_group(‘filename.nc’,…
{‘/GrpName/SubGrpName/VAR_I_Need’});
Group (
Note 3. If needs to read all variables within a) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file_group(‘filename.nc’,vnms);ncinfo() (
one can use) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file_group(‘filename.nc’,vnms);
- Returns:
ds (struct)
- If vnms not specified, ds has fields:
groups: struct that contains names of child groups as fields LongName: path to current group Attributes: Attributes of current group Variables: struct that contains each variable as a struct and
a list of all variables within this group. e.g., Variables.(varname1), Variables.(varname2), …
- Variables.(varname) includes fields:
Name: full name of the variable Data: metadata Dims: dimension names FillValue: V.FillValue or V.Attributes{‘_FillValue’} if
the former is not given.
Attrs: V.Attributes except for V.Attributes{‘_FillValue’}
- If vnms specified, ds has fields: ds.(varname).
Each ds.(varname) contains fields same as Variables.(varname) described above.
precheck:
- mhkit.utils.cmocean(ColormapName, varargin)
cmocean returns perceptually-uniform colormaps created by Kristen Thyng.
Syntax
cmocean cmap = cmocean(‘ColormapName’) cmap = cmocean(‘-ColormapName’) cmap = cmocean(…,NLevels) cmap = cmocean(…,’pivot’,PivotValue) cmap = cmocean(…,’negative’) cmocean(…)
Description
cmocean without any inputs displays the options for colormaps.
cmap = cmocean(‘ColormapName’) returns a 256x3 colormap. ColormapName can be any of of the following:
SEQUENTIAL: DIVERGING: ‘thermal’ ‘balance’ ‘haline’ ‘delta’ ‘solar’ ‘curl’ ‘ice’ ‘diff’ ‘gray’ ‘tarn’ ‘oxy’ ‘deep’ CONSTANT LIGHTNESS: ‘dense’ ‘phase’ ‘algae’ ‘matter’ OTHER: ‘turbid’ ‘topo’ ‘speed’ ‘amp’ ‘tempo’ ‘rain’
cmap = cmocean(‘-ColormapName’) a minus sign preceeding any ColormapName flips the order of the colormap.
cmap = cmocean(…,NLevels) specifies a number of levels in the colormap. Default value is 256.
cmap = cmocean(…,’pivot’,PivotValue) centers a diverging colormap such that white corresponds to a given value and maximum extents are set using current caxis limits. If no PivotValue is set, 0 is assumed. Early versions of this function used ‘zero’ as the syntax for ‘pivot’,0 and the old syntax is still supported.
cmap = cmocean(…,’negative’) inverts the lightness profile of the colormap. This can be useful particularly for divergent colormaps if the default white point of divergence gets lost in a white background.
cmocean(…) without any outputs sets the current colormap to the current axes.
Examples Using this sample plot:
imagesc(peaks(1000)+1) colorbar
Set the colormap to ‘algae’:
cmocean(‘algae’)
Same as above, but with an inverted algae colormap:
cmocean(‘-algae’)
Set the colormap to a 12-level ‘solar’:
cmocean(‘solar’,12)
Get the RGB values of a 5-level thermal colormap:
RGB = cmocean(‘thermal’,5)
Some of those values are below zero and others are above. If this dataset represents anomalies, perhaps a diverging colormap is more appropriate:
cmocean(‘balance’)
It’s unlikely that 1.7776 is an interesting value about which the data values diverge. If you want to center the colormap on zero using the current color axis limits, simply include the ‘pivot’ option:
cmocean(‘balance’,’pivot’,0)
Author Info This function was written by Chad A. Greene of the Institute for Geophysics at the University of Texas at Austin (UTIG), June 2016, using colormaps created by Kristen Thyng of Texas A&M University, Department of Oceanography. More information on the cmocean project can be found at http://matplotlib.org/cmocean/.
Citing this colormap: If you find an occasion to cite these colormaps for any reason, or if you just want some nice beach reading, check out the following paper from the journal Oceanography:
Thyng, K.M., C.A. Greene, R.D. Hetland, H.M. Zimmerle, and S.F. DiMarco. 2016. True colors of oceanography: Guidelines for effective and accurate colormap selection. Oceanography 29(3):9?13, http://dx.doi.org/10.5670/oceanog.2016.66.
See also colormap and caxis.
- mhkit.utils.sanitize_fieldname(input_str)
Convert a string into a valid MATLAB structure fieldname
- Parameters:
input_str (
string or char) – String to be converted into a valid fieldname- Returns:
valid_fieldname (char) – Valid MATLAB structure fieldname
Notes
Function performs the following conversions: 1. Removes invalid characters 2. Ensures it starts with a letter 3. Ensures it only contains letters, numbers, and underscores 4. Converts camelCase to snake_case 5. Handles MATLAB keywords by prefixing with ‘x_’
Examples
sanitize_fieldname(‘My Field (2)’) returns ‘my_field_2’ sanitize_fieldname(‘123field’) returns ‘f123field’ sanitize_fieldname(‘$special@chars#’) returns ‘special_chars’ sanitize_fieldname(‘myCamelCase’) returns ‘my_camel_case’
- mhkit.utils.uninstall_all_toolboxes()
Uninstall all currently installed toolboxes
- Parameters:
none
- Returns:
result (boolean) – 0 = success, 1 = failure, not all toolboxes uninstalled
- mhkit.utils.mhkit_nanmean(x, dim)
Compute mean ignoring NaN values
Computes the mean of input array, ignoring NaN values. This function provides equivalent functionality to Python numpy.nanmean()%
- Parameters:
x (
numeric array) – Input array containing numeric data with potential NaN valuesdim (
positive integer, optional) – Dimension along which to compute mean. If not specified, operates along first non-singleton dimension [dimensionless]
- Returns:
m (numeric array) – Mean values with NaN values excluded. Output dimensions depend on input array and specified dimension [same units as input]
Examples
% Vector example x = [1, 2, NaN, 4, 5]; m = mhkit_nanmean(x); % Returns 3 (mean of [1,2,4,5])
% Matrix example - column-wise mean x = [1 2 3; NaN 5 6; 7 8 NaN]; m = mhkit_nanmean(x, 1); % Returns [4, 5, 4.5]
- mhkit.utils.cached_webread(url, options)
Perform webread with caching to avoid redundant downloads.
- Parameters:
url (
char) – The URL from which to retrieve the data.options – (Optional) Additional options for the webread function.
- mhkit.utils.twilight(N)
Perceptually uniform cyclic colormap from MatPlotLib.
Copyright (c) 2015 Bastian Bechtold Copyright (c) 2017-2024 Stephen Cobeldick
- Syntax:
map = twilight map = twilight(N)
[map,num] = twilight(..)
Colormap designed by Bastian Bechtold.
Cyclic colormaps are designed for plotting data that repeats, therefore the colormap loops around (the first and last colors tend to the same color).
Examples %%
Plot the scheme’s RGB values: rgbplot(twilight(256))
New colors for the COLORMAP example: load spine imagesc(X) colormap(twilight)
New colors for the SURF example: [X,Y,Z] = peaks(30); surfc(X,Y,Z) colormap(twilight) axis([-3,3,-3,3,-10,5])
Input and Output Arguments %%
Inputs (**=default): N = NumericScalar, N>=0, an integer to define the colormap length.
= []**, same length as MATLAB’s inbuilt colormap functions.
Outputs: map = NumericMatrix, size Nx3, a colormap of RGB values between 0 and 1. num = NumericScalar, the length of the defining colormap (number of nodes).
- See Also CIVIDIS INFERNO MAGMA PLASMA TWILIGHT_SHIFTED VIRIDIS
TAB10 TAB20 TAB20B TAB20C BREWERMAP CUBEHELIX CMOCEAN LBMAP PARULA COLORMAP SET FIGURE
- mhkit.utils.read_nc_file(fname, varargin)
Read NetCDF data into a MATLAB struct().
- Parameters:
filename (
string) – Filename of NetCDF file to read.vnms (
cell array of characters (optional)) –Variable names to read. Read all variables if vnms not given. Otherwise, read specified variables. If the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,
’GROUP_NAME2/SUBGROUP_NAME2/VAR_NAME2’,…}
extracted (
Note 1. Variables in root group ('/') can be) – directly by ‘VARNAME’.variable (
Note 2. To find the group info of a) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.reading (
other than) – the provided user manual of the corresponding NetCDF data product, one can use ncdisp(filename) in MATLAB which shows variable info for all groups and subgroups.be (
An example ncdisp(filename.nc) output can)... –
/GrpName/SubGrpName/ Variables:
… VAR_I_Need
Size: … Dimensions: … …
filename.nc (
To get VAR_I_Need from) –- ds = read_nc_file(‘filename.nc’,…
{‘/GrpName/SubGrpName/VAR_I_Need’});
as (
one can run this function) –- ds = read_nc_file(‘filename.nc’,…
{‘/GrpName/SubGrpName/VAR_I_Need’});
Group (
Note 3. If needs to read all variables within a) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file(‘filename.nc’,vnms);ncinfo() (
one can use) – ginfo = ncinfo(‘filename.nc’,’/GrpName/SubGrpName/’); vnms = {ginfo.Variables.Name}; vnms = strcat(‘GrpName/SubGrpName/’,vnms); ds = read_nc_file(‘filename.nc’,vnms);
- Returns:
ds (struct)
- If vnms not specified, ds has fields:
groups: struct that contains names of child groups as fields LongName: path to current group Attributes: Attributes of current group Variables: struct that contains each variable as a struct and
a list of all variables within this group. e.g., Variables.(varname1), Variables.(varname2), …
- Variables.(varname) includes fields:
Name: full name of the variable Data: metadata Dims: dimension names FillValue: V.FillValue or V.Attributes{‘_FillValue’} if
the former is not given.
Attrs: V.Attributes except for V.Attributes{‘_FillValue’}
- If vnms specified, ds has fields: ds.(varname).
Each ds.(varname) contains fields same as Variables.(varname) described above.
- mhkit.utils.check_name(input_name)
- Check if a given string is a valid
field name for MATLAB struct.
If not, convert it to a valid field name
“Field names can contain ASCII letters (A–Z, a–z), digits (0–9), and underscores, and must begin with a letter. The maximum length of a field name is $namelengthmax$.” – MATLAB ref for struct
- Parameters:
input_name (
string) – Name to check.- Returns:
output_name (the same as input_name if it is valid,) – otherwise, it is the modified name.
1. All special characters are replaced by ‘_SC’
2. Names not beigining with letters now begins with ‘A’.
3. Name length exceeding $namelengthmax$ (=63) are truncated – to name with a length of $namelengthmax$.
- mhkit.utils.magnitude_phase(vector)
Calculates magnitude and phase in two or three dimensions call -> [mag, theta] = magnitude_phase({x; y}) call -> [mag, theta, phi] = magnitude_phase({x; y; z})
- Parameters:
vector (
cell array) –- cell array consisting of x, y, and optionally the z component
of vector
x: array_like x component y: array like y component z: array like z component defined positive up (Optional)
- Returns:
varargout (array) – depending on number of inputs it will either output an array for magnitude and theta or one for magnitude, theta, and phi
magnitude: array - magnitude of vector theta: array - radians from the x axis phi: array - radians from the z-axis defined as positive up
- mhkit.utils.nc_get_var_info(filename, vinfo, varpath, res)
Append NetCDF variable data and info to an existring structure (res).
- Parameters:
filename (
string) – Filename of NetCDF file to read.vinfo – NetCDF variable schema, a structure.
varpath – Variable path in the NetCDF file, E.G., ‘GROUP_NAME/SUBGROUP_NAME/VAR_NAME’
res – Structure to hold the variable info
- Returns:
res (structure that holds the variable info) – fields will include variables from nc file e.g., res.(varname1), res.(varname2), … res.(varname) includes fields:
res.(varname).Name: full name of the variable res.(varname).Data: metadata res.(varname).Dims: dimension names res.(varname).FillValue: V.FillValue or
V.Attributes{‘_FillValue’} if the former is not given.
- res.(varname).Attrs: V.Attributes except for
V.Attributes{‘_FillValue’}
- mhkit.utils.read_nc_file_var(filename, vnms, opt_out)
Read NetCDF data structure.
- Parameters:
filename (
string) – Filename of NetCDF file to read.vnms (
cell array of characters) – variable names to read. if the variable is in a group, make sure to include group name, i.e., {‘GROUP_NAME1/VAR_NAME1’,’GROUP_NAME2/VAR_NAME2’,…}opt_out – 1 will output as structure array, ds(N).
- Returns:
ds (structure or structure array) –
structure that has fields: 1.1 variable names from nc file
e.g., res.(varname1), res.(varname2), … res.(varname) includes fields:
res.(varname).Name: full name of the variable res.(varname).FillValue: V.FillValue or
V.Attributes{‘_FillValue’} if the former is not given.
res.(varname).Data: metadata res.(varname).Dims: dimension names res.(varname).Attrs: V.Attributes except for
V.Attributes{‘_FillValue’}
1.2 global attributes from nc file
2. struct array, and each struct has the same fields as res.(varname) stated above.
- mhkit.utils.reloadPy()
RELOADPY Reloads Python for use after Python code changes
- mhkit.utils.bluewhitered_colormap(m)
Creates a diverging colormap from blue to white to red The colormap is centered on white and handles the transitions separately to avoid hue wrapping issues
- mhkit.utils.bplot(x, varargin)
save the initial hold state of the figure.
- mhkit.utils.convert_numeric_struct_to_dataframe(input_struct)
Converts a MATLAB struct to a numeric pandas DataFrame.
- Parameters:
input_struct (
struct) – A MATLAB struct containing numeric arrays for each field.- Returns:
df (DataFrame) – A numeric pandas DataFrame containing the data from the struct.
- mhkit.utils.viridis_colormap(m)
From: https://www.mathworks.com/matlabcentral/fileexchange/51986-perceptually-uniform-colormaps
- mhkit.utils.get_statistics(data, freq, options)
Calculate mean, max, min and stdev statistics of continuous data for a given statistical window. Default length of statistical window (period) is based on IEC TS 62600-3:2020 ED1. Also allows calculation of statistics for multiple statistical windows of continuous data.
- Parameters:
data (
strucutre) – structure of variables to get statistics for with field called time.freq (
double or int) – Sample rate of data [Hz]period (
double/int (optional)) – Statistical window of interest [sec], default = 600 to call: get_statistics(data,freq,”period”,period)vector_channels (
cell array of strings (optional)) – List of channel names that are to be vector averaged to call: get_statistics(data,freq,”vector_channels”,vector_channels)
- Returns:
stats (structure) – Structure with mean, max, min, and stdev of each variable
- mhkit.utils.nc_file_precheck(filename)
Check NetCDF filename before working on it
- Parameters:
filename (
string) – Filename of NetCDF file to read.- Returns:
No return.
check to see if the filename input is a string