hong2p.util module

Common functions for dealing with Thorlabs software output / stimulus metadata / our databases / movies / CNMF output.

exception hong2p.util.IOPerformanceWarning

Bases: Warning

Warning that data does not seem to be read/written from fast storage

hong2p.util.add_fly_id(df, **kwargs)

hong2p.util.add_group_id(data, group_keys, name=None, dim=None, letter=False, start_at_one=True, sort=True, inplace=False)

Adds integer column to identify unique combinations of group_keys.

Parameters

• data (Union[DataFrame, DataArray]) – DataFrame or DataArray to add metadata to

• letter (bool) – if True, use letters starting from ‘A’ (rather than integers) for the new ID variable

hong2p.util.add_missing_odor_cols(df, missing_df)

hong2p.util.add_recording_id(df, **kwargs)

hong2p.util.addlevel(df, names, values, *, axis='index')

Add level(s) to pandas MultiIndex

Intended to be an inverse to pandas.DataFrame.droplevel. pandas.DataFrame.set_index with append=True would work except that there is no axis kwarg to that function, so it does not work for the columns. pandas.DataFrame.unstack is almost what I would want, but it can seemingly arbitrarily change order of rows.

Parameters

• df (DataFrame) – DataFrame to add MultiIndex levels to

• names – str/sequence-of-str name(s) for the new levels

• values – values for the new levels. If names is a sequence, this should be of the same length.

• axis – 0/’index’ or 1/’columns’, defauling to ‘index’ as in pandas

Returns: DataFrame with MultiIndex containing names/levels from input

hong2p.util.analysis_fly_dir(date, fly)

Return type

Path

hong2p.util.analysis_intermediates_root()

Return type

Path

hong2p.util.analysis_output_root(**kwargs)

Return type

Path

hong2p.util.arraylike_cols(df)

Returns a list of columns that have only lists or arrays as elements.

hong2p.util.assign_frames_to_trials(movie, presentations_per_block, block_first_frames, odor_onset_frames)

Returns arrays trial_start_frames, trial_stop_frames

hong2p.util.baselined_normed_u8(img)

hong2p.util.calculate_df_over_f(raw_f, trial_start_frames, odor_onset_frames, trial_stop_frames)

for b_start, b_end in zip(block_first_frames, block_last_frames):

for c in range(n_footprints):

# TODO TODO TODO TODO need to be (b_end + 1) since not # inclusive? (<-fixed) other problems like this elsewhere????? # TODO maybe smooth less now that df/f is being calculated more # sensibly… raw_f[b_start:(b_end + 1), c] = smooth_1d(

raw_f[b_start:(b_end + 1), c], window_len=11

)

hong2p.util.cell_ids(df)

Takes a DataFrame with ‘cell’ in MultiIndex or columns to unique values.

hong2p.util.check_dir_exists(fn_returning_dir)

hong2p.util.check_index_vals_unique(df)

Raises AssertionError if any duplicates in column/index indices.

Return type

None

hong2p.util.const_ranges(xs, *, include_val=False)

Returns tuples of indices for largest contiguous constant-value ranges in input.

All elements in input sequence should be included in exactly one (start, stop) index tuple. Range tuples should all be in order, and stop indices are inclusive, so the end of one range should be one less than the start of the next range.

Parameters

include_val (bool) – if True, each tuple in output will be length 3, with the first element the value for the range specified by (start, stop) indices in the remaining elements of the tuple. If False, each tuple will be length 2, just containing these (start, stop) indices.

>>> const_ranges(['MCH', 'MCH', 'OCT', 'OCT'])
[(0, 1), (2, 3)]

>>> const_ranges(['MCH', 'MCH', 'OCT', 'OCT'], include_val=True)
[('MCH', 0, 1), ('OCT', 2, 3)]

Return type

Union[List[Tuple[int, int]], List[Tuple[Any, int, int]]]

hong2p.util.data_root(verbose=False)

Return type

Path

hong2p.util.date_fly_list2paired_thor_dirs(date_fly_list, n_first=None, print_full_paths=True, verbose=False, **pair_kwargs)

Takes list of (date, fly_num) tuples to pairs of their Thor outputs.

Parameters

• date_fly_list (list of (date-like, int)) – (date, fly number) tuples

• n_first (None | int) – If passed, only up to this many of pairs are enumerated. Intended for testing on subsets of data.

• verbose (bool) – (default=False) If True, prints the fly/ThorImage/ThorSync directories as they are being iterated over.

• **pair_kwargs – Passed through to thor.pair_thor_subdirs. See arguments to thor.pair_thor_dirs (called by thor.pair_thor_subdirs) for most of the useful options.

Each output is of the form: ((date, fly_num), (thorimage_dir<i>, thorsync_dir<i>))

Return type

Generator[Tuple[Tuple[Timestamp, NewType()(FlyNum, int)], Tuple[Path, Path]], None, None]

hong2p.util.df_to_odor_order(df, observed=True, return_name1=False)

Takes a complex-mixture DataFrame to odor names in desired plot order.

Args: df (pd.DataFrame): should have a ‘original_name1’ column, with names of

odors from complex mixture experiments we have pre-defined odor orders for.

observed (bool): (optional, default=True) If True, only return odor names

in df.

return_name1 (bool): (optional, default=False) If True, corresponding

values in ‘name1’ will be returned for each value in ‘original_name1’.

hong2p.util.diff_dataframes(df1, df2)

Returns a DataFrame summarizing input differences, or None if no differences.

Return type

Optional[DataFrame]

hong2p.util.dir2keys(path)

Returns (date, fly_num, thorimage_id) for dir with these as last three parts.

Return type

Tuple[Timestamp, int, str]

hong2p.util.downsample_movie(movie, target_fps, current_fps, allow_overshoot=True, allow_uneven_division=False, relative_fps_err=True, debug=False)

Returns downsampled movie by averaging consecutive groups of frames.

Groups of frames averaged do not overlap.

hong2p.util.drop_incomplete_presentations()

hong2p.util.drop_orphaned_presentations()

hong2p.util.dtypes_equal(a, b, *, reorder_cols=True, cast=True, verbose=False)

Return type

bool

hong2p.util.euclidean_dist(v1, v2)

hong2p.util.exp_decay(t, scale, tau, offset)

hong2p.util.expand_array_cols(df)

Expands any list/array entries, with new rows for each entry.

For any columns in df that have all list/array elements (at each row), the column in out_df will have the type of single elements from those arrays.

The length of out_df will be the length of the input df, multiplied by the length (should be common in each input row) of each set of list/array elements.

Other columns have their values duplicated, to match the lengths of the expanded array values.

Args: df (pd.DataFrame)

Returns: out_df (pd.DataFrame)

hong2p.util.first_group(df, group_cols)

Returns key tuple and df of first group, grouping df on group_cols.

Just for ease of interactively testing out functions on DataFrames of a groupby.

hong2p.util.fit_exp_decay(signal, sampling_rate=None, times=None, numerical_scale=1.0)

Returns fit parameters for an exponential decay in the input signal.

Parameters

• signal (1 dimensional np.ndarray) – time series, beginning at decay onset

• sampling_rate (float) – sampling rate in Hz

hong2p.util.format_date(date)

Takes a pandas Timestamp or something that can be used to construct one and returns a str with the formatted date.

Used to name directories by date, etc.

Return type

str

hong2p.util.format_keys(date, fly, *other_keys)

hong2p.util.format_mixture(*args)

Returns str representing 2-component odor mixture.

Input can be any of: - 2 str names - 2 names and concs (n1, n2, c1, c2) - a pandas.Series / dict with keys name1, name2, and (optionally)

log10_concvv<1/2>

hong2p.util.format_odor_conc(name, log10_conc)

Takes str odor name and log10 concentration to a formatted str.

hong2p.util.format_params(params, *, delim=', ', sort=True, exclude_params=None, float_format='#.3g', abbrevs=None)

Formats dict with params into one string.

Parameters

sort (bool) – whether to sort keys

Return type

str

hong2p.util.format_timestamp(timestamp)

Returns human-readable str for timestamp accepted by pd.Timestamp, to minute.

Ex: ‘2022-04-07 16:53’

Return type

str

hong2p.util.frame_pdist(df, *, metric='euclidean')

Returns a DataFrame matrix of all pairwise distances between input rows.

Returns output of same shape as DataFrame.corr() (calculating distances between all pairs of input rows), so output rows and columns will be same as input columns.

Return type

DataFrame

hong2p.util.full_frame_avg_trace(movie)

Takes a (t,[z,]x,y) movie to t-length vector of frame averages.

hong2p.util.get_axis_index(df, axis)

Return type

Index

hong2p.util.get_fly_dir(date, fly)

Returns str path fragment as YYYY-MM-DD/<n> for variety of input types

Return type

Path

hong2p.util.git_hash(repo_file)

Takes any file in a git directory and returns current hash.

hong2p.util.gsheet_csv_export_link(file_with_edit_link, gid=None, no_append_gid=False, extra_search_dirs=None)

Takes a gsheet link copied from browser while editing it, and returns a URL suitable for reading it as a CSV into a DataFrame.

GID seems to default to 0 for the first sheet, but seems unpredictable for further sheets in the same document, though you can extract it from the URL in those cases.

Return type

str

hong2p.util.gsheet_to_frame(file_with_edit_link, *, gid=None, bool_fillna_false=True, convert_date_col=True, drop_trailing_bools=True, restore_ints=True, normalize_col_names=False, use_cache=False, extra_search_dirs=None)

Parameters

• file_with_edit_link (Union[str, Path]) –

• gid (Optional[int]) –

• bool_fillna_false (bool) – whether to replace missing values in columns that otherwise only contain True/False with False. will convert column dtype to ‘bool’ as well.

• convert_date_col (bool) – whether to convert the contents of any columns named ‘date’ (case insensitive) to pd.Timestamp

• drop_trailing_bools (bool) –

  whether to drop blocks of False in bool columns beyond the last row where all non-bool columns have any non-NaN values.

  If a column has data validation for a boolean, the frame will have values (False as I’ve seen it so far) through to the end of the validation range, despite the fact that no data has been entered.

• restore_ints (bool) – whether to convert columns parsed as floats (because missing data in rows where only default values for bool cols are present) to an integer type. Requires that drop_trailing_bools actually gets rid of all the NaN values in the columns to be converted to ints (float columns with only whole number / NaN values).

• normalize_col_names (bool) – whether to rename columns using the hong2p.util.to_filename (with period=False to that function) as well as lowercasing.

• use_cache (bool) – whether to try loading cached Google sheet data, if there is a connection error when trying to load the sheet data from online. Each call will unconditionally write to this cache, saved as a hidden file in the same directory as file_with_edit_link.

• extra_search_dirs (Optional[Sequence[Union[str, Path]]]) – extra directories to search for file with name file_with_edit_link. will be searched before defaults (current directory and hong2p root)

Return type

DataFrame

hong2p.util.have_all_repeats(df, n_repeats=None)

Returns True if a recording has all blocks gsheet says it has, w/ full number of repeats for each. False otherwise.

Requires at least recording_from, comparison, name1, name2, and repeat_num columns. Can also take prep_date, fly_num, thorimage_id.

hong2p.util.have_full_comparisons(df)

Requires at least recording_from, comparison, name1, name2 columns. Can also take prep_date, fly_num, thorimage_id.

hong2p.util.index2dict_list(index)

Returns list of dicts, each with index.names keys and index values.

Raises ValueError if index.names elements are not all of type str.

Return type

List[Dict[str, Any]]

hong2p.util.invert_melt_symmetric(ser, suffixes=('_a', '_b'))

hong2p.util.is_acquisition_host()

Return type

bool

hong2p.util.is_array_sorted(array)

Returns whether 1-dimensional np.ndarray is sorted.

Return type

bool

hong2p.util.is_subrecording(thorimage_id)

Returns whether a recording id matches my GUIs naming convention for the “sub-recordings” it can create.

hong2p.util.is_subrecording_tiff(tiff_filename)

Takes a TIFF filename to whether it matches the GUI’s naming convention for the “sub-recordings” it can create.

hong2p.util.latest_trace_pickles()

Returns (date, fly, id) indexed DataFrame w/ filename and timestamp cols.

Only returns rows for filenames that had the latest timestamp for the combination of index values.

hong2p.util.list_motion_corrected_tifs(include_rigid=False, attempt_analysis_only=True)

List motion corrected TIFFs in conventional directory structure on NAS.

hong2p.util.load_metadata(*args)

Returns metadata from YAML, with defaults added.

hong2p.util.load_movie(thorimage_dir, **kwargs)

Loads movie and pre-processes (e.g. flipping) if metadata requests it.

hong2p.util.load_odor_metadata(stimfile_path)

Returns odor metadata loaded from pickle and additional computed values.

Additional values are added into the dictionary loaded from the pickle. In some cases, this can overwrite the loaded values.

hong2p.util.load_stimfile(stimfile_path)

Loads odor metadata stored in a pickle.

These metadata files are generated by scripts under ejhonglab/cutpast_arduino_stimuli.

hong2p.util.mb_team_gsheet(use_cache=False, natural_odors_only=False, drop_nonexistant_dirs=True, show_inferred_paths=False, print_excluded_on_disk=True, verbose=False)

Returns a pandas.DataFrame with data on flies and MB team recordings.

hong2p.util.md5(fname)

Calculates MD5 hash on file with name fname.

Should match output of Linux utility md5sum.

Return type

str

hong2p.util.melt_symmetric(symmetric_df, drop_constant_levels=True, suffixes=('_a', '_b'), name=None, keep_duplicate_values=False)

Takes a symmetric DataFrame to a tidy Series with unique values.

Symmetric means the row and columns indices are equal, and values should be a symmetric matrix.

hong2p.util.metadata_filename(thorimage_dir)

Returns filename of YAML for extra metadata.

hong2p.util.missing_odor_pairs(df)

Requires at least recording_from, comparison, name1, name2 columns. Can also take prep_date, fly_num, thorimage_id.

hong2p.util.missing_repeats(df, n_repeats=None)

Requires at least recording_from, comparison, name1, name2, and repeat_num columns. Can also take prep_date, fly_num, thorimage_id.

hong2p.util.most_recent_contained_file_mtime(path, recurse=True, verbose=False)

Recursively find the os.path.getmtime of the most recently modified file

Parameters

• path (Union[str, Path]) – directory within which to check mtime of files

• recurse (bool) – whether to check files under descendant directories of input

• verbose (bool) – prints which file had the most recent mtime (mostly for debugging)

Returns None if there are no files in the directory.

Testing on Ubuntu, this does not recurse into symlinks to directories, as I want for at least current use case.

Return type

Optional[float]

hong2p.util.motion_corrected_tiff_filename(date, fly_num, thorimage_id)

Takes vars identifying recording to the name of a motion corrected TIFF for it. Non-rigid preferred over rigid. Relies on naming convention.

hong2p.util.movie_blocks(tif, movie=None, allow_gsheet_to_restrict_blocks=True, stimfile=None, first_block=None, last_block=None)

Returns list of arrays, one per continuous acquisition.

tif must be named and placed according to convention, and a .mat file created from ejhonglab/matlab_kc_plane (typically run via populate_db.py in what is now my kc_natural_mixes repo) must exist in the conventional path under the analysis root. This .mat file is used for the timing information (ultimately derived mainly from ThorSync data).

Total length along time dimension should be preserved from input TIFF.

hong2p.util.n_expected_repeats(df)

Returns expected # repeats given DataFrame w/ repeat_num col.

hong2p.util.nan_eq(arr1, arr2)

Elementwise equality of arrays, but True if both are NaN at a location.

Normally, NaN != NaN, but this is not the behavior I want when trying to check which parts of an array/DataFrame were changed by an operation.

hong2p.util.no_skipped_comparisons(df)

Requires at least recording_from and comparison columns. Can also take prep_date, fly_num, and thorimage_id.

hong2p.util.num_notnull(df)

Return type

int

hong2p.util.num_null(df)

Return type

int

hong2p.util.odorset_name(df_or_odornames)

Returns name for set of odors in DataFrame.

Looks at odors in original_name1 column. Name used to lookup desired plotting order for the odors in the set.

hong2p.util.pair_ordering(comparison_df)

Takes a df w/ name1 & name2 to a dict of their tuples to order int.

Order integers start at 0 and do not skip any numbers.

hong2p.util.paired_thor_dirs(matching_substrs=None, start_date=None, end_date=None, n_first=None, skip_redone=True, verbose=False, print_skips=True, print_fast=True, print_full_paths=True, **pair_kwargs)

Generates tuples of fly metadata and ThorImage output paths, in acquisition order

Parameters

• matching_substrs (Optional[Sequence[str]]) – If passed, only experiments whose ThorImage path contains at least one of these substring will be included.

• n_first (Optional[int]) – If passed, only up to this many of pairs are enumerated. Intended for testing on subsets of data.

• verbose (bool) – If True, prints the fly/ThorImage/ThorSync directories as they are being iterated over.

• **pair_kwargs – Passed through to thor.pair_thor_subdirs. See arguments to thor.pair_thor_dirs (called by thor.pair_thor_subdirs) for most of the useful options.

Each output is of the form: ((date, fly_num), (thorimage_dir<i>, thorsync_dir<i>))

Return type

Generator[Tuple[Tuple[Timestamp, NewType()(FlyNum, int)], Tuple[Path, Path]], None, None]

hong2p.util.parent_recording_id(tiffname_or_thorimage_id)

Returns recording id for recording subrecording was derived from.

Input can be a TIFF filename or recording id.

hong2p.util.pd_allclose(a, b, *, if_index_mismatch=None, **kwargs)

Parameters

• a|b – inputs to compare. must both be either DataFrames or Series.

• if_index_mismatch (Optional[str]) – passed to pd_indices_equal

• **kwargs – passed thru to np.allclose (equal_nan=True may be the most useful)

Return type

bool

hong2p.util.pd_indices_equal(a, b, *, if_index_mismatch=None)

Parameters

• a|b – inputs to compare. must both be either DataFrames or Series.

• if_index_mismatch (Optional[str]) – whether to return False (None), warn (and return False), or raise ValueError if either row/column indices do not match

Return type

bool

hong2p.util.pd_isclose(a, b, **kwargs)

Like np.isclose, but preserves input indices.

Parameters

• a|b – inputs to compare. must both be either DataFrames or Series.

• **kwargs – passed thru to np.allclose (equal_nan=True may be the most useful)

Raises ValueError if indices do not match.

Return type

bool

hong2p.util.print_all_stimfile_odorsets()

Return type

None

hong2p.util.print_block_frames(block_first_frames, block_last_frames)

Prints block numbers and the corresponding start / stop frames.

For subsetting TIFF in ImageJ / other manual analysis.

Prints frame numbers 1-indexed, but takes 0-indexed frames.

hong2p.util.print_color(color_name, *args, **kwargs)

hong2p.util.print_thor_paths(image_dir, sync_dir, print_full_paths=True)

Return type

None

hong2p.util.print_trial_odors(data, odor_onset_frames=None)

data should be as the output of load_odor_metadata

hong2p.util.raw_data_root(root=None, **kwargs)

Return type

Path

hong2p.util.raw_fly_dir(date, fly, *, warn=True, short=False)

Parameters

short (bool) – If True, returns in format YYYY-MM-DD/<fly #>/<ThorImage dir>, without the prefix specifying the full path. Intended for creating more readable paths, where absolute paths are not required.

Return type

Path

hong2p.util.recording_df2keys(df)

hong2p.util.set_data_root(new_data_root)

Sets data root, so future calls to data_root will return the input.

You may either use this function or set one of the environment variables that the data_root function checks.

Return type

None

hong2p.util.shorten_path(full_path, n=3)

Returns a string containing just the last n parts of input path.

For making IDs / easier-to-read paths, when the full path isn’t required.

Return type

str

hong2p.util.shorten_stimfile_path(stimfile_path, stimfile_dir=None)

Shortens absolute stimulus YAML path to one relative to stimfile_dir.

hong2p.util.skipped_comparison_nums(df)

Requires at least recording_from and comparison columns. Can also take prep_date, fly_num, and thorimage_id.

hong2p.util.smooth_1d(x, window_len=11, window='hanning')

smooth the data using a window with requested size.

This method is based on the convolution of a scaled window with the signal. The signal is prepared by introducing reflected copies of the signal (with the window size) in both ends so that transient parts are minimized in the begining and end part of the output signal.

input:

x: the input signal window_len: the dimension of the smoothing window; should be an odd

integer

window: the type of window from ‘flat’, ‘hanning’, ‘hamming’,

‘bartlett’, ‘blackman’ flat window will produce a moving average smoothing.

output:

the smoothed signal

example:

t=linspace(-2,2,0.1) x=sin(t)+randn(len(t))*0.1 y=smooth_1d(x)

see also:

numpy.hanning, numpy.hamming, numpy.bartlett, numpy.blackman, numpy.convolve scipy.signal.lfilter

TODO: the window parameter could be the window itself if an array instead of a string NOTE: length(output) != length(input), to correct this: return y[(window_len/2-1):-(window_len/2)] instead of just y.

hong2p.util.split_odor_w_conc(row_or_str)

hong2p.util.start_color(color_name)

hong2p.util.stimfile_odorset(stimfile_path, strict=True)

hong2p.util.stimfile_root(**kwargs)

Return type

Path

hong2p.util.stimulus_yaml_from_thorimage(thorimage_dir_or_xml, stimfile_dir=None)

Returns absolute path to stimulus YAML file from note field in ThorImage XML.

Parameters

• thorimage_dir_or_xml – path to ThorImage output directory or XML Element containing parsed contents of the corresponding Experiment.xml file.

• stimfile_dir (str) – (optional) directory containing stimulus .yaml files. If not passed, stimfile_root() is used.

Raises

• TooManyStimulusFiles if multiple substrings of note field end with .yaml –

• NoStimulusFile if no substrings of note field end with .yaml –

• IOError if stimulus file directory does not exist –

XML should contain a manually-entered path relative to where the olfactometer code that generated it was run, but assuming it was copied to the appropriate location (directly under stimfile_dir if passed or stimfile_root() otherwise), this absolute path should exist.

My typical workflow is to paste this path into the ThorImage note field, while doing the experiment. olf will automatically populate clipboard with this path, when running the olfactometer program, to facilitate this.

hong2p.util.stop_color()

hong2p.util.subrecording_tiff_blocks(tiff_filename)

Returns tuple of int (start, stop) block numbers subrecording contains.

Block numbers start at 0.

Requires that is_subrecording_tiff(tiff_filename) would return True.

hong2p.util.subrecording_tiff_blocks_df(series)

Takes a series w/ TIFF name in series.name to (start, stop) block nums.

(series.name must be a TIFF path)

Same behavior as subrecording_tiff_blocks.

hong2p.util.subset_levels(index, subset)

Return type

Index

hong2p.util.suffix_index_names(df, suffix='_b', axis='columns')

Return type

None

hong2p.util.thor2tiff(image_dir, *, output_name=None, output_basename=None, output_dir=None, if_exists='err', flip_lr=None, discard_channel_b=False, check_round_trip=False, verbose=True, _debug=False)

Converts ThorImage .raw file to .tif file in same directory

Parameters

• if_exists (str) – ‘load’, ‘ignore’, ‘overwrite’, or ‘err’

• flip_lr (Optional[bool]) – If True, flip the raw movie along the left/right axis, to make experiments including both left/right side data more comparable. If True/False, default output basename will be ‘flipped.tif’. If None, default output basename will remain ‘raw.tif’.

• check_round_trip – If True, and a TIFF was written, read it and check it is equal to data loaded from ThorImage raw.

Returns an np.ndarray movie if TIFF was created OR if if_exists=’load’ and the corresponding TIFF already exists. Returns None if if_exists=’ignore’ and the corresponding TIFF already exists.

Return type

Optional[ndarray]

hong2p.util.thorimage2yaml_info_and_odor_lists(thorimage_dir_or_xml, stimfile_dir=None)

Returns yaml_path, yaml_data, odor_lists

Parameters

• thorimage_dir_or_xml – path to ThorImage output directory or XML Element object parsed from corresponding Experiment.xml file

• stimfile_dir (str) – (optional) directory containing stimulus .yaml files. If not passed, stimfile_root() is used.

Returns

path to YAML

yaml_data (dict): loaded contents of yaml_path

odor_lists (list-of-lists-of-dicts): each list this contains is a representation

of all the odors presented together on a given trial

Return type

yaml_path (str)

hong2p.util.thorimage_dir(date, fly, thorimage_id, **kwargs)

Return type

Path

hong2p.util.thorimage_dir_input(fn)

Wraps functions taking ThorImage path and adds option to call via fly keys

Fly keys are (date, fly_num, thorimage_id)

hong2p.util.thorsync_dir(date, fly, base_thorsync_dir, **kwargs)

Return type

Path

hong2p.util.tiff_filename2keys(tiff_filename)

Takes TIFF filename to pd.Series w/ ‘date’,’fly_num’,’thorimage_id’ keys.

TIFF must be placed and named according to convention, because the date and fly_num are taken from names of some of the containing directories.

Works with TIFFs either under raw_data_root or analysis_output_root.

hong2p.util.tiff_ijroi_filename(tiff, confirm=None, gui_confirm=False, gui_fallback=False, gui='qt5')

Takes a tiff path to corresponding ImageJ ROI file, assuming a certain naming convention and folder structure.

Automatic search for ROI will only check same folder as the TIFF passed in.

Options for fallback to manual selection / confirmation of appropriate ROI file.

hong2p.util.tiff_thorimage_id(tiff_filename)

Takes a path to a TIFF and returns ID to identify recording within (date, fly). Relies on naming convention.

Works for input that is either a raw TIFF or a motion corrected TIFF, the latter of which should have a conventional suffix indicating the type of motion correction (‘_nr’ / ‘_rig’).

hong2p.util.tiff_title(tif)

Returns abbreviation of TIFF filename for use in titles.

hong2p.util.to_filename(x, period=True, extra_remove_chars=None)

Take a str and normalizes it a bit to make it a better filename prefix.

E.g. taking a plot title and using it to derive a filename for saving the plot.

Return type

str

hong2p.util.u8_color(draw_on)

hong2p.util.version_info(*args, used_for='', force_git=False)

Takes module or string path to file in Git repo to a dict with version information (with keys and values the database will accept).

hong2p.util.write_tiff(tiff_filename, movie, strict_dtype=True, dims=None, _debug=False)

Write a TIFF loading the same as the TIFFs we create with ImageJ.

TIFFs are written in big-endian byte order to be readable by imread_big from MATLAB file exchange.

Dimensions of input should be (t,[z,],y,x).

Metadata may not be correct.

Parameters

dims (Optional[str]) – may or may not have the same meaning as tifffile.imsave axes kwarg

Return type

None