playnano.processing.video_processing module

Video processing functions for AFM time-series (stacks of frames).

This module provides functions that operate on 3D numpy arrays (time-series of 2D AFM frames). These include:

  • Frame alignment to compensate for drift

  • Cropping and padding utilities

  • Temporal (time-domain) filters

  • Future extensions such as spatio-temporal denoising

All functions follow a NumPy-style API: input stacks are 3D arrays with shape (n_frames, height, width). Outputs are processed stacks and a metadata dictionary.

playnano.processing.video_processing.align_frames(stack: ndarray, reference_frame: int = 0, method: str = 'fft_cross_correlation', mode: str = 'pad', debug: bool = False, max_shift: int | None = None, pre_filter_sigma: float | None = None, max_jump: int | None = None)[source]

Align a stack of AFM frames to a reference frame using integer-pixel shifts.

Alignment is performed using either FFT-based or full cross-correlation. Jump smoothing prevents abrupt unrealistic displacements between consecutive frames by limiting the change in shift relative to the previous frame.

Parameters:

  • stack (np.ndarray[float]) – 3D array of shape (n_frames, height, width) containing the input AFM image stack.

  • reference_frame (int, optional) – Index of the frame to use as the alignment reference (default 0). Must be within [0, n_frames-1].

  • method ({"fft_cross_correlation", "full_cross_correlation"}, optional) – Alignment method (default “fft_cross_correlation”). FFT-based cross-correlation is generally faster and uses less memory for large frames.

  • mode ({"pad", "crop", "crop_square"}, optional) – How to handle borders after shifting: - “pad”: keep all frames with NaN padding (default) - “crop”: crop to intersection of all frames - “crop_square”: crop to largest centered square

  • debug (bool, optional) – If True, returns additional diagnostic outputs.

  • max_shift (int, optional) – Maximum allowed shift in pixels. Detected shifts are clipped to this range.

  • pre_filter_sigma (float, optional) – Standard deviation of Gaussian filter applied to frames before cross-correlation.

  • max_jump (int, optional) – Maximum allowed change in shift between consecutive frames. If exceeded, the shift is replaced by a linear extrapolation from the previous two frames.

Returns:

  • aligned_stack (np.ndarray[float]) – Aligned 3D stack of frames. Shape may be larger than input to accommodate all shifts.

  • metadata (dict) – Dictionary containing alignment information: - “reference_frame”: int, index of the reference frame - “method”: str, the alignment method used - “mode”: str, border approach used - “shifts”: np.ndarray of shape (n_frames, 2), detected (dy, dx) shifts - “original_shape”: tuple of (height, width) - “aligned_shape”: tuple of (height, width) of the output canvas - “border_mask”: np.ndarray[bool], True where valid frame pixels exist - “pre_filter_sigma”: float or None - “max_shift”: int or None - “max_jump”: int or None

  • debug_outputs (dict, optional) – Returned only if debug=True. Contains: - “shifts”: copy of the shifts array.

Raises:

  • ValueError – If stack.ndim is not 3.

  • ValueError – If method is not one of {“fft_cross_correlation”, “full_cross_correlation”}.

  • ValueError – If reference_frame is not in the range [0, n_frames-1].

Notes

  • Using fft_cross_correlation reduces memory usage compared to full

cross-correlation because it leverages the FFT algorithm and avoids creating large full correlation matrices. - Padding with NaNs allows all frames to be placed without clipping, but may increase memory usage for large shifts. - The function does not interpolate subpixel shifts; all shifts are integer-valued.

Examples

>>> import numpy as np
>>> from playnano.processing.video_processing import align_frames
>>> stack = np.random.rand(10, 200, 200)  # 10 frames of 200x200 pixels
>>> aligned_stack, metadata = align_frames(stack, reference_frame=0)
>>> aligned_stack.shape
(10, 210, 210)  # padded to accommodate shifts
>>> metadata['shifts']
array([[ 0,  0],
    [ 1, -2],
    ...])
playnano.processing.video_processing.crop_square(stack: ndarray) → tuple[ndarray, dict][source]

Crop aligned stack to the largest centered square region.

Parameters:

stack (ndarray of shape (n_frames, height, width)) – Input aligned stack.

Returns:

  • cropped (ndarray) – Cropped stack with square height and width.

  • meta (dict) – Metadata about cropping parameters.

playnano.processing.video_processing.intersection_crop(stack: ndarray) → tuple[ndarray, dict][source]

Crop aligned stack to the largest common intersection region.

Parameters:

stack (ndarray of shape (n_frames, height, width)) – Input aligned stack with NaN padding.

Returns:

  • cropped (ndarray) – Cropped stack containing only valid (non-NaN) pixels, or original stack if no valid pixels exist.

  • meta (dict) – Metadata about cropping parameters.

playnano.processing.video_processing.register_video_processing() → dict[str, Callable][source]

Return a dictionary of registered video processing filters.

Keys are names of the operations, values are the functions themselves. These functions should take a 3D stack (n_frames, H, W) and return either an ndarray (filtered stack) or a tuple (stack, metadata).

playnano.processing.video_processing.replace_nan(stack: ndarray, mode: Literal['zero', 'mean', 'median', 'global_mean', 'constant'] = 'zero', value: float | None = None) → tuple[ndarray, dict][source]

Replace NaN values in a 2D frame or 3D AFM image stack using various strategies.

Primarily used in video pipelines after alignment, but also applicable to single frames.

Parameters:

  • stack (np.ndarray) – Input 3D array of shape (n_frames, height, width) or 2D frame (height, width) that may contain NaN values.

  • mode ({"zero", "mean", "median", "global_mean", "constant"}, optional) – Replacement strategy. Default is “zero”. - “zero” : Replace NaNs with 0. - “mean” : Replace NaNs with the mean of each frame. - “median” : Replace NaNs with the median of each frame. - “global_mean” : Replace NaNs with the mean of the entire stack. - “constant” : Replace NaNs with a user-specified constant value.

  • value (float, optional) – Constant value to use when mode=”constant”. Must be provided in that case.

Returns:

  • filled (np.ndarray) – Stack of the same shape as stack with NaNs replaced according to mode.

  • meta (dict) – Metadata about the NaN replacement operation (e.g., count, mode, constant used).

Raises:

ValueError – If mode is unknown or if mode=”constant” and value is not provided.

Notes

  • Frame-wise operations like “mean” and “median” compute statistics per frame independently.

  • Preserves the dtype of the input stack.

playnano.processing.video_processing.rolling_frame_align(stack: ndarray, window: int = 5, mode: str = 'pad', debug: bool = False, max_shift: int | None = None, pre_filter_sigma: float | None = None, max_jump: int | None = None)[source]

Align a stack of AFM frames using a rolling reference and integer pixel shifts.

This function computes frame-to-frame shifts relative to a rolling reference (average of the last window aligned frames) using phase cross-correlation. Each frame is then placed on a canvas large enough to accommodate all shifts. Optional jump smoothing prevents sudden unrealistic displacements between consecutive frames, and optional Gaussian pre-filtering can improve correlation robustness for noisy data.

Parameters:

  • stack (np.ndarray[float]) – 3D array of shape (n_frames, height, width) containing the image frames.

  • window (int, optional) – Number of previous aligned frames to average when building the rolling reference. Default is 5.

  • mode ({"pad", "crop", "crop_square"}, optional) – How to handle borders after shifting: - “pad”: keep all frames with NaN padding (default) - “crop”: crop to intersection of all frames - “crop_square”: crop to largest centered square

  • debug (bool, optional) – If True, returns additional diagnostic outputs such as the rolling reference frames. Default is False.

  • max_shift (int, optional) – Maximum allowed shift in pixels along either axis. Detected shifts are clipped. Default is None (no clipping).

  • pre_filter_sigma (float, optional) – Standard deviation of Gaussian filter applied to both reference and moving frames prior to cross-correlation. Helps reduce noise. Default is None.

  • max_jump (int, optional) – Maximum allowed jump in pixels between consecutive frame shifts. If exceeded, the shift is replaced by a linear extrapolation from the previous two shifts. Default is None (no jump smoothing).

Returns:

  • aligned_stack (np.ndarray[float]) – 3D array of shape (n_frames, canvas_height, canvas_width) containing the aligned frames. NaN values indicate areas outside the original frames after alignment.

  • metadata (dict) – Dictionary containing alignment information: - “window”: int, rolling reference window used - “method”: str, alignment method used - “mode”: str, border approach used - “shifts”: ndarray of shape (n_frames, 2), detected integer shifts (dy, dx) - “original_shape”: tuple of (height, width) - “aligned_shape”: tuple of (canvas_height, canvas_width) - “border_mask”: ndarray of shape (canvas_height, canvas_width), True where

    valid pixels exist

    • ”pre_filter_sigma”: float or None

    • ”max_shift”: int or None

    • ”max_jump”: int or None

  • debug_outputs (dict, optional) – Returned only if debug=True. Contains: - “shifts”: copy of the detected shifts array - “aligned_refs”: deque of indices used for rolling reference

Raises:

Notes

  • The rolling reference is computed using the last window aligned frames, ignoring NaN pixels.

  • Shifts are integer-valued; no subpixel interpolation is performed.

  • Padding ensures all frames fit without clipping, but increases memory usage.

  • Internally, a deque aligned_refs tracks which patches of which frames contribute to the rolling reference. Each entry stores:

    (frame_index, y0c, y1c, x0c, x1c, fy0, fy1, fx0, fx1),

    i.e. both the region of the canvas updated and the corresponding slice in the original frame. This allows exact removal of old contributions from rolling_sum and rolling_count when the window is exceeded, ensuring consistency without recomputation.

Examples

>>> import numpy as np
>>> from playnano.processing.video_processing import rolling_frame_align
>>> stack = np.random.rand(10, 200, 200)  # 10 frames of 200x200 pixels
>>> aligned_stack, metadata = rolling_frame_align(stack, window=3)
>>> aligned_stack.shape
(10, 210, 210)
>>> metadata['shifts']
array([[0, 0],
       [1, -1],
       ...])
playnano.processing.video_processing.temporal_mean_filter(stack: ndarray, window: int = 3) → ndarray[source]

Apply mean filter across the time dimension.

Parameters:

  • stack (ndarray of shape (n_frames, height, width)) – Input stack.

  • window (int, optional) – Window size (number of frames). Default is 3.

Returns:

filtered – Stack after temporal mean filtering.

Return type:

ndarray of shape (n_frames, height, width)

playnano.processing.video_processing.temporal_median_filter(stack: ndarray, window: int = 3) → ndarray[source]

Apply median filter across the time dimension.

Parameters:

  • stack (ndarray of shape (n_frames, height, width)) – Input stack.

  • window (int, optional) – Window size (number of frames). Default is 3.

Returns:

filtered – Stack after temporal median filtering.

Return type:

ndarray of shape (n_frames, height, width)