Skip to content

Archive Summarization

This document corresponds to the Archive Summarization example folder within a VIAME desktop installation.

Overview

This example demonstrates how to process video archives to create:

  1. Searchable Index - An indexed database of video content for visual queries
  2. Detection Plots - Timeline visualizations showing organism counts over time
  3. Interactive Interfaces - Web-based tools for searching and browsing results

This is useful for processing large video archives from underwater cameras (MOUSS, drop cameras, ROVs, etc.) where you want to quickly find specific content or analyze species distributions over time.

Available Scripts

Script Description
summarize_and_index_videos Full processing: detection, plots, and index
summarize_videos Detection and plots only (no search index)
launch_timeline_interface View detection timeline plots
launch_search_interface Query the indexed database visually

Quick Start

Step 1: Process your videos

Edit the script to point to your video directory, then run:

Linux:

./summarize_and_index_videos.sh

Windows:

summarize_and_index_videos.bat

Step 2: View results

For timeline visualization:

./launch_timeline_interface.sh

For search interface:

./launch_search_interface.sh

Script Configuration

The summarize_and_index_videos script accepts several key parameters:

  • -d INPUT_DIRECTORY - Path to your video folder
  • -p pipelines/index_generic.pipe - Detection/indexing pipeline to use
  • -plot-threshold 0.25 - Minimum detection confidence for plotting
  • -frate 2 - Frame rate for processing (frames per second)
  • -plot-smooth 2 - Smoothing factor for timeline plots
  • --build-index - Enable building the search index
  • --detection-plots - Enable generation of detection plots

Output Structure

After processing, results are organized as:

database/
├── video1_detections.csv    # Detection results
├── video1_timeline.png      # Species timeline plot
├── video2_detections.csv
├── video2_timeline.png
└── ...

index/                       # Searchable index (if --build-index used)
└── ...

Pipeline Selection

The default uses pipelines/index_generic.pipe which runs a generic fish detector. For other use cases, you may need to modify the pipeline or use a different pre-trained model.

Available pipelines vary by installation but may include:

  • index_generic.pipe - Default detection and indexing
  • index_default_fish.svm.pipe - Fish detection with SVM classifier
  • index_frame.pipe - Frame-level indexing
  • Custom pipelines for your specific use case

DIVE Documentation

Code and Build Flags

Flags to enable when building VIAME from source for this example:

  • VIAME_ENABLE_PYTHON
  • VIAME_ENABLE_PYTORCH
  • VIAME_ENABLE_PYTORCH-RF-DETR
  • VIAME_ENABLE_PYTORCH-VISION
  • VIAME_ENABLE_SVM
  • VIAME_ENABLE_VXL

Add-ons providing the pipelines or models used: generic.

Command line tools:

  • tools/run.py – viame run
  • tools/search.py – viame search

Pipeline and configuration files:

  • configs/pipelines/query_retrieval_and_iqr.pipe
  • configs/add-ons/generic/index_generic.pipe
  • configs/add-ons/default-fish/index_default_fish.svm.pipe
  • configs/pipelines/index_frame.pipe

Source code:

  • plugins/core/average_track_descriptors.cxx
  • plugins/core/create_database_query_process.cxx
  • plugins/core/filter_frame_process.cxx
  • plugins/core/query_track_descriptor_set_csv.cxx
  • plugins/core/refine_detections_add_fixed.cxx
  • plugins/core/refine_detections_nms.cxx
  • plugins/core/select_database_query_process.cxx
  • plugins/core/windowed_detector.cxx
  • plugins/pytorch/rf_detr_detector.py
  • plugins/pytorch/torchvision_descriptors.py
  • plugins/svm/process_query_process.cxx