Cloud Database & RAG

Zapdos uses a sophisticated cloud database architecture built on the Entity-Attribute-Value (EAV) model to store and retrieve video indexing results. This flexible design allows for extensibility while enabling powerful Retrieval-Augmented Generation (RAG) capabilities. All processed results are automatically stored in our cloud database and can be accessed through the developer portal at app.zapdoslabs.com.

Database Architecture

Our cloud database is built using LanceDB, an efficient vector database that supports both traditional data storage and vector embeddings. The system uses an Entity-Attribute-Value (EAV) model for maximum flexibility in storing diverse types of video analysis data.

Entity-Attribute-Value (EAV) Model

The EAV model provides flexibility in storing varying types of information:

Component	Description
Entities	Media units (videos, frames, segments)
Attributes	Named properties of these units
Values	The actual data, which can be strings, numbers, or vector embeddings

This model allows us to store different types of information for different media units without requiring schema changes, making the system highly adaptable to new analysis capabilities.

Core Data Structures

Media Units

Media units represent the fundamental building blocks of indexed content:

Type	Description
Video Files	Root-level entities representing entire videos
Video Frames	Individual frames extracted from videos
Segments	Temporal segments of videos with specific characteristics

Each media unit contains metadata about its properties:

Property	Type	Description
`id`	String	Unique identifier for the media unit
`type`	String	Type of media unit (video_file, video_frame, segment)
`root_id`	String	ID of the parent video file
`from_ms`	Float	Start time in milliseconds
`to_ms`	Float	End time in milliseconds
`x`	Float	X-coordinate (for spatial data)
`y`	Float	Y-coordinate (for spatial data)
`width`	Float	Width (for spatial data)
`height`	Float	Height (for spatial data)

Fields

Fields store specific attributes and analysis results for media units:

Property	Type	Description
`media_unit_id`	String	ID of the associated media unit
`name`	String	Name of the attribute
`string_value`	String	Text value (if applicable)
`number_value`	Float	Numeric value (if applicable)
`vector`	Array	Vector embedding for semantic search

RAG Capabilities

The combination of structured metadata and vector embeddings enables powerful Retrieval-Augmented Generation capabilities:

Capability	Description
Semantic Search	Find similar frames or segments based on content meaning
Hybrid Queries	Combine traditional filters with vector similarity searches
Contextual Retrieval	Retrieve relevant video segments for generative AI applications

Example Data

Media Unit Records

Here's an example of actual records from our database showing a video file and its associated frames:

[
  {
    "id": "06db07d4-8b7d-4f88-9ae1-80b16824dd8b",
    "type": "video_file",
    "value": {
      "from_ms": 0.0,
      "to_ms": 6007000.0,
      "width": 1920.0,
      "height": 1080.0
    }
  },
  {
    "id": "a30dbab3-53a8-4393-9fd9-82c33add8ad9",
    "type": "video_frame",
    "root_id": "06db07d4-8b7d-4f88-9ae1-80b16824dd8b",
    "value": {
      "from_ms": 0.0,
      "to_ms": 0.0,
      "width": 1920.0,
      "height": 1080.0
    }
  }
]

Field Records

Associated analysis results for media units:

[
  {
    "media_unit_id": "57e5fae5-df76-4ad6-8b87-a701c648ef83",
    "name": "image-description",
    "value": {
      "string_value": "Two men are seated in front of a fireplace, engaged in a conversation. The man on the left is wearing a gray blazer over a blue shirt, while the man on the right is dressed in a black suit with a white shirt..."
    }
  }
]

When you index a video, all the structured output is stored in this database and can be queried using both traditional database operations and vector similarity searches.

Cloud Database & RAG

On this page