Driver Creation Guide

What Drivers Do

A driver is a JSON file that acts as a creative brief for Onset Engine, telling it what visual content to assign to each energy tier of your music.

Without a driver, the engine falls back to using basic raw CLIP similarity and motion scores. With a driver, you get full control over the AI’s clip selection. It steers the selection process by using CLIP embeddings to find the closest match for the current music intensity across predefined energy tiers.

JSON Structure

A driver is written in JSON format (specifically schema v3) and is composed of the following key sections:

meta: Basic information like name, version, and description.
global: Settings applied to every tier, such as a baseline min_rating, exclude_tags, and tier_thresholds.
tiers: The heart of the driver, representing four distinct energy levels (1_LOW, 2_MED, 3_HIGH, 4_MAX).

Each tier contains:

descriptions: A list of short text descriptions of the content.
subjects / tags: Specific tags you’ve identified in your library.
Optional preferences like moods, scene_types, and min_rating.

Example Driver Structure

Here is a partial look at how a driver structures its tiers based on drivers/examples/action_movie.json:

{
    "meta": {
        "name": "Action Movie",
        "version": "3.0"
    },
    "tiers": {
        "1_LOW": {
            "descriptions": [
                "character talking in a quiet room",
                "city skyline establishing shot"
            ],
            "moods": ["serene", "neutral"],
            "scene_types": ["wide"]
        },
        "4_MAX": {
            "descriptions": [
                "massive explosion with fire and debris",
                "slow-motion bullet impact"
            ],
            "moods": ["epic", "chaotic"],
            "min_rating": 4
        }
    }
}

How Text Descriptions Map to CLIP Vectors

The descriptions array in each tier is what powers the AI’s visual matching.

When the engine processes a driver, each natural language phrase (e.g., "city skyline establishing shot") is encoded into a mathematical 768-dimensional vector using a CLIP text encoder.

During editing, the engine computes cosine similarity between the description vector and the pre-computed embedding vectors for every clip in your library. It then ranks the clips to find the best match for the current tier’s text.

Optional Preferences

Beyond text matching, a tier can express preferences about the clips it picks. These are not filters. Despite the naming, none of them removes a clip from consideration — each one multiplies the clip’s similarity score, so a non-matching clip can still win if nothing better fits the moment. That is deliberate: a hard filter would leave gaps in your edit when the library runs out of qualifying clips.

moods: The AI’s mood classification (e.g., epic, serene, tense). A clip whose mood doesn’t match takes a 0.50x score penalty.
scene_types: Scene composition (e.g., wide, close-up, aerial). Mismatches take a 0.60x penalty.
min_rating: An integer (0–5). Clips rated below it take a 0.70x penalty — so the tier leans hard toward your best material without ever being unable to fill a slot.

The penalties stack: a clip missing both the preferred mood and scene type scores at 0.50 × 0.60 = 0.30x. In practice that is enough to sink it unless the alternatives are far worse matches.

Using `@Tag` References

If you have already tagged specific subjects or themes in the Onset Engine GUI, you can reference them directly in your driver using the @ syntax in the tags (or legacy subjects) array.

{
    "3_HIGH": {
        "descriptions": ["fast punches", "beam clash"],
        "tags": ["@Goku", "@Vegeta"]
    }
}

These tags do not match as plain text. Instead, they resolve directly to the library tag centroids (an averaged vector of all clips you explicitly tagged).

@Tag references are blended with the description score, not AND-ed with it. The engine takes the clip’s best similarity against any tag centroid in the tier and mixes it into the final score at the driver’s tag_weight (default 0.3, i.e. 70% description / 30% tag).

So a tag steers a tier strongly toward a subject, but it does not lock the tier to them — a clip that matches the descriptions superbly can still be picked over a weaker clip of the right person. Raise tag_weight if you want the subject to dominate.

Testing with Different Footage: Because drivers rely on CLIP embeddings rather than strict filenames, the exact same JSON driver can be used across completely different sets of footage. For example, a “Wedding Highlights” driver can be run on five different wedding folders, and the engine will dynamically adapt to find the best matching clips in each respective folder.

Sharing Drivers: Since driver files are just plain JSON, sharing them is simple. Send the .json to another Onset Engine user; it will work against their own footage library immediately.

There is no folder a driver has to live in. Onset Engine loads a driver by its full path, so save it anywhere convenient and pick it with Creative Direction → Driver → Browse. No restart is required.

The drivers bundled with the app are in C:\Program Files\Onset Engine\drivers\examples and are worth reading as references — but keep drivers you write somewhere writable, since that folder needs admin rights and a reinstall can overwrite it.