Metadata

Metadata that travels through your OptiView Live stream — whether pushed via the API or embedded as SEI — surfaces on the player as cues on a text track. The general approach is the same in every case:

1. Listen for addtrack on player.textTracks so you find out when a new metadata track shows up.
2. Subscribe to a cue event on that track.
3. Inspect the cue to make sure it carries the metadata you care about.
4. Read the payload from cue.content.

The properties you match on, the cue event you subscribe to, and the shape of cue.content all depend on which kind of metadata is being delivered.

User-data-unregistered SEI and API push

UDU SEI and API-pushed payloads surface as ID3 PRIV frames inside an ID3 v2.4 tag embedded in an emsg box, carried by the CMAF segments of the stream.

• The owner identifier of the PRIV frame will be set to: optiview.live:meta:<your-uuid>
• The frame's private data will contain your metadata as raw binary data.

(See section 4.27 of the ID3 v2.4 Native Frames specification and ID3 in CMAF for more details.)

The OptiView Player Web SDK exposes these ID3 frames on a text track with TextTrack.type equal to 'id3'. Listen for the cue event that matches when you want to react:

• addcue fires as soon as a cue becomes available on the track, before playback reaches it — useful when you want to prefetch data or update application state ahead of time.
• entercue fires when playback enters the cue — useful when you want to act in sync with the video, for example to render an overlay at the moment the cue was inserted.

The ID3 frame will be contained in cue.content as a ID3PrivateFrame with ownerIdentifier equal to optiview.live:meta:<your-uuid>.

important

Make sure to always check the frame's id and ownerIdentifier before using it, to avoid processing unrelated ID3 frames.

Suppose your UUID is 11111111-1111-1111-1111-111111111111:

player.textTracks.addEventListener('addtrack', function trackListener(event) {
  const track = event.track;
  if (track.type === 'id3') {
    // ID3 track was added. Listen for incoming ID3 cues.
    track.addEventListener('addcue' /* or 'entercue' */, (e) => {
      const frame = e.cue.content;
      if (frame.id === 'PRIV' && frame.ownerIdentifier === 'optiview.live:meta:11111111-1111-1111-1111-111111111111') {
        console.log('metadata', frame.data);
      }
    });
    track.removeEventListener('typechange', trackListener);
  } else if (track.type === '') {
    // Track type is not yet known. Check again when it becomes known.
    track.addEventListener('typechange', trackListener);
  } else {
    // Track type is known, but is not ID3. Stop listening.
    track.removeEventListener('typechange', trackListener);
  }
});

Here the frame.data carries the binary payload exactly as it was sent — your application is responsible for parsing it (for example UTF-8 JSON, protobuf, or your own format).

Using third-party players

When using a third-party player SDK, please refer to their respective documentation on how to retrieve ID3 from the stream.

For example, with Shaka Player, you can listen for the metadata event:

player.addEventListener('metadata', (event) => {
  if (event.metadataType === 'org.id3') {
    const frame = event.payload; // see https://shaka-player-demo.appspot.com/docs/api/shaka.extern.html#.MetadataFrame
    if (frame.key === 'PRIV' && frame.description === 'optiview.live:meta:11111111-1111-1111-1111-111111111111') {
      console.log('metadata', frame.data);
    }
  }
});

With hls.js, you can listen for the FRAG_PARSING_METADATA event. However, hls.js doesn't come with a built-in ID3 parser, so you'll need to parse the ID3 tag manually:

hls.on(Hls.Events.FRAG_PARSING_METADATA, (event, data) => {
  for (const sample of data.samples) {
    if (sample.type === Hls.MetadataSchema.emsg) {
      const tag = sample.data;
      // TODO Parse ID3 tag and look for ID3 PRIV frame with correct owner identifier
    }
  }
});

Picture timing SEI

Picture timing SEI cues surface on a text track with TextTrack.id equal to 'timecode'. Each cue is anchored to a frame, so entercue is usually the most useful event: it fires when playback reaches the cue, letting you react in sync with the video.

player.textTracks.addEventListener('addtrack', (event) => {
  const track = event.track;
  if (track.id === 'timecode') {
    track.mode = 'showing'; // Setting the mode to showing will enable the entercue events
    track.addEventListener('entercue', onTimeCode);
  }
});

The cue content is a structured TimeCode object:

export interface TimeCode {
  readonly hours: number;
  readonly minutes: number;
  readonly seconds: number;
  readonly frames: number;
}

A handler can then turn that timecode into whatever action your app needs:

const onTimeCode = (event) => {
  const timeCode = event.cue.content;
  // Insert other code here
};

A full example could be as follows: you distribute sports matches and want to display an overlay on the player when the score changes. Every time a score change happens, you add a picture timing SEI message to the stream and store in your backend that this time corresponds with this score. (You could also just add the timing message to all frames if this is easier, but this requires more processing both server and player side.) This information can then be retrieved by the application running on the device of a viewer. You then add the event listener and listen for the entercue events. In the listener you check whether the TimeCode corresponds with a score change event you recorded earlier and if so, display an overlay over the player.