Introduction to video annotation

  • Updated

What is the tool?

A video annotation tool is used for annotating objects in a video by drawing a bounding box around the objects and/or placing key points on the required frames. You might also have to label the objects and relate them to each other. 

 

What will you learn?

This document will contain all the information needed for you to perform video annotation tasks using our new video annotation tool. You will learn:

  1. What a workflow looks like
  2. How to create annotations: For example, key points and bounding boxes
  3. How to edit and adjust bounding boxes 
  4. How to navigate the frame layout
  5. How to hide and lock annotations
  6. How to save and submit tasks

 

What are the components?

You will be given a video and the tools needed to draw bounding boxes and/or key points. 

image30.png

image96.jpg

Here’s what it looks like on the platform

 

There are two parts to the workflow. The top half and the bottom half. Let’s take a look at the tools on the top half:

image52.jpg

Top Halfimage53.png

Top Half 

Video

  • At the center of the workflow you can see a video. 
  • The length of the video varies from different tasks. Sometimes it can be a few seconds long or even minutes. 
  • The type of video also varies depending on the task. It can be an eye-level view, bird’s eye view, CCTV, and others. 

image16.jpg

Example of video

 

Tools

On the two sides of the video, you are able to see tool symbols are used to annotate the objects and adjust the video appearances

image36.jpg

Here is the list of the tools provided along with their functions:

Tools Functions
image23.png To adjust the brightness of the video. You may drag the range from 0 - 200.  Please note the default value is set at 100.
image22.png To adjust the contrast of the video. You may drag the range from 0 - 200.
image15.png To reset the brightness and the contrast of the video to its original appearance.
image68.png To help you zoom in to the video. The video will gradually get closer every time you click.
image2.png To reset the size of the video to its original position.
image64.png To help you zoom out the video. The video will gradually get further every time you click.
image12.png To hide or show labels of objects that you have annotated. It will only work if there is at least one annotation in the video.
image1.png To switch on/off the sound of the video, if the video contains sound.

 

image62.jpg

Tools Functions
Group_50.png To draw a bounding box around the objects. 
keypoint.png To draw a keypoint on the frame.
polygon.png To draw a polygon to segment an object.
line.png To draw a polyline to segment an object.
image35.png To undo the changes you have made.
image61.png To redo the changes you have made.
delete.png

To delete the annotations that you have made.

 

Instructions 

On the left side of the workflow you can see there are three instructions on the information panel: 

  1. Hotkeys
  2. Release Notes
  3. Pop-Out Button popout.png

image48.jpg

Instructions

Hotkeys 

  • Hotkeys are shown to guide you on the keyboard shortcuts that can be useful during the task. 
  • There are two different hotkeys:
  1. For non-Mac
  2. For Mac
  • (Note: Non-mac refers to the devices used for the task that is NOT a Macbook model by Apple Inc. )

HK_general_non-mac.png

Hotkeys for non-MAC users

 

HK_shape_non-mac.png

HK_video_non-mac.png

 

HK_image_non-mac.png

 

While Mac hotkeys refer to the devices used for the task that is a Macbook model by Apple Inc.

HK_general_mac.png

Hotkeys for Mac users

HK_shape_mac.png

HK_video_mac.png

HK_image_mac.png

Release Notes

Release Notes generally provide updates on new features and enhancement made to the video annotation tool. 

image55.png

Example of release notes

 

Pop-out button

The Pop-Out button is to show the detailed instructions of the task in a separate window that can be minimized.

image16.png

Example of pop-out button

 

Bottom Half

image54.png

Bottom Half

 

Video Frames

  • Video frames are a series of photos that are combined together to form a single video. 
  • Video frames are important in the task as it requires accurate annotations on a particular millisecond of the video. Therefore, frames help the viewer capture specific frames in a video. 
  • Below the video, you can see the layout of frames in the form of layers or better known as the timeline. 
  • If you move your mouse cursor on the timelines, it will allow you to find a specific frame of your choosing. 

image21.gif

Example of frames

 

  • You can also zoom in and out of the timelines. For instance, if you prefer to see all the frames in the layout, you may zoom out to maximum until all the frames appear. 
  • The size of the layers varies depending on the frames shown.  You can find these zoom buttons on the right side of the layout.

image12.gif

Example of zooming in/out frame

 

Video Navigator 

Below the video, a navigation bar is provided at the playback toolbar to assist you with navigating the video while you annotate. 

image73.jpg

Video Navigator

image5.jpg

Here is the list of the buttons and their functions:

Tools Functions
image14.png This feature allows you to jump to a specific frame of your choosing by inserting the frame number.
image20.png This button allows you to choose the speed of the video. The speeds are:  1x, 0.75x, 0.5x, and 0.25x, with 1x being the fastest speed.
image59.png Allows you to jump to the first frame.
image60.png Allows you to jump 2 or more frames to the back. The number of frames to be skipped can be determined at the “seek __ frames at a time” bar (most right).
image50.png Go back one frame.
image71.png To play the video.
image34.png To pause the video.
image80.png To go one frame forward.
image48.png Allows you to jump 2 or more frames to the front. The number of frames to be skipped can be determined at the “seek __ frames at a time” bar (most right).
image43.png Allows you to jump to the last frame
image62.png Allows you to set the number of frames you wish to skip. After the number is set, you can click on the jump-few-frames button.

 

You can also see the length of the video and the total number of frames on the left side of the video navigator.

image41.png

 

Entity Labelling Tools 

  • Once you have annotated the object, the entity labelling tools will be activated to reflect it on the entity panel.
  • These tools can be found on the left side of the frames layout.  

image29.png

Annotation labels tools

image63.gif

 

Here are the list of buttons and their functions:

Tools Functions
image2.png To label the annotations of required objects. It will be activated once annotation is drawn. 
image72.png To hide or show all annotations on the video. If you wish to hide only one annotation, you may click the same icon in the specific annotations of your choosing.
image81.png To lock or unlock all annotations on the video. If you wish to lock only one annotation, you may click the same icon in the specific annotations of your choosing.
image35.png To hide or show the list of all annotations you have made. If you wish to hide only one annotation, you may click the same icon in the specific annotations of your choosing.
image23.png

The search bar is used to find annotations with a keyword. The annotations label tab will only show the specific annotations according to the word on the search bar. 

You can search more than one entity by separating them using a comma.

image31.png The hierarchical relationship view is used to hide or show the details of the annotation that you have made. It will be activated with at least 1 annotation.
image44.png The entity type view is used to hide or show the list in the grey colored tab of all the annotations available in the task.

 

  • Once you have annotated an object, there are a few information needed inputs (if necessary) in the grey area
  • These inputs vary according to tasks: 
  1. Add name
  2. Select classification / relationship from dropdown option

image6.png

Here’s the list of the inputs and their functions.

Tools Functions
image15.png

This feature allows you to add a descriptive name for the label to help you identify a specific annotation. 

image3.png This feature requires you to select the classification/ relationship from a dropdown option.

 

Compound Search

  • A compound search is used when you would like to show a specific entity on the entity label to help you organize or find entities easily. 
  • You can do this by typing entities or their descriptive names in the search bar
  • The other entities will be hidden once you have typed in the words.

However, please note that if there is no annotation done, you will not be able to search for it in the compound search.

image78.gif

Compound Search

image27.gif

Entity without annotation

 

You can also show two or more entities at once by placing a comma “, “ in-between the words.

image66.gif

Two or more entities

 

Hierarchical Relationship View

  • A hierarchical relationship view image31.png  is a feature to show you the main entity with its relations to other entities in a hierarchical form.
  • If an entity (e.g. hand) is related to the main entity (e.g person), it will be shown and connected under the main entity.

image88.png

Hierarchical relationship view

Entity Type View 

  • Entity type view image44.png is a feature to show you entities by categories in a list form. 
  • If two same entities (e.g. person 1 and person 2) are annotated, they will be shown in the same categories without the connection to its relationship.

 

image8.png

Entity type view

image57.gif

Hierarchical type view vs Entity type view

 
Share This