Introduction to video annotation

What is the tool?

A video annotation tool is used for annotating objects in a video by drawing a bounding box around the objects and/or placing key points on the required frames. You might also have to label the objects and relate them to each other. 

What will you learn?

This document will contain all the information needed for you to perform video annotation tasks using our new video annotation tool. You will learn:

1. What a workflow looks like
2. How to create annotations: For example, key points and bounding boxes
3. How to edit and adjust bounding boxes 
4. How to navigate the frame layout
5. How to hide and lock annotations
6. How to save and submit tasks

What are the components?

You will be given a video and the tools needed to draw bounding boxes and/or key points. 

Here’s what it looks like on the platform

There are two parts to the workflow. The top half and the bottom half. Let’s take a look at the tools on the top half:

Top Half

Top Half 

Video

• At the center of the workflow you can see a video. 
• The length of the video varies from different tasks. Sometimes it can be a few seconds long or even minutes. 
• The type of video also varies depending on the task. It can be an eye-level view, bird’s eye view, CCTV, and others. 

Example of video

Tools

On the two sides of the video, you are able to see tool symbols are used to annotate the objects and adjust the video appearances. 

Here is the list of the tools provided along with their functions:

Tools	Functions
	To adjust the brightness of the video. You may drag the range from 0 - 200.  Please note the default value is set at 100.
	To adjust the contrast of the video. You may drag the range from 0 - 200.
	To reset the brightness and the contrast of the video to its original appearance.
	To help you zoom in to the video. The video will gradually get closer every time you click.
	To reset the size of the video to its original position.
	To help you zoom out the video. The video will gradually get further every time you click.
	To hide or show labels of objects that you have annotated. It will only work if there is at least one annotation in the video.
	To switch on/off the sound of the video, if the video contains sound.

Tools	Functions
	To draw a bounding box around the objects.
	To draw a keypoint on the frame.
	To draw a polygon to segment an object.
	To draw a polyline to segment an object.
	To undo the changes you have made.
	To redo the changes you have made.
	To delete the annotations that you have made.

Instructions 

On the left side of the workflow you can see there are three instructions on the information panel: 

1. Hotkeys
2. Release Notes
3. Pop-Out Button

Instructions

Hotkeys 

• Hotkeys are shown to guide you on the keyboard shortcuts that can be useful during the task. 
• There are two different hotkeys:

1. For non-Mac
2. For Mac

• (Note: Non-mac refers to the devices used for the task that is NOT a Macbook model by Apple Inc. )

Hotkeys for non-MAC users

While Mac hotkeys refer to the devices used for the task that is a Macbook model by Apple Inc.

Hotkeys for Mac users

Release Notes

Release Notes generally provide updates on new features and enhancement made to the video annotation tool. 

Example of release notes

Pop-out button

The Pop-Out button is to show the detailed instructions of the task in a separate window that can be minimized.

Example of pop-out button

Bottom Half

Bottom Half

Video Frames

• Video frames are a series of photos that are combined together to form a single video. 
• Video frames are important in the task as it requires accurate annotations on a particular millisecond of the video. Therefore, frames help the viewer capture specific frames in a video. 
• Below the video, you can see the layout of frames in the form of layers or better known as the timeline. 
• If you move your mouse cursor on the timelines, it will allow you to find a specific frame of your choosing. 

Example of frames

• You can also zoom in and out of the timelines. For instance, if you prefer to see all the frames in the layout, you may zoom out to maximum until all the frames appear. 
• The size of the layers varies depending on the frames shown.  You can find these zoom buttons on the right side of the layout.

Example of zooming in/out frame

Video Navigator 

Below the video, a navigation bar is provided at the playback toolbar to assist you with navigating the video while you annotate. 

Video Navigator

Here is the list of the buttons and their functions:

Tools	Functions
	This feature allows you to jump to a specific frame of your choosing by inserting the frame number.
	This button allows you to choose the speed of the video. The speeds are:  1x, 0.75x, 0.5x, and 0.25x, with 1x being the fastest speed.
	Allows you to jump to the first frame.
	Allows you to jump 2 or more frames to the back. The number of frames to be skipped can be determined at the “seek __ frames at a time” bar (most right).
	Go back one frame.
	To play the video.
	To pause the video.
	To go one frame forward.
	Allows you to jump 2 or more frames to the front. The number of frames to be skipped can be determined at the “seek __ frames at a time” bar (most right).
	Allows you to jump to the last frame
	Allows you to set the number of frames you wish to skip. After the number is set, you can click on the jump-few-frames button.

You can also see the length of the video and the total number of frames on the left side of the video navigator.

Entity Labelling Tools 

• Once you have annotated the object, the entity labelling tools will be activated to reflect it on the entity panel.
• These tools can be found on the left side of the frames layout.  

Annotation labels tools

Here are the list of buttons and their functions:

Tools	Functions
	To label the annotations of required objects. It will be activated once annotation is drawn.
	To hide or show all annotations on the video. If you wish to hide only one annotation, you may click the same icon in the specific annotations of your choosing.
	To lock or unlock all annotations on the video. If you wish to lock only one annotation, you may click the same icon in the specific annotations of your choosing.
	To hide or show the list of all annotations you have made. If you wish to hide only one annotation, you may click the same icon in the specific annotations of your choosing.
	The search bar is used to find annotations with a keyword. The annotations label tab will only show the specific annotations according to the word on the search bar.  You can search more than one entity by separating them using a comma.
	The hierarchical relationship view is used to hide or show the details of the annotation that you have made. It will be activated with at least 1 annotation.
	The entity type view is used to hide or show the list in the grey colored tab of all the annotations available in the task.

• Once you have annotated an object, there are a few information needed inputs (if necessary) in the grey area. 
• These inputs vary according to tasks: 

1. Add name
2. Select classification / relationship from dropdown option

Here’s the list of the inputs and their functions.

Tools	Functions
	This feature allows you to add a descriptive name for the label to help you identify a specific annotation.
	This feature requires you to select the classification/ relationship from a dropdown option.

Compound Search

• A compound search is used when you would like to show a specific entity on the entity label to help you organize or find entities easily. 
• You can do this by typing entities or their descriptive names in the search bar. 
• The other entities will be hidden once you have typed in the words.

However, please note that if there is no annotation done, you will not be able to search for it in the compound search.

Compound Search

Entity without annotation

You can also show two or more entities at once by placing a comma “, “ in-between the words.

Two or more entities

Hierarchical Relationship View

• A hierarchical relationship view   is a feature to show you the main entity with its relations to other entities in a hierarchical form.
• If an entity (e.g. hand) is related to the main entity (e.g person), it will be shown and connected under the main entity.

Hierarchical relationship view

Entity Type View 

• Entity type view is a feature to show you entities by categories in a list form. 
• If two same entities (e.g. person 1 and person 2) are annotated, they will be shown in the same categories without the connection to its relationship.

Entity type view

Hierarchical type view vs Entity type view