AI Vision#

Introduction#

The VEX AIM Coding Robot’s AI Vision Sensor detects and tracks objects, colors, and AprilTags. This allows the robot to analyze its surroundings, follow objects, and react based on detected visual data. Below is a list of all available blocks:

Actions – Control the AI Vision feed and capture object data.

  • AI Vision viewer – Enables or disables the live AI Vision feed on the robot’s screen.

  • get object data – Captures data for a specific object type, such as colors, pre-trained objects, or AprilTags.

Settings – Adjust which detected objects are accessed.

Values – Return object presence, classification, and properties.

Actions#

AI Vision viewer#

The AI Vision viewer block enables or disables the live AI Vision feed on the robot’s screen. When enabled, the screen displays real-time sensor data, preventing other images or text from appearing. To display other content, use this block to hide the feed.

Theshow or hide AI Vision dashboard stack block.#
  AI Vision [show v] viewer on screen

Parameters

Description

status

Controls the display of the data feed on the robot’s screen:

  • show - Displays the data feed.
  • hide - Removes the data feed from the screen.

Example

A stack of blocks that begins with a when started block, followed by a comment block reading View the AI Vision Sensor’s feed for five seconds. A show AI Vision dashboard block displays the feed, then a wait block pauses execution for five seconds. After the wait, a hide AI Vision dashboard block removes the feed from view.#
  when started :: hat events
  [View the AI Vision Sensor's feed for five seconds.]
  AI Vision [show v] viewer on screen
  wait [5] seconds
  AI Vision [hide v] viewer on screen

get object data#

The get object data block filters data from the AI Vision Sensor frame. The AI Vision Sensor can detect signatures that include pre-trained objects, AprilTags, or configured colors and color codes.

Colors and color codes must be configured first in the AI Vision Utility before they can be used with this block.

The dataset stores objects ordered from largest to smallest by width, starting at index 0. Each object’s properties can be accessed using AI Vision object property block. An empty dataset is returned if no matching objects are detected.

The Get object data stack block.#
  get [sports ball v] data from AI Vision

Parameter

Description

signature

Filters the dataset to only include data of the given signature. Available signatures are:

  • sports ball
  • orange barrel
  • blue barrel
  • AIM robot
  • AprilTag ID 0
  • AprilTag ID 1
  • AprilTag ID 2
  • AprilTag ID 3
  • AprilTag ID 4
  • all cargo – Sports balls and all barrels
  • NAME – A color or color code where NAME is the name configured in the AI Vision Utility

Example

A stack of blocks that begins with a when started block, followed by a comment block reading Move forward if a Sports Ball is detected. A take a AI Vision snapshot block captures an image of a sports ball. An if block then checks if an AI Vision object exists. If true, a move forward block moves the robot forward by 10 millimeters.#
  when started :: hat events
  [Move forward if a sports ball is detected.]
  forever
  get [sports ball v] data from AI Vision
  if <AI Vision object exists?> then
  move [forward v] for [10] [mm v] ▶

Color Signatures#

A Color Signature is a unique color that the AI Vision Sensor can recognize. These signatures allow the sensor to detect and track objects based on their color. Once a Color Signature is configured, the sensor can identify objects with that specific color in its field of view.

Color Signatures are used in the Get object data block to process and detect colored objects in real-time.

The AI Vision Utility showing a connected vision sensor detecting two colored objects. The left side displays a live camera feed with a blue box on the left and a red box on the right, each outlined with white bounding boxes. Black labels display their respective names, coordinates, and dimensions. The right side contains color signature settings, with sliders for hue and saturation range for both the red and blue boxes. Buttons for adding colors, freezing video, copying, and saving the image are at the bottom, along with a close button in the lower right corner.

Example

  when started :: hat events
  [Display if any objects matching the Red_Box signature is detected.]
  forever
  set cursor to row [1] column [1] on screen
  clear row [1] on screen
  [Change the signature to any configured Color Signature.]
  get [Red_Box v] data from AI Vision
  if <AI Vision object exists?> then
  print [Color signature detected!] on screen ▶

Color Codes#

A Color Code is a structured pattern made up of 2 to 4 Color Signatures arranged in a specific order. These codes allow the AI Vision Sensor to recognize predefined patterns of colors.

Color Codes are particularly useful for identifying complex objects, aligning with game elements, or creating unique markers for autonomous navigation.

The AI Vision Utility interface shows a connected vision sensor detecting two adjacent objects, a blue box on the left and a red box on the right, grouped together in a single white bounding box labeled BlueRed. Detection information includes angle (A:11°), coordinates (X:143, Y:103), width (W:233), and height (H:108). On the right panel, three color signatures are listed: Red_Box, Blue_Box, and BlueRed, with adjustable hue and saturation ranges. The BlueRed signature combines the Blue_Box and Red_Box. Below the video feed are buttons labeled Freeze Video, Copy Image, Save Image, and Close.

Example

  when started :: hat events
  [Display if any objects matching the BlueRed code is detected.]
  forever
  set cursor to row [1] column [1] on screen
  clear row [1] on screen
  [Change the signature to any configured Color Code.]
  get [BlueRed v] data from AI Vision
  if <AI Vision object exists?> then
  print [Color code detected!] on screen ▶

Settings#

set AI Vision object item#

The set AI Vision object item block sets which item in the dataset to use.

The Set AI Vision object item stack block.#
  set AI Vision object item to (1)

Parameters

Description

item

The number of the item in the dataset to use.

Example

A stack of blocks that begins with a when started block, followed by a comment block reading Display the largest detected AprilTag ID. Inside a forever loop, a take a AI Vision snapshot block captures an image of AprilTags. An if block checks if an AI Vision object exists. If true, a set AI Vision object item block assigns the object count to a variable, and a print block displays the tag ID of the detected object on the screen.#
  when started :: hat events
  [Display the largest detected AprilTag ID.]
  forever
  get [all AprilTags v] data from AI Vision
  clear row [1] on screen
  set cursor to row [1] column [1] on screen
  if <AI Vision object exists?> then
  set AI Vision object item to (AI Vision object count)
  print (AI Vision object [tagID v]) on screen ▶

Values#

has object?#

The has object block returns a Boolean indicating whether the robot currently has a detected object against its Kicker.

  • True – The robot has the object.

  • False – The robot does not have the object.

Theshow or hide AI Vision dashboard stack block.#
  <has [sports ball v] ?>

Parameter

Description

object

The object to detect against the robot’s kicker:

  • sports ball
  • any barrel
  • orange barrel
  • blue barrel

Example

A stack of blocks that begins with a when started block, followed by a comment block reading Kick when the robot has a Ball. Inside a forever loop, an if block checks whether the robot has a sports ball. If true, a kick object block executes with medium power.#
  when started :: hat events
  [Kick when the robot has a sports ball.]
  forever
  if <has [sports ball v] ?> then
  kick object [medium v]

AI Vision object exists?#

The AI Vision object exists block returns a Boolean indicating whether any object is detected in the dataset.

  • True – The dataset includes a detected object.

  • False – The dataset does not include any detected objects.

The AI Vision object exists Boolean block.#
  <AI Vision object exists?>

Parameters

Description

This block has no parameters.

Example

A stack of blocks that begins with a when started block, followed by a comment block reading Move forward if a Ball is detected. A take a AI Vision snapshot block captures an image of a sports ball. An if block then checks if an AI Vision object exists. If true, a move forward block moves the robot forward by 10 millimeters.#
  when started :: hat events
  [Move forward if a sports ball is detected.]
  forever
  get [sports ball v] data from AI Vision
  if <AI Vision object exists?> then
  move [forward v] for [10] [mm v] ▶

AI Vision object is?#

The AI Vision object is? block returns a Boolean indicating whether a detected object matches a specific classification.

  • True – The item in the dataset is the specific object.

  • False – The item in the dataset is not the specific object.

The AI Vision AI Classification is object Boolean block.#
  <AI Vision object is [sports ball v] ?>

Parameter

Description

object

Which object to compare the item to:

  • sports ball
  • orange barrel
  • blue barrel
  • AIM robot

Example

A stack of blocks that begins with a when started block, followed by a comment block reading Display if a sports ball is detected. Inside a forever loop, a take a AI Vision snapshot block captures an image using AI Classifications. The screen is cleared, and the cursor is set to row 1, column 1. An if block checks if an AI Vision object exists, and if true, another if block verifies if the classification is a sports ball. If so, a print block displays Ball detected! on the screen, followed by a wait block pausing for 0.5 seconds before repeating the process.#
  when started :: hat events
  [Display if a sports ball is detected.]
  forever
  get [all cargo v] data from AI Vision
  clear row [1] on screen
  set cursor to row [1] column [1] on screen
  if <AI Vision object exists?> then
  if <AI Vision object is [sports ball v] ?> then
  print [Sports ball detected!] on screen ▶
  wait [0.5] seconds

AI Vision object is AprilTag ID?#

The AI Vision object is AprilTag ID? block returns a Boolean indicating whether a detected AprilTag matches a specific ID.

  • True – The AprilTag ID is the number.

  • False – The AprilTag ID is not the number.

The AI Vision detected AprilTag is Boolean block.#
  <AI Vision object is AprilTag [1] ?>

Parameters

Description

AprilTag number

The number to compare against the detected AprilTag’s ID number.

 
  when started :: hat events
  [Report if AprilTag ID 3 is detected.]
  forever
  clear screen
  set cursor to row [1] column [1] on screen
  get [all AprilTags v] data from AI Vision
  if <AI Vision object exists?> then
  if <AI Vision object is AprilTag [3] ?> then
  print [That is 3!] on screen ▶
  else
  print [That isn't 3!] on screen ▶
  end
  end
  wait [0.1] seconds

AI Vision object count#

The AI Vision object count block returns the number of detected objects in the dataset as an integer.

The Set AI Vision object item stack block.#
  AI Vision object count

Parameters

Description

This block has no parameters.

Example

  when started :: hat events
  [Display the amount of sports balls and barrels.]
  forever
  clear row [1] on screen
  set cursor to row [1] column [1] on screen
  get [all cargo v] data from AI Vision
  if <AI Vision object exists?> then
  print (AI Vision object count) on screen ▶
  end
  wait [0.5] seconds

AI Vision object property#

There are nine properties that are included with each object (shown below) stored after the Get object data block is used.

The AI Vision object property reporter block.#
  AI Vision object [width v]

Some property values are based off of the detected object’s position in the AI Vision Sensor’s view at the time that the Get object data block was used. The AI Vision Sensor has a resolution of 320 by 240 pixels.

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box.

Parameter

Description

property

Which property of the detected object to use:

width#

width returns the width of the detected object in pixels as an integer from 1 to 320.

The AI Vision object property stack block with its parameter set to width.#
  AI Vision object [width v]

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box. Two vertical red dashed lines extend from the top of the frame down to the left and right edges of the bounding box. A double-headed black arrow between these lines indicates the width measurement.

Example

  when started :: hat events
  [Move towards a blue barrel until its width is larger than 100 pixels.]
  forever
  get [blue barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [width v]) [math_less_than v] [100]> then
  move [forward v]
  end
  else
  stop all movement

height#

height returns the height of the detected object in pixels as an integer from 1 to 240.

The AI Vision object property stack block with its parameter set to height.#
  AI Vision object [height v]

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box. Two horizontal red dashed lines extend from the left side of the frame to the top and bottom edges of the bounding box. A double-headed black arrow between these lines indicates the height measurement.

Example

  when started :: hat events
  [Move towards a blue barrel until its height is larger than 100 pixels.]
  forever
  get [blue barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [height v]) [math_less_than v] [100]> then
  move [forward v]
  end
  else
  stop all movement

centerX#

centerX returns the x-coordinate of the center of the detected object in pixels as an integer from 0 to 320.

The AI Vision object property stack block with its parameter set to centerX.#
  AI Vision object [centerX v]

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box. A vertical red dashed line extends from the top of the frame down to the center of the bounding box, indicating the X-coordinate of the center.

Example

  when started :: hat events
  [Turn slowly until a blue barrel is centered in front of the robot.]
  set turn velocity to [30] %
  turn [right v]
  forever
  get [blue barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <[140] [math_less_than v] (AI Vision object [centerX v]) [math_less_than v] [180]> then
  stop all movement

centerY#

centerY returns the y-coordinate of the center of the detected object in pixels as an integer from 0 to 240.

The AI Vision object property stack block with its parameter set to centerY.#
  AI Vision object [centerY v]

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box. A horizontal red dashed line extends from the left side of the frame to the center of the bounding box, indicating the Y-coordinate of the center.

Example

  when started :: hat events
  [Move towards a blue barrel until its center y-coordinate is more than 140 pixels.]
  forever
  get [blue barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [centerY v]) [math_less_than v] [140]> then
  move [forward v]
  end
  else
  stop all movement

originX#

originX returns the x-coordinate of the top-left corner of the detected object’s bounding box in pixels as an integer from 0 to 320.

The AI Vision object property stack block with its parameter set to originX.#
  AI Vision object [originX v]

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box. A vertical red dashed line extends from the top of the frame to the left edge of the bounding box, indicating the X-coordinate of the bounding box's origin.

Example

  when started :: hat events
  [Display if an orange barrel is to the left or the right.]
  forever
  clear row [1] on screen
  set cursor to row [1] column [1] on screen
  get [orange barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [originX v]) [math_less_than v] [160]> then
  print [To the left!] on screen ▶
  else
  print [To the right!] on screen ▶
  end
  wait [0.5] seconds

originY#

originY returns the y-coordinate of the top-left corner of the detected object’s bounding box in pixels as an integer from 0 to 240.

The AI Vision object property stack block with its parameter set to originY.#
  AI Vision object [originY v]

A blue barrel with an orange stripe is centered in the frame, surrounded by a black bounding box. A horizontal red dashed line extends from the left side of the frame to the top edge of the bounding box, indicating the Y-coordinate of the bounding box's origin.

Example

  when started :: hat events
  [Display if an orange barrel is close or far from the robot.]
  forever
  clear row [1] on screen
  set cursor to row [1] column [1] on screen
  get [orange barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [originY v]) [math_less_than v] [80]> then
  print [Far!] on screen ▶
  else
  print [Close!] on screen ▶
  end
  wait [0.5] seconds

rotation#

rotation returns the orientation of the detected AprilTag or Color Code as an integer in degrees from 0 to 359.

The AI Vision object property stack block with its parameter set to rotation.#
  AI Vision object [rotation v]

A camera interface displays a vertically stacked object with a blue top half and a red bottom half, centered in the frame. A label beneath the object reads Red_Blue A:270° CX:185 CY:81 W:93 H:150, indicating the object's name, angle of rotation, center X and Y coordinates, width, and height. Gridlines and numeric axes mark the image dimensions, with X ranging from 0 to 320 and Y from 0 to 240.

Example

  when started :: hat events
  [Slide left or right depending on how the Color Code is rotated.]
  forever
  get [Red_Blue v] data from AI Vision
  if <AI Vision object exists?> then
  if <[50] [math_less_than v] (AI Vision object [rotation v]) [math_less_than v] [100]> then
  move [right v]
  else if <[270] [math_less_than v] (AI Vision object [rotation v]) [math_less_than v] [330]> then
  move [left v]
  else
  stop all movement
  end
  else
  stop all movement

bearing#

bearing returns how far an object is to the left or right of the center of the AI Vision Sensor’s view as a degree. A value of 0 means it’s centered, positive values mean the object is to the right, and negative values mean the object is to the left.

  when started :: hat events
  [Keep the blue barrel directly in front of the robot.]
  set turn velocity to [40] %
  forever
  get [blue barrel v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [bearing v]) [math_greater_than v] [5]> then
  turn [right v]
  else if <(AI Vision object [bearing v]) [math_less_than v] [-5]> then
  turn [left v]
  else
  stop all movement

tagID#

tagID returns the identification number of the detected AprilTag as an integer.

The AI Vision object property stack block with its parameter set to tagID.#
  AI Vision object [tagID v]

Example

  when started :: hat events
  [Move forward when AprilTag ID 0 is detected.]
  forever
  get [all AprilTags v] data from AI Vision
  if <AI Vision object exists?> then
  if <(AI Vision object [tagID v]) [math_equal v] [0]> then
  move [forward v]
  end
  stop all movement
On this page