Video Demos

Project Summary

Project objective

Develop a program for two robots to transport LEGO blocks from one side of an arena to a scoring platform without crossing the middle. 

Summary

Our team programmed a wheeled robot to use camera vision for determining its absolute position and the relative locations of LEGO blocks and the scoring platform. The first robot identified, picked up, and placed LEGO blocks at the arena's half-line. A second robot then retrieved the blocks and delivered them to the scoring platform, completing the task.

To achieve this, we created a dataset to train a YOLOv8 model for object detection, fine-tuned HSV values to detect the arena half-line and scoring platform, and developed homing functions to ensure consistent arm and gripper movements. The project involved extensive trial and error with different algorithms to optimize the robots' ability to reliably pick up LEGO blocks. Ultimately, our code produced the highest success rate in the final challenge.

Takeaways

• Implementing PID controls makes error prone actuators work beautifully

• Camera vision is sensitive

• Coordinate transformations don't mix well with error

• Developing a simple GUI can simplify troubleshooting

• Using homing techniques solves a lot of issues: IMU drift, inconsistent motor commands, camera position

• Hard coding a map of nodes is messy

Project 1: Positioning using AprilTags and Path Planning using uniform-cost search

Methodology:

1. Represent the maze with a graph

2. Represent the obstacles with a dictionary

3. Find the optimal path using Uniform Cost Search

4. Initialize the robot and camera

5. Find the transformation matrix from AprilTag to World

6. Find the transformation matrix from Camera to AprilTag

7. Find the location of the robot in world coordinates

8. Navigate the inputted path, updating with the path based on sensor inputs

Location 1

Location 2

Project 2: Picking up LEGOs and transferring them using YOLOv8 and OpenCV

Project 2 Plan

Prepare Computer Vision Tools

• Train YOLOv8 Model

  • Create a folder of images of LEGO towers that can be used to train a YOLOv8 model

  • Use Roboflow to label and format the dataset

  • Train a YOLOv8 object detection model on Google CoLab with the Roboflow dataset and download the model

• Determine HSV Values

  • Test the appropriate upper and lower bounds for HSV filter for both the blue line and the platform

Code the solution consisting of the following steps:

• Robot 1: 

  • Setting the arms in our desired position for finding the tower [ResetArm()]

  • Finding and detecting the tower [FindTower(model)]

  • Navigating to the tower and stopping at the right distance to pick up the tower [Go2Tower(model)]

  • Pick up the tower [PickupTower()]

  • Find the blue line at the center of the arena with CV2 [FindBlueLine()]

  • Go to the blue and stop before crossing—holding the tower out over the boundary [Go2BlueLine()]

  • Wait for the other robot to grip the tower [Wait()]

  • Release the grip from the tower [LetGo()]

• Robot 2: 

  • Setting the arms in our desired position for finding the tower [ResetArm()]

  • Find the blue line at the center of the arena with CV2 [FindBlueLine()]

  • Orient with Blue Line and move a certain distance away [Go2BlueLine()]

  • Move horizontally to stay in line with the tower/Robot1 [FollowTower()]

  • Navigating to the tower and stopping at the right distance to pick up the tower [Go2Tower(model)]

  • Pick up the tower at the right time [CloseGripper()]

  • Find the platform using computer vision filtering [FindPlatform]

  • Move until the robot is close to the platform [Move2Platform]

  • Put down the tower on the goal [Putdown]