AI in Robotics

AI vs robotics vs AI in robotics; everyday examples (robot vacuums, self-driving cars); what PyTorch and MuJoCo are

Install PyTorch and MuJoCo.

MuJoCo model and data; the simulation step loop; reading sensors; applying control inputs; resetting a run; loading a saved model

Load the foundation-track random forest and wire it in as a live controller. Run 20 trials and record the success rate.

Neurons, layers and loss in plain language; gradient descent as nudging the weights; PyTorch tensors, Dataset and DataLoader, and the training loop; switching on Colab's free GPU; experiment tracking with W&B

Build a small multi-layer perceptron (MLP) on the wall-following data; train it; log loss and accuracy curves to W&B.

Overfitting and early stopping; normalising inputs; changing one hyperparameter at a time; a fair comparison means offline metrics plus closed-loop success rate

Deploy the MLP in the simulator; run a small hyperparameter sweep in W&B; benchmark the MLP against the random forest on accuracy, success rate and speed; begin the mini-assignment.

Convolutions as pattern detectors (no maths); pretrained networks and transfer learning; data augmentation; getting camera frames out of the simulator

Write a script that drives the robot around and saves labelled camera frames (for example clear path, obstacle, goal marker); build an image dataset with at least 100 frames per class.

Fine-tuning a pretrained model (for example MobileNet or ResNet) on a small dataset; evaluating with a confusion matrix; generalising to lighting and texture changes; turning a class prediction into a drive command

Fine-tune the model on Colab's GPU; connect predictions to robot actions (stop, turn or go). Break it: change the arena lighting and colours and measure how far accuracy drops.

Classification vs detection; bounding boxes and confidence thresholds; IoU and mAP explained in plain terms; running a pretrained YOLO model with Ultralytics

Run a pretrained YOLO model on simulator camera frames; draw the boxes; compare detections against ground truth and see where a pretrained model fails on simulated objects.

Labelling images in Roboflow; fine-tuning a small YOLO model; using box position and size for steering and rough distance

Label a small dataset of the target object (a pre-labelled backup set is provided); fine-tune YOLO in Colab; make the robot steer toward the target and stop at a set distance; begin the mini-assignment.

Text as data: tokens and embeddings explained by analogy (similar meanings sit close together); intent classification; sentence embeddings for matching a command to a robot action; using a language model to turn free text into structured output (JSON); hallucination, ambiguity, and why every model output must be validated before it reaches an actuator

Build a command interpreter that maps instructions like "back up a little" or "turn toward the red box" to a small set of robot actions, first with sentence embeddings and nearest-neighbour matching, then with an LLM that returns JSON. Add a validator that rejects unknown or unsafe commands, and run the result on the simulated robot. Break it: try vague, contradictory and misspelled commands and record what gets through.

Vision-language models (image and text in, text out) vs VLAs (image and instruction in, action out); what models like CLIP, RT-2, OpenVLA and Octo do and how they were trained (survey level); zero-shot recognition; why large VLAs are often too heavy for free compute, and what to use instead; where VLAs fall short (speed, precision, safety)

Use CLIP to score the robot's camera frames against text prompts ("a red box ahead", "a clear path") and turn the scores into stop, turn or go decisions. Compare with a hand-coded colour rule on 20 frames. Watch a pre-recorded or hosted VLA demo and map its inputs and outputs onto your own pipeline.

P, I and D explained by feel; tuning by trial; why every learned controller should be compared with a simple baseline; behaviour cloning as supervised learning on observation and action pairs

Hand-tune a PID wall follower (or goal seeker) in the simulator; record its success rate, tracking error and smoothness.

Collecting demonstrations from the PID controller or teleoperation; training a policy that outputs continuous actions; compounding error, and fixing it with extra demonstrations from failure states (the idea behind DAgger)

Train a PyTorch behaviour-cloning policy and deploy it; compare with PID; collect extra demos where it fails and retrain. Submit the capstone proposal.

Agent, environment, state, action and reward; episodes; exploration vs exploitation (epsilon-greedy); the Q-table as a cheat sheet

Implement tabular Q-learning from scratch on a Gymnasium grid world (for example FrozenLake); plot reward per episode.

Designing states and actions (discretising sensor readings); the Gymnasium reset and step interface; what makes a good reward

Wrap a small grid-navigation task with obstacles as a custom Gymnasium environment; train Q-learning; evaluate over 20 episodes; compare two reward functions.

Why tables break down with continuous sensor input; DQN and PPO at an intuitive level; the Stable-Baselines3 workflow; reward shaping and reward hacking

Train PPO with Stable-Baselines3 on a provided continuous-navigation environment to confirm the pipeline; log reward curves in W&B; save checkpoints to Drive or GitHub.

Observation and termination design; evaluating a policy across multiple seeds; Colab time-outs and checkpoints; sim-to-real gaps and domain randomisation (concepts only)

Train PPO for obstacle avoidance or goal reaching; deliberately hunt for a reward hack and fix the reward; compare with PID and behaviour cloning; begin the mini-assignment. Pre-trained checkpoints are provided as a fallback.

Survey (concepts only): vision-language and foundation models in robotics, learned vs classical components, safety and testing of learned systems, ethics; choosing a success metric and a baseline

Capstone design review in breakout rooms: finalise scope, success metric and baseline; write a test plan; set up the repository skeleton and W&B project.

Evaluating on unseen seeds and maps; common failure modes; keeping the pipeline running end to end

Build the core of the project with instructor check-ins (about 10 minutes per team); run and log a first evaluation against the baseline.

Fixing the biggest failure found in evaluation; writing a README that lets someone else run your project; recording a demo

Finish core functionality; run the final evaluation; write the README; record a 2 to 3 minute demo.

Presenting results: what you built, how you measured it, and what failed; reflecting on the journey from random forest to autonomous system; where to go next (ROS 2, real hardware, sim-to-real, larger RL frameworks)

Each team presents (about 7 to 10 minutes, scaled to class size) followed by Q&A.