MAS512 Computer Vision and Embedded AI (Autumn 2026)

On successful completion of the course, the student should be able to:

• Develop, train and deploy different ML architecture from scratch, leverage existing ML models with transfer learning, and model tuning for downstream tasks such as - classification, object detection, segmentation, and tracking on data of different modalities e.g. image from camera, point cloud from lidar
• Integrate vision and 3D sensors for perception guided robotics and autonomous operations
• Formulate, develop and deploy own ML models; computation vs model parameters vs memory quantification for deployment on edge devices and real world applications
• Real world industrial applications such as - autonomous pick and place operations, dynamic object/load localization and tracking, structural integrity of critical infrastructures
• Lidar-camera, radar fusion for real worlds applications
• Working principle of image formation from RGB+D camera, Camera calibration and modelling - Modelling camera with a pin-hole camera model, perspective transformation and localizing object in 3D space for robotic application
• Working principle of 3D lidar, point cloud processing and object localization in 3D space using 3D lidar, leveraging ML approaches for point cloud processing.
• Hands on experience on sensor fusion, feature extraction for robotic autonomous systems
• Model/data-based control leveraging Reinforcement learning to train autonomous systems e.g. robots, UGVs.
• Select suitable hardware platform for a given AI task, and deploy a pre-trained neural network to the selected platform.

The Computer Vision and Embedded AI course is a comprehensive exploration of computer vision, sensor-driven data acquisition, and machine learning, tailored for real-world applications. This course provides a blend of theoretical foundations and hands-on coding experience to tackle real world challenges in vision and sensor guided robotics, autonomous operations, object detection, classification, tracking and segmentation across diverse data modalities, including images and time-series data, and reinforcement learning.

Students will gain an in-depth understanding of sensors of different modality such as cameras, spectrometers, lidar, and radar for acquiring and processing data. Through practical sessions, they will develop expertise in entire vertical stack including - hardware interface, data acquisition, exploratory data analysis (EDA) to preprocess and analyze sensor data effectively and develop AI/ML models towards real world problems.

The course emphasizes the theoretical background and practical implementation of machine learning techniques, with a particular focus on computer vision algorithms. Participants will learn how to design and deploy solutions for tasks like regression, clustering, object detection, image classification, semantic segmentation, predicting future trend for time series data, data compression, anomaly detection and data/model-based control leveraging reinforcement learning. Real-world case studies will illustrate how these techniques can be applied to solve complex problems, including processing multimodal data, model fusion and data fusion.

Key components include:

• Machine Learning: Core concepts, algorithms, and coding practices for supervised and unsupervised learning including developing models from scratch using ML frameworks - TensorFlow, leveraging existing standard model/framework for downstream tasks - object detection, classification, segmentation, regression, clustering, model fusion, sensor fusion, data compression and anomaly detection
• Real-World Applications: Development and Deployment of AI systems for decision-making
• Computer Vision and Sensor technologies: Camera, lidar, and radar
• ML leveraged Lidar 3D point cloud processing for object detection and tracking
• Model/data-based control leveraging Reinforcement learning
• Camera calibration and modelling - Modelling camera with a pin-hole camera model, camera calibration, conversion of 3D-2D and vice versa space for robotic application
• Image processing - image filtering, image segmentation, blob detection in images, image representation, Fourier transformations, convolution

The course includes substantial programming exercises using Python and popular frameworks such as TensorFlow, OpenCV, and scikit-learn to name a few. By the end of the course, students will possess the skills to design, develop, and deploy AI-driven solutions on embedded systems for computer vision and multimodal data analysis, making them industry-ready for cutting-edge AI applications.