MAS512 Computer Vision and Embedded AI (Autumn 2025)

On successful completion of the course, the student should be able to:

• describe a camera model and be able to calibrate a camera to determine the camera matrix that describes the mapping from 2D image to the 3D world
• use image processing techniques to filter the image and detect objects in the image
• use sensors such as lidar and radar to make 2D or 3D scans of objects and environments and perform point-cloud processing such as pose estimation
• configure, train, and use artificial intelligence (AI) algorithms for use in vision applications and
• select suitable hardware platform for a given AI task, and deploy a pre-trained neural network to the selected platform.

The Computer Vision and Embedded AI course is a comprehensive exploration of computer vision, sensor-driven data acquisition, and machine learning, tailored for real-world applications. This course provides a blend of theoretical foundations and hands-on coding experience to tackle real world challenges in object detection, classification, and segmentation across diverse data modalities, including images and time-series data, and reinforcement learning.Students will gain an in-depth understanding of sensors of different modality such as cameras, spectrometers, lidar, and radar for acquiring and processing data. Through practical sessions, they will develop expertise in entire vertical stack including - hardware interface, data acquisition, exploratory data analysis (EDA) to preprocess and analyze sensor data effectively and develop AI/ML models towards real world problems.The course emphasizes the theoretical background and practical implementation of machine learning techniques, with a particular focus on computer vision algorithms. Participants will learn how to design and deploy solutions for tasks like regression, clustering, object detection, image classification, semantic segmentation, predicting future trend for time series data, data compression, anomaly detection and data/model-based control leveraging reinforcement learning. Real-world case studies will illustrate how these techniques can be applied to solve complex problems, including processing multimodal data, model fusion and data fusion.Key components include:

•Machine Learning: Core concepts, algorithms, and coding practices for supervised and unsupervised learning including developing models from scratch using ML frameworks - TensorFlow, leveraging existing standard model/framework for downstream tasks - object detection, classification, segmentation, regression, clustering, model fusion, sensor fusion, data compression and anomaly detection

•Real-World Applications: Development and Deployment of AI systems for decision-making

•Computer Vision and Sensor technologies: Camera, spectrometer, lidar, and radar

•Model/data-based control leveraging Reinforcement learning

•Camera calibration and modelling - Modelling camera with a pin-hole camera model, camera calibration, conversion of 3D-2D and vice versa space for robotic application

•Image processing - image filtering, image segmentation, blob detection in images, image representation, Fourier transformations, convolution

The course includes substantial programming exercises using Python and popular frameworks such as TensorFlow, OpenCV, and scikit-learn to name a few. By the end of the course, students will possess the skills to design, develop, and deploy AI-driven solutions on embedded systems for computer vision and multimodal data analysis, making them industry-ready for cutting-edge AI applications.