Vision Based Localisation
My PhD research is on a vision based global localisation system for mobile robots. For those unfamiliar with academic buzzwords, this just describes a system that can estimate the pose (position and orientation) of a robot in a known environment using vision (as oppose to radio signals found in your everyday GPS devices).
The system uses an off the shelf webcam with a special panoramic mirror that provides a full 360 degrees field of view. A rich 3D map (similar to a first person shooter game) is used to provide the system with a working knowledge of the environment. The system works by capturing images from the webcam and determines the most likely place it was taken from relative to the 3D map. This is something we as humans do on a daily basis with ease, but is a rather challenging task for a robot.
Overall the system achieves an accuracy of within 1 metre for position and 10 degrees for orientation and can operate where a GPS is ineffective eg. indoors. A moderate size environment, measuring roughly 50x50m, was used for testing.
Below are some images and videos that have come about from my research. For some light technical details, scroll down to the bottom of the page.
System in action!
 |
The localisation system can operate in real-time and is demonstrated by tracking me using a walking frame at around 10 fps (old Pentium M laptop). The first segment of the video shows the particle filter in action (represented by blue dots). The second part shows the 3D view of the estimated pose. |
Other related videos
 |
Autonomous navigation 1 In this experiment, the robot was placed at a random location. It moves around randomly until it determines its position and automatically navigate its way to a user defined goal point. A short laser range finder was mounted at the bottom to prevent it from crashing into obstacles. |
 |
Autonomous navigation 2 Another experiment similar to above, but at a different location. The robot narrowly misses a bicycle in this experiment. For some reason the laser did not see it. |
 |
3D Walkthrough A walkthrough of the completed 3D model used in this research. This is the engineering area at Monash and measures about 50x50m. |
 |
Scanner on a motorised cart One of the earlier equipment used to make moving the scanner around easier. It is a custom motorised cart controlled by a joystick. The cart has since been dismantled and used in a different project. I did the scans in my thesis by manually moving one piece of equipment at a time to a different scan location. The equipment includes: the laser range scanner, a table, a petrol generator, a 240V inverter, and a laptop. |
 |
Stitching two point clouds together The point clouds are stitched together manually using a custom written software. The user selects at least 3 corresponding points between the two scans and the software calculates the optimal transformation using the Iterative Closet Point algorithm. Each scan is stitched to its nearest scan. |
 |
3D point cloud demo This video shows an animated re-play of a 3D scan. This scan would have taken about 35 minutes to complete, which includes the scanning and capturing of images using a Nikon SLR mounted on top of the scanner. The user can select the angular resolution of the scan, which controls the total time and number of points captured. The black circle in the middle is the scanner’s blind spot. |
Robot gallery
Robots used during my PhD.
The ER1 robot was the first robot I used. It is a commercial off the shelf robot produced by Evolution Robotics. This robot was terrible for outdoor usage due to its small wheel base and low battery life.
The next robot was a hacked up wheel chair, which turned out to be a great platform for outdoors. Plenty of power and battery life to power external equipment.
The last robot is err, well, not really a robot at all. It’s just a walking frame with the localisation system bolted onto it.
How the system works (in a bit more detail)
An overview of the process of building the localisation system is presented in Figure 1. The first step, and the most time consuming, was to build a 3D model of the environment. This was done by using a Riegl Z420i laser range scanner, shown in Figure 2. A high resolution 3D scan was taken at various locations in the environment and stitched together to build a complete model. Using the 3D model, 2D images were then generated (from the point of view of the webcam) every 0.5 metre and 10 degrees, and a unique scene signature was extracted from each image. The signatures are stored in a database to be used later on. To localise the robot, the same signature was extracted from the webcam and fed through a particle filter. The particle filter is a probabilistic algorithm that can be used to estimate the pose (position and orientation) of a robot given sensory and odometry information. The particle filter in this case, uses the signature from the webcam and odometry information from the robot to assigns a probability score of match to all the other signatures in the database, and outputs the estimated pose of the robot (x, y, θ).
