Mobile Robotics Research Group




 Group World representation / Mapping Mapping Vision Based Laser Based Object Reconstruction Perception 2D Object Recognition Obstacle Detection 3D Stereo Vision 3D Laser Planning / navigation Probabilistic Markovian Processes Deterministic A* D* Localization Object Tracking Position Estimation Relative Absolute





Vision Based

Visual Odometer System to Build Feature Based Maps

In a visual odometer system is addressed the problem of computing the displacement of a mobile robot by analyzing the consecutive stereo image pares taken during its motion. The developed algorithm (diagram in Fig. 1) measures the relative translation and rotation of the vehicle related to the surrounding environment represented by the 3D image features obtained from the stereo pairs. In the video file the robot position computed with the visual odometer system are marked with red dots on lover left corner (on an aerial perspective of the scene).

Fig. 1. Diagram of the visual odometer algorithm
Video with Results

A. Majdik; L. Tamas; M. Popa; I. Szöke; Gh. Lazea, Visual Odometer System to Build Feature Based Maps for Mobile Robot Navigation, 18th Mediterranean Conference on Control and Automation, ISBN: 978-1-4244-8090-6, pp.: 1200-1205, Marrakech, Morocco, 2010

Solving the Kidnapped Robot Problem

Selected images out of which the map was built
Generated map containing the 3D image features
Fig. 1. Selected images captured by the mobile platform during the mapping process Fig. 2. Generated map, containing the 3D image features

Images taken online

Fig. 3. Images taken by the mobile robot to localize itself online
Fig. 4. The robot automatically realizes that he is at the end of the corridor (aerial view)

A. Majdik; M. Popa; L. Tamas; I. Szöke; Gh. Lazea, New Approach in Solving the Kidnapped Robot Problem., The 41st International Symposium on Robotics and the 6th German Conference on Robotics, VDE Verlag, ISBN: 978-3-8007-3273-9, pp.: 304-309, Munich, Germany, 2010

Object Reconstruction

Keywords: Segmentation, Layering, Fitting, Correction, Reconstruction, Verification

    • RANdom SAmple Consensus (RANSAC) and Hough Transform (HT)
    • Region Growing Method for Clustering Objects
    • Identifying Boundary Points of Regions’ Footprints
    • Quadrilateral Approximation Technique (QAT) for Clusters using Principle Component Analysis (PCA)
    • Automatic Decomposition of Clusters into Layers for Improved Reconstruction
    • Hierarchical Model Fitting and Validation
    • Methods for Dealing with Over-and Under-Segmentation Problems by Merging and Splitting of Models
    • Identifying Occluded Parts and Inconsistencies of Models Objectives
    • 3D Reconstruction and Verification of Object Models using Automatic Layering
    • Calibration and Fusion of 3D and Image Data for Initial Processing
    • Feature Extraction and Classification for Object Recognition


Analyzing the 2D Footprints of Objects
Automatic Layering of 3D Objects
Automatic Layering of 3D Objects
Fig. 1. Analyzing the 2D Footprints of Objects Fig. 2. Automatic Layering of 3D Objects

Solving the Over-segmentation Problem by Merging of Models Overcoming the Under-segmentation Problem by Splitting of Models
Fig. 3. Solving the Over-segmentation Problem
by Merging of Models
Fig. 4. Overcoming the Under-segmentation Problem
by Splitting of Models
Objects vs. Table Object Segmentation
Automatic Layering
Model Fitting
CAD-like Models
Model Verification

Real World Urban Scene
Plane Segmentation
Foreground Segmentation

Automatic Layered 3D Reconstruction and Verification
of Object Models for Grasping

Overcoming the Under-segmentation Problem

leafFurniture Detection

Keywords: Fitting, Detection, Segmentation, Furniture, Fixtures

    • Segmentation of planar surfaces and furniture fixtures from real-world point cloud data-sets.
    • Based on RANSAC algorithm with axes-aligned fitting of planes.
    • Using Robotic Operating System (ROS) and Point Cloud Library (PCL ).

Segmentation of Furniture and Fixtures

Segmentation of Furniture Surfaces

L. C. Goron, Z. C. Marton, G. Lazea, M. Beetz, "Robustly Segmenting Cylindrical and Box-like Objects in Cluttered Scenes using Depth Cameras," Accepted for publication at the 7 th German Conference on Robotics (ROBOTIK), Munich, Germany, 2012.
N. Blodow, L. C. Goron, Z. C. Marton, D. Pangercic, T. Rühr, M. Tenorth, M. Beetz, "Autonomous Semantic Mapping for Robots Performing Everyday Manipulation Tasks in Kitchen Environments," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011.
L. C. Goron, Z. C. Marton, G. Lazea, M. Beetz, "Automatic Layered 3D Reconstruction of Simplified Object Models for Grasping," Proceedings of the 41 st International Symposium on Robotics (ISR), Munich, Germany, 2010.

Object Recognition

Object Recognition Based on Monovision

Object recognition in different types of images is one of the most important tasks of computer vision, improving landmarks detection, obstacles detection, map building or path planning for a mobile robot. Viola and Jones proposed a classifier based on the Adaboost algorithm, recognizing an object through the detection of a constellation of its body parts. Using Haar-like features and also a training image set, some week classifiers are computed, which maintain details about the the shape of the object's body. Then, combining these classifiers, a strong classifier is obtained. The performance of this classifier depends on the complexity of the object's shape. Applied on real world scenarious, the detection rate for regular objects is around 70%, or higher in some specific cases, like the face recognition (99.8 %). The research work developed by MRRG related with the object recognition domain, consists in finding a new weighting mechanism for Adaboost classifier, applied for people detection. The purpose is to reduce the number of false positive detection in complex scenarious, maintaining the precision obtained with the Viola's algorithm. During the training phase, the importance of the object training examples is set using a squared function computed with respect to the domains of the features defining the object.

Left: Classical weighting mechanism. Right: The proposed weighting mechanism.

Using this approach, it is guaranted that the detected objects are more likely with the objects contained in the positive training set. Some results are presented below. Using the new algorithm (right), the number of false positive detections is reduced.


False positives for people detection. Left: Classical Adaboost algorithm results. Right: The proposed algorithm results.

M. Popa; A. Majdik; Gh. Lazea, The Reduction of the False Positive Detections Number for the Adaboost Classifier, The 20th DAAAM World Symposium, Vienna, 2009.
M. Popa; Gh. Lazea; A. Majdik; L. Tamas; I. Szöke, An Effective Method for People Detection in Grayscale Image Sequences, IEEE ICCP, Cluj Napoca, 2009

Adaptive Appearance Based Loop-Closing in Heterogeneous Environments

This work addressed the problem of detecting loop-closure situations whenever an autonomous vehicle returns to previously visited places in the navigation area. An appearance-based perspective was considered by using images gathered by the on-board vision sensors for navigation tasks in heterogeneous environments characterized by the presence of buildings and urban furniture together with pedestrians and different types of vegetation. We proposed a novel probabilistic on-line weight updating algorithm for the bag-of-words description of the gathered images which takes into account both prior knowledge derived from an off-line learning stage and the accuracy of the decisions taken by the algorithm along time. An intuitive measure of the ability of a certain word to contribute to the detection of a correct loop-closure is presented. The proposed strategy was extensively tested using well-known datasets obtained from challenging large-scale environments.

Video with Results

A. Majdik; D. Gálvez-López; G. Lazea; J.A. Castellanos, Adaptive Appearance Based Loop-Closing in Heterogeneous Environments, IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.: 1256 – 1263, ISSN: 2153-0858, September, IROS 2011

Obstacle Detection

2D Laser Object Detection

The key role played by probability theory in the pattern recognition domain is demonstrated by the numerous applications from the real world, in which machines are searching for patterns in data, like recognizing handwritten digits, which needs very complex computation due to the wide variability of handwriting.

In the projects of our research group, the pattern recognition in case of the 2-dimensional laser data set is done by using the Gaussian Mixture theory.

The Gaussian model is called also as the normal distribution, and is often used for the distribution of continuous variables. The parameters of the mixture models are calculated by different algorithms like K-means and Expectation-Maximization. Although there are certain limitations applying this algorithm a Bayesian approach the framework of variational inference can be used.

 Object Separation of the Office Map

In the figure above the office map of our research group was mapped by a 2-dimensional laser sensor and the separate objects were clusterized with the afforementioned algorithms. All the separate objects are plotted with different colors.

2D Laser Object Classification

The planar laser data can also be used for detection&classification purposes. This procedure involves two phases: the training phase (the description of the forms via GMMs) and the evaluation or the classification phase(with Bayesian techniques). The next results were prepared for detecting specific forms like, human leg pair in the planar laser data.

GMM legs Classification
Fig. 1. GMM legpair
Fig. 2. Classification results for GMM legpairs
Fig. 3. Video with results

L. Tamas, M. Popa, Gh. Lazea, I. Szoke, A. Majdik, Laser and Vision Based Object Detection for Mobile Robots, International Journal of Mechanics and Control, Vol.: 11, No. 2, , 2010, pp.: 89-95, ISSN: 1590-8844

Object Detection based on Monovision

Without using any depth measurements given by the laser or stereovision sensors, object detection in structural environments is a difficult task. Trying to detect objects using monovision, we assume that any object located in the environment has a defined contour or some non-uniforme texture. Therefore, the object detection problem using monovision is reduced at the problem of edge extraction. A very low quality of the response of the standard edge extraction methods is obtained on images with strong sunlight or complex lighting conditions, frequently encountered in outdoor scenarios. We propose an adaptive algorithm for edge extraction using gradient approximations with Haar-like features, whose main contribution is the provision of a more stable result of the edge extraction process regarding the varying of the lightning conditions from complex images.

(a) Unprocessed image ; (b) edge extraction results with the proposed algorithm ; (c) edge extraction results with Canny operator .

3D Laser
Another way for the 3D perception can be achieved with a custom planar laser. The mechanical extension proposed by our group for the usual planar laser enables us to retrieve 3D information from the surrounding environment with a precision less than 1cm.

L. C. Goron, L. Tamas, I. Reti, G. Lazea, 3D Laser Scanning System and 3D Segmentation of Urban Scenes, Proceedings of the 17th IEEE International Conference on Automation, Quality and Testing, Robotics, Romania, 2010.
Markovian Processes

Motion planning is a mandatory acquirement demanded of autonomous robots. Given the start and goal coordinates as the initial input, the robot has to plan its actions including collision-free movements. The probabilistic methods became very used in the last years due to the fact that are able to solve high dimensional problems in acceptable computation time. The most known techniques are the Probabilistic Roadmap Methods(PRM) and Rapidly-exploring Random Trees(RRT). In our case the latter one was used, from the initial map a tree is grown to crate the area of the free cells. These cells form the nodes of non-directed connectivity graph. Two nodes are connected if they belong to adjacent cells.

Probabilistic Cell Decomposition of the Office Map

The cell decomposition is done until all the cells are considered possibly collision-free or possibly colliding cells, but in each case the size of a cell cannot be smaller than an imposed minimum threshold value, which in real application this is the size of the autonomous vehicle. The office map is separated into minimal size rectangles having area of 0.25, if multiple free adjacent cells are found they will be merged into cells with higher area.

I. Szoke, A. Majdik, D. Lupea, L. Tamas, Gh. Lazea, Autonomous Mapping in Polluted Environments, Hunagrian Journal of Industrial Chemistry, Veszprém, Hungary, 2010,
I. Szoke; L. Tamas; Gh. Lazea; M. Popa; A. Majdik, Path Planning With Markovian Processes, 6th International Conference on Informatics in Control, Automation and Robotics, Italy, 2009, pp.: 479-482, ISBN: 978-989-674-000-9
I. Szoke; L. Tamas; Gh. Lazea; M. Popa; A. Majdik, Path Planning and Dynamic Objects Detection, 14th International Conference on Advanced Robotics, Germany, 2009, pp. 1-6, ISBN: 978-1-4244-4855-5

Active breaking system on a commercial vehicle
The purpose of this setup was to prove the theoretical and practical concepts developed in the indoor environment. Also it was important to show the short integration process for this type of experiment on a commercial vehicle.

active beacking system

Fig. 1. Active beaking system on a comercial car

Fig. 2. Video with results

Joint work of Paul Sucala, Rosu Cristian and Levente Tamas.
Object Tracking

Laser and Vision Based Object Tracking

The purpose of this experiment was to validate the classification and tracking algorithms developed in earlier phases of our research group. The first picture shows the general architecture of the system, while next to it can be seen the results of the experiments on a video.

Fig. 1. GMM legpari
Fig. 2. Classification results for GMM legpairs

L. Tamas, M. Popa, Gh. Lazea, I. Szoke, A. Majdik, Lidar and Vision Based People Detection and Tracking, Journal of Control Engineering and Applied Informatics, Romania, Vol.: 12, No.: 2, Romania, 2010, pp.:30-35, ISSN 1454-8658