Zhaoting LI

Zhaoting LI

First year master student at ETH Zurich

Research

Learning Muti-Robot Exploration Strategy with Adaptive-QMIX algorithm

Advisor: Ali Batuhan Yardim, Prof. Niao He, Department of Computer Science, ETH Zurich

Learning Robot Exploration Strategy with Deep Reinforcement Learning

Being able to explore unknown environments is a requirement for fully autonomous robots. Many learning-based methods have been proposed to learn an exploration strategy. In the frontier-based exploration, learning algorithms tend to learn the optimal or near-optimal frontier to explore. Most of these methods represent the environments as fixed size images and take these as inputs to neural networks. However, the size of environments is usually unknown, which makes these methods fail to generalize to real world scenarios. To address this issue, we present a novel state representation method based on 4D point-clouds-like information, including the locations, frontier, and distance information. We also design a neural network that can process these 4D point-clouds-like information and generate the estimated value for each frontier. Then this neural network is trained using the typical reinforcement learning framework. We test the performance of our proposed method by comparing it with other five methods and test its scalability on maps that are much larger than maps in the training set. The experiment results demonstrate that our proposed method needs shorter average traveling distances to explore whole environments and can be adopted in maps with arbitrarily sizes.

Efficient Heuristic Generation for Robot Path planning

Robot path planning is difficult to solve due to the contradiction between the optimality of results and the complexity of algorithms, even in 2D environments. To find an optimal path, the algorithm needs to search all the state space, which costs many computation resources. To address this issue, we present a novel recurrent generative model (RGM), which generates efficient heuristic to reduce the search efforts of path planning algorithms. This RGM model adopts the framework of general generative adversarial networks (GAN), which consists of a novel generator that can generate heuristic by refining the outputs recurrently and two discriminators that check the connectivity and safety properties of heuristic. We test the proposed RGM module in various 2D environments to demonstrate its effectiveness and efficiency. The results show that, compared with a model without recurrence, the RGM successfully generates appropriate heuristic in both seen and new unseen maps with higher accuracy, demonstrating the good generalization ability of the RGM model. We also compare the rapidly-exploring random tree star (RRT) with generated heuristic and the conventional RRT in four different maps, showing that the generated heuristic can guide the algorithm to efficiently find both initial and optimal solutions in a faster and more efficient way.

A sampled-based trajectory planning method for urban autonomous vehicles

In this project, we designed a motion planning method for urban autonomous vehicles. This method have the advantages of fast sampling speed, time-bounded collison-free path generation and sampling-based decision making method.

Path planning for a laser scanning robot system

Laser scanning has been widely used in the industrial manufacturing, especially in reverse engineering, quality control and surface defect detection. Achieving automatic scanning can succeed in reducing production costs and increasing production efficiency. In our project, we proposed an automatic intelligent scanning system based on ROS, including robot arm, RGBD camera and line laser scanner.

Intelligent operation of robot arm

Smart car competition

During the second year at university, I attended the 13th Chinese national NXP CUP smart car competition, which is one of the most famous competitions in China.