Designing an optimal path has been considered as the one of the key challenges for drilling engineers. Even for a group of competent engineers, it takes several months to plan a well. A robust optimized path can influence the total cost of wellbore, wellbore quality and stability, transport efficiency, and drilling speed. A properly optimized path can be called if it is with a shortest path, avoids well-collisions, low touristy, and has maximum contact with the reservoir. Optimal design of wellbore trajectory is very demanding due to many complex and interacted drilling variables, models’ uncertainties, and design constraints. In drilling engineering, well path optimization is typically based on the objectives to minimize of the total length of wellbore, drilling cost and time in consideration of well-collision avoidance. Optimization algorithms such as genetic algorithm, dynamic programming, particle swarm optimization etc. might provide probable solutions to the optimization problem.

Recently, machine learning technology becomes more attractive in drilling engineering to deal with complicated cases, like regression, classification, control and optimization problems. The aim of this paper is to investigate the implementation feasibility and efficiency of Reinforcement Learning algorithms to design an optimal path with the obstacle detection and avoidance. Our work has applied Q-Learning of Reinforcement Learning to plan a path with minimum measure depth, and in the meanwhile to avoid different shapes of obstacles. The agent interacts with the environment, achieving maximum reward upon reaching the target with passing through the given obstacles or receiving penalties when it fails. This work shows the behavior of the algorithms to obey the main criterion of the trajectory design as an alternative solution. In addition, one engineering path plan method based on Bezier curve to modify the generated path from Q-Learning is proposed, where the minimum dog leg severity (DLS) is considered to make the designed trajectory close to the real ones. To the best of authors’ knowledge, it might be the first attempt to use Reinforcement Learning for well path design. The presented work will be a good example to introduce more and more artificial intelligent (AI) methods to petroleum engineering and explore a wise way to combine traditional engineering design methods with AI approaches.

This content is only available via PDF.
You do not currently have access to this content.