Authors - Younes Ed-doughmi, El Ayachi Rachid Abstract - The supervision of road quality is among the main lines of research in road security, exploiting the various techniques of machine learning. Our goal is to propose an efficient and accurate system to provide a complete and detailed report on the state of the road and anticipate the maintenance to be done to road or decision support if the system uses real-time in the context of an autonomous car example. For this, we propose a deep learning architecture in two steps. In the first step, we start a segmentation to determine the sides of the road and limiting the area detection of potholes. The second step to detecting potholes in this area reduces the number of false positives detected by the model. It reduces the detections of potholes that are not relevant for this detection, including potholes at the side of the road or the sidewalk. For this, each model was training on images from a public database. The result of our approach showed its effectiveness by reaching accuracy 93%.