Road Defect Detection with Artificial Intelligence: Computer Vision Engineering Applied to Active Pavement Management

Urban pavement management in Brazil has historically suffered from a shortage of structured data. Municipalities rely on slow, subjective manual inspections, or on expensive equipment that makes continuous monitoring unfeasible. This produces a purely reactive model in which interventions occur only at critical stages of degradation, demanding costly reconstructions.

Computer vision engineering integrated into urban enforcement fleets breaks this cycle. By embedding deep learning algorithms in traffic or paid-parking enforcement vehicles that already circulate daily, data collection reaches zero marginal cost. The pavement comes to be evaluated continuously, objectively and with geographic referencing, feeding Pavement Management Systems (PMS).

1. Computer Vision Engineering on the Pavement: Sensors, Models and Processing

Automated detection of asphalt pathologies requires a hardware and software architecture designed for the extreme conditions of the Brazilian urban environment. Unlike controlled laboratory tests, enforcement vehicles face abrupt variations in luminosity, shadows from trees and buildings, constant vibration, and occlusions caused by heavy traffic.

Hardware Architecture and Sensor Positioning

To ensure image capture with adequate resolution and angle for analyzing millimeter-scale cracks, Areatec uses high-speed cameras (global shutter) with a minimum resolution of 4K (3840x2160 pixels) and lenses with a focal length calibrated to minimize optical distortion at the frame edges. Common cameras with an electronic rolling shutter suffer from motion distortion (motion blur) when the vehicle moves at normal urban speeds (between 40 and 60 km/h). The use of global shutter sensors ensures that all sensor pixels are exposed simultaneously, capturing sharp images even under heavy vibration.

The cameras are mounted on anti-vibration supports (electromechanical gimbals with passive elastomer damping) installed on the roof or front of the vehicle. The tilt angle of the optical axis is fixed between 30 and 45 degrees relative to the horizontal plane of the roadway. This capture geometry covers a complete travel lane (approximately 3.5 meters wide) while maintaining a spatial resolution of up to 1.5 millimeters per pixel in the immediate area of interest (between 3 and 8 meters ahead of the vehicle).

Lighting is stabilized by Aretron's dynamic exposure compensation algorithms based on regions of interest (ROI), mitigating shadows from trees and buildings. In nighttime conditions, pulsed infrared illuminators synchronized with the shutter ensure constant contrast without disturbing the vision of drivers on the road.

Deep Learning Models: Detection and Semantic Segmentation

Processing of the structured images occurs in two consecutive stages within Aretron's artificial intelligence architecture:

• Object Detection with Adapted YOLOv8 and YOLOv11: For discrete, localized defects such as potholes and patches, convolutional neural networks of the YOLO family (You Only Look Once) are applied, optimized with spatial attention layers (Squeeze-and-Excitation or Convolutional Block Attention Module - CBAM). These layers direct the network's focus toward the high-frequency textures typical of structural failures, ignoring the visual noise of dry leaves, dirt or shadows on the road.
• Semantic Segmentation with U-Net and FCN: For linear or branching pathological manifestations such as longitudinal cracks, transverse cracks and alligator cracking, detection by bounding boxes is insufficient to calculate the affected area or the extent of the damage. In these cases, the system employs semantic segmentation networks based on U-Net architectures or Fully Convolutional Networks (FCN) with pre-trained ResNet-50 encoders. Segmentation classifies each pixel of the image individually, allowing the exact extraction of the crack geometry, its average width and its density per square meter.

The loss from imbalanced classes is handled through the Focal Loss function. Since intact pavement represents more than 95% of the pixels captured on a typical route, traditional loss functions tend to ignore the pathologies. Focal Loss adjusts the weight of gradient updates during training, focusing on the difficult examples (the road pathologies) and minimizing the influence of defect-free areas. To ensure robustness against the national reality, Aretron was trained with more than 10 million images of Brazilian roads, covering tropical climate variations, irregular patches made by concessionaires, and different types of asphalt aggregates.

High-Precision Geographic Referencing and Sensor Fusion

Each defect identified by Aretron must be located in physical space with enough precision to guide maintenance teams. The enforcement vehicle integrates a multi-frequency GNSS receiver with support for RTK (Real-Time Kinematic) corrections and an industrial-grade Inertial Measurement Unit (IMU).

Sensor fusion occurs through an Extended Kalman Filter (EKF). When the vehicle passes under viaducts, tunnels or "urban canyons" formed by tall buildings, where the GPS signal suffers degradation or total loss (multipath), the IMU takes over the calculation of the vehicle's odometry and attitude. Upon recovering the GNSS signal, the system corrects the accumulated drift. This arrangement ensures that each recorded defect coordinate presents a positional error of less than 0.5 meters, allowing perfect overlay of the data on digital maps and geographic information systems (GIS).

2. National Standardization: DNIT Standards and Defect Classification

Validating a computer vision system for the Brazilian public administration requires strict adherence to the technical standards in force. At the federal level, the National Department of Transport Infrastructure (DNIT) establishes the criteria for terminology, surveying and evaluation of flexible pavements. Aretron was calibrated to translate the outputs of the deep learning models directly into the national regulatory parameters.

DNIT Standard 005/2003 - TER: Defect Terminology

The DNIT 005/2003 - TER standard [1] standardizes the visual identification of failures in flexible and semi-rigid pavements. Areatec's system classifies and codes the following occurrences:

• Isolated Crack (T): Individual fissures divided into Longitudinal (TL), parallel to the road axis (fatigue or joint failures), and Transverse (TT), perpendicular to the axis (thermal shrinkage or propagation of underlying cracks).
• Interconnected Crack (FC): Fissures that intersect, divided into Class 2 (FC-2) ("alligator cracking" without edge erosion, indicating fatigue of the surfacing) and Class 3 (FC-3) (with edge erosion or block disintegration, indicating structural failure).
• Rutting/Depression (A): Permanent deformations, divided into Wheel-Path Rutting (ATR), along the tire trajectory caused by heavy traffic, and Localized Depression (AL), caused by subgrade settlement.
• Corrugation or Rippling (O): Transverse plastic deformations with crests and valleys, common in braking or intense acceleration zones (bus stops).
• Bleeding (EX): Migration of the asphalt binder to the surface, generating a slippery film that reduces grip.
• Raveling (D) or Disaggregation: Progressive loss of aggregate from the pavement surface, generating a rough texture and detachment of plates.
• Pothole (P): Cavities that form in the surfacing and penetrate into the base layers, caused by the action of heavy traffic and water.
• Patch (R): Corrections of previous defects, divided into Surface (RS) (replacement of the surfacing) and Deep (RP) (reconstruction of base and sub-base).

DNIT Standard 006/2003 - PRO and DNIT 008/2003 - PRO: Visual Surveys

DNIT defines two main field evaluation methods. The DNIT 008/2003 - PRO standard [2] addresses the Continuous Visual Survey (LVC), an expedited inspection carried out by a recorder aboard a vehicle at low speed. This method, while fast, is highly subjective, inconsistent between operators and dangerous, as it exposes the team to urban traffic.

The DNIT 006/2003 - PRO standard [3] establishes the Detailed Visual Survey (LVD), carried out on foot in sample segments of 20 meters every 200 meters (systematic 10% sampling). The LVD is extremely slow and expensive, making its frequent application across an entire urban road network unfeasible.

Areatec's Aretron solves this technical bottleneck by performing a survey with the total coverage (100% of the road) typical of the LVC, but with geometric precision and objectivity superior to the LVD, operating at normal urban traffic speed (up to 60 km/h). Human error and evaluator subjectivity are eliminated.

3. International Standardization: ASTM D6433, PASER and SHRP-LTPP

For municipalities that use international-standard pavement management platforms or seek financing from multilateral bodies such as the World Bank or the IDB, Aretron classifies pathologies according to North American standards.

ASTM D6433: The Pavement Condition Index (PCI)

The ASTM D6433 standard [4] governs the calculation of the Pavement Condition Index (PCI), a numerical index from 0 to 100 that qualifies the condition of pavements and parking lots. The calculation requires surveying 19 types of asphalt defects across three severity levels (Low, Medium and High).

PASER Scale (Pavement Surface Evaluation and Rating)

Developed by the University of Wisconsin-Madison [5], the PASER scale simplifies road management by classifying the pavement on a rating from 1 to 10. Aretron converts crack severity and density data to generate the PASER index of each block segment instantly:

• Ratings 9 and 10 (Excellent): New or recently resurfaced pavement. Requires only routine maintenance.
• Ratings 7 and 8 (Good): First signs of wear or hairline cracks. Suitable for crack sealing.
• Ratings 5 and 6 (Fair): Moderate cracks and initial disaggregation. Requires preventive treatments such as cold asphalt micro-surfacing.
• Ratings 3 and 4 (Poor): Severe cracks, isolated potholes and depressions. Requires milling and resurfacing (overlay) or deep patches.
• Ratings 1 and 2 (Very Poor): Complete structural failure. Requires total reconstruction of the pavement structure.

SHRP-LTPP (Strategic Highway Research Program)

The LTPP (Long-Term Pavement Performance) distress identification manual [6] serves as a visual dictionary for calibrating severities. Aretron uses the LTPP definitions to classify longitudinal cracks: low severity if the width is less than 6 mm; medium if it is between 6 and 19 mm; and high if it is greater than 19 mm.

4. Pathology Correlation and Equivalence: DNIT vs. ASTM

To ensure data interoperability, Aretron implements an engineering equivalence matrix between the DNIT and ASTM standards, correlating pathologies and computer vision metrics:

5. Integration with Pavement Management Systems (PMS)

Collecting visual data with artificial intelligence is only half the path. The real value for construction engineering and for the public manager arises when this data automatically feeds the tools for decision-making and budget planning.

Feeding Engineering Software: HDM-4 and MicroPAVER

Aretron integrates directly with software such as HDM-4 [7] (the World Bank standard for investment analysis and temporal pavement deterioration) and MicroPAVER (developed by USACE for urban management based on PCI). Through APIs, the system exports structured data (JSON, .xml or .csv) containing the complete defect matrix per segment, automating the decision-making flow.

Calculation of Quality Indices: IGG, PCI and IRI

With the computer vision data entered, the PMS instantly calculates the three main pavement quality indices:

• Global Severity Index (IGG): Weighted average of defect frequencies per section (DNIT 006/2003-PRO standard), classifying roads from Excellent (0-20) to Very Poor (above 160).
• Pavement Condition Index (PCI): Based on the density and severity of the 19 ASTM D6433 defects, ranging from 100 (perfect) to 0 (destroyed).
• International Roughness Index (IRI): Measures longitudinal roughness (m/km) [8]. Aretron estimates the IRI by correlating surface defects with vertical acceleration readings from the IMU.

Inertial Estimation of the IRI

The indirect estimation of the IRI from onboard inertial sensors is based on the "Quarter Car" mathematical model (Golden Car), defined by the ASTM E1170 standard. Aretron processes the vertical accelerations measured by the IMU in real time, applying a second-order Butterworth high-pass filter to eliminate low-frequency drift. Double temporal integration provides the relative vertical displacement of the sprung mass. By correlating these displacements with the vehicle speed (via the CAN-bus), the algorithm calculates the accumulated rate of variation per kilometer, delivering the estimated IRI in m/km with a coefficient of determination ($R^2$) greater than 0.88 relative to class 1 laser profilometers.

Automatic Works Budgeting and Input Calculation

The transformation of visual data into engineering cost spreadsheets occurs automatically in Aretron. By integrating the geometric data of road pathologies with local unit price composition tables (such as SINAPI or SICRO/DNIT), the system calculates the volumes of inputs needed for road recovery.

For pothole-patching services, the volume of Hot Mix Asphalt (CBUQ) is calculated based on the segmented area ($A$) and the average depth estimated by the severity ($p$), applying the material's bulking factor ($f_{emp} \approx 1.25$) and the apparent density ($\rho_{asf} \approx 2.4 \text{ t/m}^3$):

$$V_{CBUQ} = \sum (A_i \times p_i) \times f_{emp}$$

For surface treatments (such as asphalt micro-surfacing), the system projects the total area of the segment by multiplying the length of the degraded section by the standard width of the travel lane ($3.5 \text{ m}$), estimating the consumption of polymer-modified asphalt emulsion at an average rate of $1.2 \text{ kg/m}^2$. The system generates complete reports with SINAPI codes and estimated budgets, accelerating the planning of public tenders.

6. Defect Severity and Maintenance Strategy

Pavement engineering is based on the principle that the cost of intervention grows exponentially as the pavement deteriorates. An efficient strategy prioritizes preventive actions before the base structure is compromised.

Severity Levels (Low, Medium, High)

Aretron classifies each pathology into three severity levels based on strict geometric parameters extracted by computer vision:

• Low Severity: Thin cracks (< 6 mm) without material loss or erosion. Depressions < 10 mm. The surfacing begins to suffer fatigue or oxidation, but the support structure remains intact.
• Medium Severity: Moderate cracks (6 to 19 mm) with initial edge disaggregation. Depressions between 10 and 25 mm. Alligator cracking without block detachment. Indicates compromise of the surfacing, allowing infiltration.
• High Severity: Wide cracks (> 19 mm) with strong erosion. Alligator cracking with loose blocks. Depressions > 25 mm and deep potholes. Indicates local or generalized structural collapse of the base or subgrade.

Decision Matrix: Preventive vs. Corrective vs. Reconstructive

The classification of severity and density feeds a technical decision matrix that defines the ideal type of intervention for each road section, seeking to maximize the return on public investment:

• Preventive Maintenance: Applied to pavements with a high PCI (70 to 100) and low-severity defects. Actions include sealing isolated cracks, applying asphalt rejuvenation or cold asphalt micro-surfacing. The cost of this intervention is extremely low (R$ 15 to R$ 30 per square meter).
• Corrective Maintenance (Rehabilitation): Applied to pavements with an intermediate PCI (40 to 70) and medium-severity defects. It involves milling (mechanical removal) of the degraded asphalt layer, execution of deep patches in localized areas of base failure, and application of a new CBUQ wearing course. The cost is moderate (R$ 80 to R$ 150 per square meter).
• Reconstructive Maintenance (Reconstruction): Applied to pavements with a low PCI (0 to 40) and generalized high-severity defects. When the base and sub-base lose their support capacity due to prolonged water infiltration, the only viable technical solution is complete reconstruction. The cost is extremely high (R$ 300 to R$ 600 per square meter).

Acting preventively when the pavement is in the transition from excellent to good prevents the road from reaching the reconstruction stage, where the cost of repair is up to twenty times higher per square meter.

7. Areatec's Aretron: Real-Time Operation and Zero Marginal Cost

The major barrier to the adoption of Pavement Management Systems in medium and large municipalities has always been the recurring cost of data acquisition. Hiring manual inspections or laser profilometry vehicles costs hundreds of thousands of reais per campaign, limiting readings to intervals of two or three years.

The Zero Marginal Cost Paradigm

Areatec breaks this barrier through the concept of zero marginal cost. As the Brazilian leader in technology for traffic and paid-parking enforcement, the company already operates fleets of vehicles equipped with OCR license-plate-reading systems in dozens of municipalities. These vehicles circulate continuously through urban roads, covering the entire central network and traffic corridors several times a day.

By embedding the Aretron road defect detection module in these same vehicles, the collection of road engineering data comes to occur simultaneously with and transparently to the enforcement operation. There is no need to put additional cars on the street, hire new drivers or consume more fuel. The cost of acquiring the pavement data becomes virtually zero.

Hybrid Data Processing: Edge Computing and Cloud

The system operates in an optimized hybrid model:

1. Edge Processing (Edge Computing): Embedded industrial computers with high-performance GPUs (NVIDIA Jetson) process the 4K video stream in real time at the edge, running the YOLO and U-Net models locally and avoiding the transmission of raw video.
2. Metadata Extraction and the DATARACE Protocol: Only compact metadata (type, severity, dimensions, GPS coordinates and image crop) is sent to the cloud via the proprietary DATARACE protocol, which dynamically manages TCP/UDP connections to avoid losses in cellular shadow zones.

8. Strategic Benefits for the Public Manager and Legal Framing

The implementation of road defect detection by computer vision delivers to public works secretaries, traffic secretaries and mayors a governance tool unprecedented in the national scenario.

Legal Obligations, the New Procurement Law and the Digital Twin

Active pavement management with Aretron directly meets the principles of efficiency and economy required by the Federal Constitution of 1988 and serves as a legal shield under the aegis of Law No. 14,133/2021 (the New Procurement Law). The new legislation imposes rigor on the planning of public contracts, requiring preliminary technical studies (ETP) and detailed risk matrices. The use of objective data generated by computer vision eliminates imprecise visual estimates that generate contract amendments and findings of irregularities by Courts of Accounts (TCE and TCU).

Instead of static PDF reports, the manager has a continuously updated asphalt Digital Twin. If a concessionaire makes a cut in the road and executes a poor-quality patch, the system identifies the failure on the first pass, allowing immediate notification for correction within the legal warranty period.

ROI Prioritization and Works Auditing

With the automated calculation of the IGG and PCI, the Public Works Secretariat prioritizes interventions based on real engineering data, running financial simulations of return on investment (ROI). Each recorded defect is accompanied by a geo-referenced image authenticated with a timestamp, allowing the auditing of the execution of outsourced services:

• Quantity Auditing: Validation of whether the resurfacing area billed by the contractor corresponds exactly to the degraded area previously identified.
• Effectiveness Auditing: Monitoring of the durability of the applied asphalt. If a patch shows premature failures in less than a year, the system alerts enforcement to require redoing the work at no cost to the municipality.

Road defect detection with artificial intelligence raises urban management to a new level of technical efficiency and fiscal responsibility, proving that computer vision technology, when applied to the real world, is the shortest path to safer, smarter cities with high-quality infrastructure.

References

1. DEPARTAMENTO NACIONAL DE INFRAESTRUTURA DE TRANSPORTES. DNIT 005/2003 - TER: Defeitos nos pavimentos flexíveis e semi-rigidos - Terminologia. Rio de Janeiro: IPR, 2003. Available at: https://www.gov.br/dnit/pt-br/assuntos/planejamento-e-pesquisa/ipr/coletanea-de-normas/coletanea-de-normas/terminologia-ter/dnit_005_2003_ter-1.pdf.
2. DEPARTAMENTO NACIONAL DE INFRAESTRUTURA DE TRANSPORTES. DNIT 008/2003 - PRO: Levantamento visual contínuo para avaliação da superfície de pavimentos flexíveis e semi-rigidos - Procedimento. Rio de Janeiro: IPR, 2003. Available at: https://www.gov.br/dnit/pt-br/assuntos/planejamento-e-pesquisa/ipr/coletanea-de-normas/coletanea-de-normas/procedimento-pro/DNIT_008_2003_PRO.
3. DEPARTAMENTO NACIONAL DE INFRAESTRUTURA DE TRANSPORTES. DNIT 006/2003 - PRO: Avaliação objetiva da superfície de pavimentos flexíveis e semi-rigidos - Procedimento. Rio de Janeiro: IPR, 2003. Available at: https://www.gov.br/dnit/pt-br/assuntos/planejamento-e-pesquisa/ipr/coletanea-de-normas/coletanea-de-normas/procedimento-pro/dnit006_2003_pro.pdf.
4. AMERICAN SOCIETY FOR TESTING AND MATERIALS. ASTM D6433-20: Standard Practice for Roads and Parking Lots Pavement Condition Index Surveys. West Conshohocken: ASTM International, 2020. Available at: https://www.astm.org/d6433-20.html.
5. UNIVERSITY OF WISCONSIN-MADISON. Pavement Surface Evaluation and Rating (PASER) Manuals. Madison: Transportation Information Center, 2013. Available at: https://ltap.engr.wisc.edu/documents/paser-manual-asphalt-pubpas01/.
6. FEDERAL HIGHWAY ADMINISTRATION. Distress Identification Manual for the Long-Term Pavement Performance Program. 5th ed. Washington, D.C.: FHWA, 2013. Available at: https://www.fhwa.dot.gov/publications/research/infrastructure/pavements/ltpp/13092/index.cfm.
7. WORLD ROAD ASSOCIATION (PIARC). Highway Development and Management (HDM-4) Version 2. Paris: PIARC, 2006. Available at: http://www.hdmglobal.com/hdm-4-version-2/about-hdm-4/pavement-management-systems-and-hdm-4/.
8. SAYERS, M. W.; GILLESPIE, T. D.; QUEIROZ, C. A. V. The International Road Roughness Experiment: Establishing Correlation and a Calibration Standard for Measurements. Washington, D.C.: World Bank Technical Paper No. 45, 1986. Available at: https://en.wikipedia.org/wiki/International_roughness_index.