Autonomous heavy-duty mobile machines: Why and how can we use them in construction sites?

From manual to automatic material flow based on machine productivity in construction sites

In this newsletter, we want to discuss how fleets of (semi)autonomous Heavy-Duty Mobile Machines (HDMMs) could be used not only to complete a construction project, but to accomplish that in an optimal way. Optimization is essential to improve productivity and efficiency and reduce cost and project time. Nowadays, manual methods are employed to allocate tasks to construction machines based on the experience of worksite managers and manual observation of operations. However, these methods are costly, error prone, and depend on few experts that must always be available. Automating task planning and assignment based on fleet productivity can be a promising solution for solving the above challenges and achieving optimality. Firstly, Amirmasoud will describe how to estimate the productivity of a fleet of HDMMs during earth-moving tasks; secondly, Paolo will focus on optimizing the performance of the fleet of autonomous machines by solving task planning, task assignment, motion planning, and coordination problems jointly.


Productivity Monitoring

What is the productivity, and how can it be calculated?

HDMMs are crucial to many businesses, including the mining and construction sectors. The rapidly growing construction industry encounters several difficulties, including a need for more skilled workers, harsh environmental conditions, poor productivity, and safety [1]. Studies show that the productivity of the construction industry has only increased by 1% over the past 20 years [2]. To increase the productivity of earth-moving and construction projects, estimation of the machines’ productivity is highly required. Moreover, companies need to boost the productivity of their products to remain in business.

Figure 1: A wheel loader performing a bucket loading task [5].

A wheel loader (see Fig. 1), also referred to as a front-end loader, is a heavy machinery with numerous applications on various construction and mining sites. A wheel loader can transport materials from a few meters to a few hundred meters. Based on the distance between the loading and dumping spots, wheel loaders’ primary operational cycles are divided into two classes: the “Long Loading cycle” (also called the “Load & Carry cycle”) and the “Short Loading cycle.” The quantity of material transferred per time or work cycle is a representation of wheel loader productivity [3]. The quantity of material can be expressed as weight or volume. Several methods have been proposed to estimate the weight of material in the bucket using motion and pressure sensors or the volume of material using external perceptual sensors such as light detection and ranging (LiDAR) and cameras [4]. Additionally, several vision or motion-based algorithms have been proposed to recognize the wheel loader activities. The cycle time of the machine can be estimated based on the sequence of identified activities [5].

A wheel loader with high productivity can transport more material in less time, which can bring about a reduction in costs, time, fuel consumption, and the number of machines and operators needed to complete a project. Also, high productivity contributes to enhancing safety by minimizing the time that machines are required to work in potentially dangerous areas like on steep slopes or in unstable soil. The size and type of the machine, the engine’s power, the bucket’s design, operating conditions, such as type of material, weather, and terrain conditions, and the operator’s skill all influence wheel loader productivity. Consequently, absolute productivity cannot accurately reflect performance. To analyze absolute productivity, a benchmark or point of comparison is critical. The reference, also known as theoretical productivity, is determined based on operating factors. The theoretical volume of material in each scoop is obtained based on the nominal bucket capacity and type of material. The theoretical cycle time is calculated based on the proposed models in construction equipment manufacturers’ handbooks, the optimal path between the loading and dumping positions, and the machine’s capability. Therefore, it is crucial to take wheel loader capacity and other characteristics into account when choosing a machine. This can make sure the machine can deliver a high return on investment and satisfy the project’s requirements.

The production performance ratio of a single wheel loader is calculated by dividing the actual productivity by the theoretical productivity obtained according to the operating conditions. The closer the index is to one, the higher the performance. In the next phase, the productivity of the whole fleet consisting of several wheel loaders is calculated using the weighted average of production performance ratios of single machines. The fleet productivity can be utilized to optimally assign a set of tasks to a fleet in construction sites.

About the author

Amirmasoud Molaei
Early Stage Researcher 2

Amirmasoud is a 3rd year Ph.D. researcher working on productivity estimation of mobile machines. In his studies, he focuses on how to automatically estimate the productivity of HDMMs, including excavators and wheel loaders, in different tasks using sensor data and building information modeling (BIM).


Material flow coordination and optimization

Let the robots build it

During the last decades, interest in developing autonomous HDMMs has grown significantly because of their potential to reduce development costs, labor force, and salary costs, improve productivity, performance, and work quality, and create a safer work environment in construction sites, forestry, and mining industries. In addition to the advantages mentioned above, the interest in automating earth-moving processes is driven by three main factors: 1) the fact that construction operations, such as loading, unloading, digging, and moving material, are repetitive tasks and hence suitable for automation [6]; 2) autonomous HDMMs could potentially perform construction tasks where human presence is undesirable, unsafe, or impossible, e.g., in military applications [7], in mines, or in space; 3) automation can solve the lack of skilled operators.

Some examples include using autonomous HDMMs to perform classical construction operations, such as navigation, loading and dumping, digging, and moving material [8]. However, their applications have been limited due to the high complexity of automating the construction operations, the nontrivial kinematic models of most construction machines, and the granularity and unpredicted material properties.

Automating material flow processes, namely, deciding how to move material from one place to another in construction sites, using a fleet of (possibly different) autonomous heavy-duty mobile machines (HDDMs) poses several open challenges. To coordinate and optimize the material flow, decision-making, action control, sensing, and perception are the necessary components of an automated multi-robots system. A possible modular architecture for the overall earth-moving problem is reported in Fig.2.

Figure 2: A possible modular architecture for the overall earth-moving problem.

The decision-making module required to solve the well-known combined Task And Motion Planning (TAMP) problem, where the former is the task of selecting which machine should perform which task and in which order tasks should be performed, to optimize user desired specifications (e.g., performance, time, fuel consumption); the latter deals with computing motions for the machines to carry out the assigned tasks. In Fig.3 a simulation of a loading and dumping sequence performed by an autonomous wheel loader is reported.

Figure 3: Loading and dumping sequences performed by an autonomous machine.

Optimization is the key factor for improving productivity and reducing costs in industrial scenarios (see Fig.3). Automation already brings several advantages, such as improving the productivity of digging operations [4] or reducing operational costs by using autonomous trucks in mines [5]. However, to achieve optimality in construction sites is necessary to consider both individual machine performance and capabilities as well as the interactions between different machines and the ordering and assignment of the construction operations (i.e., which robot is performing which operation and in what order the operations are performed). If these factors are not optimized, the overall productivity decreases, the costs increase, and the whole process is inefficient. The optimization cost function could be based on different industrial user requirements, e.g., time, cost, and fleet productivity.

Figure 4: Benefits of earthmoving automation. Image courtesy of Novatron Oy.

In Fig.5, a possible shared representation of the construction site might be obtained using Building Information Modeling (BIM) [11]. Building information modeling (BIM) is a process that generates and manages digital representations of construction projects. The model includes detailed information about the project’s geometry, construction materials, systems, and other features (see Fig.5). The differences between the initial and final BIM models could be used for specifying goals for the fleet of machines available on the site, allowing to synthesize construction tasks (e.g., reaching the pile position, load material, leave the material at dump locations, etc.) from high-level specifications.

Figure 5: Building Information Modeling (BIM). Image courtesy of Novatron Oy.


Header image courtesy of Novatron Oy (An excavator and a dump truck on a work site)

[1] Geimer, M. (2020). Mobile working machines. SAE International, Warrendale, PA, 2020.

[2] Kassem, M., Mahamedi, E., Rogage, K., Duffy, K., & Huntingdon, J. (2021). Measuring and benchmarking the productivity of excavators in infrastructure projects: A deep neural network approach. Automation in Construction, 124, 103532.

[3] Schaufelberger, J. E., & Migliaccio, G. C. (2019). Construction equipment management. Routledge.

[4] Guevara, J., Arevalo-Ramirez, T., Yandun, F., Torres-Torriti, M., & Cheein, F. A. (2020). Point cloud-based estimation of effective payload volume for earthmoving loaders. Automation in Construction, 117, 103207.

[5] Zauner, M., Altenberger, F., Knapp, H., & Kozek, M. (2020). Phase independent finding and classification of wheel-loader work-cycles. Automation in Construction, 109, 102962.

[6] Xu, X., & García de Soto, B. (2020). On-site Autonomous Construction Robots: A review of Research Areas, Technologies, and Suggestions for Advancement.

[7] Ha, Q.P., Yen, L., & Balaguer, C. (2019). Robotic autonomous systems for earthmoving in military applications. Automation in Construction.

[8] Mansouri, M., Pecora, F., & Schüller, P. (2021). Combining Task and Motion Planning: Challenges and Guidelines. Frontiers in Robotics and AI, 8.

[9] Nicole Elflein (Siemens mining), 2014. ‘Digging for efficiency’. Pictures of the Future, online magazine, Spring 2014.

[10] MIT Technology Review, ‘Mining 24 hours a day with robots’ (30-11-2017).


About the author

Paolo Forte
Early Stage Researcher 3

Paolo is a 3rd year Ph.D. researcher working on deploying fleets of (semi) autonomous vehicles for material handling in construction sites. His research focuses on autonomous task planning, task assignment and motion planning, and material flow optimization.