How to Maximize the Efficiency of Your Site Acceptance Tests During Commissioning

Imagine yourself standing at an impressive project job site, surrounded by the very latest equipment and technologies that are waiting to deliver their full potential. The clock is ticking, expectations are high and you are the key to a smooth commissioning process. But how can you ensure that SATs are carried out efficiently? How can you save time, reduce costs and ensure high quality at the same time?

This blog post discusses the biggest challenges in carrying out site acceptance tests and presents a three-stage solution that opens up previously unattainable potential for you to increase the overall efficiency of your commissioning process. 

The key role of Site Acceptance Tests (SATs) in mechanical engineering

Efficient commissioning plays a major role in mechanical engineering. SATs provide the opportunity to check the functionality and performance of machines and systems on site. They ensure that all components are correctly installed and connected together, that the electrical and mechanical connections work properly and that the machines can be operated safely and efficiently. In addition to quality assurance, SATs also serve as legal and contractual protection. They provide proof that the systems meet the agreed specifications and have successfully passed all the required performance tests.

The formal tests can range from software (“smoke test”) to performance testing a complete system, a machine or individual components under real operating conditions. Acceptance criteria [RL1] serve as reference points in implementation and define the required standards that must be met to successfully complete the SAT.

What is the difference between a FAT (Factory Acceptance Test) and a SAT (Site Acceptance Test)?

It is standard practice for a site acceptance test to be preceded by a factory acceptance test. The aim of both tests is to compare machine specifications with customer requirements and to verify them through various test procedures. The test procedures and process steps are sometimes repeated identically. 

The main difference between a FAT and a SAT, however, is where the test is carried out and when it is carried out. The FAT is carried out in the factory before the machines and systems are delivered to the customer, while the SAT is carried out at the customer's or operator's site in the final location of use of the machine after the installation is complete. Some tests require real production and process parameters that cannot be reconstructed by the manufacturer. Typical checks include: 

• Visual inspection: Every delivery process involves certain risks, e.g. certain components may be lost, damaged or deformed due to improper handling or environmental influences, which make a visual inspection on location at the customer's site indispensable. The inspection also ensures that all parts, components and documentation (e.g. manuals, spare parts lists, data sheets as well as CE declarations of conformity and other certificates) are provided.
• Environmental conditions: The machine manufacturer can only imitate the environmental conditions (temperature, humidity, pressure, etc.) of the actual production process to a limited extent. Here, it usually makes sense to draw on the conditions and domain knowledge of the local operator in order to obtain reliable test results.
• Electrical wiring and safety systems: Both electrical wiring and safety systems, such as emergency stop switches and circuit breakers, must be adequately checked after the machine is installed to prevent motor and voltage overloads. 
• Machine structure and foundations: The structure of the machine itself and the foundations on which it stands are examined for their stability in order to avoid vibrations. Machine modules and subsystems must be matched with each other. In addition, the piping system is checked for leaks and improper connections.

The importance of clearly defined acceptance criteria 

The acceptance criteria for SATs can be varied and are agreed in advance. The details are often very different and range from a simple “must function” to extensive contractual terms that have been carefully negotiated between the customer and the manufacturer. It is not unusual for the manufacturer to suggest a number of proven quality and product criteria, some of which are mandatory by legal regulations and relevant standards, while others can be amended at the customer's request. These criteria differ from industry to industry. Examples of acceptance criteria may include:

Productivity

• Throughput (number of units produced per hour)
• Availability (uptime, utilization, MTBF)
• Overall plant efficiency (OEE)
• Number of successful transports of wafers within a defined period without additional instruction or human intervention
• Operating performance
• Cycles per minute or total number of cycles
• Change of format or product within a defined period 

Quality

• Process quality and process specifications
• Uniformity of a coating, coating thickness
• Yield (IO vs. NIO or quality cycle time)
• Bonus-malus arrangement

Costs

• Consumption of consumables (gases and chemicals)
• Consumption of utilities
• Time until operation (ramp-up shortening, early delivery)

At which phase of the machine life cycle does the SAT take place, and what significance does it have for machine manufacturers?

In view of the complex life cycle of a machine, whose phases are closely linked and build on each other, the SAT is an integral part of successful operation during the commissioning phase. 
This phase depends on the smooth running of the process at the customer's site. As an experienced commissioning engineer from Meyer Burger Industries once said: “You know the SAT was a success if you didn’t have any surprises.” In other words, good planning and preparation is the key to a successful SAT. 
The SAT is particularly important as it proves both the effectiveness and performance of the ordered machine in the real production environment, as negotiated in data sheets, test reports and extensive discussions between the manufacturer and operator to define the specifications. Last but not least, this is also the time when the development costs and the working time invested pay off financially. After all, successful commissioning and acceptance also means handing over the machine to the customer, at which point the final payments are due. For the manufacturer, the commissioning phase is therefore the most important milestone in the entire life cycle of the plant.

The toughest challenges

However, the road to get there is a rocky one, as carrying out a SAT can involve facing up to various challenges. 

a) Drivers: costs and time

The most important factors in on-site acceptance testing are costs and time. These two are interconnected because the longer it takes to hand over the machineto the customer, the longer it takes until the final invoice is issued and the warranty period begins. The reason for this is that until the machine or system is finally accepted, the manufacturer is still the owner and bears full responsibility. Likewise, the manufacturer’s personnel are responsible for the project over a long period of time and cannot move on to the next upcoming project.

b) Bottlenecks: Inefficiency and unclear criteria 

The biggest bottlenecks during a SAT are the lack of efficient procedures and clearly agreed criteria. Since several parties and departments are usually involved and different interests clash, it is important that these are agreed. 

This starts with the project manager at headquarters, who must always be informed about all activities and any obstacles. The project manager is responsible for the cost and time-efficient coordination of on-site personnel with the required skills such as mechanics, commissioning experts, operators, service technicians and application engineers. These personnel are usually based locally, are not necessarily on the manufacturer's payroll, but are employees of partner companies or service providers. Even if they are well trained and experienced, unforeseen events, poor preparation such as missing parts and deliveries, or anything that is not clear in the contract can lead to inefficiencies, setbacks, and delays in reaching milestones. This in turn means that on-site personnel waste traveling time, working time and contractually agreed hours, which drives up project costs even further. 

In addition, continuous and transparent communication with the customer is required, which is a real challenge due to the large number of people involved. Especially if activities are not thoroughly monitored and documented and accessible at all times centrally. Last but not least, a complicated, inefficient acceptance process that is not transparent to everyone involved also increases the customer’s costs and time spent during installation, commissioning and acceptance.

c) Consequences: Low acceptance rate and delayed cash flow 

All of the above bottlenecks make it difficult to complete projects and prevent the essential flow of income. In other words, time problems inevitably lead to higher costs, which eat up the profit margin. In summary, the two most important consequences of delay to the SAT are a low rate of acceptance and delayed income due to time issues, both of which are needed for new investments, developments, and the implementation of new projects.

How can these challenges be solved sustainably?

These challenges are not unique, but apply to any(inter)national project in mechanical engineering that requires an acceptance test.

Most activities are repeatable and simply require transparency, central documentation, constant availability of up-to-date information for everyone involved. Additionally, they need flexibility in the event of changes and, last but not least, a simple and intuitive operating concept to organize and carry out all activities on schedule. These challenges have been addressed time and again by customers from the mechanical engineering sector. We knew there had to be a solution to this problem. 

The solution

Kontron AIS has developed a three-stage approach based on the EquipmentCloud® digital transformation solution for service and after-sales in mechanical engineering, which significantly increases commissioning efficiency. The great strength of EquipmentCloud® lies in the fact that it offers a manufacturer-independent platform for exchanging information with all the people involved and makes it easy to work together. By involving everybody in the SAT phase - from the machine supplier, service personnel, customer and project manager - and ensuring the frequent input of information, the entire process is made transparent and efficient right from the start. Another strength lies in the scalable and freely selectable modules, which on the one hand enable the implementation of different use cases and, on the other hand, are perfectly coordinated.

The three-stage approach embedded in EquipmentCloud® is the basis for tapping into this potential:

1. Creating transparency

In mechanical engineering it is often the way that machines and systems are globally distributed, in some cases standardized, but often also with a high proportion of variants or customer specifications. In addition, they consist of linked modules and components. Each component therefore requires versatile and in-depth technical and process knowledge on the part of the commissioning engineers and project managers. It is therefore clear that it can only be beneficial to democratize knowledge and make it available to everyone involved. The prerequisite for this is that the knowledge is well-structured and easy to find. The equipment hierarchy within EquipmentCloud® is the key element in creating this transparency. It allows information to be organized or bundled according to a single machine, a machine type and at higher hierarchical levels such as country, customer and industry. 

In addition, the EquipmentHub module provides centralized configuration of all machines due to comprehensive master data, group and component management.

The combination of Workflows and OpenIssues modules serves as a third element for greater transparency. Checklists are used to bring order and structure to the SAT, and at the same time, any values that deviate from the acceptance criteria can be recorded immediately as soon as they are identified. The resulting tasks can be comprehensively documented (description, photos, attachments of images and documents) and the corresponding status, priority and deadline as well as responsibility can be directly assigned. The integrated subscription function makes it possible to inform all project participants and people interested in the process about deviations and changes to avoid loss of information. 

Finally, the Monitoring module ensures transparency with regard to the continuous analysis of relevant KPIs and process values, which manufacturer and customer have agreed on. This is the main focus following installation, and leads up to the most demanding task of the commissioning process: the final acceptance test. In this phase, the decisive KPIs such as uptime, throughput and output of the machines must be proven. The data that is often tracked manually over hours and weeks and leads to lengthy interpretation discussions can be easily and immediately visualized for all parties involved using a uniform standard for all relevant machine data (machine status, alarms, throughput) and key figures using the Monitoring module. This not only speeds up the decision-making process, but also prevents a delayed SAT.  Any doubt about the KPIs is therefore immediately dispelled, because now coherent data collection, process evaluation and machine comparisons across different locations is possible. Manual recordings and evaluations are a thing of the past.

2. Implementing digital process routines

In order to digitalize existing work processes, it is assumed that the software can visualize each process individually and flexibly. The Workflows module makes this possible with its practical task board design, which is used to set up complex and multi-stage processes using milestones (phases), checklists and task groups. 

Each individual phase and each checklist have a status so that the commissioning team and project management can view the current status at any time without having to hold meetings or consultations with colleagues at the site. There is another advantage: phases to be processed in parallel or simultaneously are recorded separately. This prevents employees from doing the same task twice, and it keeps track of the time taken. Since timing is usually crucial, coordinating work together helps to save time and move the project forward smoothly. 
The checklists also offer a high level of granularity, so that individual tasks can be assigned specific parameters and completed by the technicians according to clear instructions. These range from entering measurement parameters, dates and text to uploading snapshots and documents. This has the advantage that it is clear what needs to be checked, that nothing is forgotten and that direct feedback on progress is visualized. This provides operators, service technicians and commissioning personnel with a reliable source of information during the process and creates transparency about all SAT activities. In addition, these predefined activities have a positive effect on subsequent acceptance criteria. 
 

3) Simplifying documentation

The final step after implementing the process routine is to ensure consistent documentation throughout the SAT.

It is also easy and useful to be able to store and manage documents and acceptance results in one place. The eDocs module supports a variety of formats, such as *.pdf, *.doc, *.xlsm, *.zip and *.html, which can be mapped as folders, individual documents or linked to the relevant machine or hierarchy. This means that manufacturers and all parties involved always have the documents relevant to the final acceptance test at hand. Intermediate results of the acceptance criteria can be saved, completed checklists can be stored, and any data sheets, manuals or installation instructions required can be retrieved. This makes documentation a simple and convenient task that is no longer a bottleneck for project managers, commissioning engineers, installation engineers, service technicians and customers. In addition, and this is the most important thing for a mechanical engineering firm, the customer’s acceptance of SATs in general can be significantly increased if all information that is provided during the life cycle is consistently recorded and documented at the time of occurrence, not weeks later. That is what makes documentation such an important part of the SAT process.

The three-stage approach presented, in combination with the various EquipmentCloud® modules, offers enormous added value for the successful implementation of the SAT. What is decisive, however, is how mechanical engineering companies can benefit in the long term: So how can EquipmentCloud® increase overall commissioning efficiency?

Here are some of the biggest potentials:

• Process optimization: Speeding up test procedures by up to 50%.
• Time savings: Reducing the duration of the entire SAT process from delivery to final acceptance by up to 40%.
• Cost reduction: Reducing on-site costs for service engineers, project managers, and process specialists
• Improving product and service quality

A centralized and cloud-based digital transformation solution that allows access with any mobile device from anywhere in the world removes such barriers and connects people, devices and systems intelligently and intuitively. System-based implementation, guaranteed process compliance and up-to-date information are further factors that speed up SATs. Following successful implementation, organizers benefit from routines, a hierarchical structure, discreet role definition within the team and the testing of the use case using content that only needs to be entered once to reduce costs and boost interoperability across all modules used.

Overall improved customer satisfaction and enormous savings in resources and costs are the direct result of more efficient commissioning and successful site acceptance tests.

Would you like to learn how other machine and equipment builders successfully implement SATs? Then read the following success story from our customer Fabmatics!