Learning alongside fellow trainees and your instructor as you walk around a three-dimensional model of an aircraft, zoom in on components, examine system behaviour and analyse faults. All of this is possible with a new type of training that uses augmented reality (AR). It was developed and tested by NLR in association with KLM Royal Dutch Airlines. The advantages are numerous, not least because it makes training more attractive, more effective and more efficient. Trainees are actively involved in the instruction session, learn from each other, get a better insight into how the system works and are better able to put what they have learned into practice.
What is the best way for mechanics to learn how to maintain an aircraft? Is it in the traditional way, with an instructor in a classroom relying mainly on PowerPoint slides to explain how an aircraft is made up from a technical point of view, as still customary at most training schools? According to Anneke Nabben, training specialists from NLR, there are now better teaching methods. “Students who are simply required to listen quickly lose their attention and do not properly absorb the subject matter. It’s because they are not actively involved and don’t talk a lot to each other about the training.”
Use of multiple HoloLens for collaborative training
Nabben teamed up with KLM to create an innovative maintenance training that outperforms existing training in terms of efficiency and effectiveness. NLR recommended using HoloLens, or augmented reality goggles, in a collaborative way. This has hardly been done yet. The goggles enable trainees and instructor to look at each other and also at a hologram of an aircraft. Even more important is that they work actively (and interactively) on the subject matter. Trainees can walk around the 3D model of a real aircraft, point out and activate parts of the aircraft and talk to each other about them. To demonstrate just how innovative such training can ultimately be, NLR elaborated in detail one of the systems of an aircraft, the air conditioning pack.
Anneke Nabben at the NLR X-Lab
3D demonstrator
“We wanted to devise something that would give students a greater insight into how one of the aircraft systems works,” says Nabben. “To do this we focused the training more on cultivating understanding than on knowledge of facts. We came up with tasks that encouraged students to work with each other. We also got the instructor to impart the subject matter interactively to make it easier for the students to absorb. Technically, we supported this collaboration by interconnecting multiple HoloLens goggles to allow everybody to see the aircraft from his or her own perspective. You feel as if you are standing in front of a real aircraft.”
The advantage of such a 3D virtual aircraft is that it is usable both for theoretical and for practical training. In principle, NLR is able to model all complex mechanical systems in the aircraft, including systems that normally cannot be demonstrated in an aircraft on the ground, such as the landing gear. It is also possible to simulate normal and abnormal aircraft behaviour. What’s more, a virtual aircraft is available any place, any time, and nothing can wear out. The intention is for students to be able in future to exercise in making repairs by means of the 3D model without anything getting broken. And because a physical aircraft is no longer necessary, normal flight operations can continue as usual. Consideration is further being given to using HoloLens as performance support in carrying out tasks on a real aircraft during a mechanic’s normal work. Instructions about the actions required will be projected visually on to the aircraft.
More interesting and more motivating
By means of shorter but more effective training, NLR’s ambition is to increase the system knowledge of maintenance mechanics so that they are better able safely to perform aircraft maintenance. So has this been accomplished? Nabben: “We compared a group trained by means of HoloLens with a traditionally trained group. The evaluation showed that everybody found the training with HoloLens interesting, innovative and exciting. Students in this group scored higher in the final test and were better at explaining the working and function of the air conditioning pack. They said that with HoloLens they found it easier to make the transition to the real-life situation, in this particular instance having a better insight into the airflow throughout the aircraft, and said they can collaborate more with colleagues in this way. It takes equally as long to train a traditional group, but the trainees felt it was actually longer. They get too much information and even fall asleep after a while.” Thanks to the 3D component localisation with HoloLens, a piece of practical training is provided simultaneously, while the traditional group still has to do it. In short, the modernised HoloLens training has increased the motivation of students, eased the transition to real-life maintenance and shortened the total length of training.
Training with HoloLens
- More interaction and collaboration
- Variety in training method
- Transfer from 2D > 3D > practice
- Comfort: HoloLens can be exhausting
- Confidence to apply knowledge into practice
Traditional training
- No student interaction
- Losing attention
- Passive: sit back & listen
- No supporting tools
- Too much information
- Lack system knowledge
- Training time too long
Students are enthusiastic about using HoloLens, although the new technique does still have a few disadvantages. “It is still tiring to wear the goggles for 20 minutes, because they are fairly heavy. And people who wear spectacles find it difficult to read while they have HoloLens on. Technically, it remains difficult to provide training simultaneously to about twenty people”, says Nabben. “It’s a question of time before HoloLens becomes lighter and will be available in the right reading strength. The final technical limitation could be solved by giving training to smaller groups”, which is also preferable from an educational point of view.
Augmented reality for maintenance, repair and inspection
Besides the demonstrator for KLM, NLR has developed a vision of the future use of HoloLens by maintenance personnel. “We want to integrate HoloLens throughout the MRO chain and to use it in the everyday performance of maintenance”, says Nabben. “For example, using augmented reality for every repair could help ensure the correct procedural steps by visually projecting the action that must be performed on the aircraft. At the same time, information could be given during inspections about previous repairs of the aircraft. If you integrate AR in this way, it will become common practice and you can increase the insight of mechanics, supervise them remotely and make recordings that are usable as progress training. But the real challenge is to organise the training differently and to make it more interactive, because that is the only way to leverage the full added value of HoloLens. It’s not enough simply to use a new technology and leave everything else as it was.”
More information
For more information about this case study:
Anneke Nabben anneke.nabben@nlr.nl