As a young new captain for a small five-airplane wet lease airline, Nicholas Dmoch thought his leadership style was “soft and laissez-faire,” he recalled. But shortly after leaving to join a large business aviation fleet operator, a lead flight attendant at the airline did Dmoch a big favor: she gave him a far more accurate assessment of his leadership style, and that sparked the idea for what became his new service, RateMyCRM.
“She said I came across as authoritarian,” he said. “Even in that small company, no one told me [that]. No one did.” So he decided to create a feedback mechanism for flight crew to rate each other—by invitation only—and he called it RateMyCRM.
A problem with CRM or crew resource management is that the training is usually lumped in with simulator training. “The situation is artificial,” he said, “and people show their best behavior. When flying on the line with someone, there are so many opportunities for observations because people behave in a more normal way.”
What Dmoch has found during his career is that people want feedback and benefit from receiving it, but that it also is rarely given. Or sometimes it comes too late. Or never.
An example was when two pilots who flew together for 20 years reached a point where they could not be scheduled in the same cockpit. “Little by little frictions built up until there was a clash of characters and they couldn’t fly together,” he said.
The only way to fix this problem was to send the pilots to mandatory training so they could rebuild their relationship. This could have been avoided, he explained, if one of the pilots had been given some constructive feedback about one small thing he did that resulted in the broken relationship. “If he was told, ‘we all like you, there’s just this one little thing,’ then it would be so much better.”
RateMyCRM is designed not to allow feedback from anyone except those who are invited to provide it. The requester is the one who pays for the service; there is no fee for the person providing the feedback.
The existing feedback system is fragmented and inconsistent, with some mechanisms in place such as postflight debriefings and check rides, which include the European NoTechs Scale feedback mechanism. NoTechs cover non-technical skills, defined as pilots’ attitudes and behaviors not directly related to aircraft control, systems control, and standard operating procedures.
Check Rides Not the Place for Honest Feedback
But systems like NoTechs can be too formal and may result in people playing too nice. “If you modified your behavior [to try to look better],” he said, “you would only kick yourself.” And it’s been shown that during simulator check rides, “if you ask two crewmembers to feedback each other on NoTechs, [their feedback] corresponds with the examiner. Check rides can be artificial and may not reveal all true sides of a person.”
When providing feedback via RateMyCRM, the person invited to give feedback is asked to respond to up to four question sets or questionnaires. The feedback is anonymous and those providing feedback do not need to supply any identifying information.
The first question set is NoTechs, and it “is based on the European non-technical skill behavioral marker system for crew resource management evaluation,” according to RateMyCRM.
Next is Perception Self/Others. “This questionnaire aims to give users feedback on how they are seen by their peers. Note that there are no negative attributes in the twelve categories of this questionnaire. You simply indicate on a scale whether the person is leaning more, for example, towards being serious and quiet or towards being talkative and sociable," Dmoch explained. "Users can then compare their self-evaluation with the feedback of their peers to better understand if there are any gaps in how they think they come across versus how they actually come across.”
The free text questionnaire allows the respondent to give any kind of feedback. “The questions being asked to stimulate a response are, ‘What should I do more?’ and 'What should I do less?’”
Finally, respondents are asked to indicate whether their pilot colleague “unintentionally exhibits any of the five FAA hazardous attitudes: anti-authority, impulsivity, invulnerability, macho, or resignation.”
Dmoch came up with the idea for RateMyCRM because he didn’t see anything similar available and he thought it would help pilots improve as leaders. “It takes away the awkwardness of giving feedback,” he said. “It’s really about more minor things, it’s not about tyrants in the cockpit who yell at people. They will be filtered out by complaints and safety reports. This is about fine-tuning and improving CRM skills. That’s also why I recommend it for first officers who want to upgrade to captain to get an idea of how they come across as leaders.”
Inadequate Leadership Training
It’s typical in aviation and probably other industries that people are thrust into leadership roles without training and especially without feedback. “We’re not being well taught,” he said. “If you don’t have feedback on your skills, you can’t fix it. If you don’t have training and don’t have feedback, then you don’t know if you need training.”
Dmoch is planning to add two more feedback questionnaires and adjust the way feedback providers fill out the questionnaires. One of the new ones will be quick to fill out and will cover important aspects of crew interaction, he explained. Questions will include: “Were my briefings useful? Do I invite suggestions and feedback? Did I exhibit appropriate situational awareness?” The second will focus on providing feedback to those who are in a training role, such as feedback on teaching skills, appropriate teaching techniques, fairness in evaluations, and more.
Eventually, the plan is for the requester to specify which of the six questionnaires to use instead of the current setup, which requires using at least the NoTechs questionnaire. Of course, at least one will have to be selected, but any of the six will do, or a combination.
Since the launch earlier this year, the RateMyCRM user base has grown, Dmoch said, with more than 140 active users on board. Two CRM instructors “are endorsing RateMyCRM in their CRM classes as a great tool for monitoring your CRM skills.”
Feedback on RateMyCRM so far “is very good,” he said. “Experienced first officers like to use it before starting their upgrade to obtain feedback on their leadership skills.” A Canadian first officer and CRM instructor commenting on LinkedIn wrote: “I see significant value for first officers seeking upgrade opportunities. RateMyCRM provides data to the pilot that increases the actionable insight for command prep and personal development.”
While RateMyCRM is free for feedback providers, the requester pays €4 ($4.30) for a package of collated feedback (there must be at least seven responses). Three feedback sessions cost €10 and 10 sessions €25. According to Dmoch, “The idea was always that obtaining feedback should be less expensive than inviting your fellow pilot for a free drink at most crew hotels, which is a very popular technique for establishing good interpersonal relationships.”