Many networks must maintain synchrony despite the fact that they operate in noisy environments. Important examples are stochastic inertial oscillators, which are known to exhibit fluctuations with broad tails in many applications, including electric power networks with renewable energy sources. Such non-Gaussian fluctuations can result in rare network desynchronization. Here we build a general theory for inertial oscillator network desynchronization by non-Gaussian noise. We compute the rate of desynchronization and show that higher moments of noise enter at specific powers of coupling: either speeding up or slowing down the rate exponentially depending on how noise statistics match the statistics of a network's slowest mode. Finally, we use our theory to introduce a technique that drastically reduces the effective description of network desynchronization. Most interestingly, when instability is associated with a single edge, the reduction is to one stochastic oscillator.