Chemical rockets -- the only ones we've ever actually used -- explode because that's what they're intended to do. The only difference between a successful rocket firing and a catastrophic rocket failure is the speed at which the explosion happens. A nuclear rocket engine has basically no risk of explosion; tearing itself apart at speed maybe, if the aerodynamics aren't done properly or there's a structural weakness. But that's about it.
Normally at least the first stage of the rocket will still be chemical, so the risk of explosion is still there. Fortunately the risk of spreading radioactive material appears limited regardless.
Pretty much any large rocket today has an explosive self-destruct mechanism to cover failure modes where the rocket goes in an unintended direction. So that's one reason a nuclear rocket might explode at low altitude.
Chemical rockets -- the only ones we've ever actually used -- explode because that's what they're intended to do. The only difference between a successful rocket firing and a catastrophic rocket failure is the speed at which the explosion happens. A nuclear rocket engine has basically no risk of explosion; tearing itself apart at speed maybe, if the aerodynamics aren't done properly or there's a structural weakness. But that's about it.