Yes, for the bank I deal with, we GPG encrypt and sign the ACH transfer file, and then we send across scp/sftp where both sides have known public keys. Both are rotated regularly.
I imagine many banks operate similarly, but I can't say that for certain.
I'm curious if the GPG signature is properly checked. There is no easy way to validate a GPG signature (ie fail if the file is not signed by X), you have to use a verbose machine mode and look for a specific string. GPG will happily decrypt a non signed file also, so there are many ways to misuse GPG, and many of them are SO top answers ;)
Well I know my side is checked :) I've never tried to send a not properly signed file, however. GPG is indeed a mess, but it's not like there are awesome alternatives, really.
I imagine many banks operate similarly, but I can't say that for certain.