I had this issue as well, it turned out that something was multicasting frequently, which gets turned into a broadcast because of wireguard's nature. I just switched to plain wireguard with a VPS and my issues magically disappeared.