[VoiceOps] RTP Delay issue with Level 3 Viper and Asterisk based devices

On Viper we have noticed customers using Asterisk based PBX sometimes have a delay of about 4-8 secs before the Customer actually hears the calling party once they answer the call. In the past we have moved a bunch of TNs over to Level 3 Vector to resolve the issue. Anyone aware of these issues and what causes them. I am assuming some RTP SSRC, timestamp, Seq number issue. But I have a trace on which everything seems OK, but no audio for first 5 secs.... Appreciate your thoughts/feedback on this.