Would it? Or would SIP/TLS supplant that? The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything. As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides? On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka anakaoka at trinet-hi.com <mailto:anakaoka at trinet-hi.com> Direct: 808.356.2901 518 Holokahana Lane Honolulu, Hi?96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY <https://youtu.be/PdUyuf0hTYY>
A Better Solution https://www.trinet-hi.com/abettersolution.pdf <https://www.trinet-hi.com/abettersolution.pdf> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU <https://www.youtube.com/watch?v=96YWPY9wCeU>
CONFIDENTIALITY NOTICE: ?The information contained in this email and any attachments may be privileged, confidential and protected from disclosure.? Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com <mailto:ryandelgrosso at gmail.com>> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers?
I can understand them being tight lipped but some transparency helps the situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote: > On 9/27/21 13:30, Darren via VoiceOps wrote: >> I know it?s hard to be patient but I can?t imagine they?re NOT all >> hands on deck. >> >> The reality is probably that the DDoS attack is now so big, they >> can?t handle it on their own, so they?re scrambling to contract out >> with another provider who can handle it. That would explain why the >> BGP routes they advertise have shifted. These DDoS products typically >> take weeks to setup, so they?re likely having to scramble. I?ll be >> surprised if this does NOT continue tomorrow (unfortunately). > > From my understanding this is not your typical volumetric DDoS but > something specific to SIP or VoIP and thus the typical scrubbing > services aren't going to be effective against the voice side of things. > > Obviously they are keeping things close to the vest in order not to > give too much information to the bad guys but I agree that it may take > some time to resolve. > >> *From: *VoiceOps <voiceops-bounces at voiceops.org <mailto:voiceops-bounces at voiceops.org>> on behalf of Carlos >> Alvarez <caalvarez at gmail.com <mailto:caalvarez at gmail.com>> >> *Date: *Monday, September 27, 2021 at 1:23 PM > >> Generic SIP client here, and the ongoing "continue to investigate" >> notices are infuriatingly like "we have no damn clue what we're >> doing."? Try explaining to customers why it's not "our fault*" and >> that there's no way to estimate a repair time. > > I think the ongoing "continue to investigate" messages are fine. > They're obviously dealing with a major incident and trying their best > to keep their customers informed. This IMHO beats silence. > >> *Our fault for choosing them I guess, but not something we can fix in >> minutes. > > The same thing could and has affected others. Voip.ms has been dealing > with a similar attack for at least a week. We've had excellent service > from Bandwidth for years and I trust that they will be able to get > through this as well as anyone. > > It's the nature of the legacy PSTN that redundant providers or fast > failover for inbound calling isn't (yet) a thing. > _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org <mailto:VoiceOps at voiceops.org> https://puck.nether.net/mailman/listinfo/voiceops <https://puck.nether.net/mailman/listinfo/voiceops>
Bandwidth is a $2.5B publicly traded company... how is this not national news? On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides? On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka anakaoka at trinet-hi.com Direct: 808.356.2901 518 Holokahana Lane Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf <https://www.trinet-hi.com/abettersolution.pdf> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers?
I can understand them being tight lipped but some transparency helps the situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos Alvarez <caalvarez at gmail.com> *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
?I was wondering this too. They are a big piece of our nation?s communications infrastructure at this point, in my eyes anyway. Maybe I?m old. I would think people would care more. I know our customers were livid and there were a lot? From: VoiceOps <voiceops-bounces at voiceops.org> on behalf of Pete Eisengrein <peeip989 at gmail.com> Date: Tuesday, September 28, 2021 at 1:56 AM Cc: "Voiceops.org" <voiceops at voiceops.org> Subject: Re: [VoiceOps] Bandwidth - Monday Outage Bandwidth is a $2.5B publicly traded company... how is this not national news? On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote: Would it? Or would SIP/TLS supplant that? The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything. As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides? On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote: If its SIP based, Verizon standard to run a VPN would become a norm. Aryn Nakaoka anakaoka at trinet-hi.com<mailto:anakaoka at trinet-hi.com> Direct: 808.356.2901 518 Holokahana Lane Honolulu, Hi 96817 AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY A Better Solution https://www.trinet-hi.com/abettersolution.pdf <https://www.trinet-hi.com/abettersolution.pdf> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation. On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote: Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers? I can understand them being tight lipped but some transparency helps the situation. I wonder if DHS is involved yet? On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps <voiceops-bounces at voiceops.org<mailto:voiceops-bounces at voiceops.org>> on behalf of Carlos Alvarez <caalvarez at gmail.com<mailto:caalvarez at gmail.com>> *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org> https://puck.nether.net/mailman/listinfo/voiceops _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org> https://puck.nether.net/mailman/listinfo/voiceops
I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too. On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein <peeip989 at gmail.com> wrote:
Bandwidth is a $2.5B publicly traded company... how is this not national news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides? On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka anakaoka at trinet-hi.com Direct: 808.356.2901 518 Holokahana Lane Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf <https://www.trinet-hi.com/abettersolution.pdf> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers?
I can understand them being tight lipped but some transparency helps the situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos Alvarez <caalvarez at gmail.com> *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
I have been asking myself that question all day? how is this not somehow covered on the news somewhere? From: VoiceOps <voiceops-bounces at voiceops.org> On Behalf Of Carlos Alvarez Sent: Monday, September 27, 2021 8:04 PM To: Voiceops.org <voiceops at voiceops.org> Subject: Re: [VoiceOps] Bandwidth - Monday Outage I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too. On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein <peeip989 at gmail.com<mailto:peeip989 at gmail.com>> wrote: Bandwidth is a $2.5B publicly traded company... how is this not national news? On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote: Would it? Or would SIP/TLS supplant that? The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything. As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides? On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote: If its SIP based, Verizon standard to run a VPN would become a norm. Aryn Nakaoka anakaoka at trinet-hi.com<mailto:anakaoka at trinet-hi.com> Direct: 808.356.2901 518 Holokahana Lane Honolulu, Hi 96817 AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY A Better Solution https://www.trinet-hi.com/abettersolution.pdf <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwww.trinet-hi.com%2fabet...> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation. On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote: Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers? I can understand them being tight lipped but some transparency helps the situation. I wonder if DHS is involved yet? On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps <voiceops-bounces at voiceops.org<mailto:voiceops-bounces at voiceops.org>> on behalf of Carlos Alvarez <caalvarez at gmail.com<mailto:caalvarez at gmail.com>> *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fVoip.ms&c=E,1,Fp6Nhg6BP8...> has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org> https://puck.nether.net/mailman/listinfo/voiceops<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailman%2flistinfo%2fvoiceops&c=E,1,fBHre1MZgNcXdPtFt1TIUULdMwgiKht4aAPAMJhM0mPl_M-VA3vCDSHs7iyZsv7o7gRm6bJWe8PYOuPbS9aAoHnucY8irU_ifD4iwJi_pweM_mtjRYSh&typo=1> _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org> https://puck.nether.net/mailman/listinfo/voiceops<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailman%2flistinfo%2fvoiceops&c=E,1,WZcKxLiq6YefVsNZy-WVXoKlwjAB2mzajAo9hwFOVEj84NSyMn5vyODLtNELXGBIYbF-5VrLPQGbJ5jtYGCzwvgdTSxHG6gTeDj7lX5Jd8GVUtMrIwxmqOotwhE,&typo=1> _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org> https://puck.nether.net/mailman/listinfo/voiceops<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailman%2flistinfo%2fvoiceops&c=E,1,k1fqwfwzE5XRJuMAJvMCFd0Sz9yiToZpt2E_XstZ5VHbaDRO1yqqf0IjlksBmwJNHtR1bahj292vjE-bsXW8lA0hecxy3qeZJKMsFUC22F7GZoO_&typo=1>
Wow, there really isn't anything anywhere... ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com ----- Original Message ----- From: "Mark Wiles via VoiceOps" <voiceops at voiceops.org> To: "Voiceops.org" <voiceops at voiceops.org> Sent: Monday, September 27, 2021 7:27:02 PM Subject: Re: [VoiceOps] Bandwidth - Monday Outage I have been asking myself that question all day? how is this not somehow covered on the news somewhere? From: VoiceOps <voiceops-bounces at voiceops.org> On Behalf Of Carlos Alvarez Sent: Monday, September 27, 2021 8:04 PM To: Voiceops.org <voiceops at voiceops.org> Subject: Re: [VoiceOps] Bandwidth - Monday Outage I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too. On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein < peeip989 at gmail.com > wrote: Bandwidth is a $2.5B publicly traded company... how is this not national news? On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso < ryandelgrosso at gmail.com > wrote: <blockquote> Would it? Or would SIP/TLS supplant that? The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything. As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides? On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote: <blockquote> If its SIP based, Verizon standard to run a VPN would become a norm. Aryn Nakaoka anakaoka at trinet-hi.com Direct: 808.356.2901 518 Holokahana Lane Honolulu, Hi 96817 AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY A Better Solution https://www.trinet-hi.com/abettersolution.pdf Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation. On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso < ryandelgrosso at gmail.com > wrote: <blockquote> Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers? I can understand them being tight lipped but some transparency helps the situation. I wonder if DHS is involved yet? On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps < voiceops-bounces at voiceops.org > on behalf of Carlos Alvarez < caalvarez at gmail.com > *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops </blockquote> _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops </blockquote> _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops </blockquote> _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
All I have I seen is a few reddit threads on it. On Mon, Sep 27, 2021, 6:40 PM Mike Hammett <voiceops at ics-il.net> wrote:
Wow, there really isn't anything anywhere...
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
Midwest Internet Exchange http://www.midwest-ix.com
------------------------------ *From: *"Mark Wiles via VoiceOps" <voiceops at voiceops.org> *To: *"Voiceops.org" <voiceops at voiceops.org> *Sent: *Monday, September 27, 2021 7:27:02 PM *Subject: *Re: [VoiceOps] Bandwidth - Monday Outage
I have been asking myself that question all day? how is this not somehow covered on the news somewhere?
*From:* VoiceOps <voiceops-bounces at voiceops.org> *On Behalf Of *Carlos Alvarez *Sent:* Monday, September 27, 2021 8:04 PM *To:* Voiceops.org <voiceops at voiceops.org> *Subject:* Re: [VoiceOps] Bandwidth - Monday Outage
I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too.
On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein <peeip989 at gmail.com> wrote:
Bandwidth is a $2.5B publicly traded company... how is this not national news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides?
On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka anakaoka at trinet-hi.com Direct: 808.356.2901
518 Holokahana Lane
Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf
<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwww.trinet-hi.com%2fabet...>
Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers?
I can understand them being tight lipped but some transparency helps the situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos Alvarez <caalvarez at gmail.com> *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fVoip.ms&c=E,1,Fp6Nhg6BP8...> has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailma...>
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailma...>
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailma...>
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
Because they aren't att, vz, sprint or t-mobile, as well as being a voip company and thus not a 'real' phone company. Not a household name. Give it a few more days and it will pop up on a few web networks but the only mention you'll see otherwise is how you can't trust voip companies to stay up. On Mon, Sep 27, 2021, 5:32 PM Mark Wiles via VoiceOps <voiceops at voiceops.org> wrote:
I have been asking myself that question all day? how is this not somehow covered on the news somewhere?
*From:* VoiceOps <voiceops-bounces at voiceops.org> *On Behalf Of *Carlos Alvarez *Sent:* Monday, September 27, 2021 8:04 PM *To:* Voiceops.org <voiceops at voiceops.org> *Subject:* Re: [VoiceOps] Bandwidth - Monday Outage
I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too.
On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein <peeip989 at gmail.com> wrote:
Bandwidth is a $2.5B publicly traded company... how is this not national news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides?
On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka anakaoka at trinet-hi.com Direct: 808.356.2901
518 Holokahana Lane
Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf
<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwww.trinet-hi.com%2fabet...>
Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent just collateral damage in a more plebian attack aimed ad portals/apis or routers?
I can understand them being tight lipped but some transparency helps the situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
On 9/27/21 13:30, Darren via VoiceOps wrote:
I know it?s hard to be patient but I can?t imagine they?re NOT all hands on deck.
The reality is probably that the DDoS attack is now so big, they can?t handle it on their own, so they?re scrambling to contract out with another provider who can handle it. That would explain why the BGP routes they advertise have shifted. These DDoS products typically take weeks to setup, so they?re likely having to scramble. I?ll be surprised if this does NOT continue tomorrow (unfortunately).
From my understanding this is not your typical volumetric DDoS but something specific to SIP or VoIP and thus the typical scrubbing services aren't going to be effective against the voice side of things.
Obviously they are keeping things close to the vest in order not to give too much information to the bad guys but I agree that it may take some time to resolve.
*From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos Alvarez <caalvarez at gmail.com> *Date: *Monday, September 27, 2021 at 1:23 PM
Generic SIP client here, and the ongoing "continue to investigate" notices are infuriatingly like "we have no damn clue what we're doing." Try explaining to customers why it's not "our fault*" and that there's no way to estimate a repair time.
I think the ongoing "continue to investigate" messages are fine. They're obviously dealing with a major incident and trying their best to keep their customers informed. This IMHO beats silence.
*Our fault for choosing them I guess, but not something we can fix in minutes.
The same thing could and has affected others. Voip.ms <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fVoip.ms&c=E,1,Fp6Nhg6BP8...> has been dealing with a similar attack for at least a week. We've had excellent service from Bandwidth for years and I trust that they will be able to get through this as well as anyone.
It's the nature of the legacy PSTN that redundant providers or fast failover for inbound calling isn't (yet) a thing.
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailma...>
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailma...>
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailma...>
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
On 9/27/21 17:27, Mark Wiles via VoiceOps wrote:
I have been asking myself that question all day? how is this not somehow covered on the news somewhere?
I suppose Alexandria, VA counts as "somewhere". https://www.kalb.com/2021/09/27/widespread-telephone-outage-causing-problems... -- Jay Hennigan - jay at west.net Network Engineering - CCIE #7880 503 897-8550 - WB6RDV
participants (9)
-
caalvarez＠gmail.com -
d＠d-man.org -
jay＠west.net -
karl＠piratel.com -
mauricio.lizano＠gmail.com -
mwiles＠akabis.com -
peeip989＠gmail.com -
ryandelgrosso＠gmail.com -
voiceops＠ics-il.net