VoiceOps
By thread
voiceops@voiceops.org
By month
Messages by month
- ----- 2026 -----
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2025 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
September 2021
- 41 participants
- 153 messages
Bandwidth - Monday Outage
by voiceops@ics-il.net
Wow, there really isn't anything anywhere...
-----
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com
Midwest Internet Exchange
http://www.midwest-ix.com
----- Original Message -----
From: "Mark Wiles via VoiceOps" <voiceops at voiceops.org>
To: "Voiceops.org" <voiceops at voiceops.org>
Sent: Monday, September 27, 2021 7:27:02 PM
Subject: Re: [VoiceOps] Bandwidth - Monday Outage
I have been asking myself that question all day? how is this not somehow covered on the news somewhere?
From: VoiceOps <voiceops-bounces at voiceops.org> On Behalf Of Carlos Alvarez
Sent: Monday, September 27, 2021 8:04 PM
To: Voiceops.org <voiceops at voiceops.org>
Subject: Re: [VoiceOps] Bandwidth - Monday Outage
I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too.
On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein < peeip989 at gmail.com > wrote:
Bandwidth is a $2.5B publicly traded company... how is this not national news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso < ryandelgrosso at gmail.com > wrote:
<blockquote>
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides?
On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
<blockquote>
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka
anakaoka at trinet-hi.com
Direct: 808.356.2901
518 Holokahana Lane
Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf
Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso < ryandelgrosso at gmail.com > wrote:
<blockquote>
Do we know this is a SIP/RTP targeted volumetric attack and those arent
just collateral damage in a more plebian attack aimed ad portals/apis or
routers?
I can understand them being tight lipped but some transparency helps the
situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
> On 9/27/21 13:30, Darren via VoiceOps wrote:
>> I know it?s hard to be patient but I can?t imagine they?re NOT all
>> hands on deck.
>>
>> The reality is probably that the DDoS attack is now so big, they
>> can?t handle it on their own, so they?re scrambling to contract out
>> with another provider who can handle it. That would explain why the
>> BGP routes they advertise have shifted. These DDoS products typically
>> take weeks to setup, so they?re likely having to scramble. I?ll be
>> surprised if this does NOT continue tomorrow (unfortunately).
>
> From my understanding this is not your typical volumetric DDoS but
> something specific to SIP or VoIP and thus the typical scrubbing
> services aren't going to be effective against the voice side of things.
>
> Obviously they are keeping things close to the vest in order not to
> give too much information to the bad guys but I agree that it may take
> some time to resolve.
>
>> *From: *VoiceOps < voiceops-bounces at voiceops.org > on behalf of Carlos
>> Alvarez < caalvarez at gmail.com >
>> *Date: *Monday, September 27, 2021 at 1:23 PM
>
>> Generic SIP client here, and the ongoing "continue to investigate"
>> notices are infuriatingly like "we have no damn clue what we're
>> doing." Try explaining to customers why it's not "our fault*" and
>> that there's no way to estimate a repair time.
>
> I think the ongoing "continue to investigate" messages are fine.
> They're obviously dealing with a major incident and trying their best
> to keep their customers informed. This IMHO beats silence.
>
>> *Our fault for choosing them I guess, but not something we can fix in
>> minutes.
>
> The same thing could and has affected others. Voip.ms has been dealing
> with a similar attack for at least a week. We've had excellent service
> from Bandwidth for years and I trust that they will be able to get
> through this as well as anyone.
>
> It's the nature of the legacy PSTN that redundant providers or fast
> failover for inbound calling isn't (yet) a thing.
>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org
https://puck.nether.net/mailman/listinfo/voiceops
</blockquote>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org
https://puck.nether.net/mailman/listinfo/voiceops
</blockquote>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org
https://puck.nether.net/mailman/listinfo/voiceops
</blockquote>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org
https://puck.nether.net/mailman/listinfo/voiceops
Sept. 28, 2021
Bandwidth - Monday Outage
by mwiles@akabis.com
I have been asking myself that question all day? how is this not somehow covered on the news somewhere?
From: VoiceOps <voiceops-bounces at voiceops.org> On Behalf Of Carlos Alvarez
Sent: Monday, September 27, 2021 8:04 PM
To: Voiceops.org <voiceops at voiceops.org>
Subject: Re: [VoiceOps] Bandwidth - Monday Outage
I never watch news, but I would have expected my news cycle addict friends to be asking me by now. Hmm. There have to be some three letter agencies fully involved, too.
On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein <peeip989 at gmail.com<mailto:peeip989 at gmail.com>> wrote:
Bandwidth is a $2.5B publicly traded company... how is this not national news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote:
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides?
On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka
anakaoka at trinet-hi.com<mailto:anakaoka at trinet-hi.com>
Direct: 808.356.2901
518 Holokahana Lane
Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf
<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwww.trinet-hi.com%2fabe…>
Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent
just collateral damage in a more plebian attack aimed ad portals/apis or
routers?
I can understand them being tight lipped but some transparency helps the
situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
> On 9/27/21 13:30, Darren via VoiceOps wrote:
>> I know it?s hard to be patient but I can?t imagine they?re NOT all
>> hands on deck.
>>
>> The reality is probably that the DDoS attack is now so big, they
>> can?t handle it on their own, so they?re scrambling to contract out
>> with another provider who can handle it. That would explain why the
>> BGP routes they advertise have shifted. These DDoS products typically
>> take weeks to setup, so they?re likely having to scramble. I?ll be
>> surprised if this does NOT continue tomorrow (unfortunately).
>
> From my understanding this is not your typical volumetric DDoS but
> something specific to SIP or VoIP and thus the typical scrubbing
> services aren't going to be effective against the voice side of things.
>
> Obviously they are keeping things close to the vest in order not to
> give too much information to the bad guys but I agree that it may take
> some time to resolve.
>
>> *From: *VoiceOps <voiceops-bounces at voiceops.org<mailto:voiceops-bounces at voiceops.org>> on behalf of Carlos
>> Alvarez <caalvarez at gmail.com<mailto:caalvarez at gmail.com>>
>> *Date: *Monday, September 27, 2021 at 1:23 PM
>
>> Generic SIP client here, and the ongoing "continue to investigate"
>> notices are infuriatingly like "we have no damn clue what we're
>> doing." Try explaining to customers why it's not "our fault*" and
>> that there's no way to estimate a repair time.
>
> I think the ongoing "continue to investigate" messages are fine.
> They're obviously dealing with a major incident and trying their best
> to keep their customers informed. This IMHO beats silence.
>
>> *Our fault for choosing them I guess, but not something we can fix in
>> minutes.
>
> The same thing could and has affected others. Voip.ms<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fVoip.ms&c=E,1,Fp6Nhg6BP…> has been dealing
> with a similar attack for at least a week. We've had excellent service
> from Bandwidth for years and I trust that they will be able to get
> through this as well as anyone.
>
> It's the nature of the legacy PSTN that redundant providers or fast
> failover for inbound calling isn't (yet) a thing.
>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org>
https://puck.nether.net/mailman/listinfo/voiceops<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailm…>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org>
https://puck.nether.net/mailman/listinfo/voiceops<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailm…>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org>
https://puck.nether.net/mailman/listinfo/voiceops<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fpuck.nether.net%2fmailm…>
Sept. 28, 2021
Bandwidth - Monday Outage
by caalvarez@gmail.com
I never watch news, but I would have expected my news cycle addict friends
to be asking me by now. Hmm. There have to be some three letter agencies
fully involved, too.
On Mon, Sep 27, 2021 at 4:57 PM Pete Eisengrein <peeip989 at gmail.com> wrote:
> Bandwidth is a $2.5B publicly traded company... how is this not national
> news?
>
> On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com>
> wrote:
>
>> Would it? Or would SIP/TLS supplant that?
>>
>> The VZ VPN requirement was a bad solution to a paranoid delusion that
>> really didnt solve anything.
>>
>> As were all being dragged kicking and screaming into TLS based peering
>> for the sake of SHAKEN/STIR why not fully embrace what it provides?
>> On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
>>
>> If its SIP based, Verizon standard to run a VPN would become a norm.
>>
>>
>>
>>
>> Aryn Nakaoka
>> anakaoka at trinet-hi.com
>> Direct: 808.356.2901
>> 518 Holokahana Lane
>> Honolulu, Hi 96817
>>
>> AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
>>
>> A Better Solution https://www.trinet-hi.com/abettersolution.pdf
>> <https://www.trinet-hi.com/abettersolution.pdf>
>> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
>>
>> CONFIDENTIALITY NOTICE: The information contained in this email and any
>> attachments may be privileged, confidential and protected from disclosure.
>> Any disclosure, distribution or copying of this email or any attachments by
>> persons or entities other than the intended recipient is prohibited. If you
>> have received this email in error, please notify the sender immediately by
>> replying to the message and deleting this email and any attachments from
>> your system. Thank you for your cooperation.
>>
>>
>> On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com>
>> wrote:
>>
>>> Do we know this is a SIP/RTP targeted volumetric attack and those arent
>>> just collateral damage in a more plebian attack aimed ad portals/apis or
>>> routers?
>>>
>>> I can understand them being tight lipped but some transparency helps the
>>> situation.
>>>
>>> I wonder if DHS is involved yet?
>>>
>>> On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
>>> > On 9/27/21 13:30, Darren via VoiceOps wrote:
>>> >> I know it?s hard to be patient but I can?t imagine they?re NOT all
>>> >> hands on deck.
>>> >>
>>> >> The reality is probably that the DDoS attack is now so big, they
>>> >> can?t handle it on their own, so they?re scrambling to contract out
>>> >> with another provider who can handle it. That would explain why the
>>> >> BGP routes they advertise have shifted. These DDoS products typically
>>> >> take weeks to setup, so they?re likely having to scramble. I?ll be
>>> >> surprised if this does NOT continue tomorrow (unfortunately).
>>> >
>>> > From my understanding this is not your typical volumetric DDoS but
>>> > something specific to SIP or VoIP and thus the typical scrubbing
>>> > services aren't going to be effective against the voice side of things.
>>> >
>>> > Obviously they are keeping things close to the vest in order not to
>>> > give too much information to the bad guys but I agree that it may take
>>> > some time to resolve.
>>> >
>>> >> *From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos
>>> >> Alvarez <caalvarez at gmail.com>
>>> >> *Date: *Monday, September 27, 2021 at 1:23 PM
>>> >
>>> >> Generic SIP client here, and the ongoing "continue to investigate"
>>> >> notices are infuriatingly like "we have no damn clue what we're
>>> >> doing." Try explaining to customers why it's not "our fault*" and
>>> >> that there's no way to estimate a repair time.
>>> >
>>> > I think the ongoing "continue to investigate" messages are fine.
>>> > They're obviously dealing with a major incident and trying their best
>>> > to keep their customers informed. This IMHO beats silence.
>>> >
>>> >> *Our fault for choosing them I guess, but not something we can fix in
>>> >> minutes.
>>> >
>>> > The same thing could and has affected others. Voip.ms has been dealing
>>> > with a similar attack for at least a week. We've had excellent service
>>> > from Bandwidth for years and I trust that they will be able to get
>>> > through this as well as anyone.
>>> >
>>> > It's the nature of the legacy PSTN that redundant providers or fast
>>> > failover for inbound calling isn't (yet) a thing.
>>> >
>>> _______________________________________________
>>> VoiceOps mailing list
>>> VoiceOps at voiceops.org
>>> https://puck.nether.net/mailman/listinfo/voiceops
>>>
>> _______________________________________________
>> VoiceOps mailing list
>> VoiceOps at voiceops.org
>> https://puck.nether.net/mailman/listinfo/voiceops
>>
> _______________________________________________
> VoiceOps mailing list
> VoiceOps at voiceops.org
> https://puck.nether.net/mailman/listinfo/voiceops
>
Sept. 28, 2021
Bandwidth - Monday Outage
by d@d-man.org
?I was wondering this too. They are a big piece of our nation?s communications infrastructure at this point, in my eyes anyway. Maybe I?m old. I would think people would care more. I know our customers were livid and there were a lot?
From: VoiceOps <voiceops-bounces at voiceops.org> on behalf of Pete Eisengrein <peeip989 at gmail.com>
Date: Tuesday, September 28, 2021 at 1:56 AM
Cc: "Voiceops.org" <voiceops at voiceops.org>
Subject: Re: [VoiceOps] Bandwidth - Monday Outage
Bandwidth is a $2.5B publicly traded company... how is this not national news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote:
Would it? Or would SIP/TLS supplant that?
The VZ VPN requirement was a bad solution to a paranoid delusion that really didnt solve anything.
As were all being dragged kicking and screaming into TLS based peering for the sake of SHAKEN/STIR why not fully embrace what it provides?
On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
If its SIP based, Verizon standard to run a VPN would become a norm.
Aryn Nakaoka
anakaoka at trinet-hi.com<mailto:anakaoka at trinet-hi.com>
Direct: 808.356.2901
518 Holokahana Lane
Honolulu, Hi 96817
AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
A Better Solution https://www.trinet-hi.com/abettersolution.pdf
<https://www.trinet-hi.com/abettersolution.pdf>
Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
CONFIDENTIALITY NOTICE: The information contained in this email and any attachments may be privileged, confidential and protected from disclosure. Any disclosure, distribution or copying of this email or any attachments by persons or entities other than the intended recipient is prohibited. If you have received this email in error, please notify the sender immediately by replying to the message and deleting this email and any attachments from your system. Thank you for your cooperation.
On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com<mailto:ryandelgrosso at gmail.com>> wrote:
Do we know this is a SIP/RTP targeted volumetric attack and those arent
just collateral damage in a more plebian attack aimed ad portals/apis or
routers?
I can understand them being tight lipped but some transparency helps the
situation.
I wonder if DHS is involved yet?
On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
> On 9/27/21 13:30, Darren via VoiceOps wrote:
>> I know it?s hard to be patient but I can?t imagine they?re NOT all
>> hands on deck.
>>
>> The reality is probably that the DDoS attack is now so big, they
>> can?t handle it on their own, so they?re scrambling to contract out
>> with another provider who can handle it. That would explain why the
>> BGP routes they advertise have shifted. These DDoS products typically
>> take weeks to setup, so they?re likely having to scramble. I?ll be
>> surprised if this does NOT continue tomorrow (unfortunately).
>
> From my understanding this is not your typical volumetric DDoS but
> something specific to SIP or VoIP and thus the typical scrubbing
> services aren't going to be effective against the voice side of things.
>
> Obviously they are keeping things close to the vest in order not to
> give too much information to the bad guys but I agree that it may take
> some time to resolve.
>
>> *From: *VoiceOps <voiceops-bounces at voiceops.org<mailto:voiceops-bounces at voiceops.org>> on behalf of Carlos
>> Alvarez <caalvarez at gmail.com<mailto:caalvarez at gmail.com>>
>> *Date: *Monday, September 27, 2021 at 1:23 PM
>
>> Generic SIP client here, and the ongoing "continue to investigate"
>> notices are infuriatingly like "we have no damn clue what we're
>> doing." Try explaining to customers why it's not "our fault*" and
>> that there's no way to estimate a repair time.
>
> I think the ongoing "continue to investigate" messages are fine.
> They're obviously dealing with a major incident and trying their best
> to keep their customers informed. This IMHO beats silence.
>
>> *Our fault for choosing them I guess, but not something we can fix in
>> minutes.
>
> The same thing could and has affected others. Voip.ms has been dealing
> with a similar attack for at least a week. We've had excellent service
> from Bandwidth for years and I trust that they will be able to get
> through this as well as anyone.
>
> It's the nature of the legacy PSTN that redundant providers or fast
> failover for inbound calling isn't (yet) a thing.
>
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org>
https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________
VoiceOps mailing list
VoiceOps at voiceops.org<mailto:VoiceOps at voiceops.org>
https://puck.nether.net/mailman/listinfo/voiceops
Sept. 28, 2021
Bandwidth - Monday Outage
by peeip989@gmail.com
Bandwidth is a $2.5B publicly traded company... how is this not national
news?
On Mon, Sep 27, 2021 at 5:44 PM Ryan Delgrosso <ryandelgrosso at gmail.com>
wrote:
> Would it? Or would SIP/TLS supplant that?
>
> The VZ VPN requirement was a bad solution to a paranoid delusion that
> really didnt solve anything.
>
> As were all being dragged kicking and screaming into TLS based peering for
> the sake of SHAKEN/STIR why not fully embrace what it provides?
> On 9/27/2021 2:35 PM, Aryn Nakaoka 808.356.2901 wrote:
>
> If its SIP based, Verizon standard to run a VPN would become a norm.
>
>
>
>
> Aryn Nakaoka
> anakaoka at trinet-hi.com
> Direct: 808.356.2901
> 518 Holokahana Lane
> Honolulu, Hi 96817
>
> AlohaTone Mobile: https://youtu.be/PdUyuf0hTYY
>
> A Better Solution https://www.trinet-hi.com/abettersolution.pdf
> <https://www.trinet-hi.com/abettersolution.pdf>
> Aloha Tone PBX https://www.youtube.com/watch?v=96YWPY9wCeU
>
> CONFIDENTIALITY NOTICE: The information contained in this email and any
> attachments may be privileged, confidential and protected from disclosure.
> Any disclosure, distribution or copying of this email or any attachments by
> persons or entities other than the intended recipient is prohibited. If you
> have received this email in error, please notify the sender immediately by
> replying to the message and deleting this email and any attachments from
> your system. Thank you for your cooperation.
>
>
> On Mon, Sep 27, 2021 at 11:27 AM Ryan Delgrosso <ryandelgrosso at gmail.com>
> wrote:
>
>> Do we know this is a SIP/RTP targeted volumetric attack and those arent
>> just collateral damage in a more plebian attack aimed ad portals/apis or
>> routers?
>>
>> I can understand them being tight lipped but some transparency helps the
>> situation.
>>
>> I wonder if DHS is involved yet?
>>
>> On 9/27/2021 1:48 PM, Jay Hennigan via VoiceOps wrote:
>> > On 9/27/21 13:30, Darren via VoiceOps wrote:
>> >> I know it?s hard to be patient but I can?t imagine they?re NOT all
>> >> hands on deck.
>> >>
>> >> The reality is probably that the DDoS attack is now so big, they
>> >> can?t handle it on their own, so they?re scrambling to contract out
>> >> with another provider who can handle it. That would explain why the
>> >> BGP routes they advertise have shifted. These DDoS products typically
>> >> take weeks to setup, so they?re likely having to scramble. I?ll be
>> >> surprised if this does NOT continue tomorrow (unfortunately).
>> >
>> > From my understanding this is not your typical volumetric DDoS but
>> > something specific to SIP or VoIP and thus the typical scrubbing
>> > services aren't going to be effective against the voice side of things.
>> >
>> > Obviously they are keeping things close to the vest in order not to
>> > give too much information to the bad guys but I agree that it may take
>> > some time to resolve.
>> >
>> >> *From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Carlos
>> >> Alvarez <caalvarez at gmail.com>
>> >> *Date: *Monday, September 27, 2021 at 1:23 PM
>> >
>> >> Generic SIP client here, and the ongoing "continue to investigate"
>> >> notices are infuriatingly like "we have no damn clue what we're
>> >> doing." Try explaining to customers why it's not "our fault*" and
>> >> that there's no way to estimate a repair time.
>> >
>> > I think the ongoing "continue to investigate" messages are fine.
>> > They're obviously dealing with a major incident and trying their best
>> > to keep their customers informed. This IMHO beats silence.
>> >
>> >> *Our fault for choosing them I guess, but not something we can fix in
>> >> minutes.
>> >
>> > The same thing could and has affected others. Voip.ms has been dealing
>> > with a similar attack for at least a week. We've had excellent service
>> > from Bandwidth for years and I trust that they will be able to get
>> > through this as well as anyone.
>> >
>> > It's the nature of the legacy PSTN that redundant providers or fast
>> > failover for inbound calling isn't (yet) a thing.
>> >
>> _______________________________________________
>> VoiceOps mailing list
>> VoiceOps at voiceops.org
>> https://puck.nether.net/mailman/listinfo/voiceops
>>
> _______________________________________________
> VoiceOps mailing list
> VoiceOps at voiceops.org
> https://puck.nether.net/mailman/listinfo/voiceops
>
Sept. 27, 2021
thinQ is making hay while Bandwidth is down
by ross@tajvar.io
Windstream and Lumen are terrible for that on wireline.
On Mon, Sep 27, 2021, 6:35 PM Carlos Alvarez <caalvarez at gmail.com> wrote:
> Additionally, I've experienced a number of these cellular ports that
> seemed hung in mid-port, where some calls flowed to the wrong carrier and
> would die there. The most amusing have been where the losing carrier tries
> to route the call internally. Cox is also "good" at doing this shenanigan
> on wireline.
>
>
> On Mon, Sep 27, 2021 at 3:21 PM Karl Douthit <karl at piratel.com> wrote:
>
>> Mobile carriers will do an internal short term forwarding while LRN
>> changes get pushed or will use their existing agreements for porting and
>> push for an LRN change live. If you have access to NPAC LRN updates you
>> can make your changes but that does not guarantee that others see them
>> ASAP, so usually it is a bit of both.
>>
>> On Mon, Sep 27, 2021, 3:12 PM Alex Balashov via VoiceOps <
>> voiceops at voiceops.org> wrote:
>>
>>>
>>> > On Sep 27, 2021, at 6:04 PM, Aryn Nakaoka 808.356.2901 <
>>> anakaoka at trinet-hi.com> wrote:
>>> >
>>> > How do cell phone companies port instantly? I can walk into Verizon
>>> and they can port my Tmobile number to them. Or are they all sharing a back
>>> end?
>>>
>>> It?s got to be something like that. Whatever it is, VoIP ITSPs don?t
>>> have it through their intermediated supply chain of ULCs.
>>>
>>> --
>>> Alex Balashov | Principal | Evariste Systems LLC
>>>
>>> Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
>>> Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
>>>
>>> _______________________________________________
>>> VoiceOps mailing list
>>> VoiceOps at voiceops.org
>>> https://puck.nether.net/mailman/listinfo/voiceops
>>>
>> _______________________________________________
>> VoiceOps mailing list
>> VoiceOps at voiceops.org
>> https://puck.nether.net/mailman/listinfo/voiceops
>>
> _______________________________________________
> VoiceOps mailing list
> VoiceOps at voiceops.org
> https://puck.nether.net/mailman/listinfo/voiceops
>
Sept. 27, 2021
Five9 Issues / DDoS?
by pasq77@gmail.com
Five9 is not having any ddos issues we are impacted by some of the same
issues other Bandwidth customers have experienced. outbound and inbound
issues originally. Outbound has been routed around, inbound issues are
intermittent, have had issues with dtmf routing over specific bandwidth sip
trunks to be exact no issues with other traffic. We have seen intermittent
improvements on Bandwidth inbound recently but this is not the first time
we have seen improvement, and then problem returns.
If you believe that your being misled by a Five9 update please feel free to
pm me on the side and I can try and answer your questions
Thanks
TP
On Mon, Sep 27, 2021 at 3:35 PM Darren via VoiceOps <voiceops at voiceops.org>
wrote:
> Five9?s uses bandwidth.com to provide some of their services. Lookup the
> phone numbers that are having issues on inbound and check their actual
> owner.
>
>
>
> *From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Ivan
> Kovacevic via VoiceOps <voiceops at voiceops.org>
> *Reply-To: *Ivan Kovacevic <ivan.kovacevic at startelecom.ca>
> *Date: *Monday, September 27, 2021 at 3:23 PM
> *To: *VoiceOps <voiceops at voiceops.org>
> *Subject: *[VoiceOps] Five9 Issues / DDoS?
>
>
>
> So we have a Five9 SIP Trunking client who is being told by Five9 that
> their IVR/DTMF isn't working because of the Bandwidth.com outage.
>
>
>
> which is strange .... because their call path does not touch bandwidth.com
> at all... and raises the prospect of Five9s being targeted separately from
> Bandwidth?
>
>
>
> Any Five9 customers who are having issues which are definitely not related
> to Bandwidth.com? Or is that platform specific question too obscure for
> this list?
>
>
>
> [image: Star Telecom - Cloud Communications and Customer Experience
> Solutions] <https://www.startelecom.ca/>
>
> *Ivan Kovacevic*
> *Co-Founder and VP Client Services*
>
>
>
> NOTE: This email message and any attachments are for the sole use of the
> intended recipient(s) and may contain confidential and/or privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by replying to this email, and destroy all copies of the original
> message.
> _______________________________________________
> VoiceOps mailing list
> VoiceOps at voiceops.org
> https://puck.nether.net/mailman/listinfo/voiceops
>
Sept. 27, 2021
Five9 Issues / DDoS?
by abalashov@evaristesys.com
That said, lots of folks are being attacked. If another domino falls, this should not come as a shock.
> On Sep 27, 2021, at 6:28 PM, Darren via VoiceOps <voiceops at voiceops.org> wrote:
>
> Five9?s uses bandwidth.com to provide some of their services. Lookup the phone numbers that are having issues on inbound and check their actual owner.
>
> From: VoiceOps <voiceops-bounces at voiceops.org> on behalf of Ivan Kovacevic via VoiceOps <voiceops at voiceops.org>
> Reply-To: Ivan Kovacevic <ivan.kovacevic at startelecom.ca>
> Date: Monday, September 27, 2021 at 3:23 PM
> To: VoiceOps <voiceops at voiceops.org>
> Subject: [VoiceOps] Five9 Issues / DDoS?
>
> So we have a Five9 SIP Trunking client who is being told by Five9 that their IVR/DTMF isn't working because of the Bandwidth.com outage.
>
> which is strange .... because their call path does not touch bandwidth.com at all... and raises the prospect of Five9s being targeted separately from Bandwidth?
>
> Any Five9 customers who are having issues which are definitely not related to Bandwidth.com? Or is that platform specific question too obscure for this list?
>
>
> Ivan Kovacevic
> Co-Founder and VP Client Services
>
>
> NOTE: This email message and any attachments are for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by replying to this email, and destroy all copies of the original message.
> _______________________________________________
> VoiceOps mailing list
> VoiceOps at voiceops.org
> https://puck.nether.net/mailman/listinfo/voiceops
--
Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Sept. 27, 2021
Five9 Issues / DDoS?
by rschoneman@blumenthalarts.org
Didn?t Five9 use Voxbone who became part of Bandwidth.com? Also with the sale of Five9 to Zoom and Zoom using Bandwidth.com?
From: VoiceOps <voiceops-bounces at voiceops.org> On Behalf Of Ivan Kovacevic via VoiceOps
Sent: Monday, September 27, 2021 6:19 PM
To: VoiceOps <voiceops at voiceops.org>
Subject: [VoiceOps] Five9 Issues / DDoS?
So we have a Five9 SIP Trunking client who is being told by Five9 that their IVR/DTMF isn't working because of the Bandwidth.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__Bandwidth.com&d=DwQFaQ&…> outage.
which is strange .... because their call path does not touch bandwidth.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__bandwidth.com&d=DwQFaQ&…> at all... and raises the prospect of Five9s being targeted separately from Bandwidth?
Any Five9 customers who are having issues which are definitely not related to Bandwidth.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__Bandwidth.com&d=DwQFaQ&…>? Or is that platform specific question too obscure for this list?
[Star Telecom - Cloud Communications and Customer Experience Solutions]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.startelecom.ca_&d=…>
Ivan Kovacevic
Co-Founder and VP Client Services
NOTE: This email message and any attachments are for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by replying to this email, and destroy all copies of the original message.
Sept. 27, 2021
Five9 Issues / DDoS?
by ivan.kovacevic@startelecom.ca
We know the owner. It's a Canadian ILEC we work with and interconnect with
privately. So we know the traffic does not touch bandwidth.com - 100% sure
of it.
This is why I am puzzled... and wondering if Five9 is getting smacked
around as well, hiding behind the Bandwidth.com issue.
[image: Star Telecom - Cloud Communications and Customer Experience
Solutions] <https://www.startelecom.ca/>
*Ivan Kovacevic*
*Co-Founder and VP Client Services*
On Mon, Sep 27, 2021 at 6:28 PM Darren <d at d-man.org> wrote:
> Five9?s uses bandwidth.com to provide some of their services. Lookup the
> phone numbers that are having issues on inbound and check their actual
> owner.
>
>
>
> *From: *VoiceOps <voiceops-bounces at voiceops.org> on behalf of Ivan
> Kovacevic via VoiceOps <voiceops at voiceops.org>
> *Reply-To: *Ivan Kovacevic <ivan.kovacevic at startelecom.ca>
> *Date: *Monday, September 27, 2021 at 3:23 PM
> *To: *VoiceOps <voiceops at voiceops.org>
> *Subject: *[VoiceOps] Five9 Issues / DDoS?
>
>
>
> So we have a Five9 SIP Trunking client who is being told by Five9 that
> their IVR/DTMF isn't working because of the Bandwidth.com outage.
>
>
>
> which is strange .... because their call path does not touch bandwidth.com
> at all... and raises the prospect of Five9s being targeted separately from
> Bandwidth?
>
>
>
> Any Five9 customers who are having issues which are definitely not related
> to Bandwidth.com? Or is that platform specific question too obscure for
> this list?
>
>
>
> [image: Star Telecom - Cloud Communications and Customer Experience
> Solutions] <https://www.startelecom.ca/>
>
> *Ivan Kovacevic*
> *Co-Founder and VP Client Services*
>
>
>
> NOTE: This email message and any attachments are for the sole use of the
> intended recipient(s) and may contain confidential and/or privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by replying to this email, and destroy all copies of the original
> message.
>
--
NOTE: This email message and
any attachments are for the sole use of the
intended recipient(s) and may
contain confidential and/or privileged
information. Any unauthorized review,
use, disclosure or distribution is
prohibited. If you are not the intended recipient,
please contact the
sender by replying to this email, and destroy all copies of
the original
message.
Sept. 27, 2021