
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine. ~Jared

Yes, they apparently claimed an issue with their authoritative DNS which they use for resolving customer endpoints. On Apr 18, 2013, at 12:59 PM, "Jared Geiger" <jared at compuwizz.net> wrote:
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Yes. We saw it start around 1:15 Eastern time. According to my VZ rep, it was a "catastrophic network outage" (her words) caused by a DDOS. We had earlier heard that there was an issue with DNS, so I assume that was probably the target. Traffic was up and down for a while. Seems to have stablized around 3pm, though if it was a DDOS, I wouldn't be surprised to see it go back down. Interestingly, I tried emailing the group earlier and my message never went through. On Thu, Apr 18, 2013 at 1:52 PM, Jared Geiger <jared at compuwizz.net> wrote:
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Verizon had a DNS issue impacted inbound SIP DID's supposedly resolved we are still seeing intermittent issues Sent from my iPhone On Apr 18, 2013, at 10:52, Jared Geiger <jared at compuwizz.net> wrote:
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Agreed, the official response is DNS. we got a verbal answer that it was a DDoS. On Apr 18, 2013, at 4:26 PM, Pasq77 <pasq77 at gmail.com> wrote:
Verizon had a DNS issue impacted inbound SIP DID's supposedly resolved we are still seeing intermittent issues
Sent from my iPhone
On Apr 18, 2013, at 10:52, Jared Geiger <jared at compuwizz.net> wrote:
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls. David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com *From:* VoiceOps [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Jared Geiger *Sent:* Thursday, April 18, 2013 10:52 AM *To:* VoiceOps *Subject:* [VoiceOps] Verizon SIP issues Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine. ~Jared

How can it be that there was such a far reaching outage (in our rep's words, "catastrophic") yet I cannot find any news whatsoever about it and there was very, very little on the likes of Twitter. Anyone know of any news outlets that said anything? On Thu, Apr 18, 2013 at 5:40 PM, David Thompson <dthompson at esi-estech.com>wrote:
Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
*From:* VoiceOps [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Jared Geiger *Sent:* Thursday, April 18, 2013 10:52 AM *To:* VoiceOps *Subject:* [VoiceOps] Verizon SIP issues
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Because Verizon probably pays a PR firm to scrub any negative reviews or information from searches. Not sure of how it?s accomplished exactly but I heard there is a firm out there that offers money to people to remove negative feedback/information. Eg. You get a call from someone offering you 500 bucks to delete your post. David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com *From:* PE [mailto:peeip989 at gmail.com] *Sent:* Friday, April 19, 2013 11:10 AM *To:* David Thompson *Cc:* Jared Geiger; VoiceOps *Subject:* Re: [VoiceOps] Verizon SIP issues How can it be that there was such a far reaching outage (in our rep's words, "catastrophic") yet I cannot find any news whatsoever about it and there was very, very little on the likes of Twitter. Anyone know of any news outlets that said anything? On Thu, Apr 18, 2013 at 5:40 PM, David Thompson <dthompson at esi-estech.com> wrote: Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls. David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com *From:* VoiceOps [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Jared Geiger *Sent:* Thursday, April 18, 2013 10:52 AM *To:* VoiceOps *Subject:* [VoiceOps] Verizon SIP issues Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine. ~Jared _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Fact of the matter is that service providers, more so on the scale of Verizon, have 'catastrophic' outages all the time. I've seen a local fiber cut called 'catastrophic'... A DDoS attack targeting DNS that impacted calls for a few hours to a subset of their customers is not really front page news... On Fri, Apr 19, 2013 at 2:40 PM, David Thompson <dthompson at esi-estech.com>wrote:
Because Verizon probably pays a PR firm to scrub any negative reviews or information from searches. Not sure of how it?s accomplished exactly but I heard there is a firm out there that offers money to people to remove negative feedback/information. Eg. You get a call from someone offering you 500 bucks to delete your post.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
*From:* PE [mailto:peeip989 at gmail.com] *Sent:* Friday, April 19, 2013 11:10 AM *To:* David Thompson *Cc:* Jared Geiger; VoiceOps *Subject:* Re: [VoiceOps] Verizon SIP issues
How can it be that there was such a far reaching outage (in our rep's words, "catastrophic") yet I cannot find any news whatsoever about it and there was very, very little on the likes of Twitter. Anyone know of any news outlets that said anything?
On Thu, Apr 18, 2013 at 5:40 PM, David Thompson <dthompson at esi-estech.com> wrote:
Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
*From:* VoiceOps [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Jared Geiger *Sent:* Thursday, April 18, 2013 10:52 AM *To:* VoiceOps *Subject:* [VoiceOps] Verizon SIP issues
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

Understood. I'm not saying this is WSJ stuff but thought at least TMC or another trade rag would pick up on it. On Apr 19, 2013, at 4:45 PM, Christian Pena <christianpen at gmail.com> wrote:
Fact of the matter is that service providers, more so on the scale of Verizon, have 'catastrophic' outages all the time. I've seen a local fiber cut called 'catastrophic'...
A DDoS attack targeting DNS that impacted calls for a few hours to a subset of their customers is not really front page news...
On Fri, Apr 19, 2013 at 2:40 PM, David Thompson <dthompson at esi-estech.com> wrote: Because Verizon probably pays a PR firm to scrub any negative reviews or information from searches. Not sure of how it?s accomplished exactly but I heard there is a firm out there that offers money to people to remove negative feedback/information. Eg. You get a call from someone offering you 500 bucks to delete your post.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
From: PE [mailto:peeip989 at gmail.com] Sent: Friday, April 19, 2013 11:10 AM To: David Thompson Cc: Jared Geiger; VoiceOps Subject: Re: [VoiceOps] Verizon SIP issues
How can it be that there was such a far reaching outage (in our rep's words, "catastrophic") yet I cannot find any news whatsoever about it and there was very, very little on the likes of Twitter. Anyone know of any news outlets that said anything?
On Thu, Apr 18, 2013 at 5:40 PM, David Thompson <dthompson at esi-estech.com> wrote:
Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
From: VoiceOps [mailto:voiceops-bounces at voiceops.org] On Behalf Of Jared Geiger Sent: Thursday, April 18, 2013 10:52 AM To: VoiceOps Subject: [VoiceOps] Verizon SIP issues
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

TMC isn't going to make a carrier mad by spotlighting otherwise unnoticed issues. They're basically as far from Consumer Reports as you can get - almost any article you see on their sites are either industry rumors that someone wants to leak through a third party, or literally sponsored or written by industry players. The independent articles that aren't the above are generally about the industry as a whole, or are careful to remember how their bread is buttered. My 2c of course. On Apr 19, 2013, at 5:06 PM, PE <peeip989 at gmail.com> wrote:
Understood. I'm not saying this is WSJ stuff but thought at least TMC or another trade rag would pick up on it.
On Apr 19, 2013, at 4:45 PM, Christian Pena <christianpen at gmail.com> wrote:
Fact of the matter is that service providers, more so on the scale of Verizon, have 'catastrophic' outages all the time. I've seen a local fiber cut called 'catastrophic'...
A DDoS attack targeting DNS that impacted calls for a few hours to a subset of their customers is not really front page news...
On Fri, Apr 19, 2013 at 2:40 PM, David Thompson <dthompson at esi-estech.com> wrote: Because Verizon probably pays a PR firm to scrub any negative reviews or information from searches. Not sure of how it?s accomplished exactly but I heard there is a firm out there that offers money to people to remove negative feedback/information. Eg. You get a call from someone offering you 500 bucks to delete your post.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
From: PE [mailto:peeip989 at gmail.com] Sent: Friday, April 19, 2013 11:10 AM To: David Thompson Cc: Jared Geiger; VoiceOps Subject: Re: [VoiceOps] Verizon SIP issues
How can it be that there was such a far reaching outage (in our rep's words, "catastrophic") yet I cannot find any news whatsoever about it and there was very, very little on the likes of Twitter. Anyone know of any news outlets that said anything?
On Thu, Apr 18, 2013 at 5:40 PM, David Thompson <dthompson at esi-estech.com> wrote:
Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
From: VoiceOps [mailto:voiceops-bounces at voiceops.org] On Behalf Of Jared Geiger Sent: Thursday, April 18, 2013 10:52 AM To: VoiceOps Subject: [VoiceOps] Verizon SIP issues
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops

That's not really accurate. TMC would write about the outages if they knew about them. But if outages happen all the time, the news isn't the outages themselves but the fact that the new fiber/IP world isn't resilient. I blog and write a column for TMC. They don't censor anything. if you have an article or news about this, write it up and send it to me. I can put it on my blog as a guest or send it to my group editor. On 4/19/2013 11:56 PM, Paul Timmins wrote:
TMC isn't going to make a carrier mad by spotlighting otherwise unnoticed issues. They're basically as far from Consumer Reports as you can get - almost any article you see on their sites are either industry rumors that someone wants to leak through a third party, or literally sponsored or written by industry players.
The independent articles that aren't the above are generally about the industry as a whole, or are careful to remember how their bread is buttered.
If you have any questions, please contact me. Thank you. Regards, Peter Radizeski RAD-INFO, Inc. Telecom Specialist 813.963.5884 http://www.rad-info.net Read my blog at http://blog.tmcnet.com/on-rads-radar/

I have recently been aware of some carrier Non-Disclosure Agreements under which the outage, and any information related to its cause and/or resolution, would be covered. In other words, speaking publicly about the situation could be considered to be breach of contract. The content of these Non-Disclosure Agreements themselves may be covered under said NDAs. As a result, some of us who might otherwise talk about a given outage, in a forum such as this, or to a reporter, might avoid doing so. Because it can be hard to keep track of NDA provisions, one would not be blamed for creating a corporate disclosure policy that held carrier information to the standard in the "highest" NDA, so as to avoid employees having to track individual NDAs. Annoying? Yes. Commercially reasonable? Also yes. -jbn On Fri, Apr 19, 2013 at 1:10 PM, PE <peeip989 at gmail.com> wrote:
How can it be that there was such a far reaching outage (in our rep's words, "catastrophic") yet I cannot find any news whatsoever about it and there was very, very little on the likes of Twitter. Anyone know of any news outlets that said anything?
On Thu, Apr 18, 2013 at 5:40 PM, David Thompson <dthompson at esi-estech.com>wrote:
Yes I can definitely confirm that this morning Verizon was having problems completing SIP calls.
David Thompson Network Services Support Technician (O) 858.357.8794 (F) 858-225-1882 (E) dthompson at esi-estech.com (W) www.esi-estech.com
*From:* VoiceOps [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Jared Geiger *Sent:* Thursday, April 18, 2013 10:52 AM *To:* VoiceOps *Subject:* [VoiceOps] Verizon SIP issues
Did anyone else have problems with inbound Verizon SIP numbers for about the past 30 minutes? Outbound calls were working fine.
~Jared
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
participants (9)
-
ashley.winters@freedomvoice.com
-
christianpen@gmail.com
-
dthompson@esi-estech.com
-
jared@compuwizz.net
-
justin@ejtown.org
-
pasq77@gmail.com
-
paul@timmins.net
-
peeip989@gmail.com
-
peter@4isps.com