Hi folks, I'd love to hear some stories (good or bad) of hosted PBX VoIP installs on 100+ seat sites (single site). Specifically if you've done this with Broadsoft or another solidified switch. I have mixed opinions on how this type of scenario can be successful and now I'm being pressed by a client on a formal opinion. I figure having it based on experience from others on a similar product is worth hearing about. Specifically curious about how you addressed call quality issues and ensured bandwidth and uplink were sufficient. As much detail as you're willing would be great, on or off list. Thanks, Darren
Since I only use Asterisk, would my experience be useful with 100-seat sites? On Wed, Feb 1, 2012 at 9:25 AM, Darren Schreiber <d at d-man.org> wrote:
Hi folks, I'd love to hear some stories (good or bad) of hosted PBX VoIP installs on 100+ seat sites (single site). Specifically if you've done this with Broadsoft or another solidified switch. I have mixed opinions on how this type of scenario can be successful and now I'm being pressed by a client on a formal opinion. I figure having it based on experience from others on a similar product is worth hearing about.
Specifically curious about how you addressed call quality issues and ensured bandwidth and uplink were sufficient.
As much detail as you're willing would be great, on or off list.
Thanks, Darren
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
-- Carlos Alvarez TelEvolve 602-889-3003
*I have done and my company has done 100 seat installs using Hosted phones with Broadsoft. Let me know what type of questions you have* * * *Thanks-* Zak *From:* voiceops-bounces at voiceops.org [mailto:voiceops-bounces at voiceops.org] *On Behalf Of *Carlos Alvarez *Sent:* Wednesday, February 01, 2012 9:41 AM *To:* voiceops at voiceops.org *Subject:* Re: [VoiceOps] Experiences with VoIP and 100+ seat sites Since I only use Asterisk, would my experience be useful with 100-seat sites? On Wed, Feb 1, 2012 at 9:25 AM, Darren Schreiber <d at d-man.org> wrote: Hi folks, I'd love to hear some stories (good or bad) of hosted PBX VoIP installs on 100+ seat sites (single site). Specifically if you've done this with Broadsoft or another solidified switch. I have mixed opinions on how this type of scenario can be successful and now I'm being pressed by a client on a formal opinion. I figure having it based on experience from others on a similar product is worth hearing about. Specifically curious about how you addressed call quality issues and ensured bandwidth and uplink were sufficient. As much detail as you're willing would be great, on or off list. Thanks, Darren _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops -- Carlos Alvarez TelEvolve 602-889-3003
On Wed, Feb 1, 2012 at 9:25 AM, Darren Schreiber <d at d-man.org> wrote:
Hi folks, I'd love to hear some stories (good or bad) of hosted PBX VoIP installs on 100+ seat sites (single site). Specifically if you've done this with Broadsoft or another solidified switch. I have mixed opinions on how this type of scenario can be successful and now I'm being pressed by a client on a formal opinion. I figure having it based on experience from others on a similar product is worth hearing about.
I've done a couple in this range. I don't think 100 is a lot, and I don't think it's much of a challenge. The "things to do" are pretty straightforward and there are lots of sources for best practices. For example, if you do separate cabling and switches for the phones, then you can simply ignore LAN QoS (CoS) and you'll have a nice separate network for troubleshooting purposes. This is what I do.
Specifically curious about how you addressed call quality issues and ensured bandwidth and uplink were sufficient.
You need a circuit that either has QoS from the ISP, is direct to the VoIP carrier, or a separate circuit with guaranteed bandwidth to the carrier(s) if you want a guarantee. That said, I've done 70+ concurrent calls over wild internet without major issues by simply selecting an ISP with excellent upstream connectivity. I don't know your level of understanding of internet routing, so it's hard to know where to go with those details. When we deploy to our larger local customers (which to us is 25 or more), we use a local ISP who has a city-wide WiMAX network. They deliver a VLAN from the customer site directly to our presence in the same facilities they are in. This takes the guessing out of it. You can do the same with most any carrier with the right engineering. The first question would be who is the SIP provider? This isn't black magic. 100 phones really is pretty simple and the ability to give them very high levels of service is well set. -- Carlos Alvarez TelEvolve 602-889-3003
On 01/02/12 16:25, Darren Schreiber wrote:
Hi folks, I'd love to hear some stories (good or bad) of hosted PBX VoIP installs on 100+ seat sites (single site). Specifically if you've done this with Broadsoft or another solidified switch. I have mixed opinions on how this type of scenario can be successful and now I'm being pressed by a client on a formal opinion. I figure having it based on experience from others on a similar product is worth hearing about.
Specifically curious about how you addressed call quality issues and ensured bandwidth and uplink were sufficient.
I help look after a site with 140 ish SIP phones on the same site. Works very well. Phones all on www.voipfone.co.uk The sums for network links are easy. Bandwidth per call * calls. Then spec the right circuit. Things people forget: 0) To plan the user experience. You can just slap a new phone on a desk and expect people to use it. You need to do training for end users. Even if you show them how to make a call. Do not tell the end users it is voip. Just `A new phone system`. 1) IP header allowance in bandwidth sums. RTP, UDP, IP, then Ethernet or ATM depending on the circuit. 2) Consider Packets per second through the router/firewall. VoIP is lots of small packets. Many firewalls have a low session count limit. a 25$ router is not going to cope with all those phones. 3) Just buy enough bandwidth 4) Protect the ethernet infrastructure. You want to be using managed switches which can - drop rogue DHCP servers - drop a port if somebody pretends to be the default gateway - cope when somebody makes a loop in the network or attaches a device which floods then lan with broadcasts 5) Put the Router in a HA setup with 2 routers and 2 WAN connections. With VRRP or CARP or similar. Or agree with the customer in writing that if the WAN fails, the phones fail. - or sell divert to mobile as part of the solution. 6) Manage all the phones on a configuration server. Lock all the phones down so people can't mess with them. 7) Don't use wifi to connect phones. 8) Avoid Active SIP ALGs. You don't want anything modding SIP packets on the router. Passive devices which detect SIP to do traffic prioritization are ok. Anything which modifies packets is bad. - Sometimes the SIP aware routers get hacked. 9) Don't use low rate codecs. 711 all the way. Or 722. 10) Primary and failover DHCP and DNS servers onsite. Tim
On Wed, Feb 1, 2012 at 10:08 AM, Tim Bray <tim at kooky.org> wrote:
9) Don't use low rate codecs. 711 all the way. Or 722.
I completely disagree. Most of our customers are on g729 and nobody was able to hear the difference when we tested that versus 711. -- Carlos Alvarez TelEvolve 602-889-3003
On 02/01/2012 12:33 PM, Carlos Alvarez wrote:
I completely disagree. Most of our customers are on g729 and nobody was able to hear the difference when we tested that versus 711.
It comes across in hold music, and other things that fill up more of the acoustic range of the good ol' 3.1 KHz bearer spectrum. But yeah, on WiMax, that would make sense. -- Alex Balashov - Principal Evariste Systems LLC 260 Peachtree Street NW Suite 2200 Atlanta, GA 30303 Tel: +1-678-954-0670 Fax: +1-404-961-1892 Web: http://www.evaristesys.com/, http://www.alexbalashov.com/
I agree with Alex. g729 has better packet loss concealment and works better over certain types of broadband connections where there may be high latency and jitter. We were an interconnect for 30 years prior to moving to a Broadworks platform so we have a lot of experience in recreating the "key system" experience to mirror what our clients had for many years.... 1. Do a thorough review of their LAN and WAN infrastructure and perform a "network readiness" assessment...(we use AppNeta PathView)and also put a simulated VOIP load on the network and run during production time to see how the network behaves with the additional overhead..(we use AppNet AppView Voice for this) 2. Remediate any bottlenecks and network issues that you discover including re-terminating jacks and patch panels if needed or relocating CAT5 cables that may be run to close to fluorescent lights and other interference inducers. 3. either test or replace all patch cords to verify end to end performance. 4. set up VLANs to separate voice, video and data traffic so it is easier to isolate fault conditions and do proper traffic shaping on trunks 5. use a good ALG router at the edge such as Edgewater Network's Edgemarc or Adtran's netvanta series. (this allows for local tftp file storage for firmware as well as things like WAN link redundancy, traffic shaping and MOS monitoring and reporting) These also integrate into central monitoring systems that can alert on predefined criteria such as packet loss, jitter, and other undesirable conditions that can affect call quality and feature availability. 6. use SIP phones such as Aastra or Polycom that have the ability to set up multiple line appearances and LAN based paging groups. (you can also use a local appliance that has a secondary registration for paging groups and connectivity to local paging amplifiers. (we have our own patent pending product based on Asterisk for this that includes info mailboxes, conference bridges, cascade paging and sms notification and xml app hosting which we integrate with Broadworks) 7. get very detailed site information and current system setup including any "special" items, notifications, vm needs, operator functions, hunt groups workarounds and connectivity that may need to be replicated in the new system. Have your engineering department sign off on the final design before implementation so you can address any gotchas before they blow up during the install. 8. Pilot the new system along side the existing system so the users can pre train and experiment prior to the live cut date. This way they will report any nuances that may be present so you can address them prior to go live thus avoiding fire drills. We usually run the pilot for 1-2 weeks 9. build monitoring for site to proactively keep things in good health 10. Document, Document, Document -----Original Message----- From: voiceops-bounces at voiceops.org [mailto:voiceops-bounces at voiceops.org] On Behalf Of Alex Balashov Sent: Wednesday, February 01, 2012 12:39 PM To: voiceops at voiceops.org Subject: Re: [VoiceOps] Experiences with VoIP and 100+ seat sites On 02/01/2012 12:33 PM, Carlos Alvarez wrote:
I completely disagree. Most of our customers are on g729 and nobody was able to hear the difference when we tested that versus 711.
It comes across in hold music, and other things that fill up more of the acoustic range of the good ol' 3.1 KHz bearer spectrum. But yeah, on WiMax, that would make sense. -- Alex Balashov - Principal Evariste Systems LLC 260 Peachtree Street NW Suite 2200 Atlanta, GA 30303 Tel: +1-678-954-0670 Fax: +1-404-961-1892 Web: http://www.evaristesys.com/, http://www.alexbalashov.com/ _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
Your customers must have bad hearing. There's a major difference. On Feb 1, 2012, at 11:39, Alex Balashov <abalashov at evaristesys.com> wrote:
On 02/01/2012 12:33 PM, Carlos Alvarez wrote:
I completely disagree. Most of our customers are on g729 and nobody was able to hear the difference when we tested that versus 711.
It comes across in hold music, and other things that fill up more of the acoustic range of the good ol' 3.1 KHz bearer spectrum.
But yeah, on WiMax, that would make sense.
-- Alex Balashov - Principal Evariste Systems LLC 260 Peachtree Street NW Suite 2200 Atlanta, GA 30303 Tel: +1-678-954-0670 Fax: +1-404-961-1892 Web: http://www.evaristesys.com/, http://www.alexbalashov.com/ _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
Yea, most of our customers can tell the difference between g729 and g711. When it comes to business calls (as opposed to residential), clarity of the call is even more important! Some customers demand that we move back to G711 and others just put up with it because of the increased call capacity that goes along with g729. We've also found that the quality of the handset sold to the customer can make a huge difference in their acceptance of g711. Some handsets just excaserbate the poorer quality of g729 calls because the speaker/mic hardware isn't all that clear in the first place. ________________________________________ From: voiceops-bounces at voiceops.org [voiceops-bounces at voiceops.org] On Behalf Of Anthony Orlando [avorlando at yahoo.com] Sent: Saturday, February 04, 2012 12:56 PM To: Alex Balashov Cc: voiceops at voiceops.org Subject: Re: [VoiceOps] Experiences with VoIP and 100+ seat sites Your customers must have bad hearing. There's a major difference. On Feb 1, 2012, at 11:39, Alex Balashov <abalashov at evaristesys.com> wrote:
On 02/01/2012 12:33 PM, Carlos Alvarez wrote:
I completely disagree. Most of our customers are on g729 and nobody was able to hear the difference when we tested that versus 711.
It comes across in hold music, and other things that fill up more of the acoustic range of the good ol' 3.1 KHz bearer spectrum.
But yeah, on WiMax, that would make sense.
-- Alex Balashov - Principal Evariste Systems LLC 260 Peachtree Street NW Suite 2200 Atlanta, GA 30303 Tel: +1-678-954-0670 Fax: +1-404-961-1892 Web: http://www.evaristesys.com/, http://www.alexbalashov.com/ _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.
On 1 Feb 2012, at 17:33, Carlos Alvarez wrote:
On Wed, Feb 1, 2012 at 10:08 AM, Tim Bray <tim at kooky.org> wrote:
9) Don't use low rate codecs. 711 all the way. Or 722.
I completely disagree. Most of our customers are on g729 and nobody was able to hear the difference when we tested that versus 711.
I think it is a bit different in Europe. People are used to really good phone lines which are always g711a all the way. All trunks have been digital for 30 years or more. People can't put their finger on the difference on 729, but they know it is there. In the UK many telemarketing calls are delivered from India to the UK using G.729 and G.723. There is a badge on the call (the sound of background call centre noise encoded) which makes you think you are about to have your life wasted. My belief is that g729 calls last longer than g711 calls. I also believe that people ask for more clarifications. They say 'Can you repeat that?', 'Can you spell that please?', 'Sorry I missed that.' more often. It would be an interesting thing to pull the data on or some some proper research. In the early days of Skype I used to watch people come off a call and say 'That call was really hard work'. I think Skype have improved their systems since though. I'm just into things sounding good. There was a time I could answer a call and tell you which VoIP provider the call came from. Some had a distinctive sound. Tim
On 2/1/12 8:25 AM, Darren Schreiber wrote:
Hi folks, I'd love to hear some stories (good or bad) of hosted PBX VoIP installs on 100+ seat sites (single site). Specifically if you've done this with Broadsoft or another solidified switch. I have mixed opinions on how this type of scenario can be successful and now I'm being pressed by a client on a formal opinion. I figure having it based on experience from others on a similar product is worth hearing about.
We have done several with Broadsoft.
Specifically curious about how you addressed call quality issues and ensured bandwidth and uplink were sufficient.
It's pretty much the same formula as with smaller sites. As a rule, an office with lots of phones also has need for lots of data bandwidth. Some times a larger site is easier. Customers that have an office with 100+ employees understand the need for redundancy and high availability more than those with smaller offices. This allows us to provide two diverse circuits for failover and run the VoIP over one and data over the other. Only in the event of a failure of either link does QoS come into play. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
I second most everything that Tim and Carlos have said. The PBX side is rarely what turns a large site deployment sour - it usually comes down the LAN/WAN and project management. To that end, I'd add that a survey of the physical LAN should be conducted to find any dumb switches lying under desks or areas that are not yet occupied that could be hastily connected to the LAN in a suboptimal fashion. Another issue that could pop up (if the phones' TFTP server is not on the LAN) is bandwidth saturation on your upstream link when upgrading firmware. TFTP does not handle missing/out of order packets well and phones can enter a continuous download/reboot loop. Avoid the temptation to turn up the entire VoIP VLAN at once, as 100 simultaneous firmware downloads will crush a low (<= 3Mb) link. Please excuse any typos, spelling errors etc - mobile. -Sean
participants (10)
-
abalashov＠evaristesys.com -
avorlando＠yahoo.com -
carlos＠televolve.com -
d＠d-man.org -
Eric.Jastak＠adp.com -
jay＠west.net -
sean.grossman＠gmail.com -
tim＠kooky.org -
twolf＠unifiedtechnologies.com -
zak＠simplesignal.com