Do you do anything special for the DST changes? In other words, do you try to adjust for the fact that the 0200 hour occurs twice in the fall and that the 0200 hour is completely missing in the spring on reports and such? Thanks, David
GMT for the win. Colin On Nov 4, 2009, at 2:48 PM, David Hiers wrote:
Do you do anything special for the DST changes? In other words, do you try to adjust for the fact that the 0200 hour occurs twice in the fall and that the 0200 hour is completely missing in the spring on reports and such?
Thanks,
David _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
On Wed, 4 Nov 2009, Colin wrote:
GMT for the win.
Agreed, though we use UTC. While GMT and UTC are mostly the same, UTC is based on atomic time and includes leap seconds that are added occasionally to account for the irregularity in the earth and sun's movements. We decided UTC was a more accurate and "official" representation of time. GMT is based on the hypothetical average day at Greenwich, which is much less scientifically accurate. Everything in our system is stored as UTC, and we convert on a per-user basis when displaying call logs and the like. That way we avoid any issues with DST. I would make all of your CPE devices on UTC as well. It avoids confusion about what time zone any date/time in your system is on the back end. Build the conversion into your front-end systems and reporting on a per-customer basis. You also avoid calls that were really 4 minutes, but get billed as -56 minutes or 64 minutes for calls during the change from/to DST. In discussing time, one of my providers avoids syncing their servers to the correct time on an ongoing basis (i.e. with ntpd), with the argument that if they sync during calls, billing would be wrong. Some of there servers therefore skew a LOT, and sometimes their call servers can be off the real time by _HOURS_ (I suspect this is due to running Asterisk on VMs which are notoriously bad at keeping time). I've griped about it, but they say they only sync the time when there are no calls at midnight, which never happens. Has anyone else been frustrated by the lack of punctuality/analness when it comes to making sure clocks are accurate when it comes to call records? Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
On Thu, Nov 5, 2009 at 1:03 PM, Peter Beckman <beckman at angryox.com> wrote:
?In discussing time, one of my providers avoids syncing their servers to ?the correct time on an ongoing basis (i.e. with ntpd), with the argument ?that if they sync during calls, billing would be wrong. ?Some of there ?servers therefore skew a LOT, and sometimes their call servers can be off ?the real time by _HOURS_ (I suspect this is due to running Asterisk on VMs ?which are notoriously bad at keeping time). ?I've griped about it, but ?they say they only sync the time when there are no calls at midnight, ?which never happens.
I'm sure this isn't news to you, but... By default, ntpd will not snap to a new time, but rather slew slowly towards the correct time. How slowly? man 8 ntpd: "about 2,000 s for each second the clock is outside the acceptable range." -- Dan Young <dyoung at mesd.k12.or.us> Multnomah ESD - Technology Services 503-257-1562
On Thu, 5 Nov 2009, Dan Young wrote:
On Thu, Nov 5, 2009 at 1:03 PM, Peter Beckman <beckman at angryox.com> wrote:
?In discussing time, one of my providers avoids syncing their servers to ?the correct time on an ongoing basis (i.e. with ntpd), with the argument ?that if they sync during calls, billing would be wrong. ?Some of there ?servers therefore skew a LOT, and sometimes their call servers can be off ?the real time by _HOURS_ (I suspect this is due to running Asterisk on VMs ?which are notoriously bad at keeping time). ?I've griped about it, but ?they say they only sync the time when there are no calls at midnight, ?which never happens.
I'm sure this isn't news to you, but... By default, ntpd will not snap to a new time, but rather slew slowly towards the correct time.
How slowly? man 8 ntpd: "about 2,000 s for each second the clock is outside the acceptable range."
Oh, _I_ know that. I've tried to explain to them why this would be a good thing. Then again, I'm not sure how fast ntp can react to bad time skew due to Virtual Machine kernel timing issues, might take a lot of trial and error to get it right. --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
I thought that ntp doesn't know or care about DST. It works in UTC, and the OS is responsible for any DST adjustment it wants to make. If you're an hour (3600 seconds) off in NTP, and it takes 2000 seconds to adjust for every second you are off, it'll take 7200000 seconds to get back on track, which is about 83 days. Seems kinda long... David On Thu, Nov 5, 2009 at 1:55 PM, Peter Beckman <beckman at angryox.com> wrote:
On Thu, 5 Nov 2009, Dan Young wrote:
On Thu, Nov 5, 2009 at 1:03 PM, Peter Beckman <beckman at angryox.com> wrote:
?In discussing time, one of my providers avoids syncing their servers to ?the correct time on an ongoing basis (i.e. with ntpd), with the argument ?that if they sync during calls, billing would be wrong. ?Some of there ?servers therefore skew a LOT, and sometimes their call servers can be off ?the real time by _HOURS_ (I suspect this is due to running Asterisk on VMs ?which are notoriously bad at keeping time). ?I've griped about it, but ?they say they only sync the time when there are no calls at midnight, ?which never happens.
I'm sure this isn't news to you, but... By default, ntpd will not snap to a new time, but rather slew slowly towards the correct time.
How slowly? man 8 ntpd: "about 2,000 s for each second the clock is outside the acceptable range."
?Oh, _I_ know that. ?I've tried to explain to them why this would be a good ?thing. ?Then again, I'm not sure how fast ntp can react to bad time skew ?due to Virtual Machine kernel timing issues, might take a lot of trial and ?error to get it right.
--------------------------------------------------------------------------- Peter Beckman ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Internet Guy beckman at angryox.com ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? http://www.angryox.com/ --------------------------------------------------------------------------- _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
On Thu, Nov 5, 2009 at 3:16 PM, David Hiers <hiersd at gmail.com> wrote:
I thought that ntp doesn't know or care about DST. ?It works in UTC, and the OS is responsible for any DST adjustment it wants to make.
If you're an hour (3600 seconds) off in NTP, and it takes 2000 seconds to adjust for every second you are off, it'll take 7200000 seconds to get back on track, which is about 83 days. ?Seems kinda long...
On most NTP implementations that I have worked with, the service will not make jumps above a certain threshold (like more than 5 minutes off). It is the responsibility of the operator to get the clock in sync to start with, then NTP will adjust for the natural drift. Thanks for the correction Peter, I should have said UTC, not GMT. All of our hardware clocks use UTC, and then the OS/reporting/etc adjusts to make the user feel warm and fuzzy. -Jonathan
The NTP protocol uses UTC as reference time; UTC is a time zone that has no DST changes. If the real-time clock on a device is kept in a local time, the NTP implementation on the device has to translate to UTC and back. There should be no difference in the calculated UTC time before, or after time change caused by DST. On Thu, Nov 5, 2009 at 4:34 PM, Jonathan Thurman <jonathan at thurmantech.com> wrote:
On Thu, Nov 5, 2009 at 3:16 PM, David Hiers <hiersd at gmail.com> wrote: [...]?It is the responsibility of the operator to get the clock in sync to start with, then NTP will adjust for the natural drift.
Adjusting clocks by large amounts by default can be dangerous.. there might be an accident on another NTP server. An evil hacker who can forge NTP packets may have cracked an expired Kerberos TGT, or may desire that a server execute a certain cron job over and over again.... If the time is off by less than 128ms, NTP will slew. If it's off by more than that NTP will often step (make a large error correction) instead. But there is a "panic" limit to the error corrections. If the clock is too far off, NTP will simply exit. You can change the number of seconds, or set it to be unlimited [at own risk]: tinker panic 0 ... In certain *ix distros, you also get stepping at NTP startup according to entries made in an /etc/ntp/step-tickers -- -J
My system clocks are all within 1 second of time.gov just days after the DST change. We made the jump all at once, got double the number of hourly CDR files for an hour, and will get no CDR files for an hour in the spring. Don't think that we did anything special... What do your clocks look like today? We show start time and duration for customer-facing reports. David On Thu, Nov 5, 2009 at 9:28 PM, James Hess <mysidia at gmail.com> wrote:
The NTP protocol uses UTC as reference time; ?UTC ?is a time zone that has no DST changes. ? ? ?If the real-time clock on a device is kept in a local time, the NTP implementation on the device has to translate to UTC ?and back. There should be no difference in the calculated UTC time before, or after time change ?caused by DST.
On Thu, Nov 5, 2009 at 4:34 PM, Jonathan Thurman <jonathan at thurmantech.com> wrote:
On Thu, Nov 5, 2009 at 3:16 PM, David Hiers <hiersd at gmail.com> wrote: [...]?It is the responsibility of the operator to get the clock in sync to start with, then NTP will adjust for the natural drift.
Adjusting clocks by large amounts by default can be dangerous.. there might be an accident on another NTP server. ? ?An evil hacker who can forge NTP packets may have cracked an expired ?Kerberos TGT, ? or ?may desire that a server execute a certain cron job over and over again....
If the time is off by less than 128ms, ?NTP will slew. ?If it's off by more than that NTP will often step ?(make a large error correction) instead. But there is a "panic" ?limit ?to the error corrections. If the clock is too far off, ?NTP will simply exit. ? You can change the number of seconds, or set it to be unlimited ?[at own risk]:
tinker panic 0 ...
In certain *ix distros, you also get stepping at NTP startup according to entries made in an ?/etc/ntp/step-tickers
-- -J
Why aren'y you recording all cdr's in GMT and adjusting call log displays as needed based on where the customer wants to see the call log in? Colin On Nov 6, 2009, at 11:06 AM, David Hiers wrote:
My system clocks are all within 1 second of time.gov just days after the DST change. We made the jump all at once, got double the number of hourly CDR files for an hour, and will get no CDR files for an hour in the spring. Don't think that we did anything special...
What do your clocks look like today?
We show start time and duration for customer-facing reports.
David
On Thu, Nov 5, 2009 at 9:28 PM, James Hess <mysidia at gmail.com> wrote:
The NTP protocol uses UTC as reference time; UTC is a time zone that has no DST changes. If the real-time clock on a device is kept in a local time, the NTP implementation on the device has to translate to UTC and back. There should be no difference in the calculated UTC time before, or after time change caused by DST.
On Thu, Nov 5, 2009 at 4:34 PM, Jonathan Thurman <jonathan at thurmantech.com> wrote:
On Thu, Nov 5, 2009 at 3:16 PM, David Hiers <hiersd at gmail.com> wrote: [...] It is the responsibility of the operator to get the clock in sync to start with, then NTP will adjust for the natural drift.
Adjusting clocks by large amounts by default can be dangerous.. there might be an accident on another NTP server. An evil hacker who can forge NTP packets may have cracked an expired Kerberos TGT, or may desire that a server execute a certain cron job over and over again....
If the time is off by less than 128ms, NTP will slew. If it's off by more than that NTP will often step (make a large error correction) instead. But there is a "panic" limit to the error corrections. If the clock is too far off, NTP will simply exit. You can change the number of seconds, or set it to be unlimited [at own risk]:
tinker panic 0 ...
In certain *ix distros, you also get stepping at NTP startup according to entries made in an /etc/ntp/step-tickers
-- -J
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
I may not have been clear before, but that's pretty much what we do. David On Fri, Nov 6, 2009 at 8:12 AM, Colin <zavoid at gmail.com> wrote:
Why aren'y you recording all cdr's in GMT and adjusting call log displays as needed based on where the customer wants to see the call log in?
Colin
On Nov 6, 2009, at 11:06 AM, David Hiers wrote:
My system clocks are all within 1 second of time.gov just days after the DST change. ?We made the jump all at once, got double the number of hourly CDR files for an hour, and will get no CDR files for an hour in the spring. ?Don't think that we did anything special...
What do your clocks look like today?
We show start time and duration for customer-facing reports.
David
On Thu, Nov 5, 2009 at 9:28 PM, James Hess <mysidia at gmail.com> wrote:
The NTP protocol uses UTC as reference time; ?UTC ?is a time zone that has no DST changes. ? ? ?If the real-time clock on a device is kept in a local time, the NTP implementation on the device has to translate to UTC ?and back. There should be no difference in the calculated UTC time before, or after time change ?caused by DST.
On Thu, Nov 5, 2009 at 4:34 PM, Jonathan Thurman <jonathan at thurmantech.com> wrote:
On Thu, Nov 5, 2009 at 3:16 PM, David Hiers <hiersd at gmail.com> wrote: [...] It is the responsibility of the operator to get the clock in sync to start with, then NTP will adjust for the natural drift.
Adjusting clocks by large amounts by default can be dangerous.. there might be an accident on another NTP server. ? ?An evil hacker who can forge NTP packets may have cracked an expired ?Kerberos TGT, ? or ?may desire that a server execute a certain cron job over and over again....
If the time is off by less than 128ms, ?NTP will slew. ?If it's off by more than that NTP will often step ?(make a large error correction) instead. But there is a "panic" ?limit ?to the error corrections. If the clock is too far off, ?NTP will simply exit. ? You can change the number of seconds, or set it to be unlimited ?[at own risk]:
tinker panic 0 ...
In certain *ix distros, you also get stepping at NTP startup according to entries made in an ?/etc/ntp/step-tickers
-- -J
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
On Fri, 6 Nov 2009, David Hiers wrote:
I may not have been clear before, but that's pretty much what we do.
Except that you, according to your previous emails, are storing them in a time zone that has DST. Having double the (or no) records in a given hour, ever, is bad, bad, bad. How do you handle areas that don't do DST in the US? Arizona, Hawaii, Puerto Rico: They don't do DST. Maybe you don't have any customers there, but they are gonna be kind of annoyed if their call records are an hour off for 7 months of the year. Beckman --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
On Fri, 6 Nov 2009, David Hiers wrote:
My system clocks are all within 1 second of time.gov just days after the DST change. We made the jump all at once, got double the number of hourly CDR files for an hour, and will get no CDR files for an hour in the spring. Don't think that we did anything special...
Except what happened to calls active during the change? Or did you just get lucky?
What do your clocks look like today?
Average -0.02 seconds off of UTC.
We show start time and duration for customer-facing reports.
And it makes it so easy to do such things as: MySQL: select convert_tz(starttime, 'UTC', 'America/New_York') from calls PHP: date_default_timezone_set('America/New_York'); echo date('r', $calls[0]['startdate']); // UTC from DB, now displayed as US/EST The nice thing about doing it in your code rather than your MySQL is that you can easily change the time zone for the entire site on a per-user basis just by storing their time zone in a user variable. This also means it is really easy for users to change the time/date if they are travelling. We have US customers who go to Europe and simply change the Time Zone for their account while in Europe so calls are shown in their local time, and since we use POSIX Time Zone names, and we keep our servers up to date, we don't have to know that where they are is 30 minutes off. If you are running a phone company, and you aren't storing your call records (or event records for that matter) in UTC, in my opinion you're doing it wrong. --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
On Thu, 5 Nov 2009, David Hiers wrote:
I thought that ntp doesn't know or care about DST. It works in UTC, and the OS is responsible for any DST adjustment it wants to make.
If you're an hour (3600 seconds) off in NTP, and it takes 2000 seconds to adjust for every second you are off, it'll take 7200000 seconds to get back on track, which is about 83 days. Seems kinda long...
That's the default. You don't want NTP to change a production server from 1pm to 2pm in one giant leap. That would screw a lot of things up, especially on a VoIP server that does billing! By spreading an hour change over 83 days, it makes sure you don't totally screw up your billing. Ideally, if it were off by an hour, you'd take it out of production, update the clock, then put it back in. If you couldn't, NTP's solution is a pretty good balance. Ideally, NTP keeps your clock in sync +/- 1 second at all times, and since the OS does the translation from UTC to your server time zone, there will never be a time that it is an hour off, unless all of your time servers go down (or your network does). If we all worked on UTC and all had all of our clocks synced, we would all have a LOT easier time correlating CDRs between ourselves and our providers. That brings up another question -- do you display the start time or end time of the call? I display start time, but I'm curious if there is a standard among providers, both retail and wholesale. --------------------------------------------------------------------------- Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
Maybe they're relying on ntpdate in a cronjob at midnight. Dan Young wrote:
On Thu, Nov 5, 2009 at 1:03 PM, Peter Beckman <beckman at angryox.com> wrote:
In discussing time, one of my providers avoids syncing their servers to the correct time on an ongoing basis (i.e. with ntpd), with the argument that if they sync during calls, billing would be wrong. Some of there servers therefore skew a LOT, and sometimes their call servers can be off the real time by _HOURS_ (I suspect this is due to running Asterisk on VMs which are notoriously bad at keeping time). I've griped about it, but they say they only sync the time when there are no calls at midnight, which never happens.
I'm sure this isn't news to you, but... By default, ntpd will not snap to a new time, but rather slew slowly towards the correct time.
How slowly? man 8 ntpd: "about 2,000 s for each second the clock is outside the acceptable range."
-- Dan Young <dyoung at mesd.k12.or.us> Multnomah ESD - Technology Services 503-257-1562 _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
Another source of skew is multi-processor systems. I'm not sure if it applies to multi-core, but I would guess it does. Peter Beckman wrote:
On Wed, 4 Nov 2009, Colin wrote:
GMT for the win.
Agreed, though we use UTC. While GMT and UTC are mostly the same, UTC is based on atomic time and includes leap seconds that are added occasionally to account for the irregularity in the earth and sun's movements. We decided UTC was a more accurate and "official" representation of time. GMT is based on the hypothetical average day at Greenwich, which is much less scientifically accurate.
Everything in our system is stored as UTC, and we convert on a per-user basis when displaying call logs and the like. That way we avoid any issues with DST.
I would make all of your CPE devices on UTC as well. It avoids confusion about what time zone any date/time in your system is on the back end. Build the conversion into your front-end systems and reporting on a per-customer basis.
You also avoid calls that were really 4 minutes, but get billed as -56 minutes or 64 minutes for calls during the change from/to DST.
In discussing time, one of my providers avoids syncing their servers to the correct time on an ongoing basis (i.e. with ntpd), with the argument that if they sync during calls, billing would be wrong. Some of there servers therefore skew a LOT, and sometimes their call servers can be off the real time by _HOURS_ (I suspect this is due to running Asterisk on VMs which are notoriously bad at keeping time). I've griped about it, but they say they only sync the time when there are no calls at midnight, which never happens.
Has anyone else been frustrated by the lack of punctuality/analness when it comes to making sure clocks are accurate when it comes to call records?
Beckman ---------------------------------------------------------------------------
Peter Beckman Internet Guy beckman at angryox.com http://www.angryox.com/ ---------------------------------------------------------------------------
_______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
I log everything in GMT to make everything consistent. -Jonathan On Wed, Nov 4, 2009 at 12:48 PM, David Hiers <hiersd at gmail.com> wrote:
Do you do anything special for the DST changes? ?In other words, do you try to adjust for the fact that the 0200 hour occurs twice in the fall and that the 0200 hour is completely missing in the spring on reports and such?
Thanks,
David _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
I don't find its such a big deal on reports, more often the big deal is dealing with CPE. Since we are nation-wide it is a huge headache tracking what CPE needs to be configured for DST, which exists in areas without DST, and for devices that dont support DST, configuring our provisioning system to cover for the devices shortcomings, and keeping those settings in sync with our softswitch and voicemail systems. On Wed, 2009-11-04 at 11:48 -0800, David Hiers wrote:
Do you do anything special for the DST changes? In other words, do you try to adjust for the fact that the 0200 hour occurs twice in the fall and that the 0200 hour is completely missing in the spring on reports and such?
Thanks,
David _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
We've got customers all over the country that use a centralized app that shows their calls in near-realtime, centralized voicemail, etc, so there's ample opportunity to stumble on this kind of thing. Thanks for the input, everyone! David On Wed, Nov 4, 2009 at 12:05 PM, anorexicpoodle <anorexicpoodle at gmail.com> wrote:
I don't find its such a big deal on reports, more often the big deal is dealing with CPE. Since we are nation-wide it is a huge headache tracking what CPE needs to be configured for DST, which exists in areas without DST, and for devices that dont support DST, configuring our provisioning system to cover for the devices shortcomings, and keeping those settings in sync with our softswitch and voicemail systems.
On Wed, 2009-11-04 at 11:48 -0800, David Hiers wrote:
Do you do anything special for the DST changes? In other words, do you try to adjust for the fact that the 0200 hour occurs twice in the fall and that the 0200 hour is completely missing in the spring on reports and such?
Thanks,
David _______________________________________________ VoiceOps mailing list VoiceOps at voiceops.org https://puck.nether.net/mailman/listinfo/voiceops
participants (8)
-
anorexicpoodle＠gmail.com -
beckman＠angryox.com -
dyoung＠mesd.k12.or.us -
hiersd＠gmail.com -
jonathan＠thurmantech.com -
lriemer＠bestline.net -
mysidia＠gmail.com -
zavoid＠gmail.com