Yes, you read it right. I did say NTP, Chrony and Russian brides in the same breath. Why? Here’s a short tale on how an oracle database server ended up using “russianbridesnet” to tell time.
What is NTP, clock synchronization service & why do we need it?
Clock synchronization is a service that helps a device on a network show the correct time. Some organizations require it by law, and even if not, accurate timekeeping is fundamental for managing & securing servers, debugging, forensic analysis of security incidents or determining when a problem occurred on your device.
It is the method, to ensure that the time on your server is accurate.
To implement clock synchronization, Network Time Protocol (NTP) is the internet standard, and NTPD or chrony are implementations of this protocol. i.e. packages that make use of the NTP protocol to maintain time.
In the newer versions of Linux (Linux 7 & Linux 8), chrony has replaced the traditional ntpd service, and it comprises two programs chronyd (daemon) and chronyc (a command-line utility to manage chronyd)
Back to our story
As a part of Oracle Grid Infrastructure19c Installation on a Database Server (Oracle Enterprise Linux 7) the status of time synchronization daemon was reviewed, and it turned out that the server was using russianbrides.net as an NTP source to maintain time. This left us wildly amused & bewildered.
Checking the status of chrony client on our server & the NTP Sources being used for time synchronization, this is what we could see
## Checking chrony status
[root@server01 ~]# systemctl status chronyd
● chronyd.service - NTP client/server
Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-06-04 15:33:53 EDT; 5 days ago
Docs: man:chronyd(8)
man:chrony.conf(5)
Main PID: 1492 (chronyd)
CGroup: /system.slice/chronyd.service
└─1492 /usr/sbin/chronyd
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 162.159.200.123 offline
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 95.216.218.15 offline
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 62.149.0.30 offline
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 50.205.57.38 offline
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Can't synchronise: no selectable sources
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 50.205.57.38 online
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 162.159.200.123 online
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 95.216.218.15 online
Jun 09 13:35:12 server01.acme.com chronyd[1492]: Source 62.149.0.30 online
Jun 09 13:41:49 server01.acme.com chronyd[1492]: Selected source 50.205.57.38
## Checking the source
[root@server01 ~]# chronyc sources
210 Number of sources = 4
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^- support.russianbridesnet> 2 10 377 618 -1091us[-1125us] +/- 57ms
^* 50-205-57-38-static.hfc.> 1 10 377 276 -12us[ -49us] +/- 4817us
^- time.cloudflare.com 3 9 377 510 +1630us[+1594us] +/- 11ms
^- ntp.time.in.ua 1 10 377 602 +329us[ +294us] +/- 55ms
## Tells us the source was online
[root@server01 ~]# chronyc activity
200 OK
4 sources online
0 sources offline
0 sources doing burst (return to online)
0 sources doing burst (return to offline)
0 sources with unknown address
## Tells us that the source "support.russianbridesnet" was last verified 57ms
[root@server01 ~]# chronyc -m tracking sources
Reference ID : ******************
Stratum : 2
Ref time (UTC) : Thu Jun 10 17:32:12 2021
System time : 0.000018746 seconds slow of NTP time
Last offset : -0.000036971 seconds
RMS offset : 0.000058610 seconds
Frequency : 112.123 ppm slow
Residual freq : -0.001 ppm
Skew : 0.082 ppm
Root delay : 0.009632965 seconds
Root dispersion : 0.000346089 seconds
Update interval : 1034.1 seconds
Leap status : Normal
210 Number of sources = 4
MS Name/IP address Stratum Poll Reach LastRx Last sample
======================================================================
^- support.russianbridesnet> 2 10 377 633 -1091us[-1125us] +/- 57ms
^* 50-205-57-38-static.hfc.> 1 10 377 291 -12us[ -49us] +/- 4817us
^- time.cloudflare.com 3 10 377 4 +1614us[+1614us] +/- 11ms
^- ntp.time.in.ua 1 10 377 617 +329us[ +294us] +/- 55ms
As you can see the chrony source “support.russianbridesnet” was online and in active use. So how did this happen? Digging a bit we realized that chrony was configured to use the NTP pool project as the source server in the configuration file.
The NTP Pool Project is a large cluster of timeservers, and any organization across the internet can register their servers in this pool. “support.russianbrides.net” is one of the many such timeservers in the pool.
Usage of NTP Pool is the default configuration for most of the major Linux distributions including Oracle Enterprise Linux 7, and from thousands of servers in this pool, a few are randomly assigned to serve you.
Here is the default chrony configuration file, where you can see 4 clusters of the NTP Pool project listed.
## Default chrony configuration file
[root@server01 ~]# cat /etc/chrony.conf
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
server 2.pool.ntp.org iburst
server 3.pool.ntp.org iburst
We soon realized our oversight of modifying the NTPD configuration instead of chronyd during our build.
The fix was simple, we updated the chrony configuration to use the internal time servers of our organization and the chrony service was restarted. So we switched from using an external time source to an internal one used by our organization.
## Update the Chrony configuration file, to use the internal time servers
[root@server01 ~]# grep server /etc/chrony.conf | grep -v ^#
server time1.acme.com
server time2.acme.com
## Restart the chrony daemon
[root@server01 ~]# systemctl stop chronyd
[root@server01 ~]# systemctl start chronyd
## Review the chrony sources
[root@server01 ~]# chronyc sources
210 Number of sources = 2
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* time1.acme.com 1 10 377 764 -26us[ -35us] +/- 1164us
^+ time2.acme.com 1 10 377 191 -35us[ -35us] +/- 1106us
Lesson learnt
Reviewing the chrony configuration is something that should be done when transitioning to Linux 7+ OS. Another similar thing to note would be the tempfiles.d configuration for Oracle Database Servers as discussed here
Now, if you are wondering whether there any risks associated with using external sources for NTP, then the answer is – yes it appears so. NTP servers are open to misuse and potential denial-of-service (DDoS) and man-in-the-middle (MITM) attacks. As a best practice, it is recommended to configure the Internal NTP service for your organization.