Dear customers,
Yesterday, on 19.11.2014 our network suffered partial outage caused by a problem with DNS (Domain Name Service).
The root cause of the problem was that ITE, one of our main suppliers for .gr domains due to a misconfiguration on their database, lost the data of our glue entries for some of our core DNS Servers.
Along with some issues that Google DNS servers faced yesterday with their caching and the fact that we had no issues on our servers, this made it hard to pinpoint the problem.
Our 4 root servers for .gr were fully operational at all times. The fact though that ITE did not have the glue records containing their IP addresses, no traffic was being sent to them.
Most of our customers with .gr domains had name servers under the @my-dns.gr & @ans.gr domain names which were affected.
We rapidly switched most domains to use name servers under the swissns.ch domain, added the glue records that ITE lost.
In order to prevent this from happening again, we added 2 more DNS servers on different domain and updated all our records.
The problem started at around 10.00 in the morning and was completely resolved at 16.00. We had performed all changes and corrections by 12.00 and tried to force ITE to update it’s records sooner but eventually they restarted their root zones at 16.00.
We apologise for the inconvenience this issue caused which even if was beyond our management affected several of our customers.
Kind regards,
swissns GmbH Team
