Dear Valued Customers,
We regret to inform you that our US Linux Server (Earth) has caused various problems to you today, 28th February 2007.
For your information, at around 4.00am (GMT+8) this morning until around 8.00am (GMT+8), our Apache server (HTTP) appeared up and down continuously. We found out that a client has abused our server resource, whose script generated a lot of PHP processes until our Apache server crashed. When the Apache server was crashed, it was rebooted automatically after 3 minutes by our custom monitoring script installed on the server. However, that particular client’s script again generated the same amount of PHP processes to crash our Apache server, and Apache server was once again rebooted by our script. This whole process lasted until our server was rebooted manually at around 8.00am (GMT+8), and the client has now been temporarily suspended for future investigation.
At the same time, our mail server failed to deliver incoming emails to our clients’ mailboxes on the server. All the incoming emails were queued in our incoming mail queue until just now, which is around 8.30pm (GMT+8). Outgoing email however is fine. We did not realize the problem as mail server appeared up and running until I myself discovered that I have not been receiving email for the whole day, and thus I checked the mail server log, realizing that I actually have a lot of emails in the mail queue but at the same time my webmail received nothing. I then immediately informed Tris to look into the problem, and I am relieved to tell you that the mail problem has been fixed at around 8.30pm (GMT+8) and ALL your email for today are in your respective mailboxes now. This problem could be linked to the server abuse by client. We have reinstalled the mail server, Exim and that fixed the mail server issue.
The above service port downtimes (HTTPd & Mail) were not reported by Alertra & Hyperspin as both Alertra & Hyperspin monitor our server by checking the ping response. During the downtimes, our server responded to ping requests as it was still running. Hence, Alertra & Hyperspin assumes that the server was running without problem at all.
My system admins are still working closely in diagnosing the server and monitoring its resource usage. If you are experiencing any problems with the server, please do not hesitate to submit a ticket to firstname.lastname@example.org or email to me at email@example.com. We appreciate your cooperation!
Meanwhile, I would like to apologize for any inconveniences caused.
KKHost.Net Founder & Chief Executive Officer
KK Host Network (1458953-U)