Server Mess

So around 1pm today, the NOC at Radiant Networks in Deep Creek went down. We still don’t know why. Around 1:45, two of our four servers came back online. There was no power outage, as the machines uptime couters are both up for months. So one of two things happened:

1. A denial of service attack or the worm struck. We don’t know. The mail server could have been effected, but that’s the only windows box that we use up there (it’s a shared box). The database box is bsd, so it shouldn’t have been affected at all. We’re not sure why that went unresponsive, but that is why various sites that we host are halfway online (e.g. meticulous.com, nancies.org, ontaponline.com, etc).

2. A line issue with AT&T (the fiber provider there) or the router. This is possible, since it’s only certain machines that haven’t come back.

Anyway, we’re trying to get in touch with one of the owners. One of them is in Vancouver on vacation, so it’s a bit difficult. So just an FYI, we’re working on it and as soon as we have an idea, we’ll let you know.

  1. roboboshi says:

    update: the tech is en route. supposedly a power grid failure and a landlord who refuses to install power backup. They're talking a move of the NOC. we'll see.

  2. roboboshi says:

    And we're back. 6pm EST for mail, 6:30 for db.

  3. Nda says:

    Just found in my email's sent item. Nicolas is the head of IT of UK office:=================================Re : Qubee Service in BangladeshDear Nicolas Ruelle:Greeting!I would be glad if you take some of your time and go tohrugh my email. This is regards to the internet service you are providing in Bangladesh. Since I failed to get the contact email address of head of IT or some upper management of Qubee Bangladesh I had no choice but take this opportunity to write you few words of mine.In order for me to work to my remote office in Canada I needed a true reliable consumer/business high speed internet which I did not find until I learned about Qubee's WiMax as I believe wireless is the reliable link among others when it comes to Bangladesh.Incident 1:Since I started the Qubee service on 2nd week of January with 512kbs it was satisfactory with the speed accept for the VPN issue that started to occur from the day one. I was able to dial in the VPN and connect but was not able to communicate with any of the servers within the LAN, I had done some checks with other ISPs and found no issues which satisfied me that it was either the WiMax Router/Modem or the ISP's router. I called the customer care number wanted to speak to someone from the technical support. Customer care asked me provide her with the problem and so did I. Unfortunately the individual had no clear picture about VPN but asked me to check download speed, radio's parameter, and all the basic things that not related to the issue I had specified. At the end I had to introduce myself as an advance user and a technical person and asked her if I could speak to someone from the technical support. She created ticket for it and informed me that someone will get back to me soon. 36 hours past no one contacted me and I had to call the customer care number to follow up with my ticket. Few hours later I had a phone call from some individual and I had to describe the issue and I was asked to check the speed and other non-related VPN checks. It was frustrating. I beged to speak to someone who is familiar with the VPN and its usage. Next day I had a phone call from the customer care and told me that there was no issues from Qubee end but the VPN provider or my PC so I had to fight against the customer care's word. Few days later I received a call from Qubee and asking for my VPN credential for them to test from their office which surprised me big time. I refused and offered myself with my laptop at Qubee's office. I went there to prove that it is the Qubee not the VPN provider by connecting via other ISP's Edge Modem. They started tracing finally and at a point I had to ask them to test my laptop with a public IP assigned and it went successful. The issue was resolved by assigning a public IP address as all the Qubee modems are assigned with private 10.11. network.Incident 2:A week ago around 7 PM local time my internet link went down all of sudden. After investing I found it was about 70-90% packet loss to some hosts including Google and Qubee's local DNS server. I called the customer care number after my further investigation to make sure my end was good. The customer care individual wanted to me access my PC to troubleshoot and take some reports on the modem. Unfortunately I was unable to explain to her that it was not possible for her get into PC due to network issue. I was surprised that she claimed to be technical without knowing about ping utility and packet loss terms. She also did not have idea about pinging between their DNS name and my end or how it works. I requested to talk to someone from technical team, we have to follow our procedure and some will visit your premises tomorrow , she replied. I knew that the issue may recover next day or later by itself but I needed support at that particular time as I was in the middle of my regular remote work. At the end I had to forget about hoping and use my backup internet (real slow) to report the office about my unavailability for that evening (imagine if I had no backup internet).Next morning everything was resolved by itself, I suspect there were some issues within the base station during the incident period. I had a visitor from Qubee's third party support to check my modem and some of the numbers of it after 3 days. He found nothing or could not do anything but took some snapshots with him. Note: issue resolved by it self. Imagine you were down for three days and no followups were made. Next day I had another phone call from Qubee's third party contractor to schedule a time to visit my place regarding the ticket Download/upload Issue . Note the ticket description as I specifically asked her to put packet loss somewhere in the ticket. I had to tell him everything was fine, no issues.Summary:I would like Qubee to put extra attention to be a modern and standard internet service provider in Bangladesh. As a subscriber I would like to see the following in Qubee. * Valuing customers by listening and understanding their specific problems and the needs (should be one them on the top) * A 24/7 true technical support team with basic troubleshooting skills set. * An escalation procedure should be implemented as well so customers are in touch in time (no 3 days waiting periods). * Understanding the terms Customer Care and Technical Support and implementation of it. * Response time and proper follow up manner. * Downtime notifications in advance * Responding emails with their inquiries. (I never received one) * Plug-n-play account activation system, where customer will be able to activate their unit on their own after purchasing the box from the outlet so installation engineer is required to visit the customer premises.Thank you!================Reply received after 30 minutes:================Dear Sayeed,Thank you very much for this valuable input. I am forwarding it immediately to the appropriate managers at Qubee Bangladesh: the head of customer care and the head of IT.Best regards,Nicolas Ruelle============I believe it never helped.

Post a comment

Name or OpenID (required)


(lesstile enabled - surround code blocks with ---)