I’m also a photographer. Pushing large volumes of images up to my server for clients takes far less time with the second ISP, picked due to its much faster upload speeds.  I suspect they have a device buffering and shaping traffic as although speed test tools confirm the bandwidth is as advertised the 3-way TCP handshake and any DNS resolution takes far longer than on the slower ISP’s service. An ideal way to protect uplinks from those running bit-torrent services.

Now my reason for posting. The second ISP’s Hub has developed a fault. It can route traffic to the Internet just fine and do it’s own DNS resolution when running traceroute through it’s admin page, but not perform any DNS for clients. Despite countless reboots and a factory reset it wouldn’t change. Wireshark shows clients sending request packets and not getting a single reply.  It looks very much like the firmware problem that impacted many 2WIRE ASDL hubs in the past, and issue that also resulted in many calls to help desks and multiple replacements being shipped, also with the same broken firmware despite telling the help desk staff that shipping them was futile.

I’m reading in the technical press that my 2nd ISP is having clients leave them in large numbers.  From my recent experiences I think I know why and wonder how much longer they’ll survive.

To make matters worse the configuration setting on their hub to disable its local DHCP service was broken, so having another DHCP server override DNS details was impossible without putting a router between the LAN and hub. Thankfully keeping the Hub switched off for many hours somehow fixed DHCP.  DNS remains broken but as I’ve been able to regain control of leases I’ve got clients talking directly to the external DNS servers.

I see hardware, firmware and software issues all the time. It’s part of our modern life. It’s all designed by humans and we’re fallible. Where I have the necessary access and rights I’ll identify and if possible fix the issues, escalating to a product owner when needed, as in this case.

A support team failing their customers and ultimately their employer is another issue.

Any ISP wanting to retain customers should have an adequate 1st, 2nd and 3rd level support team. Teams that listen to what their customers say. I fully expect 1st to do the “Please power cycle the hub and reboot the computer” script following exercise. Anyone suitably experienced to work outside the script should be in 2nd or 3rd level support.

But when you get through to the supposed higher level support teams and tell them “I can traceroute from the client to the Internet fine but not do DNS resolution” only to get “Can you ping the default gateway?” you know something is wrong. Done Networking 101? When told that “nslookup run with server details changed from the hub address to a real DNS server works” only to have a response of “What happens if you enter a URL in a browser?” just enforces the point that the individual is in the wrong team or just not listening.

ISPs, these people are your front line. Many customers will talk to nobody else within your organisation, with maybe the exception of the billing team when they find they’ve been paying for a service they either don’t receive or don’t need. Do your best to have a properly trained workforce that listens to their customers, checks their understanding of the problem and escalates properly when needed. A continued bad impression will see your customers finding alternate suppliers.

43 views

Blog 100% blank yet the wp-admin dashboard works?

In my experience this happens after a WP version upgrade, tables are getting out of sync as to what the current theme is. Switch to the default theme then back to any custom theme you have. It’s fixed all occurrences of the problem for me.

Posts and Pages blank in the dashboard?

This is likely to be a plugin. In my case a page count plugin. It had been working fine for many days after an upgrade then suddenly broke after I’d added a new post.

Keep a close eye on your plugin versions and update them soon after new versions are out. I tend to let a week or two to pass first, if there are major bugs why not let others help find some first? You’ll not always have the dashboard warn you of updates so visit the authors web site for notices when WordPress has plugin issues.

When WordPress goes wrong tail your web servers error log. There will often be a reference to a variable or script name that points the finger at the faulty code.

If no new versions are available and the logs are not helping disable them one at a time until you find the culprit.

Then contact it’s creator for help and give them as much information as possible; what platform you’re running on, your blogs URL, the theme you use if by a third party and a list of all the active plugins that the broken one has to coexist with.

36 views

This would be of no concern if they were running off a single set of SAMs and APMs. But being the owners of a large global enterprise network and in the banking sector one isn’t enough and rightly so. They have two independent systems both doing live polling and alerting, backing each other up.

We’ve had a tool in place for a while, ensuring that any log entries, ownership and acknowledgement are replicated from one system to the other.

Today I finished the first cut of a Perl script that replicates In Maintenance setting between the systems via the SMARTS API. Just a bit of change control and everything will be in place.

If for whatever reason the Network Operations Centre have to switch from one system to the other all of the maintenance settings will then be in place allowing them to continue as if nothing had happened.

35 views

It’s the “glue” that allows many systems to share data. If no API exists the only alternative is often an XML or CSV file exchange via the most secure protocol both systems support.

But can you trust the content of the feed? It’s a good idea to use regular expressions to audit each field in the data before it’s used, ensuring bad data gets ignored. But how *old* is the data? Does the method used to generate or exchange the data ever cause an old file to be handed over?
How would you know about it?

Having the third party add a simple line like

# EOF 07/07/2011 21:02:37

to a CSV file, or the equivalent XML tag can save you a mass of time wondering why changes to one system are not being reflected in another.

44 views

Power but zero data? The guy at the till kept hitting buttons in frustration, no managers coming to his aid.

Luckily for the customers the device was connected to the till via a coiled lead that joined in the middle with a RJ45 coupler. A quick disconnect and reconnect rebooted the device (I upset the guy a little in doing it but action was needed), it reported its RS-232 status as okay and moments after it accepted my card and I was on my way..

45 views