smashing.org

While dealing with some spam, I accidentally deleted all comments on this site. Big oops on my part, but fortunately this blog is new, so I didn't lose too many comments.

I was reading Scott's (scrottie) journal, and came across his commentary on Ask's
Real World Scalability talk. I have some comments of my own, and in the end I think that Ask's presentation, when taken outside of the context of a Perlmongers group meeting, simply suffers from a bad title.

First, I applaud Ask for bringing to light a lot of the relatively unknown open source scalability and high availability solutions. It's important that people know that there are free and open source solutions out there. However, I also applaud Scott for exposing Ask's extreme prejudice toward FOSS solutions. I've been a system administrator for 12 years now, in academia, ISPs, and the corporate world, with a whole bunch of consulting on the side. If I know anything, it's that a good sysadmin will look at all of the available options, free or not, open source or not.

So let's nitpick. Ask's presentation seems to discount any solution whose cost is greater than zero, thereby eliminating the entire world of commercial solutions. And while there are plenty of good FOSS solutions out there, guess what? The commercial ones work too. Often better. And you get support. Not a mailing list, but someone to call to fix things when they break. Yes, all of this costs money. But in the real world, companies are willing to pay for this. And if they're not willing to pay, or the cost breaks the budget, or the open source solution is simply better than the commercial one, then that's where FOSS makes sense.

Let's move on to databases. Why the aversion to Oracle? I'm no fan of Oracle (I loathe their costly licensing scheme), but I've used it (along with MySQL and Postgres) since version 7, and it does the job and more. A properly tuned Oracle database will perform just as well as MySQL or Postgres, so don't tell me not to use something without showing me numbers. I let budget, in-house expertise, and feature/functionality dictate whether or not I use Oracle.

Also regarding databases:

Stored Procedures Dangerous...work in the database server bad

Wow. This couldn't be more wrong. Databases exist to work with data. If I can encapsulate complex data manipulation in an internal stored procedure, what is wrong with that? It divorces SQL from my application layer, where I don't want to see 30 lines of some complex multiple join. Now, Ask does have a point. There is a lot you can do in a stored procedure that does not belong in the database, such as HTML template processing (yes, I've seen this). But don't discount stored procedures as a whole. They are extremely valuable. And if your database's performance is suffering, get a DBA who knows what he's doing to diagnose the problem!

In the "High Availability Shared Storage" slide, Ask says this about shared storage, including Netapp filers, which provide both NAS and SAN interfaces:

all expensive and smells like "the one big server"

Expensive? Without question. "Smells like the one big server?" Yikes. Have you actually used a clustered Netapp filer like the FAS720? If one side goes down due to a hardware failure or maintenance, the other side takes over without missing a beat. And everything is redundant, down to dual loops through each disk controller to guard against FC-AL loop failure. How exactly is that bad? Where is the single point of failure that is "the one big server?"

Finally, we come to disaster recovery. This is a HUGE topic, and I hate seeing it covered so succinctly in any presentation. But at the same time you would be doing your audience a disservice by not mentioning it. So I won't really harp on this, but do I see one glaring thing missing in Ask's presentation -- testing. He never mentions testing your disaster recovery plan. This is the single most difficult and overlooked part of any DR plan, and it is absolutely critical to making sure your recovery procedures actually work in the real world, and for keeping your plan up to date when things change. And things always change.

Now, back to my original assertion that Ask's presentation suffers from a bad title. I think if it were titled, "Open source scalability solutions" or "Inexpensive scalability solutions", there wouldn't be a problem, since he most certainly focuses on FOSS and the price of commercial hardware and software. But the "real world" is a big place, and gives us a much wider range of appropriate solutions than you'd be led to believe from this presentation.

A recent Comupterworld article, which discusses the downsides of virtualization, looked interesting at first. As a cynic, I enjoy reading articles that deflate overhyped technologies and expose weaknesses, though I am a big fan of virutalization. After reading the article though, I can say that the consultant referenced therein, and possibly the author, see virtualization in a vacuum. The problems presented are perfectly valid, but they all have solutions in the real world. Let's look at each of them:

...enterprises sometimes [sic] have difficulty finding or applying adequate monitoring and management tools that work across both virtual and physical landscapes. Other issues can include support, integration and compatibility of different operating systems on the multivendor hardware being virtualized.

Absolutely true. The solution? Verify that the implementation you choose meets your requirements -- something that a system administrator should do before deploying any software.

Increased uptime requirements arise when enterprises stack multiple workloads onto a single server, making it even more essential to keep the server running.

Okay, so spread your workload around onto multiple hardware nodes, and where applicable, provision redundant services on multiple hardware nodes to minimize single points of failure if a hardware node goes down. You'll never eliminate the hardware node as a point of failure, but you can minimize the effects of any downtime with some careful planning.

Bandwidth problems are also a challenge, Mann says, and are caused by co-locating multiple workloads onto a single system with one network path...in a virtual environment, multiple workloads share a single NIC, and possibly one router or switch as well.

Sure. So make sure you have a hardware node with the capacity for multiple NICs. You can then put certain virtual machines on certain NICs, or trunk the interfaces. If using multiple NICs though, make sure your virutalization software supports that configuration. As for the router or switch, this is solved by using multiple hardware nodes connected to different routers or switches.

So what have we learned?

First, analyze your requirements. If you have 50 high-traffic web servers you'd like to consolidate, it doesn't make sense to have just one hardware node with a single NIC. At the other end of the application stack, a high-volume transactional database server probably doesn't belong on a virtual server at all.

Second, know the resources you have to work with. If all you have is a single server with a single NIC, then plan around that. You might not be able to virtualize everything that you want to, as performance might suffer. But that's a limitation of your hardware and your budget -- not virtualization.

These are all issues a good system administrator would take into account. Get yourself a good sysadmin, and take these kinds of articles with a grain of salt.

As a serial corporate Oracle user, I am intrigued by the cost savings that something like EnterpriseDB promises. I am not married to Oracle, but it's very good at what it does, so I'm reluctant to move from it unless there's a damn good reason. My manager hat tells me that long-term cost savings and straightforward licensing are two good reasons to start looking at EnterpriseDB. My sysadmin hat tells me it'll be fun to put their Oracle migration tools through the ringer and actually be running Postgres behind the scenes.

Heard from a friend last week:

...and I know you've got other talents, like sticking Perl in places where
it shouldn't be.

Apparently extproc_perl has only strengthened my reputation as a mad scientist. Now, if you'll excuse me, I must get to work on my latest creation, a four-assed server. :)

Starting in 2007, Daylight Saving Time in the US starts a few weeks earlier and ends a few weeks later than it previously has. Are your Linux boxes ready? Instead of checking the latest patches, here's a much quicker test:

$ zdump -v EST5EDT | grep 2007
EST5EDT Sun Mar 11 06:59:59 2007 UTC = Sun Mar 11 01:59:59 2007 EST isdst=0 gmtoff=-18000
EST5EDT Sun Mar 11 07:00:00 2007 UTC = Sun Mar 11 03:00:00 2007 EDT isdst=1 gmtoff=-14400
EST5EDT Sun Nov 4 05:59:59 2007 UTC = Sun Nov 4 01:59:59 2007 EDT isdst=1 gmtoff=-14400
EST5EDT Sun Nov 4 06:00:00 2007 UTC = Sun Nov 4 01:00:00 2007 EST isdst=0 gmtoff=-18000


If the the transition between isdst=0 and isdst=1 doesn't happen on March 11, you need to get the latest timezone data from your distribution!

NOTE: zdump can be found in /usr/bin or /usr/sbin, depending on your distribution.

I found this in the depths of my new Panasonic plasma TV's menus.

Welcome to the new smashing.org! This site will be the exclusive home for my blog, articles, presentations, open source projects, and general musings about the IT world. Feel free to comment -- I'm looking forward to sharing my thoughts and starting some interesting discussions.