rc3.org

Strong opinions, weakly held

Month: December 2012 (page 1 of 2)

Setting Apache MaxClients for small servers

Every once in awhile, the server this blog runs on chews up all of its RAM and swap space and becomes unresponsive, forcing a hard reboot. The problem is always the same — too many Apache workers running at the same time. It happened this morning and there were well over 50 Apache workers running, each consuming about 15 megs of RAM apiece. The server (a virtual machine provided by Linode) has 512 megs of RAM, so Apache is consuming all of the VM’s memory on its own.

At first I decided to attack the problem through monitoring. I had Monit running on the VM but it wasn’t actually monitoring anything. I figured that I’d just have it monitor Apache and restart it whenever it starts consuming too many resources. I did set that up, but I wondered how Apache was able to get itself into such a state in the first place.

The problem was that Apache was configured very poorly for my VM. Because I’m running PHP apps with the PHP module, I’m running Apache using the prefork module. For more information on Apache’s Multi-Processing Modules, check out the docs. Basically, prefork doesn’t use threads, so you don’t have to make sure your applications and libraries are thread-safe.

Anyway, here are the default settings for Apache in Ubuntu when it comes to resource limits:

StartServers          5
MinSpareServers       5
MaxSpareServers      10
MaxClients          150
MaxRequestsPerChild   0

In preform mode, Apache can handle one incoming request per process. So in this case, when Apache starts, it starts five worker processes. It also tries to keep five spare servers idle for incoming demand. If it has ten idle servers, it starts shutting down processes until the number of idle servers goes below ten. Finally, MaxClients is the hard limit on the number of workers Apache is allowed to start. So on my little VM, Apache feels free to start up to 150 workers, at 15 megs of RAM apiece, using up to 2.25 gigabytes of RAM, which is more than enough to consume all of the machine’s RAM and swap space.

This number is far, far, far too high for my machine. I had to do this once before but when I migrated from Slicehost to Linode some time ago, I forgot to manually change the Apache settings. I wound up setting my machine to a relatively conservative MaxClients setting of 8. I’m still tweaking the other settings, but for a server that’s dedicated to Web hosting, you may as well set the StartServers setting to the same as the MaxClients setting so that it never has to bother spinning up new server processes to meet increasing demand.

Currently my configuration looks like this:

StartServers          8
MinSpareServers       1
MaxSpareServers       8
MaxClients            8
MaxRequestsPerChild   0

The only danger with this low setting is that if there are more than 8 simultaneous incoming requests, the additional requests will wait until a worker becomes available, which could make the site really slow for users. Right now I only have about 60 megs of free RAM, though, so to increase capacity I’d need to either get a larger VM, move my static resources to S3, or set up a reverse proxy like Varnish and serve static resources that way.

Garann Means on JavaScript templates

using javascript templates

Garann Means explains the whys and wherefores of JavaScript templates. Reading this article reminds me that my front end development skills are so ten years ago.

The Web as an industry

Andre Torrez makes the observation that making things on the Internet isn’t for enthusiastic amateurs any more:

I think the thing that is eating industries: newspapers, music, movies, second rate mobile phone manufacturers…it’s eating us too. Being literate in tech isn’t enough anymore. As Robin said above, knowing how to put up a web page or write a little web app is fine for a niche hobby or an amateur pursuit, but if you want things to look good and work and be something more than a semi-broken thing you have to invest a real amount time and thought.

When I got started working on the Web, you could find gainful employment simply by being a person who really liked messing around with computers. I’m not talking about jobs making Web pages for local businesses, either, but for big businesses. Those days are definitely gone.

Back when I started, there were only “full stack” developers. Everybody did pretty much everything. Then the industry evolved to have “designers” and “engineers.” Things have become more specialized since. Now it’s not uncommon to find people developing software at Web companies who don’t know HTML at all. I would never have predicted that ten years ago.

Tony Horwitz on the gun lobby’s historical forebearer

The NRA and the “Positive Good” of Maximum Guns

As you know, I’m fascinated by historical patterns, so I pretty much had to link to this guest post by historian Tony Horwitz at Ta-Nehisi Coates’ blog. Comparing the “gun power” to the “slave power” is inflammatory, but the parallels are unmistakable. In fact, this pattern can be seen in most movements who fear their prerogatives being rolled back incrementally. Simply holding your ground is never enough.

The 100 year gun control project

There are many opponents of gun control who reject laws that would restrict gun ownership any further on principle or for reasons that seem practical to them. A society where law-abiding citizens can carry guns to defend themselves against criminals who have guns (or even those who don’t) seems like a sensible arrangement to them. Those people are entitled to their views, and I’m not particularly interested in arguing with them.

The people I want to argue with are those who are in favor of gun control but who reject it for practical reasons. They point out that there are at least 200 million guns in private hands in the US, and that even if we wanted to ban guns, there’s no realistic way to do so.

I would suggest that those people lengthen their time frame. What if we came up with a plan to fundamentally change America’s gun culture over the next 100 years? There are policies that we could start pursuing today that would move us in that direction, and taking those steps beats giving up in every way.

I’m actually not in favor of a blanket ban on guns. I respect hunters, and I think that target shooting and skeet can be fun. If it were up to me, though, there would be no concealed carry permits, and handguns would be completely unavailable. The idea of owning a gun for self-defense would seem completely perverse to people.

There are a lot of ideas out there for making guns more difficult to purchase, making ammunition more expensive, and taking existing guns out of circulation. If changing America’s gun culture is important, it’s worth thinking about how to do it, even if it takes 100 years. Believe me, that’s how the gun advocates think.

Observing people reading code

How do we read code?

Eric Holk writes about a study that observes programmers reading code. Unsurprisingly, naming things well makes a big difference.

How Flickr built its push notification system

Highly Available Real Time Push Notifications and You

From code.flickr.com. My main takeaway is that I need to get off my butt and improve my JavaScript skills.

On school shootings and gun control

There’s a lot to be said about school shootings. Jason Kottke said most of it today. One thing I want to add, though, is that pro-gun political groups not only fight for gun laws, they also do everything they can to prevent public health researchers from investigating the effects of gun ownership on society. The New York Times ran a story on this last year. When the I went back to look for that story recently, I noticed that the Times ran almost the same story back in 1995. People are entitled to their own philosophical views on the pros and cons of the firearms laws in this country, but it’s telling that pro-gun organizations work so hard to keep people from gathering data to use in the debate.

The Glitch shutdown

This is how you shut down your MMO’s servers for good: Glitch’s graceful exit

It was a wonderful tribute to the incredible energy people put into the online communities that they join. When an online community is shuttered, it inevitably ends any number of relationships that people build over years. The Glitch shutdown was very respectful of that.

John Myles White explains multi-armed bandit testing

If you’re interested in A/B testing on the Web, you should check out John Myles White’s talk at Tumblr on multi-armed bandit testing. You can learn a lot about standard A/B testing from the explanation he gives as a contrast to how multi-armed bandit tests work. I’ve read a lot of blog posts on multi-armed bandit tests, and this lecture is better than any of them in terms of explaining how this sort of testing actually works.

Older posts

© 2024 rc3.org

Theme by Anders NorenUp ↑