30 hours of panic; or how to survive a DDoS attack
<< July, 2014 >>
Search Blog

100+1 Uses (1) RSS
Adam Davidson (10) RSS
admissions (1) RSS
Advertising (2) RSS
Alex Florov (5) RSS
Ardor (4) RSS
Art and Pipes (4) RSS
Ashton (1) RSS
Ashton (3) RSS
Behind-the-Scenes (11) RSS
blog (9) RSS
blog (40) RSS
bloopers (2) RSS
books (2) RSS
Brad Pohlmann (2) RSS
Brad Pohlmann (2) RSS
briar (8) RSS
Brick House Cigars (1) RSS
Brigham (2) RSS
Broken Pipe (2) RSS
Bruce Weaver (2) RSS
Capstan (6) RSS
Carlos Torano (1) RSS
Castello (6) RSS
Chacom (4) RSS
cigars (20) RSS
Claudio Albieri (1) RSS
Claudio Cavicchi (3) RSS
comic strips (2) RSS
Cornell & Diehl (10) RSS
Customer Service (6) RSS
Dunhill (12) RSS
Ernie Markle (1) RSS
Ernie Markle (2) RSS
Escudo (1) RSS
Esoterica (2) RSS
estate pipes (17) RSS
events (2) RSS
Famous Pipe Smokers (11) RSS
Fitness (1) RSS
Flor de Gonzalez (1) RSS
Food (12) RSS
Gabriele (1) RSS
Gamboni (2) RSS
Gawith Hoggarth & Co (1) RSS
gift cards (1) RSS
Giveaways (6) RSS
G. L. Pease (10) RSS
grain (1) RSS
Gran Habano (1) RSS
Grant Batson (1) RSS
Grechukhin (2) RSS
hemp wick (1) RSS
Hermit Tobacco (1) RSS
Hiroyuki Tokutomi (9) RSS
history (4) RSS
Humor (23) RSS
Ikebana (1) RSS
Il Duca (1) RSS
Interview (2) RSS
Italy (3) RSS
J.Alan (1) RSS
J. Alan (11) RSS
Japan (1) RSS
Jess Chonowitsch (1) RSS
J&J (3) RSS
Johs (2) RSS
Kaywoodie (1) RSS
Kei-ichi Gotoh (2) RSS
Kristoff (1) RSS
La Gloria Cubana (1) RSS
Lars Ivarsson (3) RSS
Lasse Skovgaard (4) RSS
Leo (1) RSS
Letter (1) RSS
lighters (1) RSS
Low Country Pipe and Cigar (3) RSS
Luciano (3) RSS
Mac Baren (17) RSS
Maigurs Knets (2) RSS
McClelland (6) RSS
Michael Lindner (2) RSS
Michael Parks (1) RSS
Michail Kyriazanos (1) RSS
Michal Novak (2) RSS
Mystery Tobacco (2) RSS
Nanna Ivarsson (2) RSS
nasal snuff (1) RSS
Nathan Armentrout (1) RSS
Neerup (1) RSS
Newminster (1) RSS
newsletter (266) RSS
Oliva (1) RSS
Orlik (5) RSS
Padron (1) RSS
People (22) RSS
Pesaro (1) RSS
Pete Prevost (1) RSS
Peter Heding (2) RSS
Peter Heeschen (2) RSS
Peterson (7) RSS
Peter Stokkebye (3) RSS
photography (18) RSS
pipe accessories (3) RSS
pipe basics (4) RSS
Pipe Clubs (2) RSS
Pipe Fiesta (1) RSS
pipe making (6) RSS
pipe making (55) RSS
pipes (44) RSS
pipes (8) RSS
Pipe Shows (23) RSS
Pipes in Film (4) RSS
pipe tobacco (81) RSS
poster (1) RSS
Press (7) RSS
Rad Davis (1) RSS
Radice (6) RSS
Ray Kurusu (1) RSS
Reiner (1) RSS
Reviews (4) RSS
Rocky Patel (1) RSS
Rocky Patel (2) RSS
Sales (3) RSS
Samuel Gawith (1) RSS
Samuel Gawith (2) RSS
Savinelli (3) RSS
scott thile (2) RSS
Sebastien Beo (4) RSS
Ser Jacopo (3) RSS
Simeon Turner (1) RSS
Sixten Ivarsson (2) RSS
Smio Satou (3) RSS
smokingpipes.com (74) RSS
SPC Merchandise (1) RSS
SPC University (2) RSS
Stanwell (4) RSS
Storient (1) RSS
Summary (6) RSS
Takeo Arita (2) RSS
Tatuaje (2) RSS
technology (5) RSS
Thanksgiving (1) RSS
Three Nuns (4) RSS
tobacco (7) RSS
tobacco aging (1) RSS
tobacco blending (5) RSS
tobacco review (5) RSS
Tom Eltang (5) RSS
Tonni Nielsen (1) RSS
Torano (1) RSS
travel (70) RSS
Tsuge (3) RSS
Vauen (1) RSS
video (57) RSS
video (5) RSS
Viktor Yashtylov (1) RSS
virginia (1) RSS
YouTues (4) RSS
Photo Albums
florov (1)

14 September 2010

30 hours of panic; or how to survive a DDoS attack
 A very atypical day and a half in the life of Smokingpipes.com

As you may have noticed, Smokingpipes.com has been singularly unresponsive during the past twenty-four or so hours. At 1pm yesterday, we were hit with a DDoS attack (Distributed Denial of Service). Since then, with a nice stretch of working between 3am and 8am this morning (yeah, hardly a peak site traffic time), our servers have been slammed by millions of bogus requests from thousands of IPs. We've come up with a temporary solution, by moving Smokingpipes.com to another server and just serving up a flat html page from there, which redirects to the full site's web server on our old servers, which handles legitimate traffic normally.

We've also discovered that we are one of at least four major premium tobacco retailers (with the other three being major cigar retailers) to be hit with a DDoS attack in the last 24 hours. This appears to be directed against purveyors of tasty, high quality smokables. These are targeted attacks. Still, in the Wild West that is the internet, the good code slingers are winning this particular round over the bad ones with our stop gap measure; we shall see how it progresses over the next few hours.

So, what is a DDoS attack and what have we done to make the site available?

Normal: Happy users visit Smokingpipes.com and see a bevy of beautiful briars

In our wildly simplified diagram, this is how things are normally. Happy pipe smokers go to Smokingpipes.com, read about or purchase or drool over pictures of pipes, pipe tobacco and cigars. Your computer asks our web server (by way of various servers in between) and our server nicely responds by serving up lots of fun pictures, images, tons of wonderful information, all stored on either our database server or our assets server.

A DDoS Attack: Lots of Zombies trying to get in the door makes it impossible for the regular user to access the site.

When someone initiates a DDoS attack against us, they've used thousands of slaved computers (think of them as zombies, perhaps) to remotely make http requests to our servers, specifically to the Smokingpipes.com domain. Our servers, though they are shiny and fast, are utterly unable to serve up the information fast enough and end up getting completely bogged down trying to contend with all of the bogus traffic. The thousands of computers are innocent bystanders too-- more than likely they were infected with a trojan that causes them to make these requests at the behest of the master (evil!) computer, much like zombies at the behest of some wicked puppet master.

Keep in mind that this traffic doesn't do anything to us except just ask us questions. We've not been hacked, nothing has been compromised, everything is safe. All is normal, except that thousands of extra computers are asking our servers for information and we just can serve it up fast enough.

Our Solution: behind the splash page (bouncer) life is normal, but the bouncer is there to keep out the bogus traffic.

Part of the problem is that our regular servers, sort of like our store staff at Low Country Pipe & Cigar, like to greet folks with lots of fun stuff, show them what's new, and point out interesting odds and ends. That first page people reach is filled with dynamic content, pictures and other things that, in the normal running of things aren't taxing at all for our servers, but multiply that normal load times fifty or a hundred and things slow to a crawl or stop altogether. So, it's sort of like Ron, Kelly, Vince and Jennifer in the store all trying to show pipes to a thousand customers simultaneously, most of which really just want to stand in the middle of the store and not really do anything. Obviously, as good as our store staff is, they're going to grind to a halt in a hurry if they had to contend with this.

So, what do you do if you have this problem in real life? You hire a bouncer. Our digital bouncer lets in anyone who asks nicely, but doesn't try to help anyone or be particularly nice about his greeting. It makes it much easier for him to keep up with the multitudes. He then lets in anyone who asks nicely, and inside the store, the customer experience (and our poor, harried store staff in the metaphor) return to normal. The digital bouncer, our splash page, does this by serving up the simplest code possible (a bunch of explanatory text) and letting in those who click the link to enter Smokingpipes.com. We'll leave the bouncer out front until we're confident the throng of zombies has passed and just normal good folks are trying to get to the site again.

So, we very much appreciate your patience and kind words as you've waited for us to return. Hopefully, we'll be able to drop the splash page in the next few days and return everything completely to normal. Brian and Ted will cover the phones until 10pm tonight to accommodate extra call volume. I, however, having been at this for almost all of the past twenty four hours, will go take a shower and get some sleep!

Posted by sykes at 4:24 PM | Link | 7 comments

Re: 30 hours of panic; or how to survive a DDoS attack
Great explanation (with cool diagrams) of a very, very crappy, evil situation.

Whoever did that should have a needle stuck in their eye.

Posted by keving on September 14, 2010 at 4:59 PM

Re: 30 hours of panic; or how to survive a DDoS attack
Actually "the poke in the eye with a sharp implement" is too good for these folks, who seem to think that bragging rights in the hacker community outweighs potential serious damage to folks' livelihood

Posted by Mitch on September 14, 2010 at 6:05 PM

Re: 30 hours of panic; or how to survive a DDoS attack
Yes, I agree. They should be put through a meat grinder.

Posted by keving on September 14, 2010 at 6:16 PM

Re: 30 hours of panic; or how to survive a DDoS attack
Great explainer. I've worked on web type stuff pretty much since the beginning and I have not come across a more clear, simple explanation of a DDoS attack - with cool pictures to boot!

Posted by Caly on September 14, 2010 at 7:45 PM

Re: 30 hours of panic; or how to survive a DDoS attack
Clay, Kevin, Mitch,

Thanks everyone for the comments. I particularly appreciate Clay's; I sometimes wonder if my often tortured analogies help or hinder and it's always good to get a response!


Posted by sykes on September 15, 2010 at 12:42 AM

Re: 30 hours of panic; or how to survive a DDoS attack
Hate to tell you guys but this probably isn't "bragging rights in the hacker community". Successful botnets, such as the one hitting our dear friends at smokingpipes.com, are typically built by organized criminals and "rented" out. It is very likely that an anti-smoking zealot (of the ASP Prof Jonez troll kind) paid for this activity.

Posted by Eric on September 16, 2010 at 12:31 PM

Re: 30 hours of panic; or how to survive a DDoS attack
Eric is of course right. As I've tried to explain it to people here, I (a semi-technical type with some programming experience) could find someone and pay them to do this with about an hour (not that I ever would, of course).

This is almost certainly some sort of anti-smoking zealot. Sort of like environmental extremists or anti-abortion extremists bringing down Monsanto's or Planned Parenthood's websites.

Can't we all just respect each others property rights? Digital or otherwise?

Anyway, griping over.

Posted by sykes on September 16, 2010 at 1:12 PM

Name:   Required
Email:   Required your email address will not be publicly displayed.

Want to receive notifications when new comments are added? Login/Register for an account.

Anti-spam key

Type in the text that you see in the above image:

Your comment:

Sorry, no HTML allowed!

Subscription Options

You are not logged in, so your subscription status for this entry is unknown. You can login or register here.



New Pipes

Fresh Items



Click to verify BBB accreditation and to see a BBB report for Smokingpipes.com.

View in English View in Japanese