»
Discussion
»
Announcements & Links
»
The Forums are back
| Announcements & Links |
|
The Forums are back |
|
|
Ron
Administrator
Member Rara Avis
since 1999-05-19
Posts 8669Michigan, US 
|
'Twas an interesting evening.  The forums went down Thursday night, about 10:38 EST as best I can tell, and remained down until the wee hours of the morning (see time on this post). We can all thank Nan for letting me know there was a problem (though she blamed the problem on me "doing something"), because we would have lost a LOT of data if I hadn't shut the forums down immediately. As it is, I think we lost exactly one thread, from Open #36. It was number 003109, but I don't know the title or the author. We owe someone an apology, but I don't know who? The short story is that we ran out of disk space on the server. That has the potential to do a lot of damage, because the server keeps right on trucking, trying to write data to a disk that won't accept it. Every time it tries to update an existing data file, like thread 003109, it instead mangles it horribly. That's why shutting down the forum right away was so incredibly important. We were darn lucky Nan emailed me, and darn lucky I was still at the keyboard to get her message. The long story is a bit more technical. Every time someone responds to an existing thread, the server saves the post to a database file and then updates the HTML page with the same post. Essentially, everything is saved twice, once in a database and once as a web page. The big exception to that rule is our Archives section. Because those typically get a lot less traffic, the system only builds an HTML page when someone actually clicks on a link. Most of the data is thus only stored once, instead of twice, and that savings equates to massive gigabytes of disk space. Would you believe someone went through the forums this week and clicked on every single thread in every single archive? That's nearly 200,000 clicks, each one of which created a new HTML page. Ergo, we very quickly ran out of disk space. Ironically, when a Linux box is out of disk, it's difficult to do much of anything. Even deleting a file requires a few bytes of swap space, and the bigger the file you want to delete, the more swap space is needed. It took me almost two hours of zapping little files to get the disk from 100% used to 99% used, giving me enough free disk to actually be able to do something useful. Getting rid of all those extra HTML files in the Archives took another hour, but quickly took the disk down to only 52% used (which is about where we were last week before our mysterious culprit went click-crazy in the Archives). Once I found some elbow room, I had to rebuild virtually every single file in the forums. I'm about four hours into the process as I write this, with probably another two hours to go. Yea, it's been a real interesting night. So, who went crazy in the Archives and created almost 200K files in the process? Blame it on Google.  p.s. Please, everyone, keep your eyes open for potentially corrupt data files or web pages. I *think* everything is fixed, but a few extra sets of eyes sure won't hurt. |
||
| © Copyright 2005 Ron Carnell - All Rights Reserved | |||
|
Kit McCallum
Administrator
Member Laureate
since 2000-04-30
Posts 14774Ontario, Canada |
Thank you Nan! And thanks for working through it Ron.  Wow, that's one busy reader, lol. Maybe check for the person with a sore wrist and click-finger ... that's a LOT of clicking, lol.  |
||
|
Poet deVine
Administrator
Member Seraphic
since 1999-05-26
Posts 22612Hurricane Alley |
Yikes! Sounds like someone was busy..or created a program to open files in the archives? Thank you Nan for being 'awake'!! Whew! |
||
|
Sunshine
Administrator
Member Empyrean
since 1999-06-25
Posts 63354Listening to every heart |
Thank you, both!  |
||
|
Sunshine
Administrator
Member Empyrean
since 1999-06-25
Posts 63354Listening to every heart |
Ron, I think I see where we lost a few pages. I went to Prose, clicked on Copperbell's post, clicked on Larry's "Taco Bell" link, and poof - no page to be found. Went into Archives #3 [I think], WhtDove's post of 10/29/05 was at the top, went to click in to page 2 to see who had responded...Poof - no page to be found. There may still be a few bugs? |
||
|
nakdthoughts Member Laureate
since 2000-10-29
Posts 19200Between the Lines |
wow..I can't even imagine who would have the time to read all those in the archives. I have enough trouble finding the time to read what is recent. Maybe it was someone new who wanted to "catch up" Thank you Nan for paying such good attention and thank you Ron for this site and ALL you do.  M |
||
|
Nan
Administrator
Member Seraphic
since 1999-05-20
Posts 21191Cape Cod Massachusetts USA |
I can't take all the credit... I actually was getting into the threads with no difficulty. BUT - It was a rare moment where I happened to be on both MSN and Yahoo, as I was building webpages and needed to be able to message. If it weren't for Mysteria, Sunshine, Vicky, and Martie all reporting they were unable to access PiP all at the same time - coupled with my inability to FTP into the site - I wouldn't have known either. Teamwork - Yeah! |
||
Larry C
since 2001-09-10
Posts 10286United States |
So THAT'S what you do with your spare time! Wow! You are the most dedicated man I know. Thanks Ron. Click crazy for sure. Glad it wasn't me. |
||
Nightshade
since 2001-08-31
Posts 13962just out of reach |
 I don't quite understand all of the technical stuff .... but a big, big thanks and lots of hugs to all of those who helped get our home up and running again !! |
||
|
Ron
Administrator
Member Rara Avis
since 1999-05-19
Posts 8669Michigan, US |
Karilea, I'll check out those threads (after some coffee), but I'm not sure there's much I can do to fix them. As for our click-crazy culprit, I guess I was too tired last night to be entirely clear? The search engines all send out automated programs, called spiders or robots, to "crawl the web" and find pages that will be added to the search database. That's how all our stuff gets into a search engine in the first place. In the past few days, Google's spider (called Googlebot) went through pretty much the entire forum structure, including those critical Archive threads. This is the first time we've attracted quite so much attention from the spiders, but if my experience on the main site is any measure, it won't be the last. Now that Googlebot knows we have so much spider food hidden in the back rooms, he'll likely become a very regular visitor. I need to slip back into programming mode for a few days and redesign our Archive system so it doesn't create those extra pages and fill up the disk. There are several possible directions I still need to investigate, but it obviously just became a priority. I'm getting much too old for these all-nighters.  Incidentally, a few years ago I had a little fun with everyone as we approached our one million post milestone. Last night, as I was trudging through the database, I realized we passed our two million milestone some time in the past week or two, with nary a balloon or festive whoot in sight. Whoot, whoot? |
||
|
serenity blaze Member Empyrean
since 2000-02-02
Posts 27738 |
You are always very clear. Doesn't mean I know what the heck you are talking about though. (But I'll bet I could still beat you at game of "Break Out.") and yep, whoot-whoot! |
||
Mysteria
since 2001-03-07
Posts 18328British Columbia, Canada |
Never admit you are too old for an all-nighters, never! Now about that Whoot-whoot? Must be another American term? Is that like our Canadian whoo-hooo?  Poor you! I mean that sincerely. I was talking to Nan, and poof the lights went out in Passions just like that! Sure glad you have a supply of fuses there. |
||
|
sandgrain Member Elite
since 1999-09-21
Posts 3662Sycamore, IL, USA |
It's really hard to remember life without PIP once it becomes a daily watering hole. Many thanks, Ron, to you and all the others who keep PIP family blue pages available and so organized for all of us. Do you suppose someone actually read all those or just fell asleep clicking away? Rae |
||
|
Christopher
Moderator
Member Rara Avis
since 1999-08-02
Posts 8296Purgatorial Incarceration |
definitely wasn't me... i don't have DSL installed at our new house yet. so no pointing fingers! |
||
|
iliana Member Patricius
since 2003-12-05
Posts 13434USA |
Thanks, Ron. And, thanks to Nan, too. Wow, Ron, I cannot believe the time you devote to this place. We are all so grateful. ....jo |
||
|
Ron
Administrator
Member Rara Avis
since 1999-05-19
Posts 8669Michigan, US |
quote: http://www.woot.com/ It doesn't quite rhyme with eBay, Sharon, but it's close. They sell one and only one thing a day, usually at a good price. I bought a wireless computer speaker just the other day, so I can take my music out into the yard next Spring. It's the kind of place I suspect you would love. Unfortunately, I'm probably teasing you, 'cause I don't think they'll ship to Vancouver. |
||
|
Mysteria
since 2001-03-07
Posts 18328British Columbia, Canada |
I have literally died and gone to heaven! Been in that Woot for hours LOL, but then you knew that I would be right? If they don't ship here, I can send it to the girls - there is always a way Ron, always a way Thanks for the link, I love it! You find the neatest places and things dontca? Still laughing at November 4th item for grabs - gotta get me a bag of that for sure!Oh, and I never heard of a woot before or a whoot either LOL |
||
|
Ron
Administrator
Member Rara Avis
since 1999-05-19
Posts 8669Michigan, US |
I've changed the way our software handles Archive files, a task which turned out to be a lot easier than I anticipated. Henceforth, all the Archive threads will by dynamic cgi pages instead of static HTML ones. Unfortunately, this will break the few old links to those static pages, but is a necessary cost to protecting us from the search engine spiders. |
||
|
Kit McCallum
Administrator
Member Laureate
since 2000-04-30
Posts 14774Ontario, Canada |
I'm glad you found a way to handle the archives without taking on a few more all-nighters Ron. Let the spiders and happy clickers have at it now, lol! But ... we missed the countdown???  TWO million posts???  Woot! Whoot! Whoo Hoo!!!and a big wide-eyed WOW!!! Unbelievable numbers!!! |
||
|
Marge Tindal
since 1999-11-06
Posts 42384Florida's Foreverly Shores |
quote: HUH ? |
||
|
Christioned Member
since 2004-01-15
Posts 93Huddling in the dark |
Thank you so much Ron for keeping our happy little sanctuary up and running. Thank you also to Nan and everyone else who found out that there was a problem. Don't know what I'd do if Pip wasn't here to come back to. Also ya hoo on 2 millionth post. Hope that we will make it 3 millionth and many more. "See my wrists; |
||
|
icebox Member Elite
since 2003-05-03
Posts 4383in the shadows |
Too many clicks!?!?!!!? ...uh...so you mean I should stop writing on the Men's Room walls: "For a good time click http://www.piptalk.com/ " ????? (hey? thanks once again for the almost infinite labor of maitaining this place!) |
||
|
serenity blaze Member Empyrean
since 2000-02-02
Posts 27738 |
Hey Ron! :waving: I believe my memory really is serving me better these days... I just thought I'd bump this because I found it so very interesting in light of recent events. I'll still write the letter folks. |
||
|
Susan Caldwell Member Rara Avis
since 2002-12-27
Posts 8348Florida |
Wow, that is interesting Karen.. "too bad ignorance isn't painful" |
||
|
|
⇧ top of page ⇧ |
|
| All times are ET (US). All dates are in Year-Month-Day format. | ||
