Server issues – the end is in sight.

After 5 months of frustration, headaches, nightmares and stress, the BladeForums servers are looking like they are stable and tasks are nearly complete. We just need to get DMARC up and running and that will be the final checkbox – stability and speed are already showing great improvements.

I wish this had been a smooth process, but unfortunately it was much more drawn out than it needed to be largely because of issues that were out of my control.

The server switchover was precipitated by discovering issues with the nightly backup schema – MySQL table locking along with I/O transfer rates were resulting in nightly downtimes during peak traffic thanks to hosting provider misconfigurations. We’d specified backups begin at 0400 across the board, and they only configured that for the web server and had the db server set for midnight.

On top of that, vBulletin 4.x has been End Of Life’d by Internet Brands and further development will not occur as they focus on version 5. Unfortunately, v5 is not scalable for sites of our size, so we had to implement a platform migration strategy to ensure site longevity instead of remaining on legacy software. Step one of this was moving to more up-to-date virtual machines that were capable of running the latest packages and modern software.

So, back in May we had 3 new VMs spun up and anticipated a clean and orderly migration to the new nodes. We configured the new virtual machines with a dedicated www server and two database servers – a master and slave, where the slave would have the master’s db replicated to it. Backups would then be performed on the slave server which would completely eliminate any table locking issues resulting in downtime. We did test imports and even did trial runs of a possible successor forums platform to see whether migration would be feasible as an all in one effort.

Instead we experienced a multiple month long nightmare. Weeks were spent trying to troubleshoot what was going on, but it boiled down to the servers just…. ‘going away’ randomly every few seconds. We proceeded to import the forums data and what was noticeable during soft testing turned into complete site failures during live traffic, so multiple migration attempts (each spanning 8+ hours of downtime) failed. We had days of outages as a result, requiring reverting to the old server architecture.

Eventually it was determined that the VMs were, quite simply, built off bad templates. Only instead of completely blowing away the 3 bad VMs and generating 3 fresh ones, our hosting provider chose to only create a new web vm and wanted us to try and make it work. Needless to say, all 3 required replacement, costing me hours of technical support and thousands of dollars in time wasted, customer frustration, and more.

Once we had working VMs, we had a relatively short downtime with an easy migration that went off without any major issues. What should have taken 4 hours had dragged on for more than 4 months.


For the last few weeks we have been battling a persistent issue with Sphinx search – we could manually run the indexer and have new posts appear, but the automated reindexing and delta indexing was not occuring via cron.  Yesterday afternoon we began to get that resolved and it was completely fixed by 1AM this morning.  With that, our last obstacle for full operation is implementing DKIM / DMARC to get into compliance with email handling standards.

After that is complete, we will be ready to begin Phase II of transitioning away from vBulletin to another forum software platform, one that will offer even greater advantages and vastly benefit the membership in many ways.

For those members reading this, I appreciate your patience and support.

Advertisements

Now both cats know that the outside is filled with adventure.

Somehow the back door was left open and both made it out – Jennifer came home and discovered they were missing. One cat scaled the back deck to come back inside; he looked like he’d survived a harrowing experience. The other was nowhere to be seen. Since she’s not the brightest one, we weren’t sure she’d know to come back.

I came home and started calling for her and looking in the obvious areas – neighbor’s yard, the alleyway, etc.  None of the neighbors had seen her.  I walked around making “smooching” sounds and went back inside, trying to work up a game plan, when Jennifer said she could hear frightened meowing.

Turns out she was stuck up in the neighbor’s tree.  I got out a step ladder, but it was too short to reach her, so I had to get a larger one.  She refused to let go of the tree so I had to reach behind her and startle her into my arms, where she proceeded to dig in with all four sets of claws, hanging on for dear life.  Also, it turns out cats can yodel really loudly when they are frightened.

Anyhow, the cats are back home, looking very proud of themselves and showing a distinct lack of PTSD.  I’m sure they are already plotting their next escape.

Making an effort to write what I’m thinking

I’ve had an “on again, off again” writing habit over the years.  Typically when I’m at SHOT show or somewhere isolated, if I have a notebook handy I’ll jot down what is on my mind as a form of therapy or just organizing my head space.

This year I picked up the Rite in the Rain brand for the store, and was rather impressed with their hardcover offerings.  The only problem, however, is that my prefered writing implement is the Sharpie Pen which does not work at all in a Rite in the Rain notebook.

At the beginning of the month I picked up a Moleskine pocket journal from Target. At the time I just threw it in the cart and checked out without bothering to read what the price was. I happened to be wandering through that aisle again last night and was shocked to discover that the little Moleskine 3.5 x 5.5 Hardcover Notebook was $13.

Now don’t get me wrong, I love this thing. Small enough to fit in my pocket and go with me almost everywhere, 192 pages, classic look, but $13 for a pocket journal is a bit much so I turned to the internet.  A quick eBay search and I found someone selling a lot of 4 of these at $8.98 each, and they accepted my “best offer” of $24 for all four, shipped.  But, I needed something more long term, so I googled “moleskine wholesale” to see if there was better pricing in bulk.

One of the top results was this blog post:
http://andybrandt531.com/2012/12/notebook-alternatives-to-moleskines/

Apparently I’m not alone in sticker shock on pocket journals.  I wound up going to CR GIbson and picked up 10 of these Markings Black Bonded Leather Journals – with their spring coupon they wound up being $5 each, shipped.  We’ll see how they compare to the Moleskine; I’m not altogether sold on the sewn border so I’m hoping I haven’t made a mistake.

Wife sleep ramblings, January 2016

“Don’t you DARE!” she suddenly announces in a gravelly voice.

“Dare what?” I reply.

“You better stop what you are doing right now” she threatens, sitting up.

I look back at my Surface, looking at the book I’m reading on the Kindle app and try to keep from laughing.

Jennifer gets out of the bed, and crosses over to the desk.

“What is this?!” she demands, picking up a medium cardboard box that had, until this afternoon, contained several pairs of pants I’d ordered online. Now delivered & emptied, it was languishing on the desk, waiting for me to take it downstairs and then out to the garbage.

“An empty box, honey?”

She quickly shifted it aside, hunting for something that had drawn her ire. Her gaze narrowed and she seized the object of her hatred. “What is this?? THIS!” she screeched, thrusting her arm at me.

“It’s a mouse?” I snicker.

“Why is it flashing?” she asks, her voice becoming confused.

“Because the battery is dying” I respond.

She rubs her nose and the light begins to appear in her eyes, then heads for the bathroom. I turn back to my book…

Fun with Fixing Furniture

In the last couple days I’ve had to try my hand at repairing a couple pieces of furniture.  Last night’s chair repair was pretty successful, but tonight’s dresser is going to require further effort.

Somehow, some way, someone managed to break one of the chair legs in the addition right at where the leg mounts to the chair base itself.  The top of the chair leg had split, leaving a portion screwed to the seat and the rest of the leg.  I sized up everything, went to the shop and picked my electric screwgun & drill, clamps, wood glue and screws and some odds & ends.

This repair job wasn’t so bad; it involved:

  • Taking the remaining chair leg off the stretcher
  • Removing the mounted part that had split away from the seat’s angle bracket
  • Removing the mounting hardware from that piece
  • Gluing and screwing the split wood back onto the leg again
  • Remounting the whole shebang

Fortunately I had every tool I needed except for socket mounted allen bits for the electric driver, but home depot was still open.

Tonight’s repair experience didn’t go so well.

My bedroom dresser was purchased back when I lived in Florida; it was one of those “finish it yourself” raw wood deals that I honestly should replace but have never gotten around to actually tossing.  When KJ was a toddler, he decided to climb up it repeatedly, causing the center mounted draw slides to break, spilling ball bearings everywhere.  Naturally, the manufacturer of the drawer slide hardware has gone out of business, so ordering replacements wasn’t an option.  So, I went to Rockler and eyeballed what they had, ordered what I thought was right, and tried to effect the repairs this evening.

Naturally things didn’t go according to my wishes.

To start with, the drawers have horizontal stringers in the rear.  Normally this wouldn’t be an issue, except that the slides themselves are designed to either perpendicularly attach to a rear mounted vertical stringer, or have another 1″ of wood underneath for top down attachment.  Since the rear of the dresser looks like luan or veneered hardboard, screwing down the draw slide seems like the best option, so I’ll need to cut a thick piece of wood and attach that to the existing horizontal stringers.

But, the fun doesn’t stop there.  The previous draw slides attached to the draws themselves using a vertical tab in the back and an angled screw in the front that went into the drawer face.  I can probably rig the new slide to screw into the front face, but the new drawer slide is set up for mounting into the bottom of the drawer, which is not thick enough.  So, clearly I’m going to have to cut some sort of cleat or tab, mount it to the inside of the drawer, then attach the drawer slide to it to prevent tear out.

Honestly I’d be better of just buying a new dresser, but I don’t have a spare $1500 right now to throw at a decent piece of furniture.  I wish I had the time and skill to make my own, but I’m sure at the end of it, it’d cost as much as buying something new.

Exercise, diet control, lack of progress and more.

Let’s face it, I’m on the wrong side of 40 years old and nothing is free anymore.  I can’t eat what I want, sleep as much as I want, and life is taking its toll.  The damage from years of stupid decisions is adding up and overcoming the hurdles has become much more difficult.

I decided that I needed to start working out again in June of 2014.  I weighed myself then and the scale said 253lbs.

I love my comfort foods.  I love frozen cokes, and cake, and ice cream.  Bread. Pizza. Hamburgers. Tacos.  That’s the problem, I love them too much and have little ability to say no.

You can’t exercise your way out of a bad diet.

I weighed myself today and the scale says 252.75.

I compared pictures of last year and now.  I look exactly the same.

There’ve been significant road blocks to progress.  Every time I get my squat up to 185-190, my back injury flares up and I have to stop working out.  Mostly though, life gets in the way.  Last year, I stopped working out in December due to family issues, then I wrecked my shoulder in January and didn’t start lifting again until June of this year. 7 months of inaction and lethargy and watching my gains go down the toilet.

I need to control my diet. I’ve been using Myfitnesspal’s app and restricting my caloric intake for the last few weeks.  I’ve been trying to avoid the junk food and garbage.  When I can keep away from the frozen cokes and don’t blow my diet eating out, I lose weight.

When I fail, I don’t.

Looking at the datapoints in the iPhone and on MFP, I’m yoyoing between 258 and 252.  Since I’ve been paying attention though, it’s decreasing. We’ll see what happens.

I will say this – I’m a lot more consistent working out with the cage & weights at the shop.  I just wish I had a means to do assisted pullups and dips.