How I am reducing the cost of hosting balagan.info

As you might have seen I can’t afford to host Balagan.Info any more. Or more accurately my current hosting company (WP Engine) has pointed out that they will have to triple my current hosting cost given the storage and bandwidth I currently consume. Ouch! I can’t afford that.

I now have a plan to manage my costs down. This is what I’m doing.

Thanks. I got lots of useful suggestions, both as comments ot the original post and directly. Particular thanks to Aidan who I’ve been discussing messy technical issues with for a few weeks now.


Cost based on storage and bandwidth

The big challenge was to control cost. I definitely need to stop costs going up, which my hosting company is saying they have to. But once I’ved stopped the upward trend I can consider whether I can lower costs.

My balagan.info costs are based on storage and bandwidth. And I was way over the limits:

Storage 37 GB from a 20 GB limit
Bandwidth 412 GB from a 200 GB limit

The two are related and images are the intersection.


Image Size

Image sizes impact both storage and bandwidth. Bigger images means more storage demands and also means heavier pages hence higher bandwidth consumption.

So lots of folks suggested optimising the images. Image size was the first thing the hosting company suggested I work on.

But suggesting it and doing it are two different things. Remember, I had over 4,900 original images, which wordpress had gleefully expanded to 36,000 images, making 13 GB of bloat. Reducing images for new posts is easy. Reducing my legacy was hard. But I’ve finally managed to do it.


Shrink Old images

I confess I got sloppy when I moved to WordPress in 2013. Previously I’d been sizing all the images before i uploaded them. Obsessing about keeping the size low. It was a faff so when I moved to WordPress I assumed the machine would do that for me. And in a sense it did. WordPress gives a choice of size to display. But what I didn’t think about was that WordPress retains the original. Doh! And it creates image versions from thumbnail (160×160 pixels) all the way up to the full size one. And that it what got my storage. You see I had 10 years of history and associated bloated files. Thank made over 36,000 images. Many of these were huge. Some were 9-10 MB. Quite a few were 4-5 MB. I had 13 GB of images in total! This is wasteful since I only need a about 50-200 KB per image for my purposes. I didn’t need any of these these giant images, whether uploaded by me or created by WordPress. They could all go (mostly)

So this is what I’ve been doing over the last couple of months:

  1. Download the 13 GB of images. Urg. That was painful. Don’t use FTP for this. Four attempts. 5-6 hours per attempt. Four failures. But WP Engine support put me right. I zipped them into one 13 GM file and used the browser’s download to get it. Took just 2 hours.
  2. Extract the 160, 320, and 640 pixel versions for reuse
  3. Delete all those wasteful versions created by WP
  4. Shrink both the original image and the nasty WP generated “-scaled” version to 740 pixels
  5. Upload the new images (only 2 GB so FTP worked)
  6. Replace old images with new images (that was agonising; the Linux mv command on the SSH gateway does weird stuff)
  7. Get WP to regenerate the images so everything was sweet
  8. Manually fix up the remaining bloated images
  9. Manually fix up the high frequency pages
  10. Ask WP Engine support foor where my bandwidth is being concerned
  11. Start manually fixing up those

So far it has taken weeks of elapsed time (and counting), many hours, frustration from my family due to an absent dad, an open ticket with the support team, a bunch of advice from Aidan, relearning Automotor on Mac, learning Apple Script, discovering the SSH Gateway to my site, relearning Linux, discovering WordPress has a command line interface (WP CLI), and then using that, and fixing, fixing, fixing. Simple. 🙂

The chart shows the profile of my storage consumption. It doesn’t show the most up to date figures, and associated drop, as WP Engine only sample once per week.

Balagan Storage Timeline
Balagan Storage Timeline

I have shrunk 13 GB of images to 2 GB. Every image is smaller so pages will load faster. I won’t get polite but urgent calls from the Account Manager about Storage. Job done.

Well, actually, not so simple. And not really job done. There were some problems with my process:

  • I had a lot of PNGs for photos. That is a no-no in the web as PNG photos are huge. So I had to hand crank JPG versions of these and upload those
  • I have found a fair few pages with missing images. Something went wrong somewhere with my 640 pixel images. The size of the image and the size WordPress things the image are are out by 1 pixel. A rounding error somewhere I guess. WP might thing in image is 640×480 pixels but the actual image is 640×481. Annoying, but I can fix these when I find them. I’ll look through the more recent posts and ensure they are okay. If you spot any problems, please shout.
  • The old theme I used just showed a thumbnail on index pages for featured images. The new theme shows a big image (740 pixels) and big photos as banners at the top of articles and in index pages just looks wrong. So I have to go through and fix all those. There are a lot. No really, there are LOT!

I was also advised to limit my photos to 72 dots per inch (dpi). As it happens almost all of them are already 72 dpi. Oddly the graphics I create in powerpoint are 150 dpi, and powerpoint doesn’t offer a way to change that setting. Luckily these images tend to be small size anyway. But something to look at when I’d dealt with the gross problems. And, rather embarrassingly, I found my balagan wooden shed logo was both large and high dpi. Oops, that was sucking bandwidth on every request. Now shrunk.

So good progress but work continues.


Small images in future posts

For future posts I’ve changed my workflow so the featured image will be 740 pixels wide exactly. This is what is displayed by the new theme I adopted in May. So putting a bigger image in there is a waste. Cutting down the featured images to 740 pixels should reduce the bandwidth a lot.

I’ve been experimenting with the optimal featured image proportions and have settled on 24:5. Don’t ask me why, but it looks right to me. So 740×154 pixels.

Normal images will be limited to 630 pixels, in both height and width. I’ll display the the full image rather than a cut down version with a link. Again this smaller size will reduce storage and bandwidth. And the code will be simpler. Everything helps.

Here is an example from my recent Small Kircholm Battle Report. The original image was 4032×3052 pixels and 3.6 MB. This got reduced to 630×477 pixels and 145 KB.

Balagan Image Shrinking Example
Balagan Image Shrinking Example

From the perspective of the viewer the only difference is you can no longer click on the image to get a bigger version. This is what it now looks like:

Kircholm-1481 Polish left and centre advance
Kircholm-1481 Polish left and centre advance

I might want to allow my maps to be bigger than other images so people can print them. A4 has 3508 x 2480 px which is quite big. We’ll see. Maybe a smaller image will be fine for printed maps. I’ll play around.


Reduce bandwidth

Reducing the image sizes/storage is already reducing bandwidth but I’m not see the same level of improvement. Image storage went from 13 GB to 2 GB. But bandwidth consumption has only gone from 15 GB per day to 10 GB. And my target is about 6.5 GB per day.

I don’t have many tools available to me. The WP Engine portal just gives me this image. I think it shows an improvement. Maybe. But it definitely doesn’t tell me where my problem is.

Balagan Bandwidth Timeline
Balagan Bandwidth Timeline

As I mentioned, I have been doing this:

  1. Manually fix up the high frequency pages
  2. Ask WP Engine support foor where my bandwidth is being concerned
  3. Start manually fixing up those

This is the kind of thing I get back from the support team. It happens to be the image tha highlighted my balagan shed logo was a problem point.

Balagan Bandwidth 2021-08-10
Balagan Bandwidth 2021-08-10

Featured images

I’m guessing that the big bandwidth cost are the featured images. Most of my posts since 2013 have featured images. I just picked the most interesting image from the posts and selected that one to feature. This rather naive approach means my featured images are huge, like all the other images. In my previous theme they were displayed as 160×160 pixel thumbnails so the index pages were light weight. In my new theme are are displayed 740 pixels wide and my index pages are weighty. I have to prune the featured images.

My brutal approach to shrinking images made all my images at most 740 pixels in either width or height. I selected this so they would be okay featured images. But it looks odd to have full sized images as featured images on my index pages. So I’m starting to make specific banner images that are wide and short. I’m going for 740 x 154 pixels for these. That seems to look right. For the last couple of days i’ve been trying to give all the posts in the the Crossfire category smaller banners. This has been quite time consuming. I’ve only gone back to 2015. Lots more to do, let alone other categories.

While working on that I found, again, that PNGs are killing me. A PNG banner might be 60 KB, much better than the 600 KB I might have had before, but a JPG version of the same banner might be 30 KB. Winner. That means my little piece of automation now turns all my PNGs into JPGs.

So I have more manual, fiddly work to do. Probably a lot more manual work. But after I’ve done enough of that I can …


Turn on CDN

My final ace in the hand is the Content Distribution Network (CDN) that WP Engine provide. I haven’t turned this on yet as it hides some of the detail. As I fix things I want to see the impact and the CDN would mean there is a delayed effect.

But when I think I’ve done as much as I can, I’ll turn on the CDN. The main effect for me will be my content will be cached. All views will hit the CDN, but it will only go to WP Engine when it is asked for new content. That will have a big impact for frequently hit content, like new posts, and popular posts. It will be less useful for older stuff that is only infrequently hit.


Review hosting options

I had several suggestions for changing hosting company…

If I do drift away, Fasthosts are definitely near the top of my list of options. I already use them for my domain name management. WP Engine don’t do this, as they are pure play WordPress. So Fasthosts is a strong contender.

Pair are much cheaper than WP Engine. I have a 10 site license (for other reasons), and their offering comes in at half the price. In fact I could get a dedicated server for a smidgeon over what I currently pay. Hmm.

Pagely. Hmm, judging from their list prices they are even more expensive than WP engine.

Stratagem Hosting is a hosting company but unfortunately they don’t do WordPress hosting. I’m lazy, so the more WordPress the offering is, the better for me.

Cloudflare have some cool stuff but not hosting. And, as it happens, I think Cloudflare are part of the WP Engine stack.

But before considering a move, I figured I should get my house in order. Lower storage and bandwidth. They would make a move both easier and cheaper.

However, I have to admit, the main reason I like WP Engine is because of their brilliant support team. And they have been helping me along in my storage reduction journey. So I’m not in a rush to leave them.


Get paid for my stuff

The above is all about cost. The thing to look at is income. Currently that is zero. But I’ll reserve my thoughts on that for another day.

5 thoughts on “How I am reducing the cost of hosting balagan.info”

  1. An awful lot of work but already paying off. Thanks for making the effort to preserve such a valuable resource.

    Reply
  2. You are a legend mate! Mind you, when discussing some of the things above, I got quite lost in the jargon as I am a bit of a dinosaur when it comes to IT stuff. But thank you for all your amazing work and effort to keep the site afloat, and don’t be backward in asking for some donations to help cover your costs..

    Reply
  3. Thanks guys. It does appear to be paying off. Latest stats are bandwidth is down to 7.4-8.5 GB per day from 15 GB per day. Getting there.

    Reply

Leave a Reply