Monkeys love to express themselves and I’m no stranger to the feeling. I started dripping good old fashion fine literature for the world to enjoy back in prep school, when I opened my first blog on blogger¹. Though the concerns of that back-then teenager were, to say the least, of little interest to the rest of humanity, and I certainly hoped no-one would ever find its very existence. I torched it, or so I think. Please don’t try to look for it: I never told you about it.

Since then, I have achieved fame, accumulating more than 10 likes on LinkedIn, that I hope one day to be able to exchange at the internet bank. I’ve been nominated several times by my own mother as the person with the highest grossing blog of all of her kids, and I’m not even a single child! Though I’m the only one with a blog. This blog has existed in one shape or another for 10 years², and it represents well the sedimentation of any IT project that’s been living for that long. I created this monstrosity:

I should preface that I write this as markdown on a phone while pretending to listen to the nonsense my kid is spewing, as. You can tell me by thd autocomplete and bat punctuation. It magically publishes itself on the net, no hands-in involved! Except when it breaks, which is not often. Functionally, this is as close to perfection as this is gonna get, until I’m rich enough to pay for a scribe. And NO, DON’T TELL ME AI WILL DO IT FOR ME, I would never take bread off the mouth of Petrucciano, that beautiful specimen of a human being who doesn’t know me yet, but will one day have the privilege of typing those words in my place.

Back to the topic: How does it work, you ask? How do I accomplish such magic? To understand how I constructed that piece of art, let me tell you the whole story, because it won’t make sense otherwise. Open your shared Spotify account and start that flute piece from Peter and the Wolf, then buckle up.

my blog infra

I started this thing as a WordPress instance on a shared instance that I was already paying to host my emails. That was when Medium was still in its gender neutral parent’s sack. Yes, yes, I could have used blogger, as I always had. But you see, I don’t like doing like everyone else, I’m also cheap, and I don’t like your judging. Stop interrupting me.

I was mainly writing reading notes, and a few posts on stuff I was learning about. I mostly wrote it for myself. But a new thing was getting popular. It was called Jekyll, and static generation from markdown was becoming all the craze.

I told myself: wait a minute: I’m cool, I don’t know anything about ruby nor do I want to learn about it, but why don’t I have that too? So I did my first migration.

The thing with static generation is that you need to generate. Unlike WordPress where you just connect to a website and press publish, with Jekyll you need to write some files, run a command, push files elsewhere. So convenient! I did what any sane person would do: put that on a git repository and set a build pipeline on azure devops. You know how you’ve got too much disposable time as a young parent when you transform what could be a journal and a pen into something that you need to maintain, all that relying on a tech foundation you have no idea about. I ignored every updates coming since I started using Jekyll, unless it stopped being supported by the stuff I use to publish it. I have changed the theme at least 5 times, but god forbid I will ever have to upgrade my Jekyll version. I will take it to my grave.

The other great thing about that: when I setup a new machine where to write my blog, I need to install ruby. Figure out all the issues with gems and all that stuff, give up and just push hoping that I didn’t screw up the formatting. (which I never do). I deployed all of that on a free tier of Azure websites, which even came with a jekyll plugin (or ruby?) and integrated pipelines, so I would just push to the internal git repo of that web app, and it would automatically generate my blog (or not, if it broke). I felt smart. Those were the days!

But then I switched jobs. I started consulting for Microsoft, on all that cool azure stuff. And I became jealous of all my clients, who were doing Kubernetes stuff, who needed to wake up at night to maintain those early unstable versions³. I figured: I have this solution that works for free. I also have this shared host that I have to keep paying for my email anyway and works really well. But on the other hand I also get $200/month of free Azure credits, why am I still running this ancient architecture when I can be cool on beta software that is barely documented?!

So I used some of that credit, and shot myself in both my feet and my face with it. I created an AKS cluster while it was still in alpha, migrated my DNS to use Azure DNS, which was also alpha, all of that on a $200 budget. I felt so great, making my blog into a personal project that required an oncall rotation, with me as the only person in the pool.

And it worked! Ish! I barely had to handle the updates, recreating pools, handling letsencrypt failures all of the time, all of that on top of my pipeline that was also breaking often. Probably barely a couple hours a week. I even built myself a watch dog to make sure my blog was still up… A true success! Until one day I posted this on HN, and it got hugged to death. Or not exactly, instead the little 2 node cluster served very well, albeit with a 20s response time at peak, but the outbound bandwidth costs, and the extra compute credits burnt through my budget like a rabid umpa lumpa with a flamethrower. I ended with 5 days until renewal of my budget, with my most successful blog post so far still pulling on my server ; when suddenly everything turned off. I couldn’t even change my DNS entries because my DNS resource would remain read-only until the end of the month or I enter a credit card number, whichever comes first.

Did I mention I’m a cheap cheat? I’m also rational, I won’t enter my credit card number into a no-limit free vending machine that the crazy people on HN are actively trying to ransack!

So I moved my DNS off of Azure. Lesson learned. And I created another, simpler option on another free subscription I had laying around, using static websites on Azure storage. Don’t ask: these were the days when you could collection free subscriptions. This worked reasonably well, but didn’t support https, so I had to front it with a Traffic Manager and a single free VeriSign ssl certificate, because I love my personal tech as I love my puzzles: simple and with 200 pieces max but constructed vertically for maximum chance of failure.

Then I migrated to Azure Static Apps because my buddy was working on it, and it’s a nice buddy, I like him, so I wanted to please him. I also moved away from Azure Search (oh did I mention search was working with Azure search?) because the preview API I was using stopped working, and I couldn’t be bothered to figure out how my contraption for indexing articles worked. So instead I wrote my own, local search library and pulled it into my blog. I know this one won’t change.

Somehow I managed to once again lock myself out of that sweet, sweet Azure credit. My website became as available as Gladys, my pet cow, when she’s down 5 straight whiskies. And she doesn’t handle alcohol well. She’s a sweet cow though. I was flying dangerously close to my credit limit every month (I had deployed a lot of PoC stuff on that subscription).

I had other things to do by then, and decided to just full circle and go back to my shared host, which, once again, I was already paying for anyway. I already had a pipeline in GitHub (oh yeah, I moved to GitHub at some point because that’s where the cool kids are), I made it point to an SFTP, toggled the checkbox to get let’s encrypt, and that’s where it’s been since then.

But…

The thing with static websites, is that the UI to edit them kinda sucks. Yes, you can use whatever you want to edit the files, but in the end you somehow need to publish it. So it closes a lot of doors. With my setup, I had to use a computer to write. That’s ok, I couldn’t tell you how many I have readily available for me to write without counting. But you see, I have a disease, which goes like this:

I write something, I spend the whole evening crafting it. It’s perfect. It’s 11PM, I publish it, share it, and go to bed. Then I open my phone, and read it. Because like everyone, I like the smell of my own farts, but I don’t like proof reading myself.

And I see a typo in the first paragraph, and one of the pictures is not loading properly.

Shit.

I get out of bed, my wife asks me what’s wrong, I can’t tell her the truth, so I tell her I heard gunshots on the street. Open the computer, fix the typo and the image link, trigger a Jekyll build, which doesn’t work because ruby 2 has been outdated for years. Push. Wait for the pipeline to finish. Check my phone, everything is fine. Tell my wife it’s just a dead person in the street and go back to bed. Crisis averted.

But then…

I think about the smartest, most astute remark on whatever I was writing about. It would be a crime against humanity to keep it for myself, and more importantly, it will preemptively shut-up any arguments that could come up on HN. I will be celebrated for years as someone who knows his shit. So I rush out of the bed, tell my wife (who’s getting angrier by the minute) that I think the shooter is trying to break into the house. Type in the half baked, typo ridden garbage I was thinking about, commit and push, and go back to bed.

Generally I wake up the day after and the post just went through with no upvotes, no comments, no-one gave a shit, because of course no-one ever gives a shit about what I write.

But during counselling, the prison guard mentioned I should probably fix that bit of my blog publishing, and I’m one to respect authority. So I figured: I don’t want to move away from git because that would mean change my pipeline, and probably I’d have to look into something related to ruby, which, you know, who wants that? But also, having to handle your blog like code is not great. After all most of my great ideas come when I’m on the loo or in the shower, and most of the time I don’t carry a git capable device with me in there.

So I settled on that: I have a solution to sync files on my phone with OneDrive (foldersync, it does wonders to my battery life). I have obsidian on my phone. I also have a server at home now. Maybe I can get my server to pull files from OneDrive and commit them to GitHub. And that’s exactly what I did: install OneDrive for linux, get a cron job every 5 minutes that syncs OneDrive, then git add -A && git commit -m "pffrrt" && git push origin youre-not-my-master, done!

morale

If you’re still here, first, I have to wonder what happened to you, and why you are still reading this. I hope it’s not too bad. You’re probably one of the two people who clicked on that link, the other being me. You can stop Peter and the wolf, I was joking. But overall, thank you.

But also, what is the morale of the story? You could conclude that I should be kept as far away from architecture decisions as humanly possible. You could also tell I learned from my lessons. I’d even agree my story somewhat looks like a progression on that meme curve:

But I think you’d be wrong.

All the decisions I took made complete sense. This is not a critical infrastructure for me, it’s a thing I do for fun, I don’t care too much if it doesn’t work. I am the only person with a stake in it, and the only one making decisions and paying the literal price. When I moved to Jekyll, I just liked the idea of static websites. When I moved to kubernetes, I wanted to have a production project going on using the tech I was discussing all the time, and I learned tons of stuff. Then I wanted to use the stuff my friend made, which honestly was coincidently the most appropriate architecture leveraging Azure. The next steps were fun as well, until I settled on something where I can just write. And I can tell you it’s not done yet. Apart from the Jekyll version, which will never change. It’s not so much continuous improvement as the need for constant change. The current solution, which I have used for a little while, is the type of crafty, barely working but surprisingly reliable kind of stuff I need at the moment. Almost like a weird kink.

In a sense, I also find that it represents well the state of any aging architecture you can meet. Some of the decisions were made for the worst reasons, and you, coming after the fact, pay the price for it (even though in my case, I’m the idiot who made said decision). Some decisions were made sensibly, but the reason they were made disappeared since then. Some are made to please a friend. Some are made to improve a resume. Some are made because everyone else is doing it. Some are made by default. And some are made because it’s fun. In the end everything is always entangled, in a compounding worsening way, compensating holes by throwing in more garbage. It’s a miracle anything works, but it does work.

Until you reap it apart and install WordPress again.

notes

Discussion on HN