This project ended up taking a few months longer than I anticipated, but it was worth it. Burn rate has gone down by about 50% and, more importantly in some ways, I can now tell how much of my burn rate is being spent on Check Run Reporter itself and how much is being spent on CI specifically.
A lot of things came with this change:
AWS Service Migrations
Originally, Check Run Reporter lived on Heroku, so when that got too expensive (and I found myself with a boatload of AWS credits), I migrated to EC2, Aurora Postgres Serverless V1, an Application Load Balancer, SQS, Elasticache (for sessions) all in a VPC.
Now, I've been able to get rid of all of that in favor of DynamoDB (for both application data and as a session store), Lambda, and API Gateway. Fun fact: I had been spending a pretty flat rate of $90/month for Aurora. After my first month on DynamoDB, that's down to $35, most of which was the result of mistake that caused about 20 million more writes than I should have made.
AWS Account Isolation
When I first migrated to AWS, I didn't fully isolate my resources. I had eventually setup a separate account for CI, but Check Run Reporter's production instance still operated in the management account. Since I had to migrate data to DynamoDB anyway, this was a good opportunity to also migrate that data to a new dedicated account. It's still part of the same AWS Organization, of course, but I now all that lives in the prod account is Check Run Reporter and a buildkite agent with permission to update the CloudFormation stack. (I'd actually like to remove the agent and grant permission to an agent in the CI account to update the production stack, but that's for another time.)
Less UI on a Slimmer UI Stack
Last year, I spent a fair bit of timing building out UI to present historical results, per branch status, and so forth. In order to support all that UI, I replaced my home-grown React SSR setup with NextJS and GraphQL.
I've come to conclude that, at least for the moment, GraphQL, NextJS, and all that UI was a mistake. Check Run Reporter's key utility is in delivering results to GitHub, not in presenting them on its own site. I was using GraphQL to solve a set of problems that I'd arbitrarily created for myself. I definitely have plans to use it in the future, but much differently. More about that at a later time. At the time, NextJS kinda felt like the only game in town, but it wasn't pleasant to use (to be fair, it was better than what I'd built myself). It comes with a bunch of dependencies that fight with other dependencies (webpack from NextJS and webpack from Storybook did
_not_ get long and I had to switch sass compilers for a while because I have an M1 Macbook and the sass that NextJS wants wouldn't install).
Remix, on the other hand, has been a joy to use. Admittedly, there's a lot less UI now, but I actually migrated everything before deciding to eliminate a bunch of it. It probably took about a day to refactor the bulk of it (and the rest was spent more on the learn-to-read-from-dynamodb side of things than on anything related to remix).
I've mentioned a few times that I removed a lot of UI, but not so much why. Spending last year building UI was mostly a mistake. While Check Run Reporter's main goal is to get you out of your CI logs so that you can just see results on GitHub, there are some compelling features that it to store data. I thought it would be a good idea to be able to view that data in Check Run Reporter, which meant I really spent a bunch of time building database views (poorly) that no one has really used yet. Instead, I've eliminated that UI (at least for now) in favor of sending the results of that stored data to GitHub.
Other changesI don't know exactly how much new code I wrote, but I was able to delete over 110,000 lines of code once I launched and started the cleanup. Nearly all of my (absurdly, organically convoluted) Terraform config is gone in favor of CloudFormation. Over half of my dependencies have been removed.Â
Most notably, however, I'm more relaxed, sleeping better, and excited to move on to some other projects (including things like going outside, though, of course, there are a few things I intend to add to Check Run Reporter in the coming months now that all this yak-shaving is complete).Â
