This page is a summary of how my career has played out, since that's mostly what people are interested in. I've broken it out into sections and reversed the order so that you only need to read as far back as you'd like. To learn more about my other hobbies such as fridge jumping and friend having then I guess you'll need to ask me about that in person.

By 2023 Benthos was popping up everywhere, big corps were running massive deployments of it, and we were finally being noticed by Senpai (the data engineering industry). My rough plan at this stage was to continue living frugally off sponsors, with Benthos being my main focus, monetising Benthos Studio being my second, video games in there somewhere, and my family coming in at priority five or so. Studio was hopefully going to provide enough income for me to start hiring, and we'd no longer run the project sarcastically.

Upon failing I was to seek an acquisition, and it looked very likely to fail even back then. I was kicking butt on product, engineering, and support (all the things I enjoyed), but awful on commercial, sales and legal (all the things I didn't). Hiring in order to plug your gaps is exactly what VC funding is for, and I definitely had the option. However, with very little financial security at the time and a family to look out for, I just didn't fancy gambling everything I had on another potentially decade-long venture. Acquisition plugs those gaps perfectly well and lets daddy access some liquidity (a lil sum sum) in the process.

In 2024, after almost a decade of working on Benthos, I made the decision to sell the company that I'd built to Redpanda. This was a bit earlier than I had originally planned. It started with a wave of orgs cooling off on Kafka Connect, where Benthos is a natural alternative. Rumours of hosted Benthos being sold in various new products were increasing. This was going to be both good and bad. Good in that I'd have an increased pipeline of new users. Bad in that I was already struggling to monetise, and this could make it even harder to do so. I would likely be part-competing and part-partnering with new and bigger players, with very little bargaining power.

Thirdly, companies that I'd been casually interacting with in the past had seen how things were progressing and were more ambitious about acquiring Benthos. An acquisition is a huge commitment for a company willing to adapt their business plan to it, and you want that to be the case as that's how you secure a good deal for everyone. If a company approaches me with a good plan and they're ready to commit to it then it's a lot easier for me to drop what I'm doing and go with it than the other way around.

I had multiple potential deals on the table with lots of factors to consider. Ultimately, Redpanda had a great business plan, a lot of commitment to it, Benthos would remain open source (a mixture of MIT and Apache v2), and, most importantly, they make great swag. It took about two months to make the deal and have the integration ready for a public announcement, which is apparently very fast.

The reception to the acquisition in the Benthos community was mixed, as predicted. The branding changes were particularly harsh, as the goofiness of the Benthos brand and documentation were deeply loved. There were also some issues with licensing. The overwhelming majority of the code remains completely open source, but there are some features present in the repository which are enterprise licensed. The distinction of the licensing wasn't yet well set out within the repository, and that meant for the first few days after the announcement there was a lot of FUD being spread around.

For the most part, people seem happy for me personally and cautiously optimistic for the future of the project. All of our core maintainers were well aware that a move like this was likely, and let's just say they weren't exactly shocked at the announcement. Companies using Benthos were also happy to have an actually serious commercial entity behind it. I have definitely seen some hurt feelings, and one or two negative comments, but none of them have convinced me that this wasn't a good move for all mature adults involved.

My advice for the Negative-Nancys is that if you actually read the contents of the MIT license you'll see it's missing the clause "SOCIAL CONTRACT: THIS MAINTAINER WILL WORK FOR FREE FOREVER ON WHATEVER YOU WANT". Luckily, the remedy is easy, just copy that license and add whatever you like to it. In fact, you'll be shocked to learn that you can actually do that with the code itself. This coincidentally is what open source actually means, as opposed to indentured servitude.

All open source projects need some sort of marketing at the beginning, with word of mouth gradually pitching in afterwards. Up until this point I'd been using the Go subreddit for advertising all of my Go libraries. However, with Benthos I felt that it wasn't getting any attention because no one could understand what it was for. I moved onto the more conventional approach of doing talks at conferences but none of my proposals were accepted, probably because of all the sexual references. Instead, I decided to make a YouTube channel. Try and sensor my gyrating hips now!

It took me a couple of days of effort to put together Building a Resilient Stream Processor in Go, and I shared it on the Go subreddit. The video caught the attention of the Golang Weekly editors and the following week they featured it. Finally, the third lucky break was that the newsletter prompted someone to post the Benthos repo on Hacker News and it made it to the front page. I have no idea why it did so well as none of my own posts ever did (or have since). NOT THAT I AM BITTER, I'M JUST POINTING THIS OUT BECAUSE IT'S INTERESTING.

After my little marketing campaign I'd accumulated a decent measure of vanity metrics such as Github stars, YouTube subscribers, and so on. Finally my mother could be proud, I thought. More importantly, the repository was seeing activity and I was receiving pull requests, issues and other signals that a plurality of organisations were interested in giving it a spin. Mission accomplished, the community was forming, but now came the harder part of keeping that momentum going.

As can be seen in this neat contributors graph, my activity absolutely skyrocketed immediately following the Hacker News feature:

And that's for two reasons. Firstly, I now had an emboldened sense of purpose with the project, it was clear from the positive reception that it could go places and I was very excited to take it there. Secondly, the larger a community gets the more work it requires to keep everyone happy, which I was determined to do. Running Benthos as an open source community was great fun and I reckon I was pretty good at it.

Great, right? The dream, correct? We can do this forever, accurate? Well, the community grew eventually to the point where I had two jobs. Fun jobs, but jobs plural nonetheless. I definitely consider my work to be a happy place, and I'm good at avoiding burn out by balancing some real-life into the mix. However, I thought I'd have a go at making Benthos my only full-time job before hitting my limit. At this point I'd already left Meltwater and was working on vector.dev, another open source project and kind-of-almost-but-not-really competitor to Benthos. That experience was worthy of an entire blog post of its own, but I left towards the end of 2020 with the goal of only doing Benthos related stuff from then on.

The opportunities were plenty. Lots of companies were requesting support, issue bounties, consulting, and the like. Seeking a Venture Capital investment probably would've been the more common path for someone in my situation. I considered it a lot, and consulted a wide range of perspectives on it, and ultimately decided that "meh, nah". Nothing I was doing at the time would make it any harder to raise VC later on. In fact, the opposite was true.

So there I was, constantly managing a chaotic mixture of consulting gigs, support retainers, issue bounties, all alone. At the same time I was managing the growing Benthos community, giving me a constant trickle of fresh users to train. However, I wasn't particularly enjoying the work and it was taking a lot of time away from my open source contributions, which defeated it's purpose. As an escape hatch I started working on Benthos Studio. My plan was to either monetise it on my own for a passive income, or, failing that, I would sell it in its entirety to someone else.

Things were busy and good, but then the unthinkable happened. I procreated. At the time I was wrongly optimistic about being able to juggle all of this stuff. In hindsight, I was destined to panic about my finances and seek a solution that would've put Benthos on the back-burner. Thankfully, at this point I also had a decent base of sponsors, and Synadia stepped in at exactly the right time as the largest. This allowed me to drop lots of the baggage and focus exclusively on my family, and Benthos.

Like many people I learn most effectively when I build stuff, dump it, and start again. Eventually, the quality and ambition of the projects I was creating grew to where I figured they'd be useful as open source software. In exchange I would benefit from some feedback and it'd serve as a good portfolio for future job searches, there was probably an aspect of seeking validation there as well but I'm not ready to unpack that just yet. This also roughly coincided with me starting to experiment with Go as a programming language.

Turns out that, although nerve-racking, I had a great time sharing my code. Coming up with fun logos and mascots neatly overlapped with my desire to draw stupid pictures and force others to acknowledge them. Go at the time was very new and so I found that I could share a library on the Go subreddit and it'd actually get a decent amount of attention. Go me! Now onto something more substantial!

At some point between 2014 and 2015 I found myself at DataSift feeling confident that I could refactor some of our stream processing services in Go for:

• Better flexibility so that platform architectural changes are easier, and they can be tested in isolation
• Configuration schemas that are more powerful and easier to document
• Design from the ground up for transaction based resiliency, making brokering and vertical scaling mechanisms easier to build on top

It was also around that time that DataSift lost access to the Twitter firehose, our main source of income. After an emotionally draining period of redundancies I wasn't sure what would be coming next. I was worried that we were going to need to adapt our platform at a faster rate, and we were going to have fewer people to achieve that with.

My refactor seemed like a great potential solution and I took this as an opportunity to pitch it to my seniors. However, understandably, there wasn't any interest at this time. Instead, I asked if they would be comfortable with me creating this experiment in my own time as a personal project, and I'd make it open source. They gave me their blessing and that's when I created Benthos.

Because this was a personal project I had to write Benthos entirely from scratch, but that also allowed me to unapologetically build it exactly as I wanted it. I would occasionally showcase my progress hoping that one day the team would adopt it. It took around two years to grind them down, where Benthos entered the platform as an experiment. It was a quiet and low-impact success, but we then replaced more and more services over time with it. Each time it allowed us to simplify our platform architecture in ways that would have required code changes in the past. It was still a hobby project but I was having a great time seeing it put to good use.

DataSift was acquired by Meltwater in 2018. They were struggling with data fragmentation and the rough plan was for our platform to serve as a common data streaming infrastructure. In practice, it moved slowly. Getting teams to migrate data technologies is a gargantuan effort and our availability was also a bottleneck. We decided to trial offering Benthos as a temporary solution for select teams, where instead of waiting for our platform they could run a miniature streaming platform of their own.

The first team we trialled ended up loving it, the project took weeks instead of months and when complete they sought to replace other technologies they were maintaining with Benthos. From that point onwards things seemed to escalate very quickly and I found myself acting as a Benthos evangelist, spreading the good word to any teams struggling with data integration issues at Meltwater. It wasn't long before I was being asked to add new features to Benthos as part of my scheduled work, and that's when I truly felt like Benthos was my "job", although only to a small extent.

In 2013 I did my first job-hop into a Software Engineer (C++) role at DataSift, which was a local start up. The salary increase was only modest but what was more important to me was that it was:

• A start up, with a relatively newly built and fast growing platform
• Lots of very smart people
• A fun culture, with fully utilised board games and nerd toys scattered about their office

The job itself was very fun. We were building services that processed live feeds of social media data, the large one being every single Tweet made globally in real time. With that data we performed tasks such as enrichments (language and sentiment analysis, etc), filtering (GBs of customer rule logic) and masking, all performed within milliseconds before being delivered to a wide network of customer feeds.

At the time very few companies were doing this sort of stuff at the volumes and speed that we were. Social media companies themselves were looking at our solutions for inspiration, and we ended up with a fair few partnerships with all of the big ones over the years. Latencies were low, throughput was high, CPU usage was low, memory usage was low, there were metrics for everything, what's not to love? Well, it was almost always the user experience. Services were awkward to experiment with and test in isolation. Configuration was undocumented and hard to discover. Services were tightly coupled making our architecture brittle. Running in production often involved some ritualistic restarts and periodic manual recovery.

My trajectory through DataSift was very similar to my previous job. I joined as the most junior member of our team and consistently did my best to learn, practice and build some awesome stuff. I never turned down a challenge, and every single project that I worked with I made sure to absorb everything that I liked and disliked. In my evenings I would experiment with reproducing the things that I liked and finding solutions for the things that I didn't in my personal projects. Overall, I was learning to adapt clever but complicated engineering solutions into more usable ones, and learning where best to cut optimisations in favour of pragmatism.

I needed money for rent and had abandoned hope for doing a PhD. What else is a graduate to do but pursue employment in this disgusting capitalist society? I felt that my qualifications made me most suited for an Electronic Engineering job. However, every interview I did was an absolute disaster. Like, imagine being asked "how would you fix this?" and they show you a crude drawing of a smurf orgy. I felt even less qualified for Software Engineering jobs but started applying to them out of sheer panic.

In hindsight the problem was scarcity, I felt more comfortable in Electronic Engineering but the jobs were so few that I was only applying for roles that required experience. The places that gave me an interview probably thought they were doing me a favour, but it actually came close to ruining my confidence entirely. Thankfully, the first Junior Software Engineer job I applied for was offered to me. I took it with absolutely no hesitation.

The job was actually quite neat. I was working on an almost decade old desktop application running on Windows, written in a combination of C++ and C#. It was a great place to get my palms sweaty and do some practical learning, and I spent most of my evenings learning and practising on my own projects. Within my first six months the "superstar" developer that had been doing all of the cool green field work resigned. This left a void that I happily filled, becoming the "get this thing done quickly however you want" guy.

It's my nature that any job at this stage would have consumed my life, as I brute forced my way out of the imposter syndrome. However, in this time I discovered that I absolutely loved Software Engineering. Of all the engineering disciplines it has the shortest dopamine loop, the largest freedoms and often the coziest jobs.

I stayed in this gig for roughly two years, long enough to become more than capable in a few languages; C++, C#, Java, JavaScript. I was playing with home servers, phone apps, and confident enough to share an open source project without immediately deleting it. I wouldn't have described myself as anything other than enthusiastic and eager to learn, but I was getting itchy for a more challenging (and higher paid) job elsewhere, so I began looking.

Towards the end of my education I had two jobs. The first was a summer internship as a QA Engineer at Sony, working on their video editing tooling. The job itself was awful. My task was to locate every single text string within the product and create a map describing how to find it in the UI. The purpose was to make life easier for the full-timers when they would later need to verify translations of the product. This one task took the entire summer holiday and was so exhaustingly boring that I (quite unfairly) couldn't bare the thought of a job in QA going forwards.

The second gig was a stint teaching at a summer camp aimed at technologically interested teens, appropriately called Tech Camp. This was probably the most fulfilling job I've ever had, I loved being able to teach in an environment that was primarily about fun, engagement and absolutely chaotic. The kids were always great, there's something about willingly wanting to attend a nerd camp that filtered out the losers. The staff were similarly great, likely for the same reasons. I loved this job so much that I continued doing it in the holidays from my subsequent full-time jobs. I would have happily taught there for free. No one tell Tom.

The furthest back I'm willing to go on this page is my university days. I studied Robotics at the University of Reading because I wanted a mixture of Electronic and Software Engineering, and also robots are cool. I was not a complex person. The degree was pretty good in that it kept me engaged with lots of practical work and gave me a good entry into lots of engineering disciplines.

However, one concern that I felt towards the end was that having a jack-of-all-trades degree made me less qualified for an entry level job than my peers, since there aren't many jobs that would actually mix all of those disciplines into a single role. The reality, as I found much later, was that any degree is equally unable to make you feel qualified for a job you've never had. In fact, having a job often isn't enough to make you feel qualified for that job.

I came very close to being trapped in academia as my third year dissertation resulted in a published paper, or two. I was offered a PhD position provided I could secure the funding, which I felt was the perfect substitution for the terrifying ordeal of having to look for a real job.

Luckily I was terrible at fundraising as I also hated it. I didn't feel particularly conflicted when, after a stressful summer of cold emails and dead ends, I dropped the pursuit and looked for jobs in anything I could find local to Reading, Berkshire.