Posts

The past few weeks I’ve been experimenting with DuckDB, and as a consequence I’ve ended up talking about it a lot as well. I’m not going to lie, I really like it! However, experienced programmers will rightly be skeptical to add new technology that overlaps with something that already works great. So why not just use postgres? Well, I really like postgres too, and I think you should consider just using it! But despite both of these technologies being all about tabular data, they’re not really for the same kinds of problems. I think DuckDB is primarily an analysis or ELT tool, and it really excels in this space. postgres can do a lot of the things that DuckDB can do, but not nearly as fast or easily. I wouldn’t want to use DuckDB for a transactional workload, so it’s not going to replace postgres for anything that I use it for. ...

One of my most vivid memories is from the day in my late teens when I first got a contact lens for my left eye. It took a long time to discover that I had poor vision on this eye, you see, like many people, I chose to keep both of my eyes open when I wasn’t sleeping. It was the headaches that sent me to a doctor. I was adamant that I could see well, but when he blocked my right eye, I had the humbling experience of no longer being able to read the biggest letters on the poster. It turned out that my headaches were probably due to my brain working overtime to interpret the world using mostly only one eye. My appointment with an optician was only a few days later, and I got to try a contact lens that same day. ...

A common way that code grows difficult to reason about is increasing the number of things you must keep in your head simultaneously to understand it. Often, this simply happens by adding one attribute, one variable, one column at a time. Some people are gifted with a great capacity for working memory, but most of us aren’t – having to hold the state of 5 variables in your head simultaneously to understand a piece of code may be pushing it far, according to this article from wikipedia: ...

It’s always an exciting day for me when I get access to the source code for an entirely new application I need to work on. How does it look inside, how does it work? Sometimes, there’s some design documentation along with it, or operational procedures, or maybe some developer handbook in a wiki. I do check all of those, but I don’t expect any of those things to accurately describe how the code works, because they tend to change less frequently. It’s also fairly low-bandwidth, it takes a ton of time to ingest technical text. ...

In my post about batch operations, I used the where id = any(:ids) pattern, with ids bound to a JDBC array. I’ve gotten questions about that afterwards, asking why I do it like that, instead of using in (:id1, :id2, ...). Many libraries can take care of the dynamic SQL generation for you, so often you can just write in (:ids), just like the array example. I would still prefer to use the = any(:ids) pattern, and I decided to write down my reasoning here. ...

Throughout a career as a software developer, you encounter many patterns. Some appear just often enough to remember that they exist, but you still need to look them up every time. I’ve discovered that writing things down helps me remember them more easily. This particular pattern is very useful for my current project. So, it’s time to write it down and hopefully commit it to memory properly this time. Although this post is specific to PostgreSQL, I’m sure other databases have the necessary features to achieve the same results efficiently. ...

For this blog post, I’m trying something different. This is a jupyter notebook that I’m using to study some data, and just dumping my brain out in text. If I can easily export this to a format that works with hugo, this might become a common occurrence. For this one, I’m leaving the code in. There isn’t that much of it, but I think it’s fun to show how much visualization per line of code you can get with seaborn and pandas. ...

Database servers are usually long-lived, and important parts of the infrastructure that we build on. We rarely set them up from scratch, because we have to take such good care of them over time. I think this causes a lot of people to think that setting up a database server is some mysteriously difficult ordeal. To be clear, that’s actually true, if you need high availability and a solid recovery point objective. But there are a lot of use cases where that’s overkill, for example short-lived test environments, or CI/CD pipelines. ...

I’ve been busy working on a documentation site for eugene, and I think it’s starting to look pretty good. I wanted to write down some of my thoughts around the process so far, and some of the things I’ve learned. It’s just been a few days since I ported my blog to hugo, so since I was already feeling like I was up to speed on that, I decided I’d try using it for the eugene documentation too. I experimented with a few different setups around the hugo-book theme, but it ended up feeling like I was having to configure a bit too much for my taste. I think it looked really nice, but the setup felt a bit janky and complicated, with having the submodules and everything in the eugene repo itself. I think my problem here is just that doing this with hugo would require me to learn more about hugo than I’m ready for right now. ...

I’ve been using pelican for my blog for a while now, and I don’t really have anything negative to say about it. But for a while, I’ve been wanting a more minimal theme. I ended up on the front page of hacker news a couple of times, and the old theme had my face on all the pages, which made me feel a bit uncomfortable. I was looking at some other themes around the web, and I found PaperMod which I absolutely loved. I followed instructions for how to set it up, and it turns out that it only took me an hour or two to get everything up and running. I guess I have to give credit to where credit is due, both Hugo and PaperMod are really well-made, and easy to use. There’s a lot more stuff that Hugo can do, but I think I have everything I need set up for now. ...