Python offers a ton of ways like os.system or os.spawn* to create new processes and run arbitrary commands in your system. However, the documentation usually encourages you to use the subprocess module for creating and managing child processes. The subprocess module exposes a hig … | Continue reading
The current title of this post is probably incorrect and may even be misleading. I had a hard time coming up with a suitable name for it. But the idea goes like this: sometimes you might find yourself in a situation where you need to iterate through a generator more than once. Su … | Continue reading
Around a year ago, I ditched my fancy Linux rig for a beefed-up 16" MacBook Pro and ever since, it’s been my primary machine for both personal and work stuff. I love how this machine strikes a decent balance between power and portability. However, I often joke that this chonky bo … | Continue reading
Over the years, I’ve used the template pattern across multiple OO languages with varying degrees of success. It was one of the first patterns I learned in the primordial hours of my software engineering career, and for some reason, it just feels like the natural way to tackle man … | Continue reading
One major drawback of Python’s huge ecosystem is the significant variances in workflows among people trying to accomplish different things. This holds true for dependency management as well. Depending on what you’re doing with Python—whether it’s building reusable libraries, writ … | Continue reading
I was watching this amazing lightning talk by Karla Burnett and wanted to understand how traceroute works in UNIX. Traceroute is a tool that shows the route of a network packet from your computer to another computer on the internet. It also tells you how long it takes for the pac … | Continue reading
Recently, I purchased a domain for this blog and migrated the content from rednafi.github.io to rednafi.com. This turned out to be a much bigger hassle than I originally thought it’d be, mostly because, despite setting redirection for almost all the URLs from the previous domain … | Continue reading
Cloudflare absolutely nailed the serverless function DX with Cloudflare Workers. However, I feel like it’s yet to receive widespread popularity like AWS Lambda since as of now, the service only offers a single runtime—JavaScript. But if you can look past that big folly, it’s a de … | Continue reading
This weekend, I was working on a fun project that required a fixed-time job scheduler to run a curl command at a future timestamp. I was aiming to find the simplest solution that could just get the job done. I’ve also been exploring Google Bard recently and wanted to see how it s … | Continue reading
I needed a way to sort a Django queryset based on a custom sequence of an attribute. Typically, Django allows sorting a queryset by any attribute on the model or related to it in either ascending or descending order. However, what if you need to sort the queryset following a cust … | Continue reading
I recently gave my blog a fresh new look and decided it was time to spruce up my GitHub profile’s landing page as well. GitHub has a special way of treating the README.md file of your repo, displaying its content as the landing page for your profile. My goal was to showcase a bri … | Continue reading
One of my favorite pastimes these days is to set BingChat to creative mode, ask it to teach me a trick about topic X, and then write a short blog post about it to reinforce my understanding. Some of the things it comes up with are absolutely delightful. In the spirit of that, I a … | Continue reading
Whenever I need to deduplicate the items of an iterable in Python, my usual approach is to create a set from the iterable and then convert it back into a list or tuple. However, this approach doesn’t preserve the original order of the items, which can be a problem if you need to … | Continue reading
I needed to compare two large directories with thousands of similarly named PDF files and find the differing filenames between them. In the first pass, this is what I did:Listed out the content of the first directory and saved it in a file:ls dir1 > dir1.txt Did the same for the … | Continue reading
Whenever I need to whip up a quick command line tool, my go-to is usually Python. Python’s CLI solutions tend to be more robust than their Shell counterparts. However, dealing with its portability can sometimes be a hassle, especially when all you want is to distribute a simple s … | Continue reading
When writing shell scripts, I’d often resort to using hardcoded ANSI escape codes to format text, such as:#!/usr/bin/env bash BOLD="\033[1m" UNBOLD="\033[22m" FG_RED="\033[31m" BG_YELLOW="\033[43m" BG_BLUE="\033[44m" RESET="\033[0m" # Print a message in bold red text on a yellow … | Continue reading
Whenever I plan to build something, I spend 90% of my time researching and figuring out the idiosyncrasies of the tools that I decide to use for the project. LLM tools like ChatGPT has helped me immensely in that regard. I’m taking on more tangential side projects because they ar … | Continue reading
In multi-page web applications, a common workflow is where a user: Loads a specific page or clicks on some button that triggers a long-running task. On the server side, a background worker picks up the task and starts processing it asynchronously. The page shouldn’t reload while … | Continue reading
I’ve always had a vague idea about what Unix domain sockets are from my experience working with Docker for the past couple of years. However, lately, I’m spending more time in embedded edge environments and had to explore Unix domain sockets in a bit more detail. This is a rough … | Continue reading
While working on a multithreaded socket server in an embedded environment, I realized that the default behavior of Python’s socketserver.ThreadingTCPServer requires some extra work if you want to shut down the server gracefully in the presence of an interruption signal. The inten … | Continue reading
I was working on a project where I needed to poll multiple data sources and consume the incoming data points in a single thread. In this particular case, the two data streams were coming from two different Redis lists. The correct way to consume them would be to write two separat … | Continue reading
Consider this iterable: it = (1, 2, 3, 0, 4, 5, 6, 7) Let’s say you want to build another iterable that includes only the numbers that appear starting from the element 0. Usually, I’d do this: # This returns (0, 4, 5, 6, 7). from_zero = tuple(elem for idx, elem in enumerate(it) i … | Continue reading
I needed to write a socket server in Python that would allow me to intermittently pause the server loop for a while, run something else, then get back to the previous request-handling phase; repeating this iteration until the heat death of the universe. Initially, I opted for the … | Continue reading
Back in the days when I was working as a data analyst, I used to spend hours inside Jupyter notebooks exploring, wrangling, and plotting data to gain insights. However, as I shifted my career gear towards backend software development, my usage of interactive exploratory tools dwi … | Continue reading
I was working with a table that had a similar (simplified) structure like this:| uuid | file_path | |----------------------------------|---------------------------| | b8658dfc3e80446c92f7303edf31dcbd | media/private/file_1.pdf | | 3d750874a9df47388569a23c559a4561 | media/private/ … | Continue reading
At my workplace, I was writing a script to download multiple files from different S3 buckets. The script relied on Django ORM, so I couldn’t use Python’s async paradigm to speed up the process. Instead, I opted for boto3 to download the files and concurrent.futures.ThreadPoolExec … | Continue reading
The colon : command is a shell utility that represents a truthy value. It can be thought of as an alias for the built-in true command. You can test it by opening a shell script and typing a colon on the command line, like this: : If you then inspect the exit code by typing $? on … | Continue reading
Django has a Model.objects.bulk_update method that allows you to update multiple objects in a single pass. While this method is a great way to speed up the update process, oftentimes it’s not fast enough. Recently, at my workplace, I found myself writing a script to update half a … | Continue reading
I’ve just migrated from Ubuntu to macOS for work and am still in the process of setting up the machine. I’ve been a lifelong Linux user and this is the first time I’ve picked up an OS that’s not just another flavor of Debian. Primarily, I work with Python, NodeJS, and a tiny bit … | Continue reading
TIL that you can specify update_fields while saving a Django model to generate a leaner underlying SQL query. This yields better performance while updating multiple objects in a tight loop. To test that, I’m opening an IPython shell with python manage.py shell -i ipython command … | Continue reading
At my workplace, while working on a Lambda function, I noticed that my Python logs weren’t appearing on the corresponding Cloudwatch log dashboard. At first, I thought that the function wasn’t picking up the correct log level from the environment variables. We were using serverle … | Continue reading
Python makes it freakishly easy to load the whole content of any file into memory and process it afterward. This is one of the first things that’s taught to people who’re new to the language. While the following snippet might be frowned upon by many, it’s definitely not uncommon: … | Continue reading
After reading Simon Willison’s amazing piece on how he adds new features to his open-source softwares, I wanted to adopt some of the good practices and incorporate them into my own workflow. One of the highlights of that post was how to kick off a feature work. The process roughl … | Continue reading
My grug brain can never remember the correct semantics of quoting commands and variables in a UNIX shell environment. Every time I work with a shell script or run some commands in a Docker compose file, I’ve to look up how to quote things properly to stop my ivory tower from cras … | Continue reading
TIL that returning a value from a function in bash doesn’t do what I thought it does. Whenever you call a function that’s returning some value, instead of giving you the value, bash sets the return value of the callee as the status code of the calling command. Consider this examp … | Continue reading
While working with GitHub webhooks, I discovered a common pattern a webhook receiver can adopt to verify that the incoming webhooks are indeed arriving from GitHub; not from some miscreant trying to carry out a man-in-the-middle attack. After some amount of digging, I found that … | Continue reading
While going through the documentation of Python’s sqlite3 module, I noticed that it’s quite API-driven, where different parts of the module are explained in a prescriptive manner. I, however, learn better from examples, recipes, and narratives. Although a few good recipes already … | Continue reading
TIL from this video by @codewithanthony that Python’s urllib.parse.urlparse is quite slow at parsing URLs. I’ve always used urlparse to destructure URLs and didn’t know that there’s a faster alternative to this in the standard library. The official documentation also recommends t … | Continue reading
Python has a random.choice routine in the standard library that allows you to pick a random value from an iterable. It works like this:# src.py import random # The seed ensures that you'll get the same random choice # every time you run the script. random.seed(90) # This builds a … | Continue reading
Over the years, I’ve used Python’s contextlib.ExitStack in a few interesting ways. The official documentation advertises it as a way to manage multiple context managers and has a couple of examples of how to leverage it. However, neither in the docs nor in GitHub code search I co … | Continue reading
While reading the second version of Brian Okken’s pytest book, I came across this neat trick to compose multiple levels of fixtures. Suppose, you want to create a fixture that returns some canned data from a database. Now, let’s say that invoking the fixture multiple times is exp … | Continue reading
I was reading Ned Bachelder’s blog Why your mock doesn’t work and it triggered an epiphany in me about a testing pattern that I’ve been using for a while without being aware that there might be an aphorism on the practice.Patch where the object is used; not where it’s defined.To … | Continue reading
I just found out that you can use Python’s unittest.mock.ANY to make assertions about certain arguments in a mock call, without caring about the other arguments. This can be handy if you want to test how a callable is called but only want to make assertions about some arguments. … | Continue reading