DT Logo

Hi, I'm David.

Here's a little about me...

As a fullstack developer and systems administrator, I craft web applications and manage remote servers and deployments. My general interests span across developer tooling, cybersecurity, and physics. Yet, what truly drives me is knowing the well of knowledge will always remain full, no matter how much I draw from it.
Check out my skills here.

What I'm Up To

  • Learning Lua through Advent of Code
  • Playing The Bazaar
  • Job hunting

Latest Posts

On Prisma and Whether You Should Use ORMs

When you need to utilize a Relational Database Management system ( RDBMS ) in your application, should you use an Object Relational Mapper ( ORM ) library?

Well, as usual, the answer is ...
it depends (but technically you can't avoid using an ORM anyways)*

Table of Contents:

  • Quick Comparison

  • Why ORMs Are Evil

  • Why I Still Prefer Using Prisma

  • So What Should You Use

  • Conclusion

Quick Comparison

For a quick intro, a RDBMS organizes information in tables and columns, which is inherently different than objects in object-oriented programming languages. ORMs seek to expose an interface in which you can interact with the data in the columns/tables using the native structures of the language.

Let's say I have a simple class User and I want to store one in the database. This would mean I need to implement the typical CRUD operations for it. For brevity I'll just do the find method.

Custom SQL

python
    db = ... # Connect your SQL db class User: name: str age: int @staticmethod def find(name): with db.cursor() as cur: cur.execute(""" SELECT * FROM users WHERE name = %s AND email IS NOT NULL; """, (name) # Don't get SQL injected ya'll ) user = cur.fetchone() return user # Create a user john = User.find("John")

    Now let's see what it would like using an ORM library, like Prisma (it's originally written for NodeJS, there's also a python client)

    ORM Libraries

    Prisma
    python
      from prisma import Prisma db = Prisma() # db.user is automatically generated from a prisma.schema file async def find_user(name: str): user = db.user.find_first( where={ "name": "John", "NOT": [ {"email": None} ]} )

      However the implementation can vary across libraries. Here's what Django's ORM would look like.

      Django
      python
        from models import User john = User.objects.filter(name="John", email__is_null=False)

        Now, the ORM implementations may seem straightforward enough, but whereas SQL has a standard approach to more complex queries, ORMs generally don't, and just become a wild guessing game where you rely on your language server's autocomplete. The documentation can't cover this, because the arguments and types are totally dependent on the columns of your table--such as its data type and constraints.

        Why ORMs Are Evil

        Okay, so maybe not evil, but potentially full of trickery.

        The principal issue is due to how we don't know how the library works under the hood, and this can lead to a lot of unexpected issues.

        python
          # This is a quick example of what a model # looks like in Django for reference from django.db import models class Address(models.Model): location = models.CharField(max_length=50, blank=True, null=True) # Field . class User(models.Model): name = models.CharField(max_length=50, blank=True, null=True) # Field . age = models.IntField(null=False) # Field . address = models.ForeignKey(Address, to_field="name", db_column="name")

          1. Implicit Behavior Instead of Explicit Defintion

          When you create a model without a declared primary_key column, Django will automatically create an INT column idand make that the primary key. Although having an id INT PRIMARY KEY column is pretty standard in RDMBS, I firmly believe that any modifications to the database absolutely need to be explicit. If there is no primary key defined, Django should error fast instead of doing things on the user's behalf unknownst to them.

          2. Compatibility

          Django's ORM requires a single column primary key. I don't see why you would ever not have a primary key, but this also means you can't have a composite primary key. This makes usage very awkward has you have to pretend like one of the fields is a single primary key, and treating the other field(s) in a simple WHERE clause.

          3. Overfetching/Sub-optimal queries

          If we have something like Additionally, when you declare a foreign key like

          python
            users = User.objects.filter(age__gt=30) # No database hit yet for user in users: print(user.address.location) # Database hit happens here

            This will actually create not one query, but 1 query + an additional query for each user. This is because address is a foreign field and does not exist in the initial query. You need to use the select_related method to include the foreign fields in your query. This issue won't make itself visible until you start getting performance issues, and unless you were already aware of this, it might be a while before you figure out where the performance hit is coming from.


            In an ironic way, my terrible experience here actually led to me preferring ORMs, because it pushed me to writing my own SQL for a few applications, which had me experiencing how inconvenient that was--leading me back to trying out ORMs. And here's ...

            Why I Still Prefer Using Prisma

            1. Convenience
              The prevailing reason is simply because it reduces the amount of boilerplate code I need to write. The benefit of this during early development stages of a project cannot be overstated. There can be many tables to create models after, and it's nice to not have to create or update the interfaces after each minor change.
            2. Manages Database
              Although I'm perfectly capable of managing the database, Prisma allows for a declarative approach to the database schema via its prisma.schema. And as a NixOS user you already know anything with "declarative" is going to get my attention.
            3. Database Layer Abstraction
              What's also exceptionally nice is having your code be pseudo-database agnostic. While SQL is mostly standardized, there can be minor differences between the syntax of the different databases. Furthermore, there are also differences in the syntax of the database structure itself. For example, Postgres uses AUTOINCREMENT for its primary keys while MySQL uses "AUTO_INCREMENT". Not having to worry about these syntax differences makes life a lot easier, and also makes it very forgiving when you want to make your application compatible with multiple databases.
            4. Multi-Language Support
              Currently there are Prisma clients for Dart, Go, Python, Rust, and of course, JavaScript/TypeScript. This solves the issue where knowledge of an ORM doesn't transfer (e.g.knowing how to use TailwindCSS will only help you with specifically using TailwindCSS in a NodeJS project.)
            5. Escape Hatch
              This is a pretty prevalent feature across ORM libs, but in a way it's like having the best of both worlds--as you are still free to write raw SQL if your use case isn't covered by the library.

            So What Should You Use

            Well, I think it boils down to where you're at and what you're doing.

            Here are a few questions to help decide:

            1. Is the ORM tightly coupled with a framework you're using?
              If you're new to the framework, it's probably better to follow idiomatic practices. So despite my disinclination with Django's ORM, if you're learning Django for the first time, using its ORM will help you understand the framework as a whole better.
              However, if you are experienced, then you should have the judgement to decide if using the ORM truly fits the structure of your project, or if you should opt for an alternate.
            2. How big is your application?
              I find that the benefits greatly outweight the inconveniences if your application contains more than a handful of data models. This reduces the boilerplate code greatly. In addition, having a declarative source of truth for the database schema is very nice, and allows for easy migration and schema changes.
            3. Do you know exactly what you want to do?
              If you know exactly what your data models should look like, and how you expect them to be updated, then it probably doesn't really matter unless what you want to do isn't supported by the ORM. I would still recommend using Prisma just for convenience of managing the schema, and you can always use the raw SQL escape hatch if you don't trust the query implementation.
            4. Do you want to support multiple databases?
              Definitely an ORM, unless you really want to implement the appropriate interfaces for each database.
            5. Is your application data-driven or task-driven?
              If your application is tied closely to data, it probably makes more sense to write raw SQL as that brings you closer to your data. If you're task-driven, that layer of abstraction will come in nicely and help you with development speed and flexibility in the underlying database technology that you choose to implement.

            Conclusion

            While both have their merits, I think for most developers using an ORM library is a no-brainer--it just saves sooo much time. And if you use Rust, Dart, TypeScript/JavaScript, Go, or Python then I highly recommend checking out Prisma.


            * This is because if you're binding your native SQL to functions and objects, you're basically just making your own ORM library

            The Command Line is a Gun

            Diving into the terminal, you will find that lines of command share a surprising kinship with firearm safety. Picture this: every keystroke is a potential bullet, and just as a responsible shooter takes deliberate aim, developers and administrators can adopt a similar mindset to safeguard their digital landscapes. Let's explore the unexpected parallels between command craftsmanship and muzzle discipline, uncovering valuable lessons that go beyond the screen and into the realm of responsible coding practices

            Imagine a situation: you're working on a server when disk space starts going low. You check what's using it and find out some directory in /tmp has been accumulating junk. All right, let's clear it out.

            rm: cannot to remove '/tmp/junk': Permission denied

            All right, think carefully about what you would do in this situation. What do you need to type?

            ...

            Obviously, it's sudo rm -rf /tmp/junk.

            But what's actually important is how you type, not what you type.

            Firearm rule: Flagging

            In firearm safety, there's a general rule that you don't point the muzzle at anything you don't intend to completely destroy. If you accidentally do at, some point, have your weapon aimed in an unsafe direction (e.g. at a friendly person) that's called flagging. In this post, I propose that typing out dangerous (i.e. irreversible commands) in the traditional fashion (i.e. left-to-right) is the command-line equivalent of 'flagging'.

            Whilst typing sudo rm -rf /tmp/junk, there is a brief moment where what you have typed is sudo rm -rf /, which for the uninitiated, nukes your computer. Now while you'll carry on typing just fine probably 99.9% of the time, who knows what freak accident might happen? Maybe your pinky taps the ENTER key too early--maybe your cat jumps onto your keyboard. Uh oh. So here's what I propose as a habit to do instead: type everything without sudo first (rm -rf /tmp/junk), and write it back in once the command has been constructed. For extra added safety, you could also inspect the target file or directory beforehand--this would be the equivalent to confirming that what you're pointing your weapon at is positively what you want to destroy.

            This isn't limited to linux shells either.

            Firearm rule: The "Safety"

            Consider SQL commands (particularly UPDATE and DELETE). As a contrived example:

            sql
              DELETE FROM users WHERE name IS 'john' AND email ILIKE '%john%@example.com';

              Some tools, like SQL, have a benefit that system shells typically don't have, which is the requirement of the semi-colon at the end--this is equivalent to the safety on a gun. However, firearm safeties should also not be relied upon as an infallible defense against negligent discharges. In this case, prior to completing your WHERE filters, you essentially have a query threatening to nuke your table. In addition, even if you type the full query out, there's still the risk of having an inaccurate WHERE filter. For this, I propose constructing your query initially with SELECT instead. The structure of the query remains exactly the same, you don't flag the entirety of your table, and at the end you can get confirmation that the rows you have selected are indeed the rows you are trying to delete.

              Conclusion

              There's a surprising amount out of parallels you can draw between command-line and firearm practices. The two I covered here seem to be the most applicable. There's a lot of additional layers of safety practices you can employ, but I feel like these are the bare minimum you should do in order to not have an incident. Some people might say this is too tedious, but to me it just seems like it should be common sense--and common practice.

              Bash vs Python for Scripting

              I recently got into a small discussion on Hacker News with someone who suggested that "Using python ... is almost always a better choice than trying to write even one loop, if statement, or argument parser in a bash script."

              I've written a fair share of Python and Bash, and it is my opinion that this thought is misguided, and that Bash and Python have their own niches to fulfill, and they both excel in their own domains.

              So if you're someone who writes only Python scripts, maybe this post will encourage you to explore Bash.

              Content

              1. Two Use Cases for Bash
                • Quick and dirty scripts
                  1. External dependency
                  2. Verbosity
                  3. Complexity
                • Interacting with system configuration
              2. Best of Both Worlds

              Two Use Cases for Bash


              Here are two strong use cases for Bash. I was going to write more, but felt that the article was getting too long for what I thought was a relatively straightforward topic.

              1. Quick and dirty scripts
              2. Interacting with system configuration and tools

              Quick and dirty scripts


              If you know exactly what you need to do for a script, and only need to run it once, it just makes sense to make a Bash script. The pipe operator is an extremely powerful operator and massively reduces the amount of text you need to write. Take for example, scraping images off a website.

              This how I would do it in a Bash script.

              bash
                #!/usr/bin/env bash set -euo pipefail URL="https://example.com" curl -s "$URL" \ | pup 'img.myClass attr{src}' \ | grep -v 'myFilter' \ | xargs -n 1 -- wget -P imageDir

                Look at that beauty. Now let's say I want to do it in Python. Maybe this is just a skill issue thing, but this is how I would do it.

                python
                  from requests import get from bs4 import BeautifulSoup import os URL="https://example.com" IMAGE_DIR="imageDir" res = get(URL) soup = BeautifulSoup(res.text, 'html.parser') for img in soup.select('img.myClass'): url = img['src'] if 'myFilter' in url: continue filename = url.split("/")[-1] with get(url, stream=True) as r: with open(os.path.join(SAVE_DIR, filename), "wb") as f: for ck in r.iter_content(chunk_size=8192) f.write(ck)

                  Boy that was a handful! And I didn't even bother to add error handling with try-except!

                  Let's look at some issues:

                  1. External dependency

                  The Python script depends on the external dependency requests and BeautifulSoup. Technically, you could make the same script using the HTMLParser and the http.client that is part of Python's stdlib, but that would make the script even more complex.

                  Aha! You have committed a folly, for you have forgotten the Bash script depends on pup

                  Well, the difference lies in how each dependency is implemented. Firstly, the Python dependencies must be installed per interpreter. Now, unless you're trying to create a development environment for each Python script you write, that means installing the dependency globally.

                  And because these tools do not follow KISS, it is very likely that some time down the road, a version will be released that will break your script.

                  But I'm only using the simple APIs, those aren't going to change!

                  True, but the other issue is down the road, you might need to use a different library that depends on a specific version of these dependencies, and that's going to drop you down into shit creek without a paddle, as they say.

                  pup is a tool made with the UNIX philosophy in mind, and does a very specific job. Breaking changes are hard to fathom here, though I'm sure it's possible.

                  Most tools for the CLI are purpose-built and have minimal dependencies, if any. Whereas the design of Python inherently relies on importing modules, and while it's thankfully not too common, this can cause dependency conflicts to arise.

                  2. Verbosity

                  I don't think I need to say much here.

                  3. Complexity

                  The great thing about tools that follow KISS is that it's hard to get them wrong. The tools are designed for a very specific, common task, and as such their APIs are extremely accessible.

                  Python on the other hand, being an extremely powerful language, comes with a lot of the associated complexity. Everything is an object, and all those objects have a bazillion methods. Who has time to remember all the specific methods and their options? Why is there a res.json() and a req.text and a req.raw and a req.content?

                  And thank god the filter string was a simple string instead of a regex, because now we would have to interact with the re library and its objects and methods.

                  Interacting with system configuration


                  On a Linux system, the command line is King. In any way you want to configure your system, you will be able to do it faster on the CLI than with a GUI. A lot of services and daemons will have CLI tools to interact with them.

                  I cannot fathom why anyone would want to interact with say, cron, docker, nginx, or ssh with Python as a middleman. You would need to use the subprocess module, and then decide between using Popen() versus run(), and in both cases it's a PITA to access STDOUT and STDERR. And forget piping if you're going that route.

                  Best of both worlds


                  Now, what Python has over bash in terms of simple APIs, is data manipulation. For example, working with arrays, maps, and floats just isn't fun in Bash. Let's do something that would really suck, let's square an array of floats! We can employ Python inside our Bash script.

                  bash
                    UGLY_FLOAT_ARR=("1.1" "2.2" "3.3" "10.123") SQUARED_FLOAT_ARR=($(python <<EOF x=[$(printf '%s,' "${UGLY_FLOAT_ARR[@]}"| sed 's/,$//')] print(list([a**2 for a in x])) EOF )) echo "${SQUARED_FLOAT_ARR[@]}" # Output: # [ # 1.2100000000000002, # 4.840000000000001, # 10.889999999999999, # 102.47512899999998 # ]

                    Ah, nice to see you again my old friend floating-point inaccuracies. Now, it's also true that you could also embed bash commands inside a Python script like so:

                    python
                      import subprocess bash_command = """ cat /etc/os-release | grep -o "^NAME=.*" """ result = subprocess.run( bash_command, shell=True, capture_output=True, text=True ) print(result.stdout) # Output: # NAME=NixOS

                      And I think that works decently, too. I haven't explored Bash-in-Python as much, so it's possible there are some additional benefits here that I'm not recognizing, but I believe the most significant factor boils down to whether most of your script is suited for native Bash or native Python. Any how, hopefully this helps some people in their scripting endeavors.

                      Stop the AI Fearmongering Please

                      It has been over 4 years since the release of ChatGPT 3--arguably the catalyst responsible for popular interest in generative AI--and it remains more polarized than it should be.

                      Before I start, I want to make it clear I am by no means an "AI bro", nor do I (at the time of writing) have any financial associations with a business in the AI space. There's been a lot of arguments against the usage of AI (despite the fact that it's not going away), and while some of them are valid critiques, many of them just seem more representative of some form of luddism, tribalism, or elitism.

                      The list of arguments I address is by no means comprehensive, but it is representative of some of the arguments I find most egregious or am just tired of seeing.


                      Counterpoint: Generative AI takes jobs away from artists

                      First off, this is peak Luddism. That is not to say it isn't true, there have been many people displaced from their careers by AI, especially writers, and they all have my deepest sympathy.

                      But, consider that humans have been innovating for thousands of years now. It's the same song and dance we have been through as a civilization multiple times. Technological breakthroughs always end up displacing some careers, but they end up creating new ones.

                      But this is unprecedented

                      Big innovations are always unprecedented, that's what makes them innovations.

                      An extremely similar event was the invention of the printing press, which eliminated the need for scribes altogether. I hope we're all in consensus that this was a good thing.

                      Secondly, you can't make this argument while also saying that generative AI works are inferior (without further clarification)

                      Let's imagine a simple, but realistic scenario: An artist makes some amount of money doing commissions for people to make posters or avatars, or whatever. Now that people can generate an infinite amount of anime profile pictures, that will of course, reduce some amount of orders that artist will get.

                      And now two sub-scenarios:

                      1. The GenAI content is superior to the artist's works

                      It's generaly agreed that this is not true, but suppose that it was--what's the problem here? Is it reasonable to expect someone to pay more for an inferior product? I don't think so.

                      2. GenAI content is inferior to the artist's works

                      Okay, so now what's the issue? That people aren't paying for something they don't need? This would be like being Ferrari and getting mad at Honda because they're making cheaper, more affordable vehicles. People might not need the quality of hand-made art. This is not even a novel issue, people already buy mass manufactured art.

                      If you have a superior product, then you shouldn't get upset that a cheaper alternative is now available to others. Your market of high-quality art isn't threatened. Yeah, you're losing some customers, but those customers were only customers because they didn't have an alternative. To get upset at this is straying close to gate-keeping art. And speaking of Generative AI content being inferior ...


                      Counterpoint: Generative AI Content Has No Soul

                      You're right. It doesn't. But I don't think that's a problem. Believe it or not, I completely understand how meanings and emotions that enriches art. I took a performance art class in college that changed how I perceived art completely.

                      Yeah, when an AI makes a picture of a sunset, it doesn't come from a sense of awe or appreciation of nature, and thus the created work is void of emotional depth. But sometimes, people just want a pretty picture. The human story behind an art piece is a significant component of the piece, but it's not all of it. A picturesque scene of a meadow can still be picturesque even if it didn't come from a person.

                      It's also ridiculous that sometimes I see people that ask whether AI was used to produced media in some video or game, and then if they find out that it does, then they exclaim something along the lines of "Oh, that explains why the art seemed so generic and lifeless". If you had to ask, that means you couldn't tell. And if you couldn't tell, why does it matter?


                      Counterpoint: Generative AI can be used maliciously to disseminate harmful information

                      You know what? This is actually pretty valid. But still, my response is "and?". What's the goal here? If you're saying this to spread awareness and advocate for more research and development in fighting misinformation, then great! We're on the same side.

                      But I see some people use this as an argument against the existence of Generative AI completely. They're probably a minority, in which case, I'm probably wasting my time, but whatever, I see it enough that I'm sick of seeing it.

                      Reality check: Generative AI is not going anywhere.

                      The invention of online services like emails and banking made it easier than ever to scam people.

                      Did we get rid of emails? No, we create spam filters and advocate for public awareness.

                      Did we get rid of online banking? No, we implemented additional safety checks to mitigate the issue.

                      Did any of those get rid of the issue completely? No, but we still use them.

                      Same goes for Generative AI works. Let's focus on thinking up counter-measures and systems to protect against the spread of misinformation. That's the solution. You don't throw the baby out with the bath water.


                      Conclusion

                      Generative AI content is not leaving anytime soon. As with anything in the world, it has its pros and its cons. But it's a nothing but a pipe dream to get rid of it.

                      I see so much fearmongering or doom-speaking about how its drawbacks are going to be the end of human creativity.

                      FOSS Highlight: ntfy

                      I want to quickly share a little gem of a tool that I have been using for some time now. It's a cute, simple notification tool, and I think it's invaluable for anyone that does any form of self-hosting, but could potentially also be useful in small organizations.


                      What is ntfy ?

                      In simple terms, it's a utility that will let you send and receive notifcations via HTTP(S). You can create different "topics" (basically feeds) to subscribe to, and impose access control on each topic using accounts and api-keys (or forego it altogether, and have it be a public channel). You can learn more about the project on its website.

                      It has a bunch of delivery mechanisms you can hook it up to as the web app, text messages, phone calls, and my favorite, the mobile app.

                      The Mobile App

                      What makes ntfy worth having for me, has to be the fact that it has a mobile application that can be found on both Android and iOS platforms.

                      This means you can hack any of your existing services to use ntfy and delivery push notifications to your mobile device, giving a quasi-native app experience.

                      There is one caveat, though. If you're reading this, you're most likely interested in running the service yourself. This means that you will need to re-compile the mobile applications yourself.

                      Well, technically you don't have to. But an important aspect of notifications is their availability and punctuality. Who wants a notification that's an hour late?

                      The TL;DR is, you need to compile it yourself in order to let your device use dedicated polling utilities to efficiently listen for incoming notifications.

                      Some Usecases

                      Here's some ideas for how to use ntfy:

                      1. Server health checks
                      2. Custom email notifications
                      3. Content feed monitoring
                      4. Bonus: My absolute favorite way to use ntfy is as a cross-platform clipboard.
                      Current visitors: 1