Diving into the terminal, you will find that lines of command share a surprising kinship with firearm safety. Picture this: every keystroke is a potential bullet, and just as a responsible shooter takes deliberate aim, developers and administrators can adopt a similar mindset to safeguard their digital landscapes. Let's explore the unexpected parallels between command craftsmanship and muzzle discipline, uncovering valuable lessons that go beyond the screen and into the realm of responsible coding practices

Imagine a situation: you're working on a server when disk space starts going low. You check what's using it and find out some directory in `/tmp` has been accumulating junk. All right, let's clear it out.

> `rm: cannot to remove '/tmp/junk': Permission denied`

All right, think *carefully* about what you would do in this situation. What do you need to type?

...

Obviously, it's `sudo rm -rf /tmp/junk`.

But what's *actually* important is **how** you type, not **what** you type.

#### Firearm rule: Flagging

In firearm safety, there's a general rule that you don't point the muzzle at anything you don't intend to completely destroy. If you accidentally do at, some point, have your weapon aimed in an unsafe direction (e.g. at a friendly person) that's called **flagging**. In this post, I propose that typing out dangerous (i.e. irreversible commands) in the traditional fashion (i.e. left-to-right) is the command-line equivalent of 'flagging'.

Whilst typing `sudo rm -rf /tmp/junk`, there is a brief moment where what you have typed is `sudo rm -rf /`, which for the uninitiated, nukes your computer. Now while you'll carry on typing just fine *probably 99.9%* of the time, who knows what freak accident might happen? Maybe your pinky taps the ENTER key too early--maybe your cat jumps onto your keyboard. Uh oh. So here's what I propose as a habit to do instead: type everything without `sudo` first (`rm -rf /tmp/junk`), and write it back in once the command has been constructed. For *extra* added safety, you could also inspect the target file or directory beforehand--this would be the equivalent to confirming that what you're pointing your weapon at is *positively* what you want to destroy.

This isn't limited to linux shells either. 

#### Firearm rule: The "Safety"

Consider SQL commands (particularly **`UPDATE`** and **`DELETE`**). As a contrived example:

```sql
DELETE FROM users
WHERE name IS 'john'
  AND email ILIKE '%john%@example.com';
```

Some tools, like SQL, have a benefit that system shells typically don't have, which is the requirement of the semi-colon at the end--this is equivalent to the safety on a gun. However, firearm safeties should also not be relied upon as an infallible defense against negligent discharges. In this case, prior to completing your **`WHERE`** filters, you essentially have a query threatening to nuke your table. In addition, *even if* you type the full query out, there's still the risk of having an inaccurate **`WHERE`** filter. For this, I propose constructing your query initially with **`SELECT`** instead. The structure of the query remains exactly the same, you don't flag the entirety of your table, and at the end you can get confirmation that the rows you have selected are indeed the rows you are trying to delete.

#### Conclusion

There's a surprising amount out of parallels you can draw between command-line and firearm practices. The two I covered here seem to be the most applicable. There's a lot of additional layers of safety practices you can employ, but I feel like these are the bare minimum you should do in order to not have an incident. Some people might say this is too tedious, but to me it just seems like it should be common sense--and common practice.



tech

guide

The Command Line is a Gun

When you need to utilize a [ Relational Database Management system ]([https://en.wikipedia.org/wiki/Relational_database]) ( RDBMS ) in your application, should you use an [ Object Relational Mapper ](https://stackoverflow.com/questions/1279613/what-is-an-orm-how-does-it-work-and-how-should-i-use-one) ( ORM ) library? 

Well, as usual, the answer is ... <br/>*it depends* (but technically you can't avoid using an ORM anyways)*
<style>
h3 {
    text-align:center;
}
</style>

#### Table of Contents:

* [ Quick Comparison ](#quick-comparison)

* [ Why ORMs Are Evil ](#why-orms-are-evil)

* [ Why I Still Prefer Using Prisma ](#why-i-still-prefer-using-prisma)

* [ So What Should You Use ](#so-what-should-you-use)

* [Conclusion](#conclusion)




### Quick Comparison

For a quick intro, a RDBMS organizes information in tables and columns, which is inherently different than objects in object-oriented programming languages. ORMs seek to expose an interface in which you can interact with the data in the columns/tables using the native structures of the language.

Let's say I have a simple class `User` and I want to store one in the database. This would mean I need to implement the typical CRUD operations for it. For brevity I'll just do the find method.

#### Custom SQL

```python
db = ... # Connect your SQL db
class User:
    name: str
    age: int

    @staticmethod
    def find(name):
        with db.cursor() as cur:
            cur.execute("""
                SELECT * FROM users
                WHERE
                  name = %s
                  AND
                  email IS NOT NULL;
                """,
                (name) # Don't get SQL injected ya'll
            )
            user = cur.fetchone()
        return user

# Create a user
john = User.find("John")
```

Now let's see what it would like using an ORM library, like [Prisma](https://www.prisma.io/) (it's originally written for NodeJS, there's also a [python client](prisma-client-py.readthedocs.io))

#### ORM Libraries

<h5><center>Prisma</center></h5>

```python
from prisma import Prisma
db = Prisma()
# db.user is automatically generated from a prisma.schema file
async def find_user(name: str):
    user = db.user.find_first(
        where={
            "name": "John",
            "NOT": [ {"email": None} ]}
        )
```

However the implementation can vary across libraries. Here's what Django's ORM would look like.
<h5><center>Django</center></h5>

```python
from models import User

john = User.objects.filter(name="John", email__is_null=False)
```

Now, the ORM implementations may seem straightforward enough, but whereas SQL has a standard approach to more complex queries, ORMs generally don't, and just become a wild guessing game where you rely on your language server's autocomplete. The documentation can't cover this, because the arguments and types are totally dependent on the columns of your table--such as its data type and constraints.


### Why ORMs Are Evil

Okay, so maybe not evil, but potentially full of trickery.

The principal issue is due to how we don't know how the library works under the hood, and this can lead to *a lot* of unexpected issues. 

```python \
# This is a quick example of what a model
# looks like in Django for reference
from django.db import models

class Address(models.Model):
    location = models.CharField(max_length=50, blank=True, null=True)  # Field .

class User(models.Model):
    name = models.CharField(max_length=50, blank=True, null=True)  # Field . 
    age = models.IntField(null=False)  # Field . 
    address = models.ForeignKey(Address, to_field="name", db_column="name")
```

**1\. Implicit Behavior Instead of Explicit Defintion** 

When you create a model without a declared primary_key column, Django will automatically create an INT column `id`and make that the primary key. Although having an `id INT PRIMARY KEY` column is pretty standard in RDMBS, I firmly believe that any modifications to the database absolutely need to be explicit. If there is no primary key defined, Django should error fast instead of doing things on the user's behalf unknownst to them.


**2\. Compatibility**

Django's ORM *requires* a single column primary key. I don't see why you would ever not have a primary key, but this also means you can't have a composite primary key. This makes usage very awkward has you have to pretend like one of the fields is a single primary key, and treating the other field(s) in a simple WHERE clause.


**3\. Overfetching/Sub-optimal queries**

If we have something like 
Additionally, when you declare a foreign key like

```python
users = User.objects.filter(age__gt=30)
# No database hit yet

for user in users:
    print(user.address.location)
    # Database hit happens here
```

This will actually create not one query, but 1 query + an additional query for each user. This is because `address` is a foreign field and does not exist in the initial query. You need to use the `select_related` method to include the foreign fields in your query. This issue won't make itself visible until you start getting performance issues, and unless you were already aware of this, it might be a while before you figure out where the performance hit is coming from.
****
In an ironic way, my terrible experience here actually led to me preferring ORMs, because it pushed me to writing my own SQL for a few applications, which had me experiencing how inconvenient that was--leading me back to trying out ORMs. And here's ...

### Why I Still Prefer Using Prisma
 

1. **Convenience**<br/> The prevailing reason is simply because it reduces the amount of boilerplate code I need to write. The benefit of this during early development stages of a project cannot be overstated. There can be many tables to create models after, and it's nice to not have to create or update the interfaces after each minor change.
2. **Manages Database**<br/> Although I'm perfectly capable of managing the database, Prisma allows for a declarative approach to the database schema via its `prisma.schema`. And as a NixOS user you already know anything with "declarative" is going to get my attention. 
3. **Database Layer Abstraction**<br/>What's also exceptionally nice is having your code be pseudo-database agnostic. While SQL is mostly standardized, there can be minor differences between the syntax of the different databases. Furthermore, there are also differences in the syntax of the database structure itself. For example, Postgres uses `AUTOINCREMENT` for its primary keys while MySQL uses "AUTO_INCREMENT". Not having to worry about these syntax differences makes life a lot easier, and also makes it very forgiving when you want to make your application compatible with multiple databases.
4. **Multi-Language Support**<br/> Currently there are Prisma clients for Dart, Go, Python, Rust, and of course, JavaScript/TypeScript. This solves the issue where knowledge of an ORM doesn't transfer (e.g.knowing how to use TailwindCSS will only help you with specifically using TailwindCSS in a NodeJS project.)
5. **Escape Hatch**<br/>This is a pretty prevalent feature across ORM libs, but in a way it's like having the best of both worlds--as you are still free to write raw SQL if your use case isn't covered by the library.


### So What Should You Use

Well, I think it boils down to where you're at and what you're doing.

Here are a few questions to help decide:

1. **Is the ORM tightly coupled with a framework you're using?**<br/>
If you're new to the framework, it's probably better to follow idiomatic practices. So despite my disinclination with Django's ORM, if you're learning Django for the first time, using its ORM will help you understand the framework as a whole better.<br/>
However, if you are experienced, then you should have the judgement to decide if using the ORM truly fits the structure of your project, or if you should opt for an alternate.
2. **How big is your application?**<br/>
I find that the benefits greatly outweight the inconveniences if your application contains more than a handful of data models. This reduces the boilerplate code greatly. In addition, having a declarative source of truth for the database schema is very nice, and allows for easy migration and schema changes.
3. **Do you know exactly what you want to do?**<br/>
If you know *exactly* what your data models should look like, and how you expect them to be updated, then it probably doesn't really matter unless  what you want to do isn't supported by the ORM. I would still recommend using Prisma just for convenience of managing the schema, and you can always use the raw SQL escape hatch if you don't trust the query implementation.
4. **Do you want to support multiple databases?**<br/>
Definitely an ORM, unless you *really* want to implement the appropriate interfaces for each database. 
5. **Is your application data-driven or task-driven?**<br/>
If your application is tied closely to data, it probably makes more sense to write raw SQL as that brings you closer to your data. If you're task-driven, that layer of abstraction will come in nicely and help you with development speed and flexibility in the underlying database technology that you choose to implement.

### Conclusion

While both have their merits, I think for most developers using an ORM library is a no-brainer--it just saves *sooo* much time. And if you use Rust, Dart, TypeScript/JavaScript, Go, or Python then I highly recommend checking out Prisma.

****

\* This is because if you're binding your native SQL to functions and objects, you're basically just making your own ORM library


opinion

On Prisma and Whether You Should Use ORMs

Typical Python usage doesn't call for asynchronous code, but it's nonetheless an essential feature for libraries like discord.py or FastAPI. Even though the implementation of asynchronous code in Python has its merits, every time I am writing it I can't help but wish I was writing JavaScript promises instead. To illustrate the reason behind my preference, let's take a look at some examples. Let's say I'm writing some code that will:

1. Fetch Resource1 from ExternalServer1
2. Fetch Resource2 from ExternalServer2
3. Perform some expensive computation on Resource1 to get Processed1
4. Process Processed1 and Resource2 together to yield FinalResult and return

So let's look at how this might look in Javascript:
```javascript
async function main() {
    const Resource1Promise = fetch("ExternalServer1").then((res)=>{
        return processResource(res)
    })
    const Resource2Promise = fetch("ExternalServer2")

    // This is Processed1 because of the .then() that we invoked that immediately
    // started processing Resource1 as soon as it was retrieved
    Processed1 = await Resource1Promise

    Resource2 = await Resource2Promise
    return combineResources(Processed1, Resource2)
}
main()
```

What I find beautiful about this is that async functions (like `fetch`) can be invoked with or without `await` (the example invokes them without `await`). Either way, the function will start its execution and return a `Promise` object. Then, `await` is used to retrieve the final result of the function call stack-- so if you invoke the async function with `await` then you're basically saying you need the result of the function *now* and this parent function should wait for the async execution to finish before continuing. But the important thing is that the implementation is pretty straightforward:

**Call the async function and it'll start executing. Then `await` it *if* you want the result of the function**

Now let's see at a similar typical python implementation:

```python
import aiothttp
import asyncio
async def main():
    async with aiohttp.ClientSession() as session:
        Resource1Response = await session.get("ExternalServer1")
        Resource2Response = await session.get("ExternalServer2")
        # Assume the content we want is a json, but it could be anything
        Resource1 = await Resource1Response.json()
        Resource2 = await Resource2Response.json()
        Processed1 = processResources(Resource1)
    return combineResources(Processed1, Resource2)
    
asyncio.run(main())
```

Initially, one might think these are equivalent--at least I did when I first started out--but that could be farther from the truth. It's deceptive because of the syntax similarity and brevity. To understand the significance of this implementation, the important thing to focus on is all the `await` keywords that show up. In the python implementation. This means that each line has to finish resolving before the next one is finished. Meaning that even though retrieving Resource1 and Resource2 have nothing to do with each other, we still need to wait Resource1 to resolve first. 

> So why not just leave out the `await`? 

Because that's simply not how Python works. In python, calling `async` functions return a `Coroutine` object. Do not make the mistake that this is equivalent to JavaScript's `Promise`. A coroutine does not start executing its underlying function until it's `await`-ed. 

To get the similar behavior to our JavaScript implementation, we need to utilize the `Task` object.


```python
import aiothttp
import asyncio
async def main():
    async with aiohttp.ClientSession() as session:

        # When invoked with create_task, the data retrieve starts happening immediately
        Resource1Task = asyncio.create_task(session.get("ExternalServer1"))
        Resource2Task = asyncio.create_task(session.get("ExternalServer2"))

        # Assume the content we want is a json, but it could be anything
        Resource1Response = await Resource1Task
        Resource1 = await Resource1Response.json()
        Processed1 = processResources(Resource1)
        Resource2Response = await Resource2Task
        Resource2 = await Resource2Response.json()
    return combineResources(Processed1, Resource2)
    
asyncio.run(main())
```

 In Python's asyncio library, a `Task` is what I know to be the closest thing to a JavaScript `Promise`. Here, the we start requesting the resources from the external servers immediately. This means by the time we're done with processing `Resource1` that `Resource2` might already be ready to be parsed.

> *What's the big deal*?

Well just look at it! It takes up nearly double the lines of code, and adds additional layers of complexity that a beginner might not know about ( how would you know you need to create a Task when your code works just fine when you `await` your async function? ) 

> So what can be done?

Well, I believe a formal solution would involve modifying the python interpreter, so that's a bit out of question. Really, the solution is to just learn the python syntax. However,  I've decided to play around and see if I could create something that would replicate the implementation of the Javascript `Promise`. It's not really meant to be used, but just an experiment to see what is achievable.

First, since a `Task` is pretty close to a `Promise`, all we really need to do is create a wrapper.

```python
import asyncio
import functools
from typing import Coroutine

def promise(func_or_coroutine):
    if asyncio.iscoroutinefunction(func_or_coroutine):
        # If the input is a coroutine function, return a decorator
        @functools.wraps(func_or_coroutine)
        def wrapper(*args, **kwargs):
            return asyncio.create_task(func_or_coroutine(*args, **kwargs))

        return wrapper
    elif isinstance(func_or_coroutine, Coroutine):
        # If the input is a coroutine, wrap it in a task directly
        return asyncio.create_task(func_or_coroutine)
    else:
        raise TypeError("promise requires an async function or a coroutine")

# Usage as decorator:
@promise
async def some_function():
    await asyncio.sleep(1)
    return 1

# Imperative usage:
async def other_function():
    await asyncio.sleep(1)
    return 2
# other_function starts executing immediately
res = promise(other_function())
# get the result
print(await res)

```

So that's pretty sufficient. But then a key feature of a `Promise` is fluent-style chaining, meaning you can declare functions that will be ran successively on the results of the previous functions. You can also add a `.catch` function to handle any errors along the way.

```javascript
actionOne().then(actionTwo).then(actionThree).catch((error)=>errorHandler(error))
```

Since `.then` is a method, we need to create a class to put these methods on. Here is the full code of my implementation:

```python
class Promise:
    def __init__(self, coro):
        self.current_task = asyncio.create_task(coro)
        self.error_handler: Optional[Callable] = None
        self.pending_error = None


    def then(self, func, error_func=None):
        async def continuation(original_task):
            try:
                result = await original_task
                if asyncio.iscoroutinefunction(func):
                    res = await func(result)
                else:
                    res = func(result)
                return res
            except Exception as e:
                self.pending_error = e
                if error_func:
                    return error_func(e)
                if self.error_handler:
                    return self.error_handler(e)
                raise e

        # This critical line doesn't actually work
        if self.pending_error:  
            return self

        original_task = self.current_task
        self.current_task = asyncio.create_task(continuation(original_task))
        return self

    def catch(self, error_func):
        self.error_handler = error_func
        if self.pending_error:
            return self.error_handler(self.pending_error)
        return self

    def __await__(self):
        # Delegate to the internal awaitable coroutine
        return self.current_task.__await__()

# Rewrite the promise function as a wrapper for the class
def promise(func_or_coroutine):
    if asyncio.iscoroutinefunction(func_or_coroutine):
        # If the input is a coroutine function, return a decorator
        @functools.wraps(func_or_coroutine)
        def wrapper(*args, **kwargs):
            return Promise(func_or_coroutine(*args, **kwargs))

        return wrapper
    elif isinstance(func_or_coroutine, Coroutine):
        # If the input is a coroutine, wrap it in a task directly
        return Promise(func_or_coroutine)
    else:
        raise TypeError("promise requires an async function or a coroutine")
```

With this implementation, our old python code now looks like this:

```python
import aiothttp
import asyncio

# Kind of silly but necessary for python implementation, 
# since `.json` is a method on the response, and we don't have access to that
async def get_json(r):
    return await r.json()


async def main():
    async with aiohttp.ClientSession() as session:
        Resource1Promise = promise(session.get("ExternalServer1")).then(get_json).then(processResource)
        Resource2Promise = promise(session.get("ExternalServer2")).then(get_json)

        Processed1 = await Resource1Promise
        Resource2 = await Resource2Promise

    return combineResources(Processed1, Resource2)
    
asyncio.run(main())
```

It's pretty close, but there's a few issues. My implementation of the `.catch` method doesn't work perfectly--when an error is reached, the function chain should stop and the error should be raised. However, I haven't found a solution to checking for an error prior to scheduling a Task. I think this is because the Tasks are all scheduled before they are ran, so the error only happens after scheduling. I'd be delighted to see if anyone could make improvements on this.

Finally, if anyone is reading this that is new to Python, I want to make it clear that *actually* implementing this is probably a no-go since your code will be extremely non-idiomatic and Pythonistas are probably going to raise their pitchforks if they see you write code this way. This was simply an educational experiment to see if I could make the async Python code function similar to JavasScript's, and how much the code would benefit from a theoretical implementation.



python

javascript

exploratory

Promising Python Coroutines

In the <a href="https://dvytran.dev/blog/2023/09/19/x3framework:%20Powering%20NixOS%20with%20docker%20compose-5" target="_blank">previous post</a> I talked about <a href="https://github.com/agentx3/nix-expressions" target="_blank">my nix framework</a> that uses <a href="https://docs.docker.com/compose/" target="_blank">docker compose</a> to run services. In this post, I'll show how to write your own NixOS modules to extend my `x3framework`. It should also give you the fundamental principals in creating NixOS modules in general.

**NOTE**: This particular implementation is for a development environment, and I **advise against directly extending `x3framework` in production** as upstream changes might break your configuration. Instead, I recommend forking my repo and using that as your input url.

**Again, do not directly extend `x3framework` in production** (unless it's from a personal fork)

For this post, I will assume you have already included my flake and imported its `default` or `dockerModules` into your NixOS configuration.

```nix
# flake.nix
{
  inputs = {
    # Other inputs ...
    nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05";
    x3framework.url = "github:agentx3/nix-expressions";
  };
  outputs = {
    # Other inputs ...
    nixpkgs
    , x3framework
    , ...
  }: {
      nixosConfigurations = {
          yourSystem.nixpkgs.lib.nixosSystem = rec {
              system = "your system architecture";
              # ... 
              modules = [
                 # Either 'default' or 'dockerModules' work
                 x3framework.nixosModules.${system}.default
              ];
          };
      };
  }
```

At the core of my system, all that really needs to be done is:
  1. Setup a docker network
  2. Define the compose file
  3. Define your desired options and config output
  4. Configure the system to use the module

#### 1. Setup a docker network:
The docker containers are configured to operate on a docker network. As part of my `dockerModules`, I include an extension for the nixpkgs docker module that allows declarative networks. In my framework, I've included a wrappear for simplicity. To use: 

```nix
# configuration.nix
{config, ...}: 
{
    x3framework.docker.networks.default = {
        enable = true;
        name = "x3framework-net"; # Choose whatever you want here
        # 172.16.0.0 - 172.31.255.255 is a private IP only range so
        # it's smart probably to choose something between there.
        # Avoid 172.17.x.x because that's what docker uses for its
        # default bridge network.
        subnetBase = "172.30.0.0";
        # Allows for 172.30.0.X , but you can modify it per your requirements.
        subnetMaskLength = "24";
    };
}
```

#### 2. Define the compose file:
For this tutorial I'll cover a simpler service that I have a <a href="https://roundcube.net/" target="_blank" rel="noopener noreferrer">Roundcube</a> container.
Let's look at a normal docker-compose.yml for roundcube:

```yaml
# docker-compose.yml
version: '3.9'

services:
  roundcubemail:
    image: roundcube/roundcubemail:latest
    container_name: roundcubemail
    restart: unless-stopped
    volumes:
      - ./www:/var/www/html
      - ./db/sqlite:/var/roundcube/db
    ports:
      - 9002:80
    environment:
      - ROUNDCUBEMAIL_DB_TYPE=sqlite
      - ROUNDCUBEMAIL_SKIN=elastic
      - ROUNDCUBEMAIL_DEFAULT_HOST=tls://mail.example.org
      - ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.example.org
```

YAML relies on line-dentation for parsing, which doesn't work well with variable interpoliation in Nix multi-line strings. Fortunately, YAML is a superset of JSON, so we can use nix's `builtins.toJSON` to generate our config file.

First, make a separate file for your module. Then let's define some variables that will be used often through the configuration and then write the compose file.  Because `roundcube` already exists as part of my framework, I will call this `roundcubeDemo`.

```nix
# roundcubeModule.nix
{config, lib, ...}:
let
  inherit (lib) mkEnableOption mkOption mkIf types mdDoc;
  service = "roundcubeDemo";
  # Define this to be the container IP suffix. 
  # If your subnet mask was /16, then you can also do "X.X" e.g. "12.1"
  IP4 = "234"; 
  cfg = config.x3framework.docker.services.roundcubeDemo;
  # This function merges the subnet with the provided IP suffix
  # E.g. `mkIp 172.30.0.0 12.1` becomes `172.30.12.1`
  mkIp = config.lib.x3framework.mkIPFromSubnetAndSuffix;
  SERVICE_IP = mkIp cfg.network.subnetBase IP4;

  # This will be used specifically just for Roundcube to help us
  # bind additional config files into the container. 
  mkExtraConfig = extraConfig: pkgs.writeTextFile {
    name = "${service}-extra-config.inc.php";
    text = extraConfig;
  };

  composeFile = pkgs.writeTextFile {
    name = "${service}-compose.yml";
    text = builtins.toJSON {
      version = "3.9";
      services = {
        roundcube = {
          restart = "unless-stopped";
          container_name = "${service}";
          image = cfg.image;
          environment = {
            # See https://github.com/roundcube/roundcubemail-docker/blob/master/README.md
            # for explanations of the environment variables
            ROUNDCUBEMAIL_DEFAULT_HOST = cfg.host;
            ROUNDCUBEMAIL_DEFAULT_PORT = toString cfg.port;
            ROUNDCUBEMAIL_SMTP_SERVER = cfg.smtpServer;
            ROUNDCUBEMAIL_SMTP_PORT = toString cfg.smtpPort;
          } // cfg.environment;
          networks."${cfg.network.name}".ipv4_address = SERVICE_IP;
          volumes = [
            "${service}-db:/var/roundcube/db"
            "${service}-config:/var/roundcube/config"
            "${service}-temp:/tmp/roundcube-temp"
            "${service}-html:/var/www/html"
          # These next few aren't strictly necessary, but are useful if
          # if you want to utilize features like sieves and plugins
          ] ++ lib.optionals (cfg.extraConfig != "") [
            "${mkExtraConfig cfg.extraConfig}:/var/roundcube/config/extra-config.inc.php"
          ] ++ lib.optionals (cfg.extraOauthConfigFile != "") [
            "${cfg.extraOauthConfigFile}:/var/roundcube/config/extra-oauth.inc.php"
          ];
        };
      };
      volumes = {
        "${service}-db" = { };
        "${service}-config" = { };
        "${service}-temp" = { };
        "${service}-html" = { };
      };
      networks."${cfg.network.name}".external = true;
    };
  };
in
{
 # Next parts go here
}
```
<br/>
#### 3. Define desired and necessary options
We need to define all the variables that were used in the compose file (anything that's an attribute of `cfg`). Additionally, to be compatible with the `x3framework` the **following options must also be created**:
 1. `containerIp`
 2. `network`

Now, let's create the options in the actual attribute set portion of the file.

```nix
# roundcubeModule.nix
{config, ...}:
let
  # previous section
in
{
  # This part is what creates the new options to be used in the rest of the system configuration
  options.x3framework.docker.services.roundcubeDemo = {
    enable = mkEnableOption "" // {
      description = ''
        Whether to enable the container task. Enable this if the configuration
        is deployed to a NixOS container.
      '';
    };
    containerIp = mkOption {
      type = types.str;
      default = SERVICE_IP;
      description = mdDoc "IP address of the container. Needs to be in subnet of the assigned network";
    };
    smtpServer = mkOption {
      type = types.str;
      description = mdDoc "Location of the smtp server";
    };
    host = mkOption {
      type = types.str;
      description = mdDoc "Location of the host mail server";
    };
    port = mkOption {
      type = types.int;
      default = 143;
      description = mdDoc "Port to use for IMAP. Usually 993 for forced TLS, 143 for STARTTLS";
    };
    image = mkOption {
      type = types.str;
      description = mdDoc "Image to use for the container";
      default = "roundcube/roundcubemail";
    };
    smtpPort = mkOption {
      type = types.int;
      default = 587;
      description = mdDoc "Port to use for SMTP. Usually 465 for forced TLS, 587 for STARTTLS";
    };
    environment = mkOption {
      type = types.attrs;
      default = { };
      description = mdDoc "Additional environment variables to pass to the container";
    };
    envFile = mkOption {
      type = types.nullOr types.str;
      default = null;
      description = mdDoc "Path to an environment file for the docker container.";
    };
    extraConfig = mkOption {
      type = types.str;
      default = "";
      description = mdDoc "Additional configuration to pass to the roundcube php config file";
    };
    extraOauthConfigFile = mkOption {
      type = types.oneOf [ types.path types.str ];
      default = "";
      description = mdDoc "Additional configuration file to pass to the roundcube oauth config file";
    };
    network = mkOption {
      type = config.lib.x3framework.types.x3NetworkType;
      default = config.x3framework.docker.networks.default;
      description = mdDoc "The docker network to use for the container";
    };
  };

  # Now this next part is used to define how the system should be configured based on the options.
  config = mkIf cfg.enable {
    # You can add additional assertions to enforce here. These will cause errors be thrown
    # during the system build process if the assertion doesn't resolve to a boolean true.
    assertions = [
      {
        assertion = cfg.network.enable;
        message = "The framework's docker network must be enabled to use this";
      }
    ];

    # This generates a helper script for root to use as shell alias
    x3framework.helper.${service}.composeFile = composeFile;

    # This generates a systemd unit that automatically starts the container as well as
    # introduces a dependency on the network, so that the container will automatically be 
    # stopped and removed if the docker-network systemd services needs to stop or change.
    # The container will start back up afterwards after the docker-network service unit is re-established.
    systemd.services."${service}-x3framework" = config.lib.x3framework.mkSystemdUnit {
        inherit config service composeFile; network = cfg.network;
    };
  };
}
```

Now to use roundcubeDemo, import the module in your configuration.nix and configure the module options.


#### 4. Configure the system to use the module

```nix
# configuration.nix
{config, ...}:
{
  imports = [
      # If using flakes, this needs to be a relative import
      ./path/to/roundcubeModule.nix
  ]
  # Configure the module
  x3framework.docker.services.roundcubeDemo = {
    enable = true;
    # Use ssl:// prefix if using implicit SSL
    # If using STARTTLS, use tls://
    host = "ssl://<YOUR IMAP MAIL SERVER HERE>";
    port = 993; # Typically 993 for IMAPS, 143 for IMAP
    smtpServer = "ssl://<YOUR SMTP SERVER HERE>";
    smtpPort = 465; # Typically 465 for SMTPS, 587 for STARTTLS
    environment = {
      ROUNDCUBEMAIL_PLUGINS = "archive,zipdownload";
    };
    env_file = "/path/to/envFile.env";
  };
}
```
Do a system rebuild and if all is well, then a systemd unit for the network will start, followed by the systemd unit for the docker container. Roundcube will now be listening on port 80 on its containerIp. So if using nginx, you should proxy_pass to `${config.x3framework.docker.services.roundcubeDemo.containerIp}:80`.

As an final exercise to the reader: see if you can bind the container to a port of your choice on your system's localhost.

Feel free to open pull requests if you have useful docker compose services you'd like to add. Thanks for reading!


nixos

How to: Extend x3framework (and write NixOS modules)

In this article, I'll go over briefly what NixOS is, discuss some problems I encountered with it while trying to setup some services, and how I solved it.

<a target="_blank" rel="noopener noreferrer" href="https://nixos.org/">NixOS</a> is an incredibly powerful tool. It's a declarative OS managed by configuration files, in a way it's like Terraform but on an individual OS level. I've been using it as a daily driver for almost a year now. I'll have to write a dedicated post for NixOS one day. But for the uninitiated, the TL;DR on what makes NixOS so unique, is that essentially the entire OS is managed by json-like configuration files, meaning everything, from users, system packages, nginx settings to filesystems, grub, and even the kernel can be managed via these files. It's great because once you figure out how to setup your desired system, you won't ever have to go through that process again, of trying to remember what you did--it's all there in your config files. Let's move on now  to the <a target="_blank" rel="noopener noreferrer" href="https://github.com/agentx3/nix-expressions">nix "framework"</a> I made. 

Some definitions before we get started:
* **Derivation**: You can essentially consider this an "atomic unit" in Nix. Basically, to make a derivation, you:
    1. Declare all the necessary inputs
    2. Declare all necessary build steps/commands
    3. Declare desired output from the build results
* **Module**: A broad term that technically encompasses any general sort of self-contained nix configuration. However, in conversation, *generally* means a purpose-built nix configuration that specifically adds new options to use in the configuration files.


Nix has an enormous cache of applications and packages ready to be used, and Nix makes it easy(ish) enough to write your own derivations for any packages you want. However, not all packages are meant to be, well, packaged with Nix. Unfortunately, some applications write to files in their source directory as port of normal operations, instead of using an external data/config location to store stateful data. This makes things very challenging, as the `/nix/store` is read-only. An example of this is `services.matomo`, which if I recall correctly, tried to write to one of its config files during normal operation. This made for me, the service unusable.

So, what to do? Download the source and run it directly with PHP? Sounds messy and imperative. Write a Dockerfile or compose file, and run it manually? Clean, but still imperative. What if I told you this was all I needed to get an instance of matomo running?

```nix
# config.nix
x3framework.docker.services.matomo = {
  enable = true;
  envFile = config.age.secrets.matomo-env.path;
  domain = domain;
};
```

Let's explore the module to see what is happening. There's two core files: <a target="_blank" rel="noopener noreferrer" href="https://github.com/agentx3/nix-expressions/blob/main/nixos/modules/dockerModules/services/matomo.nix">matomo.nix</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/agentx3/nix-expressions/blob/main/nixos/modules/dockerModules/services/mkSystemdUnit.nix">mkSystemdUnit.nix</a>. There's a third supplementary file <a target="_blank" rel="noopener noreferrer" href="https://github.com/agentx3/nix-expressions/blob/main/nixos/modules/dockerModules/helper.nix">helper.nix</a> that just adds shell aliases.

Inside `matomo.nix` I have a docker-compose file defined similar to the <a target="_blank" rel="noopener noreferrer" href="https://github.com/matomo-org/docker/blob/master/.examples/nginx/docker-compose.yml">example compose</a> file from matomo. The containers are bound to the newly available declarative docker network, which then get referenced inside the systemd unit generated by `mkSystemdUnit.nix`. All the systemd unit needs to do is spin up and spin down the containers. The extra addition that I made is since the docker network is also tied to systemd, I'm able to reference the container and the network services to each other and establish a dependency hierarchy, enabling a clean tear-down and spin-up of containers and networks as necessary.

```nix
# mkSystemdUnit.nix
{   
   # ...
  requires = [ "docker.service" "docker-network-${network.name}.service" ];
  after = [ "network.target" "docker.service" "docker-network-${network.name}.service" ];
  script = /* bash */ ''
    compose="${composeFile}"
    command="${docker} compose -f $compose -p ${service}"
    $command up -d
  '';
  preStop = /* bash */ ''
    compose="${composeFile}"
    command="${docker} compose -f $compose -p ${service}"
    $command down
  '';
}
```

And for quality of life, my matomo module also creates an nginx virtualhost for the specified domain. 

Using services with Docker might not be the best practice of Nix philosophy since it's not immutable nor exactly reproducible. However, I think it's a fair trade off, since I would rather have a functioning serve that might break later, than a completely non-functional one. 


x3framework: Powering NixOS with docker compose

 *Hey you. You're finally reading this. You were trying to find some good tech content, right? Stumbled right onto that search result, same as me, and that google web crawler over there.*

Boy it's about time I wrote an actual post. Funny enough, I've been more obsessed with constructing my own content platform rather than providing content for said  platform. *The feature creep is real my dudes.*

Anyways, this site will serve as a general platform for me to share thoughts/insights/findings that hopefully, at least one soul will find useful before the heat death of the universe.

Per the title card, I'm a <span style="color: var(--base08)">fullstack developer</span> and <span style="color: var(--base08)">systems administrator</span>. The very beginning of my journey began about 1.5 years ago. It started innocently enough, with Python discord bots. I didn't even know how to disown a shell process. About 6 months of shenanigans later, our (actual) SysAdmin of my non-profit group bailed out. It just happens I was the only one in the position to take over. Probably not very ideal when the person who's supposed to take over barely knows their bash-- made me "grow up real fast", as they say. Crash coursed myself on some containerization with <span style="color: var(--base0C)">Docker</span>, dived into some nginx, bashed myself with <span style="color: var(--base0F)">bash</span>, and developed Stockholm syndrome with <span style="color: var(--base0A)">Nix</span>/<span style="color: var(--base0A)">NixOS</span>. Of course, there's like about half-a-million other things that I learned (*and an infinite amount of things to be learned, I learned*), but I can save that for another day.

If you've read this far, thanks! Have this seal of thanks.

<img style="width: auto; height: 10em" aria-label="A seal of thanks (not the animal)" src="https://res.cloudinary.com/dbtfu4e73/image/upload/v1694424658/sealofthanks_6b989cbacc.jpg"/>





Blogs

Tags