329 lines
10 KiB
Markdown
329 lines
10 KiB
Markdown
---
|
|
title: Unprivileged deployments with Nix
|
|
date: 2025-04-17
|
|
tags: NixOS
|
|
---
|
|
|
|
> This post assumes familiarity with the Nix ecosystem.
|
|
|
|
Nix and NixOS make managing servers much easier than bespoke scripts or
|
|
complex Ansible playbooks. While I don't think NixOS is ready for
|
|
personal computers, it absolutely makes sense on less dynamic devices
|
|
like servers or embedded-ish machines.
|
|
|
|
|
|
We can use NixOS and nixpkgs to derive a fully defined operating system and services.
|
|
Then, we can:
|
|
|
|
- switch to new configurations live or on next boot
|
|
- build disk images for use as an installer or VM
|
|
- push the new configuration to a remote server using [deploy-rs]()
|
|
|
|
The latter is interesting. With a bit of setup, we can do GitOps with rollback
|
|
and push-deployments. `deploy-rs` is like `nix profile` with helper scripts
|
|
to "activate" the copied closure. Activation typically means running `home-manager switch`
|
|
or switching the NixOS profile.
|
|
|
|
When starting this blog, I wanted to have it deployed automatically like the popular
|
|
SAAS options. This could be pretty simple if I wanted it to be:
|
|
|
|
```bash
|
|
npm run build
|
|
scp -r _site/ blog@my-remote-server:_site/
|
|
```
|
|
|
|
And then have `nginx` use `/home/blog/_site/` as the site root. This works, but doesn't feel
|
|
"Nix" enough.
|
|
|
|
Other users have tied their blog to the NixOS system-level configuration, like
|
|
[this user](https://jeancharles.quillet.org/posts/2023-08-01-Deploying-a-static-website-with-nix.html).
|
|
While I think this is a good solution, it has the issue of relying on root
|
|
access for updating the site. To me, this feels excessive. I should be able to
|
|
deploy my site from a lesser-privileged user. This method also ties
|
|
sysadmin-tasks to our blog, which isn't keeping our concerns separate.
|
|
|
|
# Scaffolding the Solution
|
|
|
|
Let's recap:
|
|
|
|
1. Build the site using a nix derivation
|
|
2. Configure `nginx` to serve from a well-known path (`/var/lib/site/public`)
|
|
3. Copy the derivation to a server using a locked down non-root account.
|
|
4. Symlink `/var/lib/site/public` to our static site in the nix store.
|
|
|
|
I'll do it manually to get an idea of how it should work.
|
|
|
|
## nix build blog
|
|
|
|
I build my blog using [11ty](https://www.11ty.dev), and I use `npm run build`
|
|
to generate the static sources. Nix supports building npm-like packages with
|
|
`buildNpmPackage`. This uses a [Fixed-output
|
|
derivation](https://nix.dev/manual/nix/2.28/store/derivation/outputs/content-address#fixed)
|
|
to store the dependencies of the project if it is stored elsewhere, but I found
|
|
a better method that imports the `package.lock` directly. Then, for the
|
|
`installPhase`, we just copy the built contents to `$out`. I need to add `vips`
|
|
and `pkg-config` as well because 11ty processes my images. You should add any
|
|
other tools you'd need to build your site.
|
|
|
|
The end result looks like this:
|
|
|
|
```nix
|
|
{
|
|
nodejs,
|
|
pkg-config,
|
|
buildNpmPackage,
|
|
importNpmLock,
|
|
vips,
|
|
}: buildNpmPackage {
|
|
name = "myblog";
|
|
version = "unstable";
|
|
buildInputs = [
|
|
nodejs
|
|
vips
|
|
];
|
|
nativeBuildInputs = [
|
|
pkg-config
|
|
];
|
|
npmDeps = importNpmLock {
|
|
npmRoot = ./.;
|
|
};
|
|
npmConfigHook = importNpmLock.npmConfigHook;
|
|
src = ./.;
|
|
installPhase = ''
|
|
mkdir -p $out/public
|
|
cp -ar _site/* $out/public/
|
|
'';
|
|
}
|
|
```
|
|
|
|
Then, running `nix build` I get my site files in `result/public`, which is what we want.
|
|
I put the files in a subdirectory so that we can use `buildEnv` to merge paths later without
|
|
adding random scripts to the site contents
|
|
|
|
## Copy to server
|
|
|
|
This is a little bit silly because the static site has no dependencies. We could just
|
|
use `scp` to copy the output over. `nix copy` goes further and also copies all the runtime
|
|
dependencies to the target, which is useful for actual programs or entire systems.
|
|
We can still use it for practice.
|
|
|
|
```
|
|
nix copy --to ssh://my-server '.#default'
|
|
```
|
|
|
|
When I log into my server, I can grep the store to find the blog:
|
|
|
|
```bash
|
|
$ ls /nix/store | grep myblog
|
|
mqhssdlmg9f03avpajwcqaah2apknl02-myblog
|
|
```
|
|
|
|
|
|
Before we go any further, let's set up the nginx server,
|
|
as well as a well-known path for our website. I'll also
|
|
add a user here that we can use to deploy.
|
|
|
|
|
|
```nix
|
|
{
|
|
config,
|
|
lib,
|
|
...
|
|
}:
|
|
let
|
|
cfg = config.my.static-site;
|
|
sitesDir = "/var/lib/static-site";
|
|
|
|
user = config.users.users.static-site.name;
|
|
group = config.users.groups.static-site.name;
|
|
|
|
in
|
|
{
|
|
options = with lib; {
|
|
my.static-site = {
|
|
enable = mkEnableOption "Enable static site deployments";
|
|
keys = mkOption {
|
|
description = "list of ssh keys to give push access";
|
|
type = with types; listOf str;
|
|
};
|
|
};
|
|
};
|
|
config = lib.mkIf cfg.enable {
|
|
users.users.static-site = {
|
|
inherit group;
|
|
isSystemUser = true;
|
|
# need shell access for deploys
|
|
useDefaultShell = true;
|
|
home = sitesDir;
|
|
openssh.authorizedKeys.keys = cfg.keys;
|
|
};
|
|
|
|
# make this user trusted (spooky)
|
|
# you'll see why we need this in a moment.
|
|
nix.settings.trusted-users = [ user ];
|
|
|
|
|
|
users.groups.static-site = { };
|
|
services.nginx.virtualHosts = {
|
|
"saji.dev" = {
|
|
root = "${sitesDir}/public";
|
|
forceSSL = true;
|
|
useACMEHost = "saji.dev";
|
|
locations."/" = {
|
|
tryFiles = "$uri $uri/ =404";
|
|
};
|
|
};
|
|
};
|
|
# create the base static site directory, owned by the static-site user
|
|
systemd.tmpfiles.settings."static-site" = {
|
|
"${sitesDir}".d = {
|
|
user = user;
|
|
group = group;
|
|
mode = "0755";
|
|
};
|
|
# Create a dummy symlink to /dev/null
|
|
# this will not override an existing symlink, but it will
|
|
# make sure that the nginx configuration is valid
|
|
"${sitesDir}/public".L = {
|
|
argument = "/dev/null";
|
|
};
|
|
|
|
|
|
};
|
|
};
|
|
}
|
|
```
|
|
|
|
|
|
Now we have a scoped user, with an ssh key authorized. It needs a shell so we can actually log
|
|
in remotely.
|
|
The last step is creating that symlink. This is where the concept of "activation" comes into play.
|
|
|
|
|
|
For NixOS, `deploy-rs` activation just calls `switch-to-configuration` to make
|
|
the system change the profile. We can effectively do whatever we want here.
|
|
|
|
|
|
Reading the [custom activator](https://github.com/serokell/deploy-rs/blob/aa07eb05537d4cd025e2310397a6adcedfe72c76/flake.nix#L58C13-L96C17) source:
|
|
```nix
|
|
custom = {
|
|
__functor = customSelf: base: activate:
|
|
final.buildEnv {
|
|
name = ("activatable-" + base.name);
|
|
paths =
|
|
[
|
|
base
|
|
(final.writeTextFile {
|
|
name = base.name + "-activate-path";
|
|
text = ''
|
|
#!${final.runtimeShell}
|
|
set -euo pipefail
|
|
|
|
if [[ "''${DRY_ACTIVATE:-}" == "1" ]]
|
|
then
|
|
${customSelf.dryActivate or "echo ${final.writeScript "activate" activate}"}
|
|
elif [[ "''${BOOT:-}" == "1" ]]
|
|
then
|
|
${customSelf.boot or "echo ${final.writeScript "activate" activate}"}
|
|
else
|
|
${activate}
|
|
fi
|
|
'';
|
|
executable = true;
|
|
destination = "/deploy-rs-activate";
|
|
})
|
|
(final.writeTextFile {
|
|
name = base.name + "-activate-rs";
|
|
text = ''
|
|
#!${final.runtimeShell}
|
|
exec ${final.deploy-rs.deploy-rs}/bin/activate "$@"
|
|
'';
|
|
executable = true;
|
|
destination = "/activate-rs";
|
|
})
|
|
];
|
|
};
|
|
};
|
|
```
|
|
|
|
|
|
This is a bit difficult to parse because there's the whole `__functor` bit.
|
|
Essentially, we use `buildEnv` to add some new scripts to the `base` package,
|
|
which are then used to call `activate`, which can be a shell script or a binary.
|
|
|
|
I'll try a simple custom activation that creates a symlink. Of note is that
|
|
the `$PROFILE` variable points to the path of the `buildEnv` derivation.
|
|
|
|
```bash
|
|
# remove link to old site.
|
|
rm -rf /var/lib/static-site/public
|
|
ln -sn $PROFILE/public /var/lib/static-site/public
|
|
```
|
|
|
|
Then when deploying with `deploy-rs`:
|
|
|
|
```bash
|
|
error: cannot add path '/nix/store/2sad737aglfzmil72phv0j8s34zzmvzi-myblog'
|
|
because it lacks a signature by a trusted key
|
|
```
|
|
|
|
Drat. This makes sense though, since it would be a bit dangerous to allow any old user write access to
|
|
the nix store. We have two options:
|
|
|
|
1. Make `static-site` a trusted user
|
|
2. Create a trusted keypair to sign our closure when it's built.
|
|
|
|
Pick your poison - the keypair mechanism is more secure, as it means that
|
|
the `static-site` user can't upload arbitrary packages.
|
|
|
|
I just want to get this working, so I made `static-site` a trusted user.
|
|
Note that if you wanted to use the keypair instead, `deploy-rs` has a
|
|
[secret environment variable](https://github.com/serokell/deploy-rs/blob/aa07eb05537d4cd025e2310397a6adcedfe72c76/src/push.rs#L131)
|
|
called `LOCAL_KEY` which is a file that contains the signing key.
|
|
|
|
Regardless of the outcome you choose, when you re-deploy, it should work properly.
|
|
|
|
Let's see what happened on the server:
|
|
|
|
```bash
|
|
$ ls /var/lib/static-site/ -lah
|
|
total 20K
|
|
drwxr-xr-x 4 static-site static-site 4.0K Apr 19 23:32 .
|
|
drwxr-xr-x 17 root root 4.0K Apr 21 03:00 ..
|
|
drwxr-xr-x 3 static-site static-site 4.0K Apr 17 19:57 .local
|
|
drwxr-xr-x 2 static-site static-site 4.0K Apr 17 19:57 .nix-defexpr
|
|
lrwxrwxrwx 1 static-site static-site 60 Apr 19 23:32 public -> /var/lib/static-site/.local/state/nix/profiles/mysite/public
|
|
|
|
$ ls /var/lib/static-site/.local/state/nix/profiles/ -lah
|
|
total 40K
|
|
drwxr-xr-x 2 static-site static-site 4.0K Apr 19 23:32 .
|
|
drwxr-xr-x 3 static-site static-site 4.0K Apr 17 19:57 ..
|
|
lrwxrwxrwx 1 static-site static-site 13 Apr 19 23:32 mysite -> mysite-8-link
|
|
lrwxrwxrwx 1 static-site static-site 62 Apr 17 19:57 mysite-1-link -> /nix/store/kbw9mna3934zqj0saz1snw1pbmxi95aq-activatable-myblog
|
|
lrwxrwxrwx 1 static-site static-site 62 Apr 17 19:59 mysite-2-link -> /nix/store/aa0ai7vwv59alfmhrk29frcbipr6iv9f-activatable-myblog
|
|
...
|
|
lrwxrwxrwx 1 static-site static-site 62 Apr 17 21:33 mysite-8-link -> /nix/store/f1qsglj5zm6v0vzlllci3jqsay476d5l-activatable-myblog
|
|
```
|
|
|
|
The chain looks like this:
|
|
|
|
1. `/var/lib/static-site/public` points to the public folder in the `myblog` profile
|
|
2. The profile is itself a link to `mysite-8-link`.
|
|
3. `mysite-8-link` is again a link, this time to a derivation in the nix store.
|
|
|
|
A profile is just a symlink to a derivation in the nix store. One layer of indirection
|
|
exists to make rollbacks easier.
|
|
|
|
|
|
# End
|
|
|
|
I hope this was useful for you. I think non-root deployment is
|
|
under-explored for nix since NixOS makes it easy to reconfigure.
|
|
|
|
As a follow on, you can probably host arbitrary services like this,
|
|
if you used [home-manager](https://nix-community.github.io/home-manager/)
|
|
to manage the user and the systemd services. Or you could hack together
|
|
something similar yourself.
|
|
|
|
ok bai
|