I previously wrote an article on headscale unlimited scaling, covering how to transform headscale into a clustered deployment with near-unlimited device capacity.

The core idea is to assign each headscale instance a client_id, and have all instances share a single database or database cluster.

Headscale Series: Building a Headscale Cluster for Near-Unlimited Device Capacity

Today I’d like to share how to transform a single-instance headscale into a multi-tenant deployment (this post focuses mainly on showcasing results rather than deep technical explanation).

Multi-Tenant Transformation

Headscale was originally designed as a single-instance, single-tailnet system. To support multiple tailnets within a single instance, the key is to isolate each tenant’s ACL, tailnet, routes, DNS, and more.

tailnet

Each tenant has its own independent tailnet. The default address pool is 10.64.0.0/10, but it can be customized — for example, changed to 192.168.6.0/24 — depending on your requirements.

A default tenant is reserved in the system. Users who do not need multi-tenancy can keep using headscale as before — all users will be automatically assigned to the default tenant.

ACL

Each tenant has a fully independent ACL with access to the complete set of ACL features.

Routes

Each tenant’s routes are also independent and can be configured separately.

MagicDNS

Host FQDNs follow the format hostname.<tenant_key>.<dns.base_domain>, where dns.base_domain is shared across all tenants.

The default tenant uses the default subdomain.

CLI

The command-line tools have been updated to support tenant functionality. Use -t to specify a tenant.

Relay Servers

Each tenant can use the shared relay servers, or configure its own dedicated relay servers for complete isolation from other tenants.

Screenshots

A tenant field has been added to the system to distinguish between tenants.

users

node

node-t

Overview

This article explains how to build and package the Headscale source code yourself, and covers common errors that may occur during the build process along with their solutions.

  • OS: Windows 11 Home (Chinese edition); WSL 2 with Ubuntu 24.04 LTS. The build was performed on Ubuntu 24 installed via WSL 2.
  • Source directory: C:\Users\XXX\Documents\develop\0me\headscale. Source code stored on the Windows filesystem.
  • Source download: https://github.com/juanfont/headscale
  • Version: v0.26.1

Prerequisites

Downloading the Code

It is recommended to clone the repository using git rather than downloading a zip archive. When I tried to build from a zip archive, many errors appeared. I was ultimately able to produce an executable via go build -o headscale ./cmd/headscale, but make build could not complete successfully.

Use the following commands to download the source:

1
2
3
4
git clone https://github.com/juanfont/headscale.git

# Check out the v0.26.1 tag
git checkout -b release-v0.26.1 v0.26.1

Then copy config-example.yaml from the root directory and rename it to config.yaml.

WSL

Enable WSL and install the Ubuntu distribution.

In your Windows user directory (e.g. C:\Users\XXX), create a .wslconfig file with the following content:

1
2
3
4
5
6
7
8
9
[wsl2]
nestedVirtualization=true
ipv6=true
[experimental]
autoMemoryReclaim=gradual # gradual | dropcache | disabled
networkingMode=mirrored
dnsTunneling=true
firewall=true
autoProxy=true

Then restart WSL. This allows the Ubuntu environment to share your host machine’s VPN connection — which is required during the build, as many dependencies need to be downloaded from the internet.

Installing Nix

Install the Multi-user edition of nix using the root account.

1
sh <(curl --proto '=https' --tlsv1.2 -L https://nixos.org/nix/install) --daemon

Reference: https://nixos.org/download/

Building

Switch from the root account to a regular user account before building.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Enter the directory. Windows paths are accessible under /mnt/c/ in WSL
cd /mnt/c/Users/XXX/Documents/develop/0me/headscale

# Set up the build environment
nix develop

make generate
make test
make build

# Check the output
ls -la result/

# Verify the version
cd ./result/bin
./headscale version # prints the version number

Setting up the environment takes a while — please be patient.

Once make build completes successfully, a result directory will appear in the project folder. When viewed from Windows it appears as a 0 KB file, but it is actually a directory in the Linux filesystem.

r
wr

Running

The compiled headscale binary can be run directly on the development machine. The following describes how to copy it to another server and run it there.

  • Target server: Ubuntu 22.04 Server
1
2
3
4
5
6
7
8
9
10
# Inspect the binary using ldd or file
ubuntu@VM-16-7-ubuntu:~/doc/headscale$ ldd headscale
linux-vdso.so.1 (0x00007ffc991a1000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x000079e2a91e5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000079e2a91e0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000079e2a8e00000)
/nix/store/vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000079e2a9201000)
ubuntu@VM-16-7-ubuntu:~/doc/headscale$ file headscale
headscale: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/store/vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66/lib/ld-linux-x86-64.so.2, stripped
ubuntu@VM-16-7-ubuntu:~/doc/headscale$

From the console output above, we can see that headscale requires /nix/store/vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66/lib/ld-linux-x86-64.so.2 at runtime. This path is symlinked to /lib64/ld-linux-x86-64.so.2. For the program to run correctly, you need to manually create the /nix/store/vbrdc5wgzn0w1zdp10xd2favkjn5fk7y-glibc-2.40-66/lib/ directory and copy ld-linux-x86-64.so.2 into it.

In addition, a config.yaml file is required, along with the directories /root/.headscale/ and /var/lib/headscale/, which must be created and granted appropriate permissions.

1
2
3
4
5
6
7
8
9
10
11
12
# Start command: headscale serve

ubuntu@VM-16-7-ubuntu:~/doc/headscale$ sudo ./headscale serve
2025-08-05T17:20:01+08:00 INF Opening database database=sqlite3 path=/var/lib/headscale/db.sqlite
2025-08-05T17:20:01+08:00 INF Using policy manager version: 2
2025-08-05T17:20:01+08:00 INF Starting Headscale commit=6b6daf389bd11624c4036de525740a0568d5f72f-dirty version=6b6daf3-dirty
2025-08-05T17:20:01+08:00 INF Clients with a lower minimum version will be rejected minimum_version=v1.62.0
2025-08-05T17:20:01+08:00 INF github.com/juanfont/headscale/hscontrol/derp/server/derp_server.go:106 > DERP region: {RegionID:999 RegionCode:headscale RegionName:Headscale Embedded DERP Latitude:0 Longitude:0 Avoid:false NoMeasureNoHome:false Nodes:[0xc00051a090]}
2025-08-05T17:20:01+08:00 INF github.com/juanfont/headscale/hscontrol/derp/server/derp_server.go:107 > DERP Nodes[0]: &{Name:999 RegionID:999 HostName:124.232.181.156 CertName: IPv4:124.232.181.156 IPv6:2406:da18:d4c:c000:8d2b:1775:f73f:7c2f STUNPort:3478 STUNOnly:false DERPPort:8081 InsecureForTests:false STUNTestIP: CanPort80:false}
2025-08-05T17:20:01+08:00 INF STUN server started at [::]:3478
2025-08-05T17:20:01+08:00 INF listening and serving HTTP on: 0.0.0.0:8080
2025-08-05T17:20:01+08:00 INF listening and serving debug and metrics on: 0.0.0.0:9090

Packaging as a Docker Image

If you want to package the source into a Docker image, refer to the following link:

https://ownding.com/2025/08/08/headscale%E7%B3%BB%E5%88%97%EF%BC%9A%E5%A6%82%E4%BD%95%E5%B0%86%E6%BA%90%E7%A0%81%E7%BC%96%E8%AF%91%E7%9A%84headscale%E6%89%93%E5%8C%85%E6%88%90docker%E9%95%9C%E5%83%8F/

Build Error Reference

All of the following errors occurred when building from a downloaded zip archive. None of these issues appear when building from a git clone.

Issue 1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Following the official build steps with nix. make test succeeds, but make build fails with the error below:
djc@jetron-djc:/mnt/c/Users/DJC/Documents/develop/0me/headscale-0.26.1$ make build
nix build
error:
while calling the 'derivationStrict' builtin
at <nix/derivation-internal.nix>:37:12:
36|
37| strict = derivationStrict drvAttrs;
| ^
38|

while evaluating the derivation attribute 'name'
at /nix/store/qmm7hgw60vp7vj9lma95hl329d0j3n6n-source/pkgs/stdenv/generic/make-derivation.nix:438:13:
437| // (optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
438| name =
| ^
439| let

(stack trace truncated; use '--show-trace' to show the full, detailed trace)

error: attribute 'dirtyShortRev' missing
at /nix/store/wrx2gzxp6f5sdha4kswnpn6j8sqmfbnk-source/flake.nix:15:41:
14| }: let
15| headscaleVersion = self.shortRev or self.dirtyShortRev;
| ^
16| commitHash = self.rev or self.dirtyRev;
make: *** [Makefile:20: build] Error 1
  • Solution:

Edit the flake.nix file.

Change headscaleVersion = self.shortRev or self.dirtyShortRev; to:

1
2
3
4
5
headscaleVersion = if self ? shortRev 
then self.shortRev
else if self ? dirtyShortRev
then self.dirtyShortRev
else "v0.26.1";

Issue 2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
djc@jetron-djc:/mnt/c/Users/DJC/Documents/develop/0me/headscale-0.26.1$ make build
nix build
warning: Git tree '/mnt/c/Users/DJC/Documents/develop/0me/headscale-0.26.1' is dirty
error: Cannot build '/nix/store/x0151wjd71c3icbvqrjrhmah4039cxrc-headscale-1d8abba-dirty.drv'.
Reason: builder failed with exit code 1.
Output paths:
/nix/store/w1kqv2j36dsixf5pm4v5lwf2wiwkrkhz-headscale-1d8abba-dirty
Last 25 log lines:
> Running phase: buildPhase
> Building subPackage ./cmd/headscale
> buildPhase completed in 52 seconds
> Running phase: checkPhase
>
> ----------------------------------------------------------------------
> FAIL: headscale_test.go:29: Suite.TestConfigFileLoading
>
> headscale_test.go:54:
> c.Assert(err, check.IsNil)
> ... value *fmt.wrapError = &fmt.wrapError{msg:"fatal error reading config file: open /build/headscale246087243/config.yaml: no such file or directory", err:(*fs.PathError)(0xc0003dfcb0)} ("fatal error reading config file: open /build/headscale246087243/config.yaml: no such file or directory")
>
>
> ----------------------------------------------------------------------
> FAIL: headscale_test.go:73: Suite.TestConfigLoading
>
> headscale_test.go:96:
> c.Assert(err, check.IsNil)
> ... value *fmt.wrapError = &fmt.wrapError{msg:"fatal error reading config file: Config File \"config\" Not Found in \"[/build/headscale4241716010]\"", err:viper.ConfigFileNotFoundError{name:"config", locations:"[/build/headscale4241716010]"}} ("fatal error reading config file: Config File \"config\" Not Found in \"[/build/headscale4241716010]\"")
>
> OOPS: 0 passed, 2 FAILED
> --- FAIL: Test (0.00s)
> FAIL
> FAIL github.com/juanfont/headscale/cmd/headscale 0.020s
> FAIL
For full logs, run:
nix log /nix/store/x0151wjd71c3icbvqrjrhmah4039cxrc-headscale-1d8abba-dirty.drv
make: *** [Makefile:20: build] Error 1

This error occurs because the build is interrupted by a test failure. The test fails because it cannot find the configuration file.

Build the binary directly instead:

1
2
3
cp config-example.yaml config.yaml
go mod tidy
go build -o headscale ./cmd/headscale

After a successful build, a headscale binary approximately 80+ MB in size will appear in the project directory.

Issue 3

1
2
root@jetron-djc:~# nix develop
error: experimental Nix feature 'nix-command' is disabled; add '--extra-experimental-features nix-command' to enable it

This error occurs because the nix-command feature in Nix is experimental and disabled by default.

1
2
# Edit the Nix configuration file
echo 'experimental-features = nix-command flakes' >> /etc/nix/nix.conf

Run the above command as root.

Issue 4

1
2
3
root@jetron-djc:~# nix develop
path '/root' does not contain a 'flake.nix', searching up
error: could not find a flake.nix file

This error occurs because nix develop must be run from a directory that contains a flake.nix file. cd into your project directory and run nix develop from there.

Issue 5

1
2
3
4
=== Failed
=== FAIL: hscontrol/db TestConstraints/no-duplicate-username-if-no-oidc-postgres (0.05s)
db_test.go:404: start postgres: initdb: initdb: error: cannot be run as root
initdb: hint: Please log in (using, e.g., "su") as the (unprivileged) user that will own the server process.

This error occurs because PostgreSQL’s initdb command cannot be run as root for security reasons. The test code attempts to initialize a database as root, which is rejected. Run make build as a regular (non-root) user.

Installing Dependencies Manually

1
2
3
4
5
6
# Install Buf
go install github.com/bufbuild/buf/cmd/buf@v1.55.1

# Install Protobuf
sudo apt update
sudo apt install protobuf-compiler

Purpose

To speed up Bing’s indexing of your website, you can proactively submit URLs using IndexNow.

    1. Get an API key at https://www.bing.com/indexnow/getstarted
    1. Place the key .txt file in the website root directory so it can be accessed publicly
    1. Generate a txt file containing all URLs on your site, then submit them using the script
    1. Check the submission status in Bing Webmaster Tools

Script

The following is a shell script for batch-submitting URLs to Bing IndexNow:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
#!/bin/bash

# === Configuration (modify to match your setup) ===
HOST="www.ownding.com" # Your domain
KEY="878777754f5740419ae123455c77d8ca" # Your API key
KEY_LOCATION="http://www.ownding.com/878777754f5740419ae123455c77d8ca.txt" # Key verification file URL
URL_FILE="/xxx/baidu_urls.txt" # Path to your URL file

# === Check if the file exists ===
if [ ! -f "$URL_FILE" ]; then
echo "Error: file $URL_FILE does not exist"
exit 1
fi

# === Read and process the URL list ===
# Filter blank lines, wrap in double quotes, convert to JSON array format
URLS=$(grep -v '^$' "$URL_FILE" | sed 's/.*/"&"/' | paste -sd ',' -)

echo "-------"
echo "-------"
echo $URLS
echo "-------"
echo "-------"

# === Build the JSON request body ===
JSON_BODY=$(cat <<EOF
{
"host": "$HOST",
"key": "$KEY",
"keyLocation": "$KEY_LOCATION",
"urlList": [$URLS]
}
EOF
)

echo ""
echo "-------"
echo "-------"
echo $JSON_BODY
echo "-------"
echo "-------"


# === Send the POST request ===
echo "Submitting URLs for $HOST ..."
RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "https://www.bing.com/indexnow" \
-H "Content-Type: application/json; charset=utf-8" \
-d "$JSON_BODY")

# === Parse the response ===
HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
RESPONSE_BODY=$(echo "$RESPONSE" | sed '$d')

if [ "$HTTP_CODE" -eq 200 ]; then
echo "Submission successful! Bing response: $RESPONSE_BODY"
else
echo "Submission failed! HTTP status code: $HTTP_CODE"
echo "Response body: $RESPONSE_BODY"
exit 1
fi

URL format inside baidu_urls.txt:

1
2
3
4
5
6
7
http://www.ownding.com/2025/06/12/%E5%9C%A8%E6%9C%89%E5%85%AC%E7%BD%91IP%E7%9A%84%E6%83%85%E5%86%B5%E4%B8%8B%E5%A6%82%E4%BD%95%E5%AE%89%E5%85%A8%E5%9C%B0%E8%BF%9B%E8%A1%8C%E8%BF%9C%E7%A8%8B%E6%A1%8C%E9%9D%A2%E8%BF%9E%E6%8E%A5/
http://www.ownding.com/2025/06/12/%E5%9C%A8%E4%BA%91%E7%AB%AF%E9%81%A8%E6%B8%B8%EF%BC%8C%E4%BB%A3%E7%A0%81%E5%A6%82%E9%A3%9E%EF%BC%81%E6%9C%8D%E5%8A%A1%E5%99%A8%E8%BF%9C%E7%A8%8B%E5%BC%80%E5%8F%91%E6%8C%87%E5%8D%97/
http://www.ownding.com/2025/06/11/zlmediakit%E9%87%8D%E5%90%AF%E6%8B%89%E6%B5%81%E9%85%8D%E7%BD%AE%E4%B8%A2%E5%A4%B1%E4%B8%80%E7%A7%8D%E7%AE%80%E5%8D%95%E8%A7%A3%E5%86%B3%E6%96%B9%E6%B3%95/
http://www.ownding.com/2025/06/11/%E7%B3%BB%E7%BB%9F%E9%98%B2%E6%AD%A2%E8%BF%9C%E7%A8%8B%E6%9A%B4%E5%8A%9B%E7%A0%B4%E8%A7%A3%E6%96%B9%E6%B3%95/
http://www.ownding.com/2025/06/10/nginx%E9%85%8D%E7%BD%AEmap%E5%A4%9A%E4%B8%AA%E5%9F%9F%E5%90%8D%E8%BD%AC%E5%8F%91%E5%88%B0%E4%B8%8D%E5%90%8C%E5%90%8E%E7%AB%AF/
http://www.ownding.com/2025/06/10/ubuntu%E6%9B%B4%E6%96%B0%E6%A0%B9%E8%AF%81%E4%B9%A6/

Usage Instructions:

  1. Save the script as submit_to_bing.sh
  2. Update the configuration parameters:
    • HOST: Your website domain
    • KEY: Your Bing IndexNow API key
    • KEY_LOCATION: The URL of your key verification file
    • URL_FILE: The path to your URL file
  3. Grant execute permission:
    1
    chmod +x submit_to_bing.sh
  4. Run the script:
    1
    ./submit_to_bing.sh

OpenClaw is very popular right now. I’ve already deployed and tested it on Ubuntu and Windows Server (taking advantage of free trial promotions from Unicom and Telecom to get one month of server access and tokens).
However, I hadn’t tried it on macOS yet. The problem is I only have one MacBook Air, so a bare-metal deployment on this machine isn’t really an option. Then it occurred to me — Windows has virtual machines, so why not install a VM on the Mac and run macOS inside it? Easier said than done — and it turned out to be extremely simple, just clicking Next all the way through.

1. Install a Virtual Machine

1.1 Download UTM

For running a macOS virtual machine on a Mac, UTM is the go-to choice — no hesitation needed. Download: https://mac.getutm.app/
Note: Do NOT download UTM from the Mac App Store — the store version requires payment. The version from the official website is completely free and functionally identical.

Download

1.2 Install the OS

Once UTM is installed, create a new virtual machine in UTM, select a system image, and proceed with the installation.

utm-app

Port

Port

Port

The OS installation process won’t be covered in detail here — it’s straightforward, just click Next all the way through.

2. Initialize macOS

Setting up macOS inside the VM is the same as setting up a brand-new Mac for the first time — nothing special to explain here.

macOS initialized inside the VM

3. Prevent the MacBook Air from Sleeping and Turning Off the Screen

  1. Keep the MacBook Air plugged into its charger at all times. Don’t worry about damaging the battery — when plugged in, the battery isn’t in use. Go to System Settings → Battery → Battery Health → Enable Optimized Battery Charging. That said, do keep an eye on temperature: running too hot will shorten battery life. A cooling stand is recommended, and in summer, a small desk fan aimed at the machine helps.

  2. The MacBook Air will auto-sleep by default. To prevent this, go to Battery → Options → Enable “Prevent automatic sleeping when the display is off while connected to power adapter”. Also set Wake for network access → Always.

  3. Download Amphetamine from the App Store, and in its Quick Settings check Allow Display to Sleep.

  4. Turn the MacBook Air’s screen brightness all the way down, and let the MacBook keep running in the background.

Install OpenClaw

There are many installation guides available — please refer to other tutorials for the details. I won’t repeat them here.

0%