Self-hosting RustDesk: a remote-desktop server you run yourself

RustDesk is an open-source remote desktop tool ^[1]. Unlike TeamViewer or AnyDesk, the part that brokers connections, the server, can run on hardware you already own. I run mine on a Linux PC at home, so reaching my desktop never routes through a company's infrastructure and costs nothing beyond the electricity the PC was already drawing.

This is a write-up of what that server is: the two daemons it runs, how a connection forms between two devices behind home routers, and the two ways to expose it. The whole thing installs from one script.

hbbs and hbbr: the two halves

The server is two small daemons, both from the rustdesk/rustdesk-server image.

hbbs is the ID and rendezvous server, the matchmaker. Every client registers with it on startup. When your phone goes looking for your PC, hbbs is what introduces them. It also handles NAT-type detection and carries the registration and signalling traffic.

hbbr is the relay. It forwards a session's traffic only when the two devices can't form a direct link. Most sessions never touch it, which is the point.

In Compose, hbbs runs hbbs -r ${RUSTDESK_RELAY_HOST}:21117 and depends on hbbr. Both mount ./data, restart unless-stopped, and run a dependency-free /proc/net/tcp liveness probe so status can tell a healthy listener from a process that is up but wedged.

How a connection forms

A direct connection is the goal; the relay is the fallback. The mechanism that makes the direct path possible is NAT hole-punching.

Your phone and your PC are each behind a home router, and neither router accepts unsolicited inbound connections. hbbs hands each side the exact public address and port the other one's router is currently using, the temporary opening its NAT assigned. Both devices then fire a packet at that address at the same moment. An outbound packet briefly holds the router's own port open for a reply, so the two packets cross in the middle, each walking through the hole the other just made. From that point the stream is direct, peer to peer.

Hole-punching runs over UDP on port 21116. Without that one port open on both ends, the two devices never get to trade the addresses their routers opened, so the direct path is never attempted and every session falls back to hbbr. On a fast line that looks like one or two frames a second. The session does not fall back because the direct path was slow; it falls back because it was never tried.

Hole-punching has a known failure case: a symmetric NAT assigns a fresh external port for every destination, so the address hbbs advertised is already stale by the time your peer aims at it. The packet lands on a closed port and the session relays instead.

Two ways to expose it

You choose how clients reach the server before anything installs. The deciding question is whether devices that aren't yours ever need to connect. The wizard asks once and you can switch later by re-running setup.

Public-IP mode

Public mode reaches your server from any network, at the cost of opening ports and putting your home IP in a public DNS record. Three things decide whether it works.

A real public IP

This is the step most guides skip, and the reason a lot of home setups never connect at all.

curl -s https://api.ipify.org

Compare that against the WAN IP in your router's admin page. If they match, the internet can reach you. If they differ and you see a private address like 10.x or 100.64.x, your ISP has you behind carrier-grade NAT, sharing one public address across many customers. Port forwarding cannot punch through that.

Cloudflare in DNS-only mode

Cloudflare's orange-cloud proxy looks like protection you'd want. For RustDesk it has to be off.

The orange cloud is a reverse proxy. It terminates the connection itself, reads it as an HTTP request, and builds a fresh one to your server. RustDesk's handshake is raw TCP and UDP with no HTTP in it, so the proxy finds bytes it can't parse and drops them without a word. UDP it can't carry at all, which is the same reason a Cloudflare Tunnel won't work. The DDNS script enforces this by writing the record with proxied: false.

The trade-off of grey cloud is that your home IP is visible to anyone who resolves the record. There's a sensible way to live with that, covered under security below.

Forward four ports

RustDesk needs a small range, and the entries aren't interchangeable.

21116/UDP is the one to watch. It's the lane the hole-punch negotiation runs on, so a missing UDP rule is the single most common reason a connection that looks fine feels broken: it forms, then crawls because every frame is relayed. The Compose file also binds a few optional web ports on the host (21114 for the admin console, 21118 and 21119 for the browser client), but those don't need forwarding for a normal desktop session.

Keep the DNS record current

Residential IPs rotate without warning, and a stale record looks exactly like an offline server: the lookup succeeds, your client gets a confident answer, and it dials an address your ISP handed to someone else last week.

cf-ddns.sh prevents that. A systemd timer runs it every five minutes. It reads your current public IP (api.ipify.org, falling back to ifconfig.me) and compares it against a cached copy in .cf-ddns-last-ip. When nothing has changed, which is most of the time, it exits in milliseconds without calling the Cloudflare API. When the IP moves, it rewrites the A record (ttl 120, proxied false) and caches the new value. In Tailscale mode the script exits immediately, since there's no DNS to maintain.

Tailscale mode

Tailscale mode needs no domain, no router rules, and no DNS to keep current. The wizard installs Tailscale, signs you in, reads your tailnet IPv4 with tailscale ip -4, and binds the RustDesk ports to that 100.x.x.x address and nothing else. As far as the public internet is concerned, the ports aren't there.

One command confirms the binding stayed private:

ss -tlnp | grep '2111'
# every line should read 100.x.x.x, never 0.0.0.0

Tailscale reaches a PC with no open ports using the same hole-punching idea as the public path. Its coordination server plays the role hbbs does, telling each device where the other's NAT opened; WireGuard rides the hole that gets punched; and a DERP relay stands in for hbbr when a network is too strict. It's the same physics, wrapped in an encrypted tunnel and an account instead of four router rules.

Two operational details are specific to this mode:

• A rustdesk-reconcile.timer runs every five minutes. If your tailnet IP changes it rewrites BIND_ADDR and rebinds the containers, and restarts them if they're down. The one thing it can't fix is an expired node key.
• Tailscale node keys expire after about 180 days by default. When that happens the machine deauthenticates and needs an interactive re-login, which no timer can do for you. Disable key expiry on the server in the Tailscale admin console. The wizard prints this reminder at the end of setup.

The cost of this mode is that every device you connect from needs Tailscale installed and signed in. For your own phone and laptop that's a one-time setup.

Setup, end to end

However you answer the mode question, the shape is the same: one script does the heavy lifting, and public mode adds four router rules on top.

./rustdesk_selfhost.sh setup

./rustdesk_selfhost.sh status

The command surface

Everything runs through one script. The verbs map to the lifecycle.

Keeping it running

The containers carry restart: unless-stopped, so a reboot or a power cut brings the server back on its own. Run the update every month or so, or whenever there's a RustDesk security advisory:

./rustdesk_selfhost.sh update

Back up one directory. data/ holds the Ed25519 keypair and the server's state. The built-in backup writes a chmod 600 tarball of the whole directory to ./backups/, which keeps the SQLite write-ahead log consistent, and restore defaults to the newest snapshot. If you lose id_ed25519, every client you've ever set up has to be re-keyed by hand, so this is the one file worth copying somewhere safe.

One detail about that key is worth knowing. The container writes the private key world-readable (644). The script re-locks it to 600 on every up, restart, update, and restore, so it never sits exposed on disk for long.

When it doesn't work

Three failures cover almost everything, and each traces back to a single cause.

Red dot, server unreachable

The client never finds hbbs. In public mode, work backwards: does the hostname resolve, does port 21116 answer from outside your network (nc -zv host 21116), and does your router's WAN IP still match api.ipify.org? A mismatch usually means CGNAT, or an IP that changed before the DDNS timer caught it. In Tailscale mode it's almost always simpler: Tailscale isn't connected on the device you're calling from.

Green dot, but "failed via relay server"

The server is reachable, but the relay address it hands out is wrong. The classic version of this bug advertises the literal container name hbbr, which only resolves inside Docker, so an outside client can't reach it. This repo avoids it by handing out RUSTDESK_RELAY_HOST instead, so the fix is to confirm that value in your .env points at your public or Tailscale hostname, not a bare container name.

It connects, but the video crawls

One or two frames a second on a fast line means every session is relaying. 21116/UDP isn't getting through, so the direct path never forms. Check it from outside your network:

nmap -sU -p 21116 rustdesk.example.com
# open: good.  filtered: the router's UDP rule is missing or wrong.

Security

Two things keep strangers out of your desktop, and both are yours to set.

If you ever think the private key has leaked, rotating it takes a few minutes: nuke the keypair, up again so the server generates a fresh one, then paste the new public key into each client. The old identity is dead the moment you do.

What it costs

RustDesk, Docker, Cloudflare DNS, and Tailscale are all free at the level this needs. The only real cost is the electricity the PC was already drawing.

Words worth knowing

A few terms come up along the way. None are required reading. If one snagged you earlier, here's the longer version.

Run your own

A script, one network choice, and either four router rules or a single VPN app. That's most of what it takes.

The whole project is open and MIT-licensed ^[4], and so is everything it builds on: RustDesk, Tailscale, and Cloudflare DNS. Pick the mode that fits, leave the PC on, and the server is yours to reach whenever you need it.