Setting up a single domain with muliple DNS servers¶

NextCloud is one of those applications that can only take 1 domain. How how can we optimize our domain so that it can be used locally, over the tailnet, AND over the public internet, with different considerations for each?? This is where context aware DNS comes in handy.

Setting up DNS servers for both the Local Network and the Tailnet.¶

Now we need 2 dns servers, one for the local network, and one for the tailnet. That's because each need to resolve our domain, nextcloud.wildebeastmedia.com, to their respective server IP address (the local IP address of the server as well as the tail scale IP address of the server).

DNS servers need to listen on port 53 though, so how do we accomplish running two of them when even our host's port 53 is already taken. MacVLan! So with the MacVLan, each container will actually be it's own "machine" on our network with it's own IP address. Cool! Let's run two different DNS servers. Pi-Hole and AdguardHome. We'll use AdguardHome from our home network and Pi-Hole for the Tailscale Network.

We are going to need our Pi-Hole to be accessible to our Tailnet, so let's also include a Tailscale node that advertises Pi-Hole on our MacVLan network.

Adguard, PiHole, & Tailscale docker-compose.yml

docker-compose.yml

services:
  adguardhome:
    image: adguard/adguardhome
    container_name: adguardhome
    restart: unless-stopped
    volumes:
      - ./work:/opt/adguardhome/work
      - ./conf:/opt/adguardhome/conf
    # ports:
      # - 53:53/tcp     # Standard DNS
      # - 53:53/udp     # Standard DNS
      # - 67:67/udp     # if using as a DHCP server
      # - 68:68/udp     # if using as a DHCP server
      # - 3000:3000/tcp # Initial Web Interface
      # - 4422:80
      # - 4433:433 # Web interface to be binding to host over bridge
      # - 853:853/tcp   # DNS over TLS (DoT)
      # - 784:784/udp   # DNS-over-QUIC
      # - 853:853/udp   # DNS-over-QUIC
      # - 8853:8853/udp # DNS-over-QUIC
      # - 5443:5443/tcp # add if you are going to run AdGuard Home as a DNSCrypt⁠ server.
      # - 5443:5443/udp # add if you are going to run AdGuard Home as a DNSCrypt⁠ server.
    networks:
      adguard_macvlan_network:
        ipv4_address: 192.168.4.100  # Choose a static IP within the subnet

  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      # DNS Ports
      - "53:53/tcp"
      - "53:53/udp"
      # Default HTTP Port
      - "80:80/tcp"
      # Default HTTPs Port. FTL will generate a self-signed certificate
      - "443:443/tcp"
      # Uncomment the line below if you are using Pi-hole as your DHCP server
      #- "67:67/udp"
      # Uncomment the line below if you are using Pi-hole as your NTP server
      #- "123:123/udp"
    environment:
      # Set the appropriate timezone for your location (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g:
      TZ: 'America/Denver'
      # Set a password to access the web interface. Not setting one will result in a random password being assigned
      FTLCONF_webserver_api_password: 'correct horse battery staple'
      # If using Docker's default `bridge` network setting the dns listening mode should be set to 'all'
      FTLCONF_dns_listeningMode: 'all'
    # Volumes store your data between container upgrades
    volumes:
      # For persisting Pi-hole's databases and common configuration file
      - './etc-pihole:/etc/pihole'
      # Uncomment the below if you have custom dnsmasq config files that you want to persist. Not needed for most starting fresh with Pi-hole v6. If you're upgrading from v5 you and have used this directory before, you should keep it enabled for the first v6 container start to allow for a complete migration. It can be removed afterwards. Needs environment variable FTLCONF_misc_etc_dnsmasq_d: 'true'
      #- './etc-dnsmasq.d:/etc/dnsmasq.d'
    cap_add:
      # See https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
      # Required if you are using Pi-hole as your DHCP server, else not needed
      # - NET_ADMIN
      # Required if you are using Pi-hole as your NTP client to be able to set the host's system time
      # - SYS_TIME
      # Optional, if Pi-hole should get some more processing time
      - SYS_NICE
    restart: unless-stopped
    networks:
      adguard_macvlan_network:
        ipv4_address: 192.168.4.101  

  tailscale:
    container_name: tailscale
    hostname: PiHole Tailnet
    image: tailscale/tailscale:latest
    environment:
      TS_AUTHKEY: ${tailscale_auth_key}
      TS_ACCEPT_DNS: "true"
      TS_ROUTES: 192.168.4.101/32 # Optional: Add subnet routes if needed
    networks:
      adguard_macvlan_network:
        ipv4_address: 192.168.4.103  
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    volumes:
      - ./tailscale:/var/lib/tailscale

networks:
  adguard_macvlan_network:
    driver: macvlan
    driver_opts:
      parent: eno1  # Replace with the correct host network interface
    ipam:
      config:
        - subnet: 192.168.4.0/24  # Adjusted subnet based on your network
          gateway: 192.168.4.1    # Correct gateway (router IP)

A couple notes on our docker-compose.yml file. We're running these on a macvlan network, so we don't actually need to map any ports, they'll be exposed by default. Secondly, you do need to make sure you use the right host network interface, you can find this by running. You can also set the IP address to whatever you like as long as they're not taken on the router.

Taking between the Host and the MacVLan¶

By default even though your PC network can talk directly to the containers, you're host cannot. Go ahead and give it a try.

From your PC

ping 192.168.4.100

It works! Oh yeah, you can also just navigate the websites in your browser.

How about from your host? It doesn't work. How about taking from the container to the host?

docker exec -it adguardhome ping 192.168.4.142

Uh oh, it doesn't work. See if it can ping your PC!

Ok so if we want the host and the macvlan to be able to communicate, which we do so that the host can use the DNS server as well as our nextcloud and collabora containers, we need to manually set up a link. I know, painful. It's not that bad though. Let's create a network interface for them to talk.

sudo ip link add macvlan0 link eno1 type macvlan mode bridge
sudo ip addr add 192.168.4.254/24 dev macvlan0
sudo ip link set macvlan0 up

Ok and now let's add routes to both of the DNS servers, although I suppose we only really need to do it with the AdguardHome.

sudo ip link add macvlan0 link eno1 type macvlan mode bridge
sudo ip addr add 192.168.4.254/24 dev macvlan0
sudo ip link set macvlan0 up

Now see if they can talk!

The Local Network¶

Inside AdguardHome we need to create a DNS record in Filters -> DNS Rewrites.

Domain -> nextcloud.wildebeastmedia.com
Answer -> 192.168.4.142 (Local IP of the server)

Do the same with collabora.wildebeastmedia.com and onlyoffice.wildebeastmedia.com.

Windows¶

Ok, on your PC, you may need to tell set DNS to "Automatic". Go to

Settings -> Network & internet -> DNS server assignment

This should be set to automatic for your domain to work.

Android¶

Note with android, you will need to change Private DNS to Automatic if you want the domain to resolve to the local IP.

NextCloud, Collabora, OnlyOffice¶

These need to be able to communicate with each other over the local network via the dns, so we need to add taht to each docker container

dns:
  - 192.168.4.100

NextCloud Docker Compose

services:
  # Note: MariaDB is external service. You can find more information about the configuration here:
  # https://hub.docker.com/_/mariadb
  db:
    # Note: Check the recommend version here: https://docs.nextcloud.com/server/latest/admin_manual/installation/system_requirements.html#server
    image: mariadb:lts
    container_name: nextcloud-compose-db
    restart: always
    networks:
      nc-net:
    command: --transaction-isolation=READ-COMMITTED
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=${DB_ROOT_PW}
      - MYSQL_PASSWORD=${DB_PW}
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=${DB_USER}

  # Note: Redis is an external service. You can find more information about the configuration here:
  # https://hub.docker.com/_/redis
  redis:
    image: redis:alpine
    restart: always
    networks:
      nc-net:

  app:
    image: nextcloud
    container_name: nextcloud-compose
    dns:
      - 192.168.4.100
    restart: always
    networks:
      nc-net:
    ports:
      - 12000:80
    depends_on:
      - redis
      - db
    volumes:
      - ${NEXTCLOUD_LIBRARY}:/var/www/html
    environment:
      - MYSQL_PASSWORD=${DB_PW}
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=${DB_USER}
      - MYSQL_HOST=db
      #
      - NEXTCLOUD_DEFAULT_PHONE_REGION=US
      - TRUSTED_DOMAINS={nextcloud.wildebeastmedia.com}
      - OVERWRITECLIURL=https://nextcloud.tail.wildebeastmedia.com
      - OVERWRITEPROTOCOL=https

  collabora:
    image: collabora/code:24.04.12.3.1
    dns:
      - 192.168.4.100
    networks:
      nc-net:
    ports:
      - 127.0.0.1:9980:9980
    container_name: collabora
    # release notes: https://www.collaboraonline.com/release-notes/
    # networks:
    #   ocis-net:
    environment:
      aliasgroup1: ${alias11}
      aliasgroup2: ${alias21} # Remove this line if you don't have any other services using Collabora
      DONT_GEN_SSL_CERT: "YES"
      extra_params: |
        --o:ssl.enable=false \
        --o:ssl.ssl_verification=true \
        --o:ssl.termination=true \
        --o:welcome.enable=false
      username: admin
      password: admin
    cap_add:
      - MKNOD
    logging:
      driver: ${LOG_DRIVER:-local}
    restart: always
    command: ["bash", "-c", "coolconfig generate-proof-key ; /start-collabora-online.sh"]
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:9980/hosting/discovery" ]

networks:
  nc-net:
    driver: bridge

Tailscale¶

We should now see our pihole-tailnet machine. It should say that it advertises subnets.

On the DNS tab, set the Global nameserver to 192.168.4.101 and check to Override DNS servers.

Make sure on your windows and mobile clients that they are set to use the Tailscale DNS in the Tailscale App.

Open up Pi-Hole and create some DNS records just like with Adguard home, except this time use the Tailscale IP address of your server.

Domain -> nextcloud.wildebeastmedia.com
Answer -> 100.x.x.x (Tailscale IP of the server)

Do the same with collabora.wildebeastmedia.com and onlyoffice.wildebeastmedia.com.

Nginx¶

Go ahead and generate some certs for Collabora and OnlyOffice.

docker exec -it certbot certbot certonly --non-interactive --quiet --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
      --email thomas@thomaswildetech.com --agree-tos --no-eff-email --expand \
      --domains collabora.wildebeastmedia.com;

Nginx Blocks

# Nextcloud Docker Compose
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name nextcloud.wildebeastmedia.com;

    # Let's Encrypt Certs generated by certbot container
    ssl_certificate        /etc/letsencrypt/live/nextcloud.wildebeastmedia.com/fullchain.pem;
    ssl_certificate_key    /etc/letsencrypt/live/nextcloud.wildebeastmedia.com/privkey.pem;

    # Enable HTTP Strict Transport Security (HSTS) for one year, including subdomains.
    include /etc/nginx/conf.d/hsts.conf;

    # Include proxy headers
    include /etc/nginx/conf.d/proxy-headers.conf;

    # allow large file uploads
    client_max_body_size 50000M;

    # Enable Web Sockets
    include /etc/nginx/conf.d/websocket.conf;

    # Proxy settings for different subdomains
    location / {
        proxy_pass http://127.0.0.1:12000;
    }
}

# Collabora
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    # http2 on;
    server_name collabora.wildebeastmedia.com;

    # Let's Encrypt Certs generated by certbot container
    ssl_certificate        /etc/letsencrypt/live/collabora.wildebeastmedia.com/fullchain.pem;
    ssl_certificate_key    /etc/letsencrypt/live/collabora.wildebeastmedia.com/privkey.pem;

    # Enable HTTP Strict Transport Security (HSTS) for one year, including subdomains.
    include /etc/nginx/conf.d/hsts.conf;

    # Include proxy headers
    include /etc/nginx/conf.d/proxy-headers.conf;

    # Enable Web Sockets
    include /etc/nginx/conf.d/websocket.conf;


    # allow large file uploads
    client_max_body_size 50000M;

    # Proxy settings for different subdomains
    location / {
        proxy_pass http://127.0.0.1:9980; # Use HTTP since SSL is terminated at Nginx
    }
}

# OnlyOffice
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    # http2 on;
    server_name onlyoffice.wildebeastmedia.com;

    # Let's Encrypt Certs generated by certbot container
    ssl_certificate        /etc/letsencrypt/live/onlyoffice.wildebeastmedia.com/fullchain.pem;
    ssl_certificate_key    /etc/letsencrypt/live/onlyoffice.wildebeastmedia.com/privkey.pem;

    # Enable HTTP Strict Transport Security (HSTS) for one year, including subdomains.
    include /etc/nginx/conf.d/hsts.conf;

    # Include proxy headers
    include /etc/nginx/conf.d/proxy-headers.conf;

    # Enable Web Sockets
    include /etc/nginx/conf.d/websocket.conf;

    # allow large file uploads
    client_max_body_size 50000M;

    # Proxy settings for different subdomains
    location / {
        proxy_pass http://127.0.0.1:9981; # Use HTTP since SSL is terminated at Nginx
    }
}

Great! Now test out NextCloud on the local network without tailscale, then test out nextcloud on the mobile network WITH tailscale.

Cloudflare Tunnel¶

Cloudflare tunnel can route directly the the ports of each of the services OR you can set up another nginx instance and listen on port 8080. I found it to just be too tricky to try to route the same domain in cloudflare to the same nginx instance. I ran into issues with too many redirects, or cert issues. It's much easier to just set up another nginx instance.

nginx.conf

# Set the user that runs the Nginx worker processes.
user nginx;

# Automatically determine the number of worker processes based on available CPU cores.
worker_processes auto;

# Define the file where the master process ID is stored.
pid /run/nginx.pid;

# Include any additional module configurations located in /etc/nginx/modules-enabled/.
include /etc/nginx/modules_enabled/*.conf;

# Load the geoip2 module for GeoIP2 database support.
load_module /etc/nginx/modules/ngx_http_geoip2_module.so;

events {
    # Maximum number of simultaneous connections per worker process.
    worker_connections 1024;
}

http {

    # Add global configuration settings here.
    include /etc/nginx/conf.d/global.conf;

    # GeoIP2 Configuration
    include /etc/nginx/conf.d/GeoLite2.conf;

    # Log format
    log_format main '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" "$http_x_forwarded_for" ';

  # Access Log
  access_log /var/log/nginx/access.log main;

    # json Access Log
    include /etc/nginx/conf.d/json_log.conf;


    # Immich over Cloudflare Tunnel
    # Nextcloud via Cloudflare Tunnel (no HTTPS redirect)
    server {
        listen 80;
        listen [::]:80;
        server_name nextcloud.wildebeastmedia.com;

        # Redirect only if not from Cloudflare Tunnel
        # if ($is_from_cloudflared = 0) {
        #     return 301 https://$host$request_uri;
        # }
        # Include proxy headers
        include /etc/nginx/conf.d/proxy_headers.conf;

        # Websocket support
        include /etc/nginx/conf.d/websocket.conf;

        # Real IP support (if behind Cloudflare Tunnel)
        include /etc/nginx/conf.d/real_ip_from_cf_tunnel_docker.conf;

        # allow large file uploads
        client_max_body_size 50000M;

        location / {
            proxy_pass http://192.168.4.142:12000;  # Still HTTPS here if your Nextcloud is behind its own SSL
        }
    }

    server {
        listen 80;
        listen [::]:80;
        server_name collabora.wildebeastmedia.com;

        # Redirect only if not from Cloudflare Tunnel
        # if ($is_from_cloudflared = 0) {
        #     return 301 https://$host$request_uri;
        # }
        # Include proxy headers
        include /etc/nginx/conf.d/proxy_headers.conf;

        # Websocket support
        include /etc/nginx/conf.d/websocket.conf;

        # Real IP support (if behind Cloudflare Tunnel)
        include /etc/nginx/conf.d/real_ip_from_cf_tunnel_docker.conf;

        # allow large file uploads
        client_max_body_size 50000M;

        location / {
            proxy_pass http://192.168.4.142:9980;  # Still HTTPS here if your Nextcloud is behind its own SSL
        }
    }

    server {
        listen 80;
        listen [::]:80;
        server_name onlyoffice.wildebeastmedia.com;

        # Redirect only if not from Cloudflare Tunnel
        # if ($is_from_cloudflared = 0) {
        #     return 301 https://$host$request_uri;
        # }
        # Include proxy headers
        include /etc/nginx/conf.d/proxy_headers.conf;

        # Websocket support
        include /etc/nginx/conf.d/websocket.conf;

        # Real IP support (if behind Cloudflare Tunnel)
        include /etc/nginx/conf.d/real_ip_from_cf_tunnel_docker.conf;

        # allow large file uploads
        client_max_body_size 50000M;

        location / {
            proxy_pass http://192.168.4.142:9981;  # Still HTTPS here if your Nextcloud is behind its own SSL
        }
    }

}