Complete Guide on How to Setup a Nextcloud Spreed Signaling Server (i.e. the Talk High-Performance Backend) on Debian (11) Bullseye

New blog entry on how to set up the the Nextcloud Spreed Signaling server on Debian 12 can be found here.

This is a tutorial on how to setup the High-performance backend for the Talk/Spreed app within a Nextcloud instance. While I found several tutorials online that guide through parts of the process, those I found were insufficient for allowing access to users behind strict firewalls or NATs, and additionally were mostly written for Ubuntu. This tutorial aims at creating a fully functional High-Performance backend installed on Debian Bullseye, and which also clients from within big organisations or companies can access.

This tutorial will explain how to prepare and set up a server to use the standalone spreed signaling server, plus, how to combine it with a TURN server. I have been hosting video conferences and workshops with for an academic audience, where the users often sit in universities or big organisation, and their internet traffic is limited by NAT and firewall settings. This is, because in big organisation sometimes only the most necessary ports for webbrowsing (53, 80, 443) are open, and the default ports of turn and signaling servers are outside that range. So in order to allow their access, both the TURN server and the signaling server need to be accessible on port 443. You can achieve that by either running the TURN server on a dedicated VPS, or by adding it to an existing server (such as the signaling server you are going to set up) while using an additional IPv4 address.

This tutorial will show both options, but most importantly, it will provide an inclusive setup to also allow access from clients behind strictly set firewalls.

Requirements:

• You will need a server (VPS) with Debian 11 minimal pre-installed for the signaling server.   
  (Nextcloud suggests to use a server with dedicated cores; For my purposes [<30 video conference participants], a small VPS with 2vCores and 2GB RAM has been sufficient so far)
• A Domain and access to the zone file (DNS settings)
• For the TURN server you will either need a second VPS or an additional IPv4 address that you can bind to an existing server (see chpt. 2 for more details), such as the signaling server you are going to set up.

Skill-Level:

• You know how to edit your DNS settings
• You know how to install a VPS
• You know how to open a Linux terminal

Text between the <less/more marks> indicate a variable you will create or choose (such as a user name or secret), but please leave out the marks themselves when entering the values into the configuration files.

1. Prepare Server

Let’s start with creating the DNS entries for your signaling and TURN servers. Go to your domain’s DNS settings and create two A entries, like

signaling A <XXX.XXX.XXX.XXX>

for your signaling server, and one for your TURN server like

turn A <YYY.YYY.YYY.YYY>

(Let’s do that at the beginning, so when we get to creating certificates, the subdomains should already be accessible). You may want to change the TTL to a low value, such as 600.

After renting your server, you should have gotten some root login credentials. Open the terminal on your local machine and SSH into your new server using those credentials:

ssh root@XXX.XXX.XXX.XXX

(Optionally) you may want to rename the hostname of your VPS:

hostnamectl set-hostname signaling.your.domain

First important step: Change your root password with

passwd

(If you want a random password, you can use pwgen; If you do, please write it down somewhere :-)

1.a) Create a SSH User

Create a new user

apt install sudo

useradd -m <ssh-user>

passwd <ssh-user>

Add the user to the sudo group

usermod -a -G sudo <ssh-user>

And prepare for ssh usage

su <sshuser>

ssh-keygen

Lets make some changes to the ssh server to use a different than the default port, , to disallow root login, limit ssh access to IPv4, and to allow password-less authentication at ssh login:

sudo nano /etc/ssh/sshd_config

Uncomment (i.e. take out the "#" in front of a line), modify, or add the lines accordingly

Port 2222
PermitRootLogin no
AddressFamily inet
PubkeyAuthentication yes

and restart the ssh server

sudo systemctl restart sshd

Now you can log out, or open a second ssh session on your desktop/local computer, pass on your ssh-key and, login again.

ON YOUR LOCAL MACHINE:

Create a ssh key

ssh-keygen (press "enter" when asked for a password to leave it empty)

and copy it to the VPS:

ssh-copy-id -i ~/.ssh/id_rsa.pub -p 2222 <ssh-user>@signaling.your.domain   

 

After that you can login without a password to your VPS

ssh -p 2222 <ssh-user>@signaling.your.domain

Go back into the ssh server configuration

sudo nano /etc/ssh/sshd_config

so you can disable password authentication for ssh, and only allow login for your <ssh-user>

PasswordAuthentication no
AllowUsers <ssh-user> 

Restart the ssh server

sudo systemctl restart sshd

While you could continue using sudo to make further changes, we will switch back to su for convenience

sudo -s

1.b) Firewalling and Brute Force Mitigation

Let's further enhance the VPS security by adding basic firewall and brute force mitigation

apt install ufw fail2ban -y

Configure the firewall to allow only incoming traffic via IPv4 to your ssh server

ufw allow proto tcp to 0.0.0.0/0 port 2222

and open port 443 for apache (and the TURN server)

ufw allow 443

Enable the firewall

ufw enable

Increase brute force attack prevention, by creating a new file (for local settings)

nano /etc/fail2ban/jail.local

and add

[sshd] 
enable = true 
port = 2222 
logpath = %(sshd_log)s 
backend = %(sshd_backend)s
 
[sshd-ddos] 
enable = true 
port = 2222
logpath = %(sshd_log)s 
backend = %(sshd_backend)s

and restart fail2ban

systemctl restart fail2ban   
 

2. TURN server

According to the Nextcloud doumentation

A TURN server running on port 443 (or 80) is required in almost all scenarios

Even if the Nextcloud Talk High Performance Backend is used and publicly accessible, a TURN server might be needed:

The High Performance Backend uses a certain range of ports for WebRTC media connections (20000-40000 by default). A client could be behind a restrictive firewall that only allows connections to port 443, so even if the High Performance Backend is publicly accessible the client would need to connect to a TURN server in port 443, and the TURN server will then relay the packets to the 20000-40000 range in the High Performance Backend.

For maximum compatibility the TURN server should be configured to listen on port 443. Therefore, when both a TURN server and the High Performance Backend are used each one should run in its own server, or in the same server but each one with its own IP address, as the High Performance Backend will need to bind to port 443 too.

Hence, we can either run a dedicated VPS for the TURN server, or add a second IP to an existing and running VPS, such as the signaling server you are setting up. In the past, I used a dedicated VPS for this purpose, but my server load was too little to justify that (I had sessions with up to 30 participants at once, and the turnserver hardly used any memory or cpu).

IF YOU INSTALL YOUR TURN SERVER ON A DEDICATED VPS, REPEAT chpt. 1 ON THE SECOND VPS, AND JUMP TO 2.b)

2.a) Add additional IP address (Optional)

If you decided to use an additional IPv4, you will most likely have to add it to the network interfaces of the machine you want to add it, in the control panel of your VPS host.

To utilise an additional IPv4 address on an existing server, edit your existing config file or add a new one

nano /etc/network/interfaces.d/<your-config-file>.cfg

and add a new network interface

auto eth0:1
iface eth0:1 inet static
	address YYY.YYY.YYY.YYY
	netmask 255.255.255.255

If you use an existing VPS that already has apache installed, we can make the following modifications to apache; if not, then install apache now (which we will need later anyway) 

apt install apache

Make Apache to listen only to original IP address by editing

nano /etc/apache2/ports.conf

and add your original IP address (from the network interface eth0 in your /etc/network/interfaces.d/<your-config-file>.cfg or by looking it up using “hostname -I") in front of the ports apache is listening to

	Listen XXX.XXX.XXX.XXX:80
<IfModule ssl_module>
	Listen XXX.XXX.XXX.XXX:443
</IfModule>
<IfModule mod_gnutls.c>
	Listen XXX.XXX.XXX.XXX:443
</IfModule>

Reboot server

reboot

Potentially you may have to shut down and restart the VPS from the control panel of your VPS host to enable the additional IPv4 address.

Login again

ssh -p 2222 <ssh-user>@signaling.your.domain

2.b) Install and configure coTURN

For installing the TURN server (we will use coTURN) as well as for other software later on, we add the bullseye-backport repository to get later software than the default one. For this purpose, open your apt configuration

nano /etc/apt/sources.list

and add the backports repo to your apt settings:

#Bullseye-backports 
deb http://deb.debian.org/debian/ bullseye-backports main 
deb-src http://deb.debian.org/debian/ bullseye-backports main 

Update and install coTURN

apt update && apt install -t bullseye-backports coturn -y

The coTURN service is executed as an unprivileged user like turnserver. Due to this by default coTURN can not use privileged ports, like port 443. Linux kernel capabilities could be used to overcome this limitation. Capabilities can be associated with executable files using setcap, so you could allow the /usr/bin/turnserver executable to bind sockets to privileged ports with:

setcap cap_net_bind_service=+ep /usr/bin/turnserver

Back up the original configuration file

cp /etc/turnserver.conf /etc/turnserver.conf_ORIG

Create a "static-auth-secret" for you turnserver with

openssl rand -hex 32

and add it along with the other required configuration to

nano /etc/turnserver.conf

listening-port=443 
tls-listening-port=443
listening-ip=YYY.YYY.YYY.YYY
relay-ip=XXX.XXX.XXX.XXX
fingerprint 
use-auth-secret
static-auth-secret=<your-turnserver-secret-created-above>
realm=turn.your.domain
total-quota=0 
bps-capacity=0
stale-nonce
no-multicast-peers
no-stdout-log
log-file=/var/log/coturn.log
#verbose
simple-log
cert=/etc/coturn/certs/turn.your.domain/cert.pem 
pkey=/etc/coturn/certs/turn.your.domain/privkey.pem
no-sslv2
no-sslv3
no-tlsv1
no-tlsv1_1

If you need more log verbosity when testing your TURN server, uncomment the "#verbose" flag.

Restart your turn server

systemctl restart coturn

2.c) SSL Certificates with Letsencrypt

In order to use TLS encrypted data transfer, install certbot

apt-get install certbot

and create lLtsencrypt certificates

certbot certonly --standalone --preferred-challenges http --deploy-hook "systemctl restart coturn" -d turn.your.domain

Because coTURN doesn't have privileged rights, it is not allowed to read the certificates. Hence, create new folder where to copy the certificates to

sudo mkdir -p /etc/coturn/certs

Change the ownership and access rights for the coTURN user & group

sudo chown -R turnserver:turnserver /etc/coturn/

sudo chmod -R 700 /etc/coturn/

Because Letsecnrypt certificates are only valid for 3 months, they regularly need to be renewed. So you it is wise to automatise the certificate copy process. Create a script that can do that and place it in the renewal-hooks directory of certbot

nano /etc/letsencrypt/renewal-hooks/deploy/coturn-certbot-deploy.sh

#!/bin/sh
set -e
for domain in $RENEWED_DOMAINS; do
case $domain in
turn.your.domain)
daemon_cert_root=/etc/coturn/certs
 
# Make sure the certificate and private key files are
# never world readable, even just for an instant while
# we're copying them into daemon_cert_root.
umask 077
 
cp "$RENEWED_LINEAGE/fullchain.pem" "$daemon_cert_root/$domain.cert"
cp "$RENEWED_LINEAGE/privkey.pem" "$daemon_cert_root/$domain.key"
 
# Apply the proper file ownership and permissions for
# the daemon to read its certificate and key.
chown turnserver "$daemon_cert_root/$domain.cert" \
"$daemon_cert_root/$domain.key"
chmod 400 "$daemon_cert_root/$domain.cert" \
"$daemon_cert_root/$domain.key"
service coturn restart >/dev/null
;;
esac
done 

While the simple log configuration of coTURN will overwrite the log file on every start-up, if your server is up and running for a while, we can configure the log to auto-rotate, by creating anew file

nano /etc/logrotate.d/coturn

and add

/var/log/coturn.log {
rotate 7
daily
missingok
notifempty
compress
postrotate
endscript
}

 3. Install WebRTC Gateway: Janus 

A Janus server (from https://github.com/meetecho/janus-gateway) can be used to act as a WebRTC gateway. To install, use

apt install -t bullseye-backports janus -y

create a turn-rest-api key

openssl rand -base64 16

and change the Janus configuration in

nano /etc/janus/janus.jcfg

Look for the section nat to add your turn-rest-api key and enable "full_trickle"

turn_rest_api_key = <turn-rest-api-key>
full_trickle = true

Edit

nano /etc/janus/janus.transport.http.jcfg

and in the section [general] set interface = "lo" 

Edit

nano /etc/janus/janus.transport.websockets.jcfg 

and in the section [general] set ws_interface = "lo"

Restart Janus service

systemctl restart janus

It should automagically be enabled on boot, but just to make sure

systemctl enable janus

4. Install and Setup NATS Server

There is a handy docker version available for NATS, which you can take use of.

Install docker

apt install curl -y

curl -sSL https://get.docker.com/ | CHANNEL=stable sh

Enable and start docker

systemctl enable docker.service && systemctl start docker.service   

And run NATS in a docker container

docker run --restart=always --name='natsserver' -d -p 4222:4222 -ti nats:latest   

  

5.Setup Nextcloud Spreed Signaling Server

Now all the components are in place for your Nextcloud signaling server, so let's get to this puzzle piece. First install the dependencies

apt install -t bullseye-backports git automake golang build-essential python3 protobuf-compiler -y

(protobuf-compiler is required since Talk15/NC25; golang need to be from backports, as the version from the default repo is too old for building the signaling server)

Download the signaling server from github and build it

cd /opt 

git clone https://github.com/strukturag/nextcloud-spreed-signaling.git

cd nextcloud-spreed-signaling/ 

make build

5.a) Running the Signaling Server as Daemon

To run the signaling server as a daemon, copy the binary to Debian's default binary directory

cp bin/signaling /usr/bin/

and create a new user that will run the signaling server

useradd --system --shell /usr/sbin/nologin --comment "Standalone signaling server for Nextcloud Talk." signaling

Create a configuration file for the signaling server with appropriate ownership settings

mkdir /etc/signaling/

touch /etc/signaling/server.conf

chown signaling: /etc/signaling/server.conf

chmod 600 /etc/signaling/server.conf

and copy the shipped sample file for the daemon to Debian's systemd directory

cp dist/init/systemd/signaling.service /etc/systemd/system/signaling.service

Make sure that the signaling server only starts up after Janus, by editing

nano /etc/systemd/system/signaling.service

so that the section [unit] looks like this

[Unit] 
Description=Nextcloud Talk signaling server 
After=janus.service

Reload the daemon and enable the signaling server daemon

systemctl daemon-reload && systemctl enable signaling

5.b) Configure the Signaling Server

 For the signaling server configuration we will need a couple of secrets, so first create those   
 

Nextcloud Secret Key

openssl rand -hex 16

Block-Key

openssl rand -hex 16

Hash-Key

openssl rand -hex 16

Now we have five keys in total (incl. those from chapter 2 and 3)

Turn-Key from chpt. 2: <your-turnserver-secret-created-above>   
api-key from chpt. 3: <turn-rest-api-key>   
Nextcloud-Secret-Key: <nextcloud-secret-key>   
Block-Key: <block-key>   
Hash-Key: <hash-key>   
 

Open the signaling server configuration file

nano /etc/signaling/server.conf

and copy, paste and adjust the file (change keys to yours)

[http]
listen = 127.0.0.1:8080

[app] 
debug = false

[sessions] 
hashkey = <hash-key>
blockkey = <block-key>

#You may define several nextcloud instances for which you want to use the signaling server, here we only add one
[backend]
backends = backend-1
allowall = false 
timeout = 10 
connectionsperhost = 8

#add your nextcloud instance
[backend-1] 
url = https://your.nextcloud.domain
secret = <nextcloud-secret-key>

[nats] 
url = nats://localhost:4222

[mcu] 
type = janus 
url = ws://127.0.0.1:8188

[turn] 
apikey = <turn-rest-api-key>
secret = <your-turnserver-secret-created-in-chapter-2> 
servers = turn:turn.your.domain:443?transport=tcp

Start the signaling server

systemctl start signaling

and check status of the service:

systemctl status signaling

5.c) Reverse Proxy Server

As you can see from section [http] in the server config, the server listens to 127.0.0.1 on port 8080, so we (install and) configure a webserver and use it to proxy to the signaling server. You can use Nginx, Caddy, or some other server, personally I prefer Apache. If you haven't installed apache already (from chapter 2), you can do it now with

apt install apache2 -y

Create a virtual host by creating the file

nano /etc/apache2/sites-available/signaling.conf

and add

<IfModule mod_ssl.c>
     <VirtualHost _default_:443> 
               ServerAdmin admin@localhost 
               ServerName localhost 
               ServerAlias signaling.your.domain

       SSLEngine on

       # Enable http2
       Protocols h2 http/1.1

       # Use HSTS
       Header always set Strict-Transport-Security "max-age=63072000; preload"

       <FilesMatch "\.(cgi|shtml|phtml|php)$"> 
               SSLOptions +StdEnvVars 
       </FilesMatch> 

       <Directory /usr/lib/cgi-bin> 
               SSLOptions +StdEnvVars 
       </Directory>

       ProxyPass   "/standalone-signaling/""ws://127.0.0.1:8080/" 

       RewriteEngine On

       # Websocket connections from the clients. 
       RewriteRule ^/standalone-signaling/spreed$ - [L]

       # Backend connections from Nextcloud. 
       RewriteRule ^/standalone-signaling/api/(.*) http://127.0.0.1:8080/api/$1 [L,P] 

       Include /etc/letsencrypt/options-ssl-apache.conf 
       SSLCertificateFile      /etc/ssl/certs/ssl-cert-snakeoil.pem 
       SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key 

       ErrorLog ${APACHE_LOG_DIR}/error.log 
       CustomLog ${APACHE_LOG_DIR}/access.log combined 

	</VirtualHost> 
</IfModule>

(NOTE: settings http2 and HSTS are not necessarily required for the signaling server, but you may add it to enhance speed and transport security)

Enable the virtual host

a2ensite signaling.conf

as well as the required apache modules

a2enmod proxy proxy_http rewrite proxy_wstunnel headers http2

and restart apache

systemctl restart apache2

5.d) SSL/TLS Certificates

If you haven't already installed certbot from in chapter 2, then do it now; Certainly you will want to add the apache plugin for certbot:

apt install certbot python3-certbot-apache -y   
 

Run certbot

certbot

Select your new virtual host and let certbot install your Letsencrypt certificates.

Make sure that  certbot will run once a week to check in time before your certs expire, and to get new ones. Run

crontab -e

And add

 23 2 * * 1 certbot renew --quiet

at the bottom of the file.

6.Add Signaling and TURN Server to Nextcloud

Open your Nextcloud instance, login as admin, and go to the admin settings (https://your.nextcloud.domain/settings/admin/talk).

In section STUN servers, add

stun:  turn.your.domain:443

In section Turn servers, add

turn and turns turn.your.domain:443  <your-turnserver-secret-created-in-chapter-2>  UDP and TCP

In section High-performance backend   
https://signaling.your.domain/standalone-signaling  check "Validate SSL certificate" and in the field shared secret add <nextcloud-secret-key>

7. Problems? Check your Logs

If you don't see a green check behind the Turn and high-performance backend settings, check your settings.

• Make sure you used the created secrets in the right place
• Check your firewall settings
• check if all services are up and running
  • systemctl status coturn
  • systemctl status signaling
  • systemctl status janus
  • systemctl status apache2
  • docker ps
• Check your logs
  • tail -f /var/log/coturn.log
  • tail -f /var/log/syslog
  • tail -f /var/log/janus.log
  • tail -f /var/log/apache2/error.log
  • tail -f /var/log/apache2/access.log
  • docker logs <docker-id>

!!! Happy Video Conferencing !!!

8. Sources:

I didn't come up with all of this. There are a couple of good tutorials that cover at least parts of the process, and I want to thank the authors as of those for sharing their knowledge! Yet, none of the tutorials I found covered my use case, so I compiled my own tutorial for easy reproduction. The resulting tutorial is quite compressed and doesn't provide all the details on the settings. If you want to know more about the background, you may want to check the sources I used for compiling this tutorial:

https://nextcloud-talk.readthedocs.io/en/latest/system-requirements/

https://nextcloud-talk.readthedocs.io/en/latest/TURN

https://github.com/strukturag/nextcloud-spreed-signaling

https://morph027.gitlab.io/blog/nextcloud-spreed-signaling/

https://nichteinschalten.de/signalisierungsserver-fuer-nextcloud-aufsetzen-how-to/

https://markus-blog.de/index.php/2020/07/30/how-to-install-nextcloud-talk-high-performance-backend-with-stun-turnserver-on-ubuntu/

https://serverfault.com/questions/849683/how-to-setup-coturn-with-letsencrypt

9. Updates:

• 17. September 2023:
  Changed the setting in chapter 6 "in section High-performance backend" from " https://signaling.your.domain" to"https://signaling.your.domain/standalone-signaling" (Thanks Evgeniy for pointing out the mistake).
   
• 28. December 2023
  I recently discovered coturn not establishing a connection & throwing the error message: "TLS/TCP socket error: Connection reset by peer" due to the "reason: TLS/TCP socket buffer operation error (callback)." This appeared to be related to an interoperability issue of the TLS versions. Hence, I have added in chapter 2.b) to the configuration in /etc/turnserver.conf: 
  no-tlsv1 
  no-tlsv1_1 
  to make sure that TLSv1.2 or higher is being used. Coturn works for me now as expected again.