This is a continuation of my earlier post on migrating my mailserver to kubernetes. The next component of my mail setup to migrate is webmail. In the past I used squirrelmail for this, so I started with investigating that. However, it turns out that the squirrelmail project does not provide any docker containers. There are some containers you can get from docker hub, but these are largely unmaintained and are mostly private projects. After some looking around I found roundcube. Quick prototyping with roundcube containers, provided by the roundcube project, using docker compose showed that it was not difficult to get it to work.
And most important of all, roundcube provides a much more mature user interface than squirrelmail and is also quite fast. In fact, on my mobile phone, I find that roundcube competes directly with K9 mail on Android. It has for instance support for writing HTML mails and provides things like address books. For the latter functionality it requires some persistent storage in an SQL database. By default it will use sqlite, allowing users to get a working system without having to run a separate database, and it also supports mysql and postgressql as external databases.
Setup
Webmail requires access to SMTP, IMAPS, and managesieve. To keep things simple, it is required to use one and the same IP address for all these services. This means that at the kubernetes level, we need one service that provides SMTP, IMAPS, and managesieve. Additionally, we need to use a hostname for which SMTP, IMAPS, and managesieve have the correct certificates. So, in summary, we need a valid host (e.g. mail.example.com if the mail server has a valid wildcard certificate for *.example.com). To keep the setup simple, I will use the loadbalancer service which has a fixed IP and use network policies to limit access to managesieve to only the roundcube component. Alternatively, it would perhaps be possible to use invalid certificates and have roundcube accept them, but that would be going against the flow.
The setup is as follows:
A StatefulSet is used for deployment instead of a Deployment. This is done to avoid problems with multiple containers running at the same time using the same storage at the same time. This can be problematic since roundcube initializes when it starts up for the first time and I am not sure that multiple roundcube containers can do this at the same time. Given the setup with an external database, it would be ok to run multiple pods for roundcube, each connecting to the same database.
Roundcube itself is configured by several environment variables in the roundcube-env ConfigMap, by a config.php in the the roundcube-config ConfigMap, and by database credentials in the mysql-roundcube-credentials Secret.
These configmaps and secrets are generated using kustomize:
kind: Kustomization
namespace: exposure
generatorOptions:
disableNameSuffixHash: true
configMapGenerator:
- name: roundcube-env
envs:
- config.env
- name: roundcube-config
files:
- config.php
secretGenerator:
- name: mysql-roundcube-credentials
literals:
- username=DBUSER
- password=DBPASSWD
- host=SERVICE.NAMESPACE
- port=DBPORT
resources:
- volumes.yaml
- statefulset.yaml
- service.yaml
The config.env file contains general environment variables for roundcube:
ROUNDCUBEMAIL_DEFAULT_HOST=ssl://k8smail.example.com # A ROUNDCUBEMAIL_SMTP_SERVER=tls://k8smail.example.com # A ROUNDCUBEMAIL_DEFAULT_PORT=993 ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve # B ROUNDCUBEMAIL_UPLOAD_MAX_FILESIZE=25M ROUNDCUBEMAIL_DB_TYPE=mysql # C ROUNDCUBEMAIL_DB_NAME=roundcube
- # A: We need to use ssl:// and tls:// to indicate we want to use secure connections. Also note that roundcube supports a single host for SMTP, IMAPS, and managesieve out of the box. The host name k8smail.example.com will be configured in the pod definition.
- # B: the managesieve plugin is not enabled by default so we add it, next to some other plugins
- # C: configure the database type and database name. Leave this out in an initial setup when using sqlite or when you don’t want to use an external database.
The config.php is additional config for roundcube and we need this to provide some custom settings:
<?php $config['managesieve_host'] = 'tls://%h'; # A $config['enable_caching'] = FALSE; # B ?>
- # A: define the hostname for managesieve to be the same as the default host configured earlier.
- # B: do not cache mails in the database. This is done since the mail server is running on the same host and there wouldn’t be a big advantage in caching mails in the database. Also, it would require more storage.
Finally, the secret is defined. The setup above is just an illustration, but it shows the keys that I am putting in the secret. In practice I am using a different way to generate the secret using a custom operator, but that is something for another time. The hostname is based on the service name of the MySQL service running in the same kubernetes cluster and the namespace where it is running. The database and user are setup by logging into MySQL as root and running:
create database roundcube; create user 'DBUSER'@'%' identified with mysql_native_password by 'DBPASSWD'; grant all on roundcube.* to 'DBUSER'%';
Access to the database can be done using kubectl port-forward or by logging in to the MySQL container. Since this is quite tedious I am currently using a home grown custom resource for this that performs these tasks.
Finally, the deployment of the StatefulSet for roundcube is as follows (based on docker-compose examples from the roundcube docker project):
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: roundcube
namespace: exposure
spec:
serviceName: roundcube
replicas: 1
selector:
matchLabels:
app: webmail
template:
metadata:
labels:
app: webmail
spec:
hostAliases:
- ip: 192.168.178.123 # A
hostnames:
- k8smail.example.com
containers:
- name: mailserver
image: roundcube/roundcubemail:1.6.0-apache # B
ports:
- containerPort: 80
env:
- name: ROUNDCUBEMAIL_DB_HOST # C
valueFrom:
secretKeyRef:
name: mysql-roundcube-credentials
key: host
- name: ROUNDCUBEMAIL_DB_PORT # C
valueFrom:
secretKeyRef:
name: mysql-roundcube-credentials
key: port
- name: ROUNDCUBEMAIL_DB_USER # C
valueFrom:
secretKeyRef:
name: mysql-roundcube-credentials
key: username
- name: ROUNDCUBEMAIL_DB_PASSWORD # C
valueFrom:
secretKeyRef:
name: mysql-roundcube-credentials
key: password
envFrom: # D
- configMapRef:
name: roundcube-env
volumeMounts:
- name: roundcube-html # E
mountPath: /var/www/html
- name: roundcube-db # E
mountPath: /var/roundcube/db
- name: roundcube-config # F
mountPath: /var/roundcube/config
volumes:
- name: roundcube-config
configMap:
name: roundcube-config
- name: no-update-override
configMap:
name: roundcube-entrypoint-override
defaultMode: 0555
volumeClaimTemplates:
- metadata:
name: roundcube-html
spec:
volumeName: roundcube-html
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
- metadata:
name: roundcube-db
spec:
volumeName: roundcube-db
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
- # A: Add the internal hostname that resolves to the loadbalancer IP to /etc/hosts. In kubernetes it is recommended never to modify /etc/hosts directly and to use hostAliases instead.
- # B: Use a fixed version of roundcube mail. Note that roundcube does an automatic update at startup (which I would prefer could be disabled). In my case, I am relying on network policies to disable internet access for roundcube so it cannot update when it is started. See later
- # C: Database configuration. Leave this out when just using sqlite.
- # D: Environment variables from the roundcube-env ConfigMap.
- # E: The standard, initially empty, installation directories for roundcube.
- # F: The additional configuration file config.php discussed above is mounted here.
The service for roundcube is a cluster IP service:
---
apiVersion: v1
kind: Service
metadata:
name: webmail
namespace: exposure
spec:
type: ClusterIP
selector:
app: webmail
ports:
- name: http
port: 80
A cluster IP service is sufficient here since it will be exposed through an apache server that uses the hostname webmail.exposure for proxying traffic to it. In my setup, I created a new hostname for webmail and all traffic for that hostname is routed to roundcube. This is required since I could not find any options in roundcube to configure a different context root to host it under for example example.com/webmail. The apache snippet looks like this:
<VirtualHost *:80> ServerName roundcube.example.com ProxyPreserveHost on RequestHeader set "X-Forwarded-Proto" https RequestHeader set "X-Forwarded-Port" 443 ProxyPass / http://webmail.exposure/ disablereuse=On ProxyPassReverse / http://webmail.exposure/ </VirtualHost>
Note that HTTPS is not used in this snippet, this is because SSL termination is done using ingress, see my earlier post. The RequestHeader directives are there to explicitly tell roundcube that we are in fact using HTTPS. It is not certain whether roundcube will work without these, but I have encountered so many issues with proxied applications in setups like this that I use these directives as standard.
Persistent volume claims and persistent volumes can be defined as usual, see for example my earlier post.
Network policies
Network policies are required to achieve my goal of microsegmentation, where I define rules to specifically only allow traffic that is required.
First is to allow access from the apache server for the domain where it is running and to allow access to the database at port 3306. It also requires DNS access to find the IP of the database based on its hostname (port 53), and access to SMTP, IMAPS, and managesieve.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: webmail-ingress-egress
namespace: exposure
spec:
podSelector:
matchLabels:
app: webmail
ingress:
- ports:
- port: 80
from:
- podSelector:
matchLabels:
app: httpd-brakkee-org
egress:
- to:
- podSelector:
matchLabels:
app: mail
ports:
- port: 587 # SMTP
- port: 993 # IMAPS
- port: 4190 # managesieve
- to:
- namespaceSelector:
matchLabels:
purpose: database
ports:
- port: 3306 # mysql
- ports:
- port: 53 # DNS
protocol: TCP
- port: 53
protocol: UDP
In addition, we need to allow managesieve on port 4190 on the mail server from roundcube:
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: mail-ingress-managesieve
namespace: exposure
spec:
podSelector:
matchLabels:
app: mail
ingress:
- ports:
- port: 4190
from:
- podSelector:
matchLabels:
app: webmail
The other required mail ports are already allowed by the mailserver network policies.
Finally, the database must accept incoming connections from roundcube:
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: default-allow-roundcube
namespace: database
spec:
podSelector:
matchLabels:
app: mysql-main
ingress:
- from:
- podSelector:
matchLabels:
app: webmail
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: exposure
ports:
- port: 3306
Note that with the current network policies, roundcube cannot update itself since I did not add rules for accessing the internet (ipBlock rules for 0.0.0.0/0). If you want roundcube to update itself, you should add rules to allow this traffic.
Final thoughts
It was relatively easy to setup roundcube. The main bottlenecks in the beginning were getting the secure connections working. This required some work in finding the tls:// and ssl:// prefixes for the host names and to add entries to /etc/hosts to get the host to IP translation to work. Squirrelmail has served me well in the past 10 years, but it looks like roundcube is a worthy successor. The user interface, features, and performance are all exactly what you want.