This is a continuation of my earlier post on migrating my mailserver to kubernetes. The next component of my mail setup to migrate is webmail. In the past I used squirrelmail for this, so I started with investigating that. However, it turns out that the squirrelmail project does not provide any docker containers. There are some containers you can get from docker hub, but these are largely unmaintained and are mostly private projects. After some looking around I found roundcube. Quick prototyping with roundcube containers, provided by the roundcube project, using docker compose showed that it was not difficult to get it to work.
And most important of all, roundcube provides a much more mature user interface than squirrelmail and is also quite fast. In fact, on my mobile phone, I find that roundcube competes directly with K9 mail on Android. It has for instance support for writing HTML mails and provides things like address books. For the latter functionality it requires some persistent storage in an SQL database. By default it will use sqlite, allowing users to get a working system without having to run a separate database, and it also supports mysql and postgressql as external databases.
Webmail requires access to SMTP, IMAPS, and managesieve. To keep things simple, it is required to use one and the same IP address for all these services. This means that at the kubernetes level, we need one service that provides SMTP, IMAPS, and managesieve. Additionally, we need to use a hostname for which SMTP, IMAPS, and managesieve have the correct certificates. So, in summary, we need a valid host (e.g. mail.example.com if the mail server has a valid wildcard certificate for *.example.com). To keep the setup simple, I will use the loadbalancer service which has a fixed IP and use network policies to limit access to managesieve to only the roundcube component. Alternatively, it would perhaps be possible to use invalid certificates and have roundcube accept them, but that would be going against the flow.
The setup is as follows:
A StatefulSet is used for deployment instead of a Deployment. This is done to avoid problems with multiple containers running at the same time using the same storage at the same time. This can be problematic since roundcube initializes when it starts up for the first time and I am not sure that multiple roundcube containers can do this at the same time. Given the setup with an external database, it would be ok to run multiple pods for roundcube, each connecting to the same database.
Roundcube itself is configured by several environment variables in the roundcube-env ConfigMap, by a config.php in the the roundcube-config ConfigMap, and by database credentials in the mysql-roundcube-credentials Secret.
These configmaps and secrets are generated using kustomize:
kind: Kustomization namespace: exposure generatorOptions: disableNameSuffixHash: true configMapGenerator: - name: roundcube-env envs: - config.env - name: roundcube-config files: - config.php secretGenerator: - name: mysql-roundcube-credentials literals: - username=DBUSER - password=DBPASSWD - host=SERVICE.NAMESPACE - port=DBPORT resources: - volumes.yaml - statefulset.yaml - service.yaml
The config.env file contains general environment variables for roundcube:
ROUNDCUBEMAIL_DEFAULT_HOST=ssl://k8smail.example.com # A ROUNDCUBEMAIL_SMTP_SERVER=tls://k8smail.example.com # A ROUNDCUBEMAIL_DEFAULT_PORT=993 ROUNDCUBEMAIL_PLUGINS=archive,zipdownload,managesieve # B ROUNDCUBEMAIL_UPLOAD_MAX_FILESIZE=25M ROUNDCUBEMAIL_DB_TYPE=mysql # C ROUNDCUBEMAIL_DB_NAME=roundcube
- # A: We need to use ssl:// and tls:// to indicate we want to use secure connections. Also note that roundcube supports a single host for SMTP, IMAPS, and managesieve out of the box. The host name k8smail.example.com will be configured in the pod definition.
- # B: the managesieve plugin is not enabled by default so we add it, next to some other plugins
- # C: configure the database type and database name. Leave this out in an initial setup when using sqlite or when you don’t want to use an external database.
The config.php is additional config for roundcube and we need this to provide some custom settings:
<?php $config['managesieve_host'] = 'tls://%h'; # A $config['enable_caching'] = FALSE; # B ?>
- # A: define the hostname for managesieve to be the same as the default host configured earlier.
- # B: do not cache mails in the database. This is done since the mail server is running on the same host and there wouldn’t be a big advantage in caching mails in the database. Also, it would require more storage.
Finally, the secret is defined. The setup above is just an illustration, but it shows the keys that I am putting in the secret. In practice I am using a different way to generate the secret using a custom operator, but that is something for another time. The hostname is based on the service name of the MySQL service running in the same kubernetes cluster and the namespace where it is running. The database and user are setup by logging into MySQL as root and running:
create database roundcube; create user 'DBUSER'@'%' identified with mysql_native_password by 'DBPASSWD'; grant all on roundcube.* to 'DBUSER'%';
Access to the database can be done using kubectl port-forward or by logging in to the MySQL container. Since this is quite tedious I am currently using a home grown custom resource for this that performs these tasks.
Finally, the deployment of the StatefulSet for roundcube is as follows (based on docker-compose examples from the roundcube docker project):
apiVersion: apps/v1 kind: StatefulSet metadata: name: roundcube namespace: exposure spec: serviceName: roundcube replicas: 1 selector: matchLabels: app: webmail template: metadata: labels: app: webmail spec: hostAliases: - ip: 192.168.178.123 # A hostnames: - k8smail.example.com containers: - name: mailserver image: roundcube/roundcubemail:1.6.0-apache # B ports: - containerPort: 80 env: - name: ROUNDCUBEMAIL_DB_HOST # C valueFrom: secretKeyRef: name: mysql-roundcube-credentials key: host - name: ROUNDCUBEMAIL_DB_PORT # C valueFrom: secretKeyRef: name: mysql-roundcube-credentials key: port - name: ROUNDCUBEMAIL_DB_USER # C valueFrom: secretKeyRef: name: mysql-roundcube-credentials key: username - name: ROUNDCUBEMAIL_DB_PASSWORD # C valueFrom: secretKeyRef: name: mysql-roundcube-credentials key: password envFrom: # D - configMapRef: name: roundcube-env volumeMounts: - name: roundcube-html # E mountPath: /var/www/html - name: roundcube-db # E mountPath: /var/roundcube/db - name: roundcube-config # F mountPath: /var/roundcube/config volumes: - name: roundcube-config configMap: name: roundcube-config - name: no-update-override configMap: name: roundcube-entrypoint-override defaultMode: 0555 volumeClaimTemplates: - metadata: name: roundcube-html spec: volumeName: roundcube-html accessModes: - ReadWriteOnce resources: requests: storage: 10Gi - metadata: name: roundcube-db spec: volumeName: roundcube-db accessModes: - ReadWriteOnce resources: requests: storage: 10Gi
- # A: Add the internal hostname that resolves to the loadbalancer IP to /etc/hosts. In kubernetes it is recommended never to modify /etc/hosts directly and to use hostAliases instead.
- # B: Use a fixed version of roundcube mail. Note that roundcube does an automatic update at startup (which I would prefer could be disabled). In my case, I am relying on network policies to disable internet access for roundcube so it cannot update when it is started. See later
- # C: Database configuration. Leave this out when just using sqlite.
- # D: Environment variables from the roundcube-env ConfigMap.
- # E: The standard, initially empty, installation directories for roundcube.
- # F: The additional configuration file config.php discussed above is mounted here.
The service for roundcube is a cluster IP service:
--- apiVersion: v1 kind: Service metadata: name: webmail namespace: exposure spec: type: ClusterIP selector: app: webmail ports: - name: http port: 80
A cluster IP service is sufficient here since it will be exposed through an apache server that uses the hostname webmail.exposure for proxying traffic to it. In my setup, I created a new hostname for webmail and all traffic for that hostname is routed to roundcube. This is required since I could not find any options in roundcube to configure a different context root to host it under for example example.com/webmail. The apache snippet looks like this:
<VirtualHost *:80> ServerName roundcube.example.com ProxyPreserveHost on RequestHeader set "X-Forwarded-Proto" https RequestHeader set "X-Forwarded-Port" 443 ProxyPass / http://webmail.exposure/ disablereuse=On ProxyPassReverse / http://webmail.exposure/ </VirtualHost>
Note that HTTPS is not used in this snippet, this is because SSL termination is done using ingress, see my earlier post. The RequestHeader directives are there to explicitly tell roundcube that we are in fact using HTTPS. It is not certain whether roundcube will work without these, but I have encountered so many issues with proxied applications in setups like this that I use these directives as standard.
Persistent volume claims and persistent volumes can be defined as usual, see for example my earlier post.
Network policies are required to achieve my goal of microsegmentation, where I define rules to specifically only allow traffic that is required.
First is to allow access from the apache server for the domain where it is running and to allow access to the database at port 3306. It also requires DNS access to find the IP of the database based on its hostname (port 53), and access to SMTP, IMAPS, and managesieve.
kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: webmail-ingress-egress namespace: exposure spec: podSelector: matchLabels: app: webmail ingress: - ports: - port: 80 from: - podSelector: matchLabels: app: httpd-brakkee-org egress: - to: - podSelector: matchLabels: app: mail ports: - port: 587 # SMTP - port: 993 # IMAPS - port: 4190 # managesieve - to: - namespaceSelector: matchLabels: purpose: database ports: - port: 3306 # mysql - ports: - port: 53 # DNS protocol: TCP - port: 53 protocol: UDP
In addition, we need to allow managesieve on port 4190 on the mail server from roundcube:
kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: mail-ingress-managesieve namespace: exposure spec: podSelector: matchLabels: app: mail ingress: - ports: - port: 4190 from: - podSelector: matchLabels: app: webmail
The other required mail ports are already allowed by the mailserver network policies.
Finally, the database must accept incoming connections from roundcube:
kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: default-allow-roundcube namespace: database spec: podSelector: matchLabels: app: mysql-main ingress: - from: - podSelector: matchLabels: app: webmail namespaceSelector: matchLabels: kubernetes.io/metadata.name: exposure ports: - port: 3306
Note that with the current network policies, roundcube cannot update itself since I did not add rules for accessing the internet (ipBlock rules for 0.0.0.0/0). If you want roundcube to update itself, you should add rules to allow this traffic.
It was relatively easy to setup roundcube. The main bottlenecks in the beginning were getting the secure connections working. This required some work in finding the tls:// and ssl:// prefixes for the host names and to add entries to /etc/hosts to get the host to IP translation to work. Squirrelmail has served me well in the past 10 years, but it looks like roundcube is a worthy successor. The user interface, features, and performance are all exactly what you want.