These days, it is ill-advised to run a website (such as this one), over HTTP, even if there is no security risk at all. When hosting your website on HTTP, users will see a warning triangle in the address bar and many users will simply turn away. This is especially painful if the website is blogging on devops and security related things.<\/p>\n
Currently, I am migrating everything I have locally to a more secure and future proof setup from VMs to kubernetes. The approach I decided to take is to start with the front-facing services and move everything from there step by step. Therefore, the first step is to move the reverse proxy from a VM to kubernetes. Since I don’t want to pay for certificates, I will be using Let’s Encrypt<\/a>. Unfortunately, Let’s Encrypt only supports 90 day certificates so this means a lot of certificate renewals.<\/p>\n <\/p>\n To support this Let’s Encrypt supports the ACME protocol<\/a>, which allows for automatic certificate renewal. The ACME protocol automates the complete process. A certificate renewal process looks like this:<\/p>\n There are various tools for automatically certificates updates based on the ACME protocol, but most of them are for VMs and have options for directly updating apache of nginx configuration files. Imagine what kind of horror this could be if such a script would corrupt your special\/home-grown configuration. On kubernetes things are a lot more clean using cert-manager<\/a>.<\/p>\n Cert-manager introduces a number of concepts:<\/p>\n Install cert-manager by following the instructions on the website. In my case, I did the following:<\/p>\n After this, I used cmctl<\/em> to verify the installation<\/p>\n The cert-manager website contains\u00a0 more instructions to verify the installation using a self-signed certificate. It is recommended to follow these instructions to make sure that everything works.<\/p>\n Next up is the installation of the DNS madeeasy webhook.<\/p>\n The values-dnsmadeeasychallenger.yaml<\/em> configuration file is used to define the groupName<\/em> which is used by the issuer to identify the webhook:<\/p>\n values-dnsmadeeaychallenger.yaml<\/em><\/p>\n Note that I installed the webhook in the same namespace as cert-manager since it is an integral part of certificate management and belongs in the same namespace.<\/p>\n We need to define some essential information for everything to work:<\/p>\n Examples for brakkee.org<\/em> are as follows:<\/p>\n brakkee-org-certificate.yaml<\/em><\/p>\n Note that I am defining the root domain brakkee.org<\/em> as a separate name. This is because the wildcard comain *.brakkee.org<\/em> only covers subdomains of brakkee.org<\/em> but not the root domain brakkee.org<\/em>.<\/p>\n dnsmadeeasy-apikey.yaml<\/em><\/p>\n Apart from this, we need to configure the issuer to use the webhook and to pass the DnsMadeEasy API key and secret on to the webhook to comply to the challenge:<\/p>\n dnsmadeeasy-issuer.yaml<\/em><\/p>\n In the above file, note the server attribute where you can use the staging URL for Let’s Encrypt if you want to test. This is useful because of the strict rate limits on the Let’s Encrypt API. Note the groupName<\/em>\u00a0 which links back to the webhook we installed earlier. Also note the reference to the API key sedret to pass on configuration values to the web hook. The private key is stored in a separate secret (hosting-key-secret). This secret is created by the Issuer.<\/p>\n After applying the above resources, first a temporary secret will be created in the exposure namespace. After it is finished, you should have a new TLS secret brakkee-org-tls-secret in the exposure namespace that can be used in an ingress rule. During the process you can do a kubectl describe<\/em> on the certificate resource to see the progress.<\/p>\n Note that I am creating all these resources in a separate exposure<\/em> namespace. This is the namespace where I will have all SSL termination using ingress. Backend applications will typically be running in different namespaces. This will turn out to be important in a future post where I will go into setting up network policies to improve security.<\/p>\n Using the TLS secret in an ingress rule is straightforward. In my case I am forwarding all traffic for brakkee.org<\/em> and *.brakkee.org<\/em> to the same HTTPD backend service. For example:<\/p>\n A nice trick here is to use a YAML anchor (&proxy_rules). This allows me to avoid duplicating the same rules for brakkee.org<\/em> and *.brakkee.org<\/em>. The backend service httpd-reverse-proxy-brakkee-org<\/em> is not shown in this post but it is not too hard to adapt the example to use your own backend service. It is easy to add more domains with their own certificates simply by adding hosts<\/em> entries to the tls section and by adding rules.<\/p>\n I have tested certificate renewal by using the spec.renewBefore field in the Certificate to force earlier renewal. For example:<\/p>\n It is not possible to request a shorter duration of the certificate since Let’s Encrypt always generates 90 day certificates. Just choose renewBefore counting back from the 90 day expiry time of the certificate.<\/p>\n To force regeneration of the secret you can just delete the generated secret and it will get generated again. The same applies to the host key secret.<\/p>\n If you run into a rate limit at Let’s Encrypt, note that the rate limit applies to the list of requested DNS alternative names (*.brakkee.org<\/em>, brakkee.org<\/em> in my case). The current rate limit is 5 times per week for each unique combination of DNS alternative names. If you run into this limit, then temporarily add another DNS alternative name such as\u00a0x.y.brakkee.org<\/em> to the list. This name does not fall under *.brakkee.org<\/em>. Using this it is easy to work around the limit. It is of course always better to use the staging URL of Let’s Encrypt before you start using tricks.<\/p>\n To use the DnsMadeEasy APIs I had to upgrade my membership to a business membership which costs 75 USD per year. But at least I am not manually updating certificates and not paying a similar amount per domain.<\/p>\n","protected":false},"excerpt":{"rendered":" These days, it is ill-advised to run a website (such as this one), over HTTP, even if there is no security risk at all. When hosting your website on HTTP, users will see a warning triangle in the address bar … Continue reading The ACME protocol<\/h2>\n
\n
Automatic renewal: cert-manager<\/h2>\n
\n
Installation<\/h2>\n
helm repo add jetstack https:\/\/charts.jetstack.io\nhelm repo updatehelm install \\\n cert-manager jetstack\/cert-manager \\\n --namespace cert-manager \\\n --create-namespace \\\n --version v1.8.0 \\\n --set installCRDs=true<\/pre>\n
OS=$(go env GOOS); ARCH=$(go env GOARCH); curl -sSL -o cmctl.tar.gz https:\/\/github.com\/cert-manager\/cert-manager\/releases\/download\/v1.7.2\/cmctl-$OS\n-$ARCH.tar.gz\ntar xvf cmctl.tar.gz\ncmctl check api<\/pre>\n
helm repo add k8s-at-home https:\/\/k8s-at-home.com\/charts\/\nhelm repo updatehelm install dnschallenger k8s-at-home\/dnsmadeeasy-webhook --namespace cert-manager -f values-dnsmadeeasychallenger.yaml \n\n<\/pre>\n
# This name will need to be referenced in each Issuer's `webhook` stanza to\n# inform cert-manager of where to send ChallengePayload resources in order to\n# solve the DNS01 challenge.\n# This group name should be **unique**, hence using your own company's domain\n# here is recommended.\ngroupName: dnsmadeeasy.challenger<\/pre>\n
Configuration<\/h2>\n
\n
apiVersion: cert-manager.io\/v1\nkind: Certificate\nmetadata:\n name: wildcard-brakkee-org\n namespace: exposure\nspec:\n dnsNames:\n - \"*.brakkee.org\"\n - \"brakkee.org\"\n issuerRef:\n name: dnsmadeeasy-issuer\n kind: Issuer\n secretName: brakkee-org-tls-secret\n<\/pre>\n
apiVersion: v1\nkind: Secret\nmetadata:\n name: dnsmadeeasy-apikey\n namespace: cert-manager\ntype: Opaque\nstringData:\n key: your_key_here\n secret: your_secret_here\n\n<\/pre>\n
apiVersion: cert-manager.io\/v1\nkind: Issuer\nmetadata:\n name: dnsmadeeasy-issuer\n namespace: exposure\nspec:\n acme:\n #server: https:\/\/acme-staging-v02.api.letsencrypt.org\/directory\n server: https:\/\/acme-v02.api.letsencrypt.org\/directory\n email: info@brakkee.org\n privateKeySecretRef:\n name: hosting-key-secret\n solvers:\n - dns01:\n webhook:\n groupName: dnsmadeeasy.challenger\n solverName: dnsmadeeasy\n config:\n apiKeyRef:\n name: dnsmadeeasy-apikey\n key: key\n apiSecretRef:\n name: dnsmadeeasy-apikey\n key: secret\n\n<\/pre>\n
Using the TLS secret in an ingress rule<\/h2>\n
apiVersion: networking.k8s.io\/v1\nkind: Ingress\nmetadata:\n name: ingress-brakkee-org\n namespace: exposure\nspec: \n tls: \n - hosts:\n - \"*.brakkee.org\"\n - \"brakkee.org\"\n secretName: brakkee-org-tls-secret\n rules:\n - host: \"brakkee.org\"\n http: &proxy_rules\n paths:\n - path: \/\n pathType: Prefix \n backend:\n service:\n name: httpd-reverse-proxy-brakkee-org\n port: \n number: 80\n - host: \"*.brakkee.org\"\n http: *proxy_rules\n\n<\/pre>\n
Certificate renewal<\/h2>\n
apiVersion: cert-manager.io\/v1\nkind: Certificate\nmetadata:\n name: wildcard-brakkee-org\n namespace: exposure\nspec:\n renewBefore: 2136h\n ...<\/pre>\n
Force regeneration of the certificate<\/h2>\n
Rate limits<\/h2>\n
Costs<\/h2>\n