3KeyPKITechnical

Certificates Monitoring and Alerting

In the previous post regarding the EJBCA Dashboards and Reporting we have shown how easily you can setup monitoring for your certification authorities and create visualizations based on that.

Now we will go further and create a tool for monitoring and alerting of certificate attributes. In our example below we will focus on monitoring certificate expiration date and will create e-mail notification if certificates are going to be expired. However, the same approach can be used in order to monitor compliance of the certificates, for example:

  • signature algorithms, public key type and size
  • extension data
  • subject name conventions
  • self-signed certificates

We will build it as an extension of our ELK stack, with the additional parsing rules for the certificate data. Moreover, we will setup and configure alerting, which will be running periodically and will trigger e-mail notification in case some of the certificates are before expiration.

Extending Logstash configuration

We need to extend our Logstash pipeline configuration in order to parse the base64 encoded certificate and index its attributes. There are some options how we can do that:

  • log the data on application and index attributes as they are through Logstash
  • use Logstash ruby filter plugin and get the attributes from the certificate through OpenSSL
  • develop custom Logstash filter plugin which will output the attributes into Elasticsearch
  • pre-process data in between the application and Logstash, or send it directly to Elasticsearch

I believe there are many more alternatives how to achieve the same. We will use the ruby filter plugin as it is out of the box supported by the Logstash.

The following is a simple filter to get the basic attributes of the certificate:

ruby {
  code => '
    cert = Base64.decode64(event.get("[ejbca][additional][cert]"))
    certificate = OpenSSL::X509::Certificate.new cert

    event.set("[ejbca][certificate][issuer]", certificate.issuer.to_s(OpenSSL::X509::Name::RFC2253))
    event.set("[ejbca][certificate][subject]", certificate.subject.to_s(OpenSSL::X509::Name::RFC2253))
    event.set("[ejbca][certificate][not_after]", certificate.not_after)
    event.set("[ejbca][certificate][not_before]", certificate.not_before)
    event.set("[ejbca][certificate][serial][dec]", certificate.serial.to_s)
    event.set("[ejbca][certificate][serial][hex]", certificate.serial.to_s(16))
    event.set("[ejbca][certificate][signature_algorithm]", certificate.signature_algorithm)
    event.set("[ejbca][certificate][version]", certificate.version)

    if certificate.public_key.is_a?(OpenSSL::PKey::RSA)
      event.set("[ejbca][certificate][public_key][modulus]", certificate.public_key.n.to_s(16))
      event.set("[ejbca][certificate][public_key][exponent]", certificate.public_key.e.to_s(16))
      event.set("[ejbca][certificate][public_key][size]", certificate.public_key.n.num_bytes * 8)
      event.set("[ejbca][certificate][public_key][type]", "RSA")
      event.set("[ejbca][certificate][public_key][base64]", certificate.public_key.to_s.lines[1..-2].join)
    else
      event.set("[ejbca][certificate][public_key][type]", "not implemented")
    end
  '
}

As you can see, it takes as an input base64 encoded certificate and creates fields containing attributes like issuer DN, subject DN, not before date, not after date, serial number in decimal and hexadecimal format, version, and public key attributes in case the public key is RSA, modulus, exponent, bit size of the key, and its base64 encoded format.

When all of this information are available and indexed, we can start to implement our certificate tool in Kibana.

Preparing visualizations and dashboard

Using the certificate attributes, we can prepare dashboard containing the following visualizations. You can use your imagination and prepare different types of visualizations to include in the final dashboard, it is up to you.

  • number of expired and non-expired certificates
  • number of certificates expiring in 30 days, 60 days, and 90 days
  • distribution of public key sizes
  • distribution of signature algorithms
  • heat map of public key sizes in relation with signature algorithms
  • table of certificates with nearest expiration date
  • distribution of public key types

The result can look like this:

Alerting

What would be our monitoring tool without ability to alert and notify events? Alerting is a very important feature in order to achieve full automation of certificate lifecycle management. Again, we have some options here how to build it and configure:

  • using Watcher functionality of ELK stack
  • implementing our own solution using the alerting rules stored inside index
  • integrating with open source plugins and add-ons (like elastalert or Sentinl)
  • development of custom alerting module

We will show how to use Watcher functionality of the ELK stack, as it is out of the box solution without additionally developing or installing other components. Watcher is used to create actions based on conditions, called Watches, which are periodically evaluated using queries on your data. It has a simple structure in JSON:

  • trigger – contains definition of schedule when the Watcher should be triggered
  • input – loads the data that should be part of checking. It can be defined as search through the data we have indexed
  • condition – specification of rules and controls whether the actions should be executed
  • actions – specifies what should happen when conditions are met

In case you would like to know more, you can always read the complete documentation of the Watcher.

Expiring certificates notification

Let’s create a Watch which will be checking our certificates once a day at midnight and will send us e-mail notification about the certificates that are going to be expired in 30 days with serial numbers we can use for identification.

Start by creating an advanced Watch in Management > Elasticsearch > Watcher, put the name of the Watch and start implementing its behaviour in JSON. As a trigger we can use the following:

"trigger": {
  "schedule": {
    "daily": {
      "at": [
        "midnight"
      ]
    }
  }
}

The Watch will be triggered each day at midnight, now continue with the input part. As an index we will take our ejbca-* index (actually it consists of all indexes starting with ejbca-) and we are searching for all certificates which has the not_after filed less than or equal to current date + 30 days:

"input": {
  "search": {
    "request": {
      "search_type": "query_then_fetch",
      "indices": [
        "ejbca-*"
      ],
      "rest_total_hits_as_int": true,
      "body": {
        "query": {
          "range": {
            "ejbca.certificate.not_after": {
              "lte": "now+30d"
            }
          }
        }
      }
    }
  }
}

The condition part of the Watch will take the data from input (if any) and will apply the rules and controls. In our case we would like to know if there are any certificates which will be expired in 30 days, so technically we would like to know if the input part contains some payload which is greater than or equal to 1:

"condition": {
  "compare": {
    "ctx.payload.hits.total": {
      "gte": 1
    }
  }
}

And finally, we need to define action, what will happen when the condition is true. We will construct a simple e-mail template with html, which will iterate through all certificates that are going to be expired, and notify us about it:

"actions": {
  "send_email": {
    "email": {
      "profile": "standard",
      "from": "notifications@3key.company",
      "to": [
        "info@3key.company"
      ],
      "subject": "Certificate Expiration Watcher Notification",
      "body": {
        "html": "The following certificates identified by serial number will be expired in 30 days:

{{#ctx.payload.hits.hits}}{{_source.ejbca.certificate.serial.hex}}
{{/ctx.payload.hits.hits}}
Please renew the certificates before it will be late!" } } } }

And that’s it! We can simulate the Watch in order to confirm the configuration and save it.

Conclusion

We have shown how easy is to extend reporting of the certificates with the alerting functionality. This is very important step toward automation of the PKI, towards reducing outages associated with the expired certificates and reducing operational costs.

Keep in mind that the PKI depends not only on the technology used but relies very heavily on people and procedures. You can easily lose trust when you will not maintain it properly!

Get in touch with us to know more!

You can find the configuration and setup code on the GitHub page.

We are also on LinkedIn, follow us!

If you like what we are doing than do not hesitate and contact us for more information! We believe that we have experience and knowledge that can be a benefit for your business.