Install Rsyslog 8 on Elastic Beanstalk

How to install Rsyslog 8 on AWS Elastic Beanstalk

Recently at vagon we needed to experiment with ways to centrally collect logs. One of the setups to be benchmarked was pushing logs to Elasticsearch. We soon realized that we had to jump through a couple of hoops to get this done.

AWS released Amazon Linux 2 almost two years ago in 2018. At the time I’m writing this it’s still in beta for Elastic Beanstalk. Why that’s the case is beyond me but the documentation scary clear:

“If you’re using an Amazon Linux 2 platform version that is in beta for your evaluation, do not go to production. Wait until we release a supported platform version. Beta platform versions aren’t final, and we may change some naming and implementation details before we fully support these platforms.”

Which ordinarily wouldn’t be a problem since Amazon Linux is still a supported platform. Unfortunately this decade old release’s main yum repository only has Rsyslog 5 in it and doesn’t have rsyslog-elasticsearch plugin at all. So we turned to our usual tools for customizing Elastic Beanstalk environments: EB Extensions.

So we’ve created an ebextension to setup rsyslog 8.2:

commands: 01_install_rsyslog: command: yum install rsyslog --disablerepo=amzn-main -y 02_install_rsyslog-gnutls: command: yum install rsyslog-gnutls --disablerepo=amzn-main -y 03_install_rsyslog-elasticsearch: command: yum install rsyslog-elasticsearch --disablerepo=amzn-main -y files: "/etc/yum.repos.d/rsyslog.repo": mod: "000644" owner: root group: root content: | [v8-stable] name=Adiscon CentOS-6 - local packages for \$basearch baseurl=http://rpms.adiscon.com/v8-stable/epel-6/\$basearch enabled=1 gpgcheck=0 gpgkey=http://rpms.adiscon.com/RPM-GPG-KEY-Adiscon protect=1 view raw 21_rsyslog.config hosted with ❤ by GitHub

Let’s walk through what’s going on there. As per AWS documentation the ebextensions’ order of execution is as follows:

• Packages

• Groups

• Users

• Sources

• Files

• Commands

• Services

• Container commands

So files section gets executed first!

That section configures yum to tap into the Adiscon RPM repository which has rsyslog 8 ready to go.

Then the commands section gets executed. The tricks is to bypass amzn-main repository so that we don’t end up installing the old version of rsyslog from there. It’s not really necessary to specify it for rsyslog-elasticsearch package but we did it for the sake of consistency.

When you add 21_rsyslog.config to .ebextensions/ either at build time or directly in your codebase, things will kick into gear and voila! You get what you want.