Effective log management is a critical aspect of maintaining any IT infrastructure. Logs serve as a valuable source of information for monitoring system health, troubleshooting errors, and ensuring security compliance. However, managing and analyzing large volumes of log data manually can become overwhelming, especially in modern systems with multiple applications and servers. This is where Graylog, a powerful open-source log management platform, makes life easier. Graylog enables centralized log collection, real-time data analysis, and streamlined alerting, making it a crucial tool for system administrators and DevOps teams. Its web-based interface used in combination with Elasticsearch and MongoDB as its backend, provides the solution for efficiently managing log data in dynamic environments.
This guide will walk you through the process of installing and configuring Graylog on Ubuntu 22.04, along with setting up Elasticsearch and MongoDB server.
Prerequisites
- A Virtual Machine (such as the ones provided by NodeShift) with at least:
- 2 vCPUs
- 8 GB RAM
- 20 GB SSD
- Ubuntu 22.04 VM
Note: The prerequisites for this are highly variable across use cases. A high-end configuration could be used for a large-scale deployment.
Step-by-step process to install Graylog on Ubuntu 22.04
For this tutorial, we’ll use a CPU-powered Virtual Machine by NodeShift, which provides high-compute Virtual Machines at a very affordable cost on a scale that meets GDPR, SOC2, and ISO27001 requirements. It also offers an intuitive and user-friendly interface, making it easier for beginners to get started with Cloud deployments. However, feel free to use any cloud provider you choose and follow the same steps for the rest of the tutorial.
Step 1: Setting up a NodeShift Account
Visit app.nodeshift.com and create an account by filling in basic details, or continue signing up with your Google/GitHub account.
If you already have an account, login straight to your dashboard.
Step 2: Create a Compute Node (CPU Virtual Machine)
After accessing your account, you should see a dashboard (see image), now:
- Navigate to the menu on the left side.
- Click on the Compute Nodes option.
- Click on Start to start creating your very first compute node.
These Compute nodes are CPU-powered virtual machines by NodeShift. These nodes are highly customizable and let you control different environmental configurations, such as vCPUs, RAM, and storage, according to your needs.
Step 3: Select configuration for VM
- The first option you see is the Reliability dropdown. This option lets you choose the uptime guarantee level you seek for your VM (e.g., 99.9%).
- Next, select a geographical region from the Region dropdown where you want to launch your VM (e.g., United States).
- Most importantly, select the correct specifications for your VM according to your workload requirements by sliding the bars for each option.
Step 4: Choose VM Configuration and Image
- After selecting your required configuration options, you’ll see the available VMs in your region and as per (or very close to) your configuration. In our case, we’ll choose a ‘4vCPUs/8GB/160GB SSD’ as the closest match to the “Prerequisites”.
- Next, you’ll need to choose an image for your Virtual Machine. For the scope of this tutorial, we’ll select Ubuntu, as we will install Graylog on the Ubuntu server.
Step 5: Choose the Billing cycle and Authentication Method
- Two billing cycle options are available: Hourly, ideal for short-term usage, offering pay-as-you-go flexibility, and Monthly for long-term projects with a consistent usage rate and potentially lower cost.
- Next, you’ll need to select an authentication method. Two methods are available: Password and SSH Key. We recommend using SSH keys, as they are a more secure option. To create one, head over to our official documentation.
Step 6: Finalize Details and Create Deployment
Finally, you can also add a VPC (Virtual Private Cloud), which provides an isolated section to launch your cloud resources (Virtual machine, storage, etc.) in a secure, private environment. We’re keeping this option as the default for now, but feel free to create a VPC according to your needs.
Also, you can deploy multiple nodes at once using the Quantity option.
That’s it! You are now ready to deploy the node. Finalize the configuration summary; if it looks good, go ahead and click Create to deploy the node.
Step 7: Connect to active Compute Node using SSH
As soon as you create the node, it will be deployed in a few seconds or a minute. Once deployed, you will see a status Running in green, meaning that our Compute node is ready to use!
Once your node shows this status, follow the below steps to connect to the running VM via SSH:
- Open your terminal and run the below SSH command:
(replace root with your username and paste the IP of your VM in place of ip after copying it from the dashboard)
ssh root@ip
2. In some cases, your terminal may take your consent before connecting. Enter ‘yes’.
3. A prompt will request a password. Type the SSH password, and you should be connected.
Output:
Step 8: Install dependencies
- Update the Ubuntu package source-list.
apt update
Output:
2. Install Dependencies required for the installation of Graylog.
apt install curl wget apt-transport-https
Output:
3. Another dependency is Java; we’ll install OpenJDK, which is the open-source implementation of Java.
apt install openjdk-17-jre-headless -y
Output:
4. Confirm Java installation.
java --version
Output:
Step 9: Install Elasticsearch
Elasticsearch is essential in Graylog as it stores, indexes, and enables efficient searching of the large volumes of log data collected. It powers Graylog’s querying and visualization capabilities for log analysis and monitoring.
- Download and add the Elasticsearch GPG key.
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Output:
2. Add the Elasticsearch repository to the Ubuntu package list.
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
Output:
3. Update the package source-list once again for changes to take effect.
apt update
4. Install Elasticsearch.
apt install elasticsearch -y
Output:
5. Edit the Elasticsearch configuration file.
a) Open the yml file using Nano.
nano /etc/elasticsearch/elasticsearch.yml
b) Add the following content to the Cluster section.
(replace <CLUSTER_NAME> with your preferred cluster name.)
cluster.name: <CLUSTER_NAME>
action.auto_create_index: false
This is how it looks in the file:
Save and close the editor (Ctrl+O > ENTER > Ctrl+X).
6. Reload the system, start the Elasticsearch service, and check the status.
Run the following commands one by one:
systemctl daemon-reload
systemctl start elasticsearch
systemctl status elasticsearch
Output:
Then, enable the Elasticsearch service to allow it to start on system boot.
systemctl enable elasticsearch
Output:
Finally, you may send a GET request to the Elasticsearch node to check the information.
curl -X GET http://localhost:9200
Output:
Step 10: Install the MongoDB server
In Graylog, MongoDB serves as the metadata storage for configurations, user accounts, dashboards, and other internal settings. It does not store log data but maintains the operational data necessary for Graylog to function efficiently.
- Add the MongoDB GPG key.
curl -fsSL https://pgp.mongodb.com/server-6.0.asc | \
sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/mongodb-server-6.0.gpg
2. Add the MongoDB repository to the Ubuntu package source-list.
echo "deb [ arch=amd64,arm64 signed=/etc/apt/trusted.gpg.d/keyrings/mongodb-server-6.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
3. Update the package source-list once again for changes to take effect.
apt update
4. Install the MongoDB server.
apt install mongodb-org -y
Output:
5. Confirm the installation.
mongod --version
Output:
6. Start the MongoDB service and check the status.
Run the following commands one by one:
systemctl start mongod
systemctl status mongod
Output:
You may enable the service to allow it to start on boot.
systemctl enable mongod
Output:
Step 11: Install the Graylog server
Now, we’ll finally start the installation and setup of the Graylog server from this step onwards.
- Download the Graylog package.
wget https://packages.graylog2.org/repo/packages/graylog-5.2-repository_latest.deb
Output:
2. Run the package.
(replace <YOUR_GRAYLOG_FILENAME> with the downloaded Graylog’s package filename)
dpkg -i <YOUR_GRAYLOG_FILENAME>
Output:
3. Update the local APT cache for changes to take effect.
apt-get update
Output:
4. Install the Graylog server.
apt install graylog-server -y
Output:
Once the installation is complete, we’ll need to generate a password secret and encrypted password for the admin user. With this password we’ll be able to login to Graylog web UI.
5. Generate password secret for securing user passwords.
< /dev/urandom tr -dc A-Z-a-z-0-9 | head -c${1:-96};echo;
Output:
6. Generate an encrypted password for the admin.
Run the following command and enter a strong password when the prompt opens:
echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Output:
As you enter your password, it will generate an encrypted version. Copy the above password secret and encrypted password and note them somewhere, as we’ll need them in the next step.
7. Open the Graylog configuration file and scroll down to paste the above two passwords at their designated places.
nano /etc/graylog/server/server.conf
This is where the passwords should be in the file:
In the same file above, scroll down more, and you’ll find the http_bind_address attribute. Replace its value with <YOUR_SERVER_IP>:9000, then save and close the editor.
8. Reload the system, start the Graylog service, and check the status.
Run the following commands one by one:
systemctl daemon-reload
systemctl start graylog-server
systemctl status graylog-server
Finally, enable the Graylog service to allow it to start on system boot.
systemctl enable graylog-server
Step 12: Configure NGINX as reverse proxy
In the following steps, we’ll configure a reverse proxy using NGINX. Configuring a reverse proxy for Graylog will help in secure and simplified access to its web interface. It allows you to use a custom domain, enable HTTPS, handle load balancing, and much more.
- Install NGINX.
apt install nginx
Output:
2. Create a virtual host file for Graylog.
nano /etc/nginx/sites-available/graylog.conf
3. Add the following configuration script to the file.
(replace graylog.example.org with your domain name for Graylog or your remote server’s IP, and <YOUR_SERVER_IP> in the proxy_pass with your server’s IP).
server {
listen 80;
server_name graylog.example.org;
location /
{
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Graylog-Server-URL http://$server_name/;
proxy_pass http://<YOUR_SERVER_IP>:9000;
}
}
This is how the file looks:
4. Check the syntax.
nginx -t
Output:
As you may see in the above image, the output says that the test has failed. This can occur because of NGINX not working properly. To fix this we’ll take the reference from the “Troubleshoot NGINX Errors” section in the NGINX article which works most of the time.
After the fix, we’ll run the above command once again, and this time, it’s a success.
5. Moving forward, enable the NGINX virtual host file.
ln -s /etc/nginx/sites-available/graylog.conf /etc/nginx/sites-enabled/
And delete the default virtual host file to prevent it from overriding the new virtual host file.
rm -rf /etc/nginx/sites-enabled/default
6. Finally, restart the NGINX service, and check the status.
systemctl restart nginx
systemctl status nginx
Output:
Step 13: Access the Graylog web interface
Now, you can finally access Graylog’s web interface on the browser with the following URL.
(replace <YOUR_SERVER_IP> with your server’s IP address)
http://<YOUR_SERVER_IP>
We’ll need to enter the “admin” as the username and the password that we entered in Step 11 > Substep 6.
Once you’ve successfully logged in, you’ll see an interface, as shown in the below image. You can use this Graylog dashboard to monitor, search, and for real-time data analysis of logged data for effective troubleshooting and system insights.
Conclusion
Installing Graylog on Ubuntu 22.04 helps you with a powerful tool for centralized log management and real-time analysis. By leveraging Elasticsearch and MongoDB, Graylog ensures efficient log storage and querying, while NGINX, as a reverse proxy, secures and simplifies access. We have deployed our Graylog server on NodeShift, which helps in enhancing this whole setup by offering a reliable, scalable infrastructure that supports Graylog’s performance needs.
