Complete guide to Linux server optimization

Informatec Digital » Resources » Complete guide to Linux server optimization

If you manage Linux servers in production, you know that simply building a powerful machine and running it isn't enough. Truly maximizing performance, security, and stability requires fine-tuning the kernel , network, storage, services, and even the organization of the administrative workload.

This article brings together and reorganizes many of the best ideas on Linux server optimization, security hardening, and automation into a single practical guide . You'll see how to reduce latency, avoid bottlenecks, strengthen your system against attacks, and automate resource-intensive tasks so your infrastructure stays running smoothly without you having to write countless commands every day.

Why is it so important to optimize and strengthen a Linux server?

In many companies, Linux is the heart of the infrastructure: it hosts databases, web applications, VoIP services, microservices, and compute-intensive workloads . If the server is poorly configured, latency, crashes, I/O blocking, and, as a bonus, easily exploitable security vulnerabilities will appear.

System hardening aims to close attack vectors, limit privileges, and reduce the attack surface . Simultaneously, performance optimization seeks to ensure that every CPU cycle, every megabyte of RAM, and every disk operation is used efficiently, avoiding waste that reduces the capacity of critical applications.

Furthermore, managing dozens or hundreds of servers manually is unthinkable today. Tools like Ansible, Prometheus, Zabbix, Nagios, and TSplus Server Monitoring allow you to automate deployments, apply consistent configurations, and monitor system health in real time, much more reliably than accessing each server individually via SSH.

Taken together, strengthening, optimizing, and automating transform a generic Linux server into a robust, fast, and easily scalable platform , capable of responding to load spikes and threats without breaking down at the first sign of trouble.

Keep the system lightweight: services, packages, and ports

The first step to ensuring a Linux server performs well is to avoid unnecessary tasks . No matter how powerful the hardware, if you have half a dozen pointless daemons hogging CPU, RAM, and disk space, everything will suffer.

Start by checking the services enabled at startup with something like ` systemctl list-unit-files --state=enabled` . From there, disable anything that doesn't make sense on a server: Bluetooth, print services, automatic discovery daemons like Avahi, graphical environments, etc.

The idea is that on a production server, only SSH, the firewall, the monitoring agent, application daemons, storage services, and little else should remain active . This not only frees up resources but also reduces the number of processes exposed to potential vulnerabilities.

The same principle applies to installed packages. It's advisable to list the software using your distribution's tools (rpm, yum, dnf, apt, etc.) and remove unused packages that increase your attack surface . However, always check your dependencies to avoid accidentally breaking something critical.

Finally, it's good practice to occasionally audit listening ports using commands like `ss -tuln` or `netstat -tulnp` . If you find irrelevant services or open ports you don't use, close them or restrict them with a firewall and specific configuration.

CPU optimization: priorities, affinity, and scheduling

Linux uses the Completely Fair Scheduler (CFS) by default , designed to distribute CPU resources fairly among processes. It works very well for general use, but on servers that support databases, streaming systems, VoIP, or near real-time workloads, low latency and predictability are sometimes prioritized over absolute fairness.

One of the most straightforward tools is renice, which allows you to raise or lower the priority (nice) of running processes. By lowering the priority of secondary tasks and raising it for key processes (such as the database server), you'll reduce latency spikes when there's a lot of CPU contention.

Another important lever is the use of real-time priorities with `chrt` for certain highly sensitive processes (always with great care), and CPU affinity with taskset, which allows you to assign processes to a subset of cores. This can reduce context switching between CPUs and improve performance in very intensive tasks.

In ultra-demanding environments, low-latency kernels or PREEMPT_RT kernels are also considered , designed for high-frequency trading, telecom, or industrial applications where milliseconds matter. They don't make sense on every server, but they are another tool in the arsenal.

Fine-tuning RAM memory: swap, caches, and HugePages

Memory is one of the most critical resources: when it's scarce, the system resorts to swapping, and performance plummets. That's why it's crucial that the server uses RAM efficiently and consistently with the workload.

The kernel's `vm.swappiness` parameter controls how much the system uses swap space. On servers with ample memory, it's common to lower its value to minimize swapping , allowing the kernel to remain in RAM longer before writing pages to disk.

Also worth noting is `vm.vfs_cache_pressure`, which affects the behavior of inode and dentry caches. Adjusting it appropriately allows you to balance memory allocation between file system cache and application memory , which is crucial for file servers or databases with many small operations.

For large databases or JVM-intensive applications, static HugePages or Transparent Huge Pages (THP) make a significant difference. Configuring `vm.nr_hugepages` to reserve appropriately large pages reduces memory management overhead and can improve performance under very heavy workloads, provided the application is designed to take advantage of them.

Finally, the overcommit control (vm.overcommit_memory) determines the extent to which the kernel promises more memory than it actually has. On database servers and mission-critical applications, adjusting this behavior helps prevent unexpected out-of-box (OOM) kills at the worst possible moment.

Disks and I/O: file systems, schedulers and RAID

In many environments, the main bottleneck is not the CPU but the input/output (I/O) subsystem . Databases, logging systems, or analytics applications are commonly hampered by a slow disk or poor configuration.

The first step is to choose the I/O scheduler best suited to the hardware . On modern SSDs, the "none" or "mq-deadline" scheduler usually works very well, as it reduces reordering logic because disk latencies are already minimal. On mechanical hard drives, however, other schedulers make more sense.

At the mount level, options like noatime and nodiratime prevent the constant updating of file and directory access timestamps, resulting in fewer unnecessary writes. This is declared both in mount commands and persistently in /etc/fstab.

Choosing the right file system is also important. XFS is generally a great option for high-concurrency loads and large files , while a well-tuned ext4 remains very robust and versatile. In any case, tools like tune2fs allow you to tweak parameters to get even more performance.

The RAID layer should not be forgotten: RAID 10 offers a very attractive balance of performance and redundancy for critical databases and data , whereas RAID 0 only makes sense for transient loads where you can assume losing the data (e.g., temporary compute storage).

Network and TCP/IP stack: making the most of bandwidth

When a server handles heavy HTTP traffic, persistent connections, or high concurrency, the network becomes a critical component of performance. A sensible tuning of the TCP/IP stack can reduce latency, improve throughput, and prevent unnecessary packet queuing.

A classic is to increase the maximum number of file descriptors with ulimit -ny parameters such as fs.file-max, so that the system can handle thousands or tens of thousands of simultaneous connections without crashing.

Similarly, increasing the send and receive buffers using net.core.rmem_max, net.core.wmem_max and the tcp_rmem / tcp_wmem series allows the TCP stack to more easily handle long-distance, high-performance connections or many concurrent sessions.

Other interesting settings include TCP Fast Open to speed up handshakes , or the activation and proper configuration of RSS/RPS on multi-core network cards, so that the packet processing load is well distributed between CPUs.

Finally, daemons like irqbalance help in general scenarios to distribute interrupts between cores; in ultra-low latency configurations they are sometimes disabled and IRQs are set manually, demonstrating that there is no single recipe, but rather the configuration must be adapted to the type of traffic.

Account security, SSH, firewall, and SELinux

A fast server is of little use if it's vulnerable to attack. Security starts with the basics: user accounts and authentication policies.

It is highly recommended to avoid generic usernames like "admin" or "oracle" and opt for less obvious names. Furthermore, password policies should enforce long, complex passwords that are rotated regularly . System tools allow you to review expiration dates, minimum length, and other restrictions.

It's also a good idea to customize the range of UIDs in /etc/login.defs to avoid trivial patterns that an attacker might assume . Ultimately, all these details add small layers of resistance to any brute-force attempt.

At the service level, SSH is the primary entry point. It's advisable to disable direct root login , change the default port, enable public key authentication, and, if appropriate for your environment, completely disable password login.

The firewall (using firewalld, nftables, or iptables) — or an implementation of Netfilter and Suricata — becomes the first line of network defense: it defines clear rules that limit incoming and outgoing traffic to only what is strictly necessary . Explicitly adding services like SSH to the public zone and reloading the configuration should be part of the checklist for any deployment.

SELinux adds an extra layer of mandatory access control. Configuring it in enforcing mode, reviewing denial logs, and adjusting policies with utilities like semanage helps contain the impact of a potential intrusion, preventing a vulnerable service from accessing resources that don't belong to it.

Automation and mass management with Ansible

When you're only managing one or two servers, it's tempting to do everything manually. But as the infrastructure grows, the only sensible solution is to automate as much as possible . This is where Ansible shines, thanks to its simplicity and agentless nature.

Simply install Ansible on a control node and maintain an inventory in /etc/ansible/hosts with the IPs or hostnames of the managed machines . Communication is done via SSH, so it's best to generate key pairs and distribute the public key using ssh-copy-id to avoid interactive passwords.

YAML playbooks allow you to describe tasks such as installing packages, modifying configuration files, restarting services, or deploying applications. A classic example would be a playbook that installs and secures a set of common utilities (tmux, monitoring agents, diagnostic tools) on all web servers in a cluster.

The great advantage is that Ansible is declarative: you describe the desired state, and the tool takes care of bringing each node to that state, avoiding divergent configurations and repetitive human errors . This applies to both performance optimizations and security measures.

Automatic optimization with Tuned profiles

In addition to manual adjustments, many distributions include Tuned, a utility that applies predefined performance profiles tailored to different types of workloads: general servers, virtual guests, storage, low latency, etc.

After installing and enabling the service, you can list the available profiles and activate the one that best suits your needs. A classic in virtualization environments is the profile designed for guest machines or high-performance hosts , which modifies kernel parameters, power management, and some I/O settings without you having to adjust them individually.

The beauty of Tuned is that it offers a sensible and reproducible starting point. From there, you can always customize specific parameters for your circumstances , but you're no longer starting from scratch; you're starting from a configuration tested in similar scenarios.

Metrics and monitoring tools: CPU, memory, disk, and network

There's no optimization without data. To know if a change improves or worsens the system, you need reliable, real-time metrics . Otherwise, you'll just be tinkering with things blindly.

For CPUs, tools like top and htop offer an interactive view of the most resource-intensive processes, the load on each core, and the distribution of states (user, system, I/O waiting, etc.). Analyzing this information helps detect runaway processes or imbalances between cores.

In memory, commands like free, vmstat, or reading /proc/meminfo provide details about actual RAM usage, caches, buffers, swap consumed , and more. This allows you to locate memory leaks or applications that are reserving more memory than is reasonable.

For disk I/O, iostat displays aggregated statistics per device (utilization, average service time, operations per second), while iotop shows, process by process, which service is generating the most traffic. This combination makes it very easy to identify which service is saturating the storage.

On a network, tools like iftop or nethogs allow you to see live which connections and processes are using the most bandwidth, which is essential for detecting anomalous traffic, abuses, or simple load imbalances.

Continuous monitoring, alerts and preventive maintenance

Beyond interactive commands, a serious environment needs a centralized monitoring system. Solutions like Nagios, Zabbix, Prometheus, or TSplus Server Monitoring allow you to collect metrics from many servers, store them as time series, and trigger alerts when something deviates from the norm.

Typical configurations include monitoring CPU load, available memory, disk latency, network usage, service status, and certificates . Based on predefined thresholds, notifications are sent when dangerous values ​​are reached or failures are detected.

Over time, these tools accumulate valuable historical data for trend analysis: usage growth, peak times, effects of updates… This opens the door to predictive maintenance strategies , in which action is taken before a failure occurs.

In parallel, it is essential to schedule routine tasks using cron or similar systems: updates, backups, log rotation, file system integrity checks, etc. Automating these tasks ensures they are always executed and prevents oversights that can be costly later.

Service optimization: web, databases and applications

A large part of overall performance depends not only on the operating system, but also on how you configure the services running on top of it. A flawless Linux server can be hampered by a poorly configured MySQL or Nginx instance.

In MySQL or MariaDB databases, the my.cnf file contains most of the important parameters: the size of the different buffers, connection limits, query caches, InnoDB behavior, etc. Adjusting values ​​such as innodb_buffer_pool_size, query_cache_size, key_buffer_size, or tmp_table_size makes a huge difference between a system that stutters and one that responds smoothly.

Tools like MySQLTuner analyze server statistics after it has been running for a while and suggest specific configuration changes to improve performance and stability . Their suggestions shouldn't be followed blindly, but they serve as a very useful guide to know where to start.

Hardware also plays a role: moving highly active databases to SSDs or running them in optimized containers greatly reduces read and write latency, especially noticeable under workloads with many random queries or large volumes of data . It's one of the most welcome performance improvements.

On web servers like Nginx or Apache, parameters such as the number of workers, connection limits, keepalive, buffer sizes, and caching strategies are key. Adjusting them to the actual traffic pattern ( many small requests, a few very large ones, lots of static content , etc.) prevents unnecessary server overload.

Java applications benefit from good JVM tuning: choice of garbage collector (G1GC, ZGC…), heap sizes, and specific parameters that reduce pauses and improve latency. Something similar applies to other runtime environments that allow you to fine-tune their memory and concurrency management.

Identifying and resolving bottlenecks

With all these pieces in place, there is still a fundamental task to be done: learning to pinpoint exactly where the bottleneck is when the system is not performing as it should.

The process begins by collecting monitoring data: CPU peaks, disk latency histogram, memory usage, network response times, database metrics… By analyzing this data together, clear patterns of saturation or anomalous behavior can be located.

Once the problem area has been identified (CPU, RAM, disk, network, specific application), the next step is to decide whether optimizing the configuration is sufficient or if additional resources are needed . Sometimes changing a kernel parameter, adjusting the cache size, or splitting a database into multiple instances is enough; other times, the only solution is to add more CPU, more RAM, or faster storage.

After implementing the changes, load tests must be repeated and measurements taken. Only in this way can you determine if the problem has truly been solved or simply moved elsewhere. This continuous cycle of measuring, adjusting, and verifying is the essence of serious optimization.

When these decisions are also supported by extensive historical data and, where appropriate, by trend analysis models or even machine learning, it is possible to anticipate future incidents and schedule expansions or adjustments before the system starts to suffer.

This entire set of practices—cleaning services, tuning the kernel, managing memory and disk, fine-tuning the network, strengthening security, automating with Ansible, using Tuned profiles, monitoring with modern tools, and polishing key services—transforms any ordinary Linux server into a robust, fast, and long-term healthy platform that better withstands spikes, defends against attacks, and requires far less manual intervention to maintain performance.