Diagnosing Linux boot problems and hardware failures

Informatec Digital » Resources » Diagnosing Linux boot problems and hardware failures

When a Linux system fails to boot or starts behaving erratically, the natural reaction is to think that "Linux is broken." In practice, however, most of the time the underlying issues are configuration errors, full disks, file system corruption, or hardware problems that can be located with a little methodology. The purpose of this article is precisely that: to demonstrate a clear procedure, supported by specific commands, for diagnosing and repairing the most common boot and hardware problems in Linux, both on physical and virtual machines.

If you encounter messages like “root file system not found,” “No space left on device,” “EXT4-fs error,” “XFS: Metadata CRC error,” or a dreaded Kernel Panic , here you'll learn how to interpret them, which commands to use, and in what order to act. We'll also see how to determine if the problem is truly software-related or if the hardware (RAM, SSD/HDD, network card, etc.) is on its last legs, and how to minimize the risk of data loss while troubleshooting.

Typical causes of boot problems in Linux

Before you start messing around with things haphazardly, it's a good idea to understand the most common reasons why Linux might not start or start incorrectly . Understanding the root cause will save you hours of troubleshooting and trying solutions that lead nowhere.

The first set of problems stems from the boot manager and BIOS/UEFI configuration : incorrectly generated GRUB entries, an overwritten MBR after installing Windows in a dual boot setup , Secure Boot incompatible with your distribution, or the BIOS attempting to boot from the wrong disk. In these cases, you often won't even see the GRUB menu, or when you select your Linux distribution, it will freeze or revert to the firmware.

Another important block has to do with the disk and file systemDamaged partitions, bad sectors, EXT4 or XFS corruption, full system disks, or LVM errors. This usually manifests as messages during boot such as “Failed to mount /…","EXT4-fs error","XFS: Unmount and run xfs_repair"or even end up in emergency mode asking you to execute journalctl -xb.

Problems are also very common after a kernel update or a poorly applied patch . Incomplete downloads, unloaded modules, drivers that are no longer compatible with your hardware, or changes to the initramfs can cause the system to freeze mid-boot or trigger a Kernel Panic at startup.

We must not forget the section on the configuration of the system itself : overly strict secure boot rules (for example, auditd configured with HALT when /var/log/audit is full), critical services that fail due to lack of space, or changes in configuration files that prevent essential daemons from starting (network, systemd, cloud-init in clouds like Azure, etc.).

Finally, there's the hardware layer: disks with red SMART flags, RAM with ECC errors, network cards that are losing packets, or SSDs that are overheating . Often, the clue lies in kernel messages (dmesg) or in the SMART and EDAC counters, so we'll see how to read them carefully.

How to identify the source of the starting failure

The key to avoiding blind starts is to enable verbose boot mode and review the logs . By default, many distributions display a pretty splash screen that hides the messages, which looks good but is of no help when something goes wrong.

On systems using GRUB, you can disable silent mode by editing /etc/default/grub . Look for the line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

and replace it with:

GRUB_CMDLINE_LINUX_DEFAULT=""

Then run `update-grub` (or the equivalent command in your distribution) to regenerate the configuration. On the next boot, you'll see all the kernel and systemd messages displayed, allowing you to pinpoint the exact error causing the system to crash.

If the system manages to partially boot or you can access it from a Live environment, you will have several key logs available in /var/log :

• /var/log/boot.log: records everything that happens during the startup phase; if the fault is in this stage, it is the first place to look.
• / var / log / messages o / var / log / syslog (depending on the distro): general system events, very useful for viewing errors in services, daemons, etc.
• dmesg: displays kernel messages, including hardware failures, module problems, bus errors, PCI errors, etc.
• journalctlIn systems using systemd, it is the primary tool for traversing the entire registry; with journalctl -xb You see the final start in all its glory.

If the system doesn't reach the desktop but does reach a TTY or recovery mode, you can run `journalctl -xb` directly from there. If that doesn't work, you'll have to boot from a Live USB , mount the system disk, and read those files from the live environment.

Check if the problem is hardware or software-related

Before you start racking your brain editing configurations, it's worth checking if the problem isn't directly with the hardware: disk, memory, CPU, motherboard, or power supply . Many installations that become corrupted "randomly" are actually revealing a failing SSD or faulty RAM.

The first step, if the computer doesn't even recognize the drive, is to enter the BIOS/UEFI and confirm that the drive is listed. If the drive doesn't appear at all, or appears intermittently, check the connections, SATA cables, and power supply, or assume that the drive may need to be recycled.

If you reach GRUB or a Live boot, you can launch MemTest86+ from the GRUB menu to check your RAM. Let it run for at least eight passes; if you see red lines, the memory is faulty and the modules need to be replaced. Faulty RAM is a breeding ground for data corruption.

To check the status of disks in Linux, use smartmontools . After installing it, enable SMART and review the most important attributes:

• Relocated_Sector_Ct: number of reassigned sectors; if it is greater than 0, that disk has already started to have bad sectors.
• Current_Pending_Sector_Ct: sectors pending reassignment; any value greater than 0 indicates a high risk of imminent failure.
• Power_On_Hours: accumulated hours of use; the higher the number, the greater the likelihood of the disk failing.

Using `smartctl -H /dev/sdX` you can quickly see if the disk passes or fails the health test. If the result is not PASSED, you need to back up your data as soon as possible and start thinking about replacing it.

If you suspect high temperatures, install lm-sensors and run sensors (o watch -n 2 sensors (to view it in real time). For SSDs and SATA drives, hddtemp It shows you the current temperature, so you can detect units that spike in temperature even when idle, useful for... server optimization.

Specialized hardware diagnostics in Linux

In addition to CPU, RAM, and disk, Linux has specific tools for evaluating ECC memory, network cards, GPUs, and other components . Spending a few minutes checking these points can save you many hours of debugging.

If your server or workstation has ECC memory , the kernel typically logs corrected and uncorrected errors using EDAC. A simple:

dmesg | grep EDAC

It allows you to see if there are CE (Corrected Errors) or UE (Uncorrected Errors) . CEs indicate that the RAM has had faults, but the hardware has been able to correct them; it's advisable to monitor these modules because it's usually only a matter of time before they become uncorrectable errors. UEs often end with Kernel Panic to prevent serious data corruption .

To check the memory structure and capacity, `dmidecode -t memory` shows you the type of modules (DDR3, DDR4, DDR5), capacities, occupied banks, and more. And with `dmidecode -t 16` you can see the maximum RAM capacity supported by the motherboard, which is useful if you're thinking about upgrading.

In the area of ​​temperature and visual monitoring, you can support sensor and HDD Temp data with graphical tools such as psensor or xsensors , which plot the temperature over time to detect unusual spikes or equipment that is perpetually at the heat limit.

For diagnosing disks and USB drives , in addition to lsblk and fdisk -l to list devices, it is very useful to combine:

• df -h y df -i: percentage of space and inode usage; you can have gigabytes free but 100% of inodes occupied and the system will still complain of "no space".
• fsck /dev/sdXN -y: searches for and repairs logical errors in file systems (EXT2/3/4, among others).
• badblocks /dev/my_disk: scans for defective sectors and marks them so that the system does not use them (it is highly recommended to make a backup before doing this).

In the case of a network card , typical symptoms include disconnections, latency, or unexplained packet loss. You can use ethtool to review statistics and netstat to check packet loss rates (RX-DRP and TX-DRP). If the percentage of packet loss exceeds approximately 0,2% , network performance is clearly affected, and the NIC may be faulty or poorly supported by the driver.

Disk full and space errors: the classic “No space left on device”

On many machines (especially servers and virtual machines ), one of the most common problems is that the system disk fills up completely. When this happens, services, logs, boot processes, and even the kernel itself start throwing I/O errors.

In environments like Azure, it's common to see messages like this in the console or in startup diagnostics:

• No space left on device in cloud-init, which can prevent the VM from finishing booting.
• Repeated messages from “There is no space left on the device” that affect critical services, including the cloud agent.
• Errors in the system logs indicating that audit or network logs cannot be written.

To quickly locate what is occupying the space, you can use commands such as:

• du -ks /* | sort -n: lists the root directories sorted by size; repeats within the heaviest ones until the real culprit is found.
• ls -altSr /var/log: displays log files sorted by size, from smallest to largest; you will often see huge old logs that can be rotated or deleted.
• find / -size +500M -exec ls -alFh {} \;: search for large individual files; adjust that 500M as needed.

Besides the typical filling up of /var/log or /tmp, there are security configurations such as auditd that can cause a machine to shut down or fail to start when runs out of space in /var/log/audit. If in /etc/audit/auditd.conf you have:

admin_space_left_action = HALT
disk_full_action = HALT
disk_error_action = HALT

The system may shut down in a controlled manner or refuse to boot if it cannot write to the audit logs. A temporary solution is to change those values ​​to SUSPEND, IGNORE , or other valid options (never SINGLE in this context) to allow the system to boot and free up space . After resolving the issue, you should revert to the original policy if compliance is required.

When there's no way to delete things because the system literally won't boot, you can resort to rescue modes : automatic cloud repair commands (e.g., az vm repair in Azure), recovery virtual machines, or single-user mode to mount the disk and delete unnecessary files until you have at least 10% free space in the file system that hosts /var/log and the rest of the critical directories.

Corrupted EXT4 and XFS: How to Repair Damaged File Systems

If at startup you see things like “EXT4-fs error (device sda1)” , “bad extra_isize” , “no journal found” or XFS messages such as “Metadata CRC error detected… Unmount and run xfs_repair” , you are not dealing with a one-off problem: the file system is corrupt and until you repair it, the machine will not boot normally.

The first step is to identify which device is affected . In the boot logs, look at the text that appears in parentheses in the kernel messages: sda1, sdc1, dm-0, dm-2, /dev/mapper/vgname/lvname, etc. This will tell you whether you are dealing with a direct partition (sdXN) or an LVM logical volume (dm-N, /dev/vgname/lvname).

Once you have access to a shell (emergency mode, single user, or rescue machine), run `lsblk -f` to see the complete structure: disks, partitions, LVM, and filesystem types. It's very important to confirm here whether that partition is actually ext4, xfs, vfat, LVM2_member , etc., and not rely solely on what `/etc/fstab` says if you suspect it might be misconfigured.

To repair EXT4 file systems, fsck is used . As a general rule:

• Make sure the file system is unmounted (if it is a data disk) or that you are working from a rescue environment where it is not in use.
• Lanza fsck /dev/sdXN o fsck /dev/vgname/lvnameIt will ask you if you want to correct inconsistencies, recreate the resize inode, adjust block counters, etc.
• If many questions appear, interrupt with CTRL+C and run again with fsck -y so that it automatically answers "yes"; that way you don't miss anything.
• If you move files to lost+foundYou will have to check them later and put them back where they belong.
• Run fsck again until the output indicates that the file system is clean.

In the case of XFS , the tool is xfs_repair . The typical workflow here is:

• First, a dry check: xfs_repair -n /dev/vgname/homelv to see what damage there is without modifying anything.
• If the analysis is reasonable, repeat without -n so it can attempt to correct: xfs_repair /dev/vgname/homelv.
• If the command complains that the filesystem has "valuable metadata changes in a journal that need to be reproduced," try mounting it: on XFS systems, many pending changes are applied precisely upon mounting. If you're on a rescue VM, you can do this at a point like /recovery.
• If there's no other way and the journal errors aren't fixed, the last resort is to use xfs_repair -L To discard the journal and force the assembly as if all changes had been applied. This may result in the loss of recent data, so only do this as a last resort.

In all cases, it's crucial to understand that fsck and xfs_repair aren't magic : they fix the file system structure, but they can't always recover all the data. That's why it's so critical to have previous backups and, if you're in the cloud or a VM, to work from a disk snapshot or a copy attached to a rescue machine.

GRUB, UEFI, Secure Boot and other boot classics

When you turn on the computer and don't even see the GRUB menu, or the BIOS/UEFI throws errors like "Failed to open \EFI\ubuntu\grubx64.efi - Not found" and enters a loop trying to boot from that entry, the problem is almost certainly in the boot manager or the firmware configuration.

These errors can occur after installing Windows in a dual-boot configuration (which often takes over the MBR or overwrites UEFI entries), after accidentally deleting an EFI partition, or after tampering with the boot order in the BIOS. On modern laptops, it's not uncommon for the machine to reboot in a loop if the UEFI entry "ubuntu" points to a file that no longer exists, attempting to load it.

The easiest way to repair a broken GRUB bootloader is to boot from a Live distribution (for example, Ubuntu) and use the Boot-Repair tool . The typical procedure is:

• Start from a USB Live and open a terminal.
• Add the repository and update: sudo apt-add-repository ppa:yannubuntu/boot-repair && sudo apt update.
• Install the tool: sudo apt install -y boot-repair.
• Run it with boot-repair and choose the option “recommended repair”.

Boot-Repair analyzes partitions, locates installed systems, reconfigures GRUB, regenerates the configuration file, and, if necessary, adjusts UEFI entries so that the computer boots from the appropriate boot manager.

On systems with UEFI, Secure Boot, and Fast Boot enabled, there are several incompatibilities worth being aware of. Not all distributions support Secure Boot correctly, and some designed for older hardware don't even work well with UEFI. If your distribution is unsigned or the boot shim is incompatible, UEFI may refuse to load the kernel.

In these cases, the solution usually involves accessing the UEFI settings, enabling Legacy/CSM mode to allow classic BIOS-style booting, and disabling Secure Boot . However, if you also use Windows 11 on that computer or want to migrate from Windows to Linux , disabling Secure Boot can break Windows' boot requirements, so you should consider which system takes priority or look for a Linux distribution compatible with Secure Boot to avoid having to access the BIOS every time.

Windows Fast Boot also causes problems in dual-boot scenarios. When shutting down with Fast Boot enabled, Windows doesn't completely power off: it leaves part of the kernel hibernated on disk, blocking full access to the NTFS file system. When Linux tries to mount these partitions, errors may occur or the system may even freeze. It's recommended to disable Fast Boot in both the Windows power options and, if your UEFI allows it, in the BIOS itself.

Use of recovery modes and built-in repair tools

If GRUB appears but your Linux system fails to boot, or if you suspect that a package has been left unfinished or that the system has been corrupted after an update, you can take advantage of GRUB's advanced options and the recovery modes included in most distributions.

The GRUB menu usually has an entry called " Advanced options " or something similar. Inside, you'll typically see all available kernel versions and, for each one, a Recovery mode . Select the recovery mode for the latest version (and if that fails, try the previous one).

Recovery mode shows you a menu of very useful utilities for:

• fsck: Check and repair the file system (similar to chkdsk in Windows).
• clean: free up space by deleting temporary files and other residual files.
• dpkg: fix broken packages, unmet dependencies, or stuck installations.
• grub: regenerate the boot manager configuration.

Running these options in order usually resolves many problems caused by power outages during updates, corrupted packages, or disks that were about to fill up . Afterward, the system will typically offer to restart to check if the boot process has returned to normal.

In clouds like Azure , in addition to emergency and single-user modes, you have specific tools like Azure Linux Automatic Repair (ALAR) and the az vm repair command , which automate part of the process: they mount the system disk in a rescue VM, run actions like “auditd” to fix typical configurations, and allow you to easily revert if something goes wrong.

Strategies for reinstalling Linux without losing data

There are situations where, no matter how much you investigate, the system is so corrupted or your hardware is so unusual (like that HP N150 or modern laptops with exotic controllers) that the most sensible thing to do is reinstall the distro . However, this doesn't necessarily mean losing all your data.

Many distributions, such as Ubuntu and its derivatives, offer an option during installation called "reinstall the operating system while keeping documents and settings ." This reinstalls the base system but attempts to preserve your /home directory and, sometimes, even some installed applications. It's a convenient option, though not foolproof, so it's advisable to manually back up anything you don't want to lose.

The most robust way to protect against future reinstallations is to have the disk organized into several separate partitions :

• One for / (root): base system.
• Another one for / Boot (and /boot/efi in UEFI) if you want more control over booting.
• An exclusive section for / Home or for data.

This way, if your Linux system becomes unusable, you can format only the system and boot partitions, leaving the data partition untouched . Even if you only have one partition for everything, you can boot from a Live environment, mount the disk, copy documents to an external drive or the cloud, and then, once everything is safe, perform a clean installation.

In virtual machines, whether from VirtualBox, VMware, or the cloud, it's a good idea to also separate data disks from the system disk . In Azure, for example, it's recommended that LVM data volumes not be mixed in the same volume group as the system disk to prevent an OS failure from also affecting the data.

Best practices to prevent problems from recurring

Once you've dealt with a broken boot and recovered your system (or at least your data), the smart thing to do is take some steps to make the next time the scare less severe or prevent it from happening altogether.

The first is to keep your system and software up to date , but do so sensibly. In rolling release distributions like Arch, frequent updates are essential; in more conservative ones (Debian Stable, Ubuntu LTS) you can space them out a bit, but it's not advisable to leave them untouched for years. Before major kernel or version changes, always make a decent backup.

Another very healthy habit is document any major configuration changes: which file you touched, which line you changed, and what it was like before. A simple trick is to always save a copy of the original file with the suffix. .Capricorn (for example, sshd_config.bak) so you can easily restore it if you mess it up.

Regarding data, regular backups are essential . You can use rsync for incremental backups, tar to package entire directories, or cloud and NAS solutions. Furthermore, if your data is on a different partition than the system partition, you can reinstall without much trouble even if the disk won't boot.

Finally, it's always a good idea to monitor the system's health regularly: check SMART monthly, monitor inodes and free space on critical partitions (/, /var, /home), look at logs for repetitive messages, and keep temperature and dust levels down on physical equipment. A simple maintenance routine saves many early mornings fixing machines that have "suddenly" stopped booting.

With this arsenal of techniques, commands, and best practices, diagnosing and repairing boot problems and hardware errors in Linux ceases to be an act of faith and becomes a fairly reasonable process: first, you identify whether the failure is due to the disk, file system, GRUB, kernel, configuration, or hardware; then you apply the appropriate tool (fsck, xfs_repair, Boot-Repair, smartctl, MemTest, recovery modes, etc.); and, in the worst-case scenario, you have a well-organized Plan B with separate partitions and backups that allow you to reinstall without the world ending.