Linux Admin — Cheat Sheet

Tags: linux, bash, systemd, networking, storage, security, troubleshooting, devops

Level: intermediate

TL;DR: Quick reference for Linux system administration, service management, process troubleshooting, networking, and disk management during technical interviews.

Key concepts

Everything in Linux is represented as a file or a process descriptor.

Systemd manages services, startup targets, timers, and unified logging through journald.

Standard file permissions use read, write, and execute bits across user, group, and others.

Linux processes operate in user space or kernel space, communicating via system calls.

Load average measures active and waiting threads over 1, 5, and 15-minute intervals.

The VFS abstracts underlying filesystems like ext4, XFS, and Btrfs behind a uniform API.

Memory management uses virtual memory, page caches, swap space, and an OOM killer mechanism.

Linux networking operates across kernel networking sockets, netfilter tables, and network namespaces.

Sudo provides granular privilege delegation without exposing the root password directly.

SSH key pairs replace password authentication and enforce strict identity verification on remote hosts.

Typical questions and ready answers

Q: How do you troubleshoot a server that is reporting high load average?R: I start with uptime and top to identify whether CPU or I/O wait is saturating resources. Then I check memory and swap with free -m, followed by vmstat 1 and iostat -xz 1 to locate disk bottlenecks.

Q: What is the difference between a hard link and a soft (symbolic) link?R: A hard link points directly to the underlying inode and shares permissions and file data across the same filesystem. A symlink stores the textual path to the target file and can cross filesystem boundaries.

Q: How does systemd handle a failed service restart?R: Systemd evaluates the unit's Restart= directive and restart timers. I inspect failures using systemctl status <unit> and journalctl -u <unit> -xe to identify the exit code and root cause.

Q: What happens when a disk shows 100% full, but du shows free space available?R: Deleted files are often held open by running processes. I run lsof +L1 to identify unlinked files with open file descriptors, then restart or terminate the owning process.

Q: How do you secure an SSH server on a freshly installed Linux instance?R: I disable root login and password authentication in /etc/ssh/sshd_config, enforce ed25519 public keys, change the default port if needed, and restrict inbound traffic via iptables or UFW.

Pitfalls and common mistakes

Modifying /etc/fstab without validating syntax via mount -a before rebooting.

Running chmod -R 777 instead of applying least-privilege permissions and specific group ownership.

Killing processes blindly with kill -9 instead of allowing graceful shutdown with SIGTERM.

Forgetting that deleting an open log file does not release disk space until the process closes the descriptor.

Overriding systemd vendor units in /lib/systemd/system instead of using drop-ins under /etc/systemd/system.

Editing /etc/resolv.conf directly when managed dynamically by systemd-resolved or NetworkManager.

Snippets and commands

# Check system resource utilization and load
uptime
top -b -n 1 | head -n 20
free -h
vmstat 1 5

# Disk usage and open deleted file descriptors
df -hT
du -sh /* 2>/dev/null | sort -hr | head -n 10
lsof +L1

# Service management and journal inspection
systemctl status nginx.service
journalctl -u nginx.service --since "1 hour ago" -xe --no-pager
systemctl restart nginx.service

# Network sockets, ports, and interface status
ss -tulpn
ip addr show
ip route show
curl -Iv https://example.com

# File permissions and ownership
chmod 640 /var/www/html/config.php
chown -R www-data:www-data /var/www/html
find / -perm -4000 -type f 2>/dev/null

# Process inspection and signals
ps aux | grep node
pgrep -fl nginx
kill -15 <pid>
kill -9 <pid>

To review

[ ] Advanced systemd unit file configuration (Type=notify, sandboxing, and resource limits)

[ ] Network namespace isolation and routing policies via ip netns and iptables/nftables

[ ] Epoll vs select/poll I/O multiplexing internals in Linux kernel

[ ] Core dumps and kernel panic diagnostic logs via kdump and dmesg

[x] Standard file permissions, sticky bits, SUID, and SGID execution flags

[x] Process lifecycle states (running, sleeping, zombie, stopped)

[x] Basic SSH hardening and key distribution workflows

References

Linux man-pages (kernel.org man-pages project)

The Linux System Administrator's Guide (LDP)

Systemd System and Service Manager Documentation (freedesktop.org)

Brendan Gregg - Linux Performance Analysis and Tools