journald and application logs

Logs tell you what software reported

The journal is systemd's structured collection of entries from services and other sources. An entry can carry a timestamp, priority, unit, process identity, and message. These fields help narrow a failure; the last message is not automatically its cause.

Start with the failed operation, its time and timezone, and the actual unit name. In the Linux machine, substitute the existing service you inspected in the previous bite:

journalctl -u SERVICE.service -b --since '15 minutes ago' --no-pager
journalctl -u SERVICE.service -n 30 -o short-iso --no-pager
journalctl --disk-usage

-u selects a unit, -b the current boot, --since a time window, and -n the most recent entries. short-iso prints a readable timestamp format. --no-pager prints directly instead of opening an interactive viewer. For a user unit, add --user; the next lab uses that form.

Read a timeline, not an isolated line

Consider these illustrative messages, not output from a running lab:

10:00:00 site[4210]: opening /home/learner/site/private/settings.txt
10:00:00 site[4210]: permission denied
10:00:00 systemd[1]: site.service: Main process exited, status=1/FAILURE
10:00:02 systemd[1]: site.service: Scheduled restart job

The manager's restart message is an effect. The useful earlier clue is the attempted file open. Check the configured service user, path, and parent-directory permissions. A missing file, an unreadable file, and invalid file content need different repairs. The process ID links the first two messages to this attempt; a later attempt might have a different PID.

If the logs instead contain a connection timeout to another service, first identify that endpoint. Changing local file permissions would not follow from that evidence. Time-correlated events can suggest a relationship, but a timestamp alone does not prove causation.

Practice a bounded investigation

Choose a service that already exists and make no change to it. Read its last 20 entries with journalctl -u SERVICE.service -n 20 --no-pager. For one entry, identify the time, emitter, and message. Then narrow the query around that timestamp using --since and --until with your actual values. If there are no entries, record that result rather than manufacturing an error.

To follow a service while performing the later lab, journalctl --user -u learnwithsk-check.service -f displays new entries until Ctrl-C. Ctrl-C stops the viewer, not the service. journalctl -k -b selects kernel entries for the current boot, subject to permissions; these may help with memory or device errors rather than application syntax.

No visible entries can mean insufficient journal access, the wrong unit/time window, a different boot, or a program writing to a separate log file. Read the service/application logging configuration. -b -1 requests the previous boot, but cannot recover logs that were never persisted or have already expired.

Retention and verification are different tasks

Journal storage may be volatile or persistent according to configuration. Its size and retention compete with other disk needs. Do not remove old logs as a repair for a service's incorrect configuration, and do not copy credentials or personal content from logs into a public site.

After a targeted fix, repeat the original failed operation, inspect the new exit/runtime result, and read the new messages in a defined window. Quiet logs alone do not show that the application serves requests. In the oneshot lab, exit status 0 is appropriate; for the personal site, also fetch the expected page.

Check yourself: what does “Scheduled restart job” establish? That systemd scheduled another attempt according to its configuration, not that the attempt will succeed. This bite only reads existing logs. Continue to the separate user-service fixture to create a controlled failure and recovery timeline.

Sources

Primary references: journalctl reference↗.