Uh oh!
There was an error while loading. Please reload this page .
systemd
/
systemd **
Public
Uh oh!
There was an error while loading. Please reload this page .
-
Notifications You must be signed in to change notification settings
-
Fork 4.6k
-
Star 16.6k
Excessive IO caused by systemd-journald #40262
New issue Copy link New issue Copy link Open Open Excessive IO caused by systemd-journald #40262 Copy link Labels bug 🐛 Programming errors, that need preferential fixing Programming errors, that need preferential fixing journal
Description
[图片](https://avatars.githubusercontent.com/u/147394?v=4&size=48) XANi opened on Jan 3, 2026 Issue body actions
systemd version the issue has been seen with
257.9
Used distribution
Debian 13
Linux kernel version used
6.12.57+deb13-amd64
Component
systemd-journald
Expected behaviour you didn't see
Log writes should be within order of magnitude of syslog
Unexpected behaviour you saw
VM doing ~50 IOPS when writing 2 lines of log per second
Steps to reproduce the problem
this is exactly same issue #15292 that was closed without good reason
Step 1. Use journald in mode where it writes to hard drive. The FS is XFS
Step 2. have constant stream of log entries going on a VM
Jan 03 13:37:01 cthylla haproxy[03/Jan/2026:13:37:01.392] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:03 cthylla haproxy[03/Jan/2026:13:37:03.403] fwww bicinga/web 0/0/0/7/7 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:05 cthylla haproxy[03/Jan/2026:13:37:05.416] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:07 cthylla haproxy[03/Jan/2026:13:37:07.427] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:09 cthylla haproxy[03/Jan/2026:13:37:09.439] fwww bicinga/web 0/0/0/7/8 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:11 cthylla haproxy[03/Jan/2026:13:37:11.454] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:13 cthylla haproxy[03/Jan/2026:13:37:13.465] fwww bicinga/web 0/0/0/7/7 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:15 cthylla haproxy[03/Jan/2026:13:37:15.476] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:17 cthylla haproxy[03/Jan/2026:13:37:17.488] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0" Jan 03 13:37:19 cthylla haproxy[03/Jan/2026:13:37:19.500] fwww bicinga/web 0/0/0/6/6 302 153 - - ---- 2/2/0/0/0 0/0 "GET / HTTP/1.0"
Step 3. Observe the VM IO traffic.
[图片](https://private-user-images.githubusercontent.com/147394/531625159-41d31a0c-2963-4d22-8d17-73bafa87e08b.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3ODY2ODE0MzAsIm5iZiI6MTc4NjY4MTEzMCwicGF0aCI6Ii8xNDczOTQvNTMxNjI1MTU5LTQxZDMxYTBjLTI5NjMtNGQyMi04ZDE3LTczYmFmYTg3ZTA4Yi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwODE0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDgxNFQwNDE4NTBaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1jNTU3ODFiZGRkMTliNzRkYzZmZjlkMDU1NjlmMmEwYWIwZDRhMmQ3MjgwOTIwOGY1NjA3Y2QzMjM0ZTAxNzZmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.19yfDZfMtYSX9uhj4w8hOgKbQBrpxZluMI7NN1lJcWY) I used VM as example because the complaint in #15292 was "iotop is not accurate" (I can believe that, it's before any OS write coaelscing) but this clearly shows traffic after every kernel mechanism was used. So no, it isn't "kernel making lotsa iops out of it", it's slow.
Journald just uses extremely inefficient format (also I've seen it corrupt on unclean reboot enough times to declare it's not even all that resilient) as files are also multiple times the size of what's actually written in them.
Reactions are currently unavailable
Metadata
Metadata
Assignees
No one assigned
Labels
bug 🐛 Programming errors, that need preferential fixing Programming errors, that need preferential fixing journal
Type
No type
Projects
No projects
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions
- 图片 Open in GitHub Copilot app