Hi ! I have strange situation with low disk space on /var/log/, yet I can't figure out what have consumed so much space. Look at the du output. Thanks in advance for any suggestion(s). # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-var 15G 1.3G 13G 9% /var /dev/mapper/centos-var_log 7.8G 7.1G 293M 97% /var/log /dev/mapper/centos-var_log_audit 2.0G 41M 1.8G 3% /var/log/audit # # du -ah /var/log | sort -n -r | head -n 20 664K /var/log/vdsm 660K /var/log/vdsm/vdsm.log 624K /var/log/gdm 580K /var/log/anaconda/storage.log 480K /var/log/anaconda/packaging.log 380K /var/log/gdm/:0.log.4 316K /var/log/anaconda/syslog 276K /var/log/tuned 252K /var/log/libvirt/qemu/NextCloud-Collabora-LVM.log-20210806 220K /var/log/Xorg.0.log 168K /var/log/gdm/:0.log 156K /var/log/yum.log-20191117 156K /var/log/secure-20180726 132K /var/log/libvirt/qemu/NextCloud-Collabora-Active.log 128K /var/log/anaconda/anaconda.log 120K /var/log/hp-snmp-agents 116K /var/log/hp-snmp-agents/cma.log 112K /var/log/vinchin 104K /var/log/vinchin/kvm_backup_service 104K /var/log/tuned/tuned.log.2 # # find . -printf '%s %p\n'| sort -nr | head -10 16311686 ./rhsm/rhsm.log 4070423 ./cron-20180726 3667146 ./anaconda/journal.log 3409071 ./secure 3066660 ./rhsm/rhsm.log-20180726 2912670 ./audit/audit.log 1418007 ./sanlock.log 1189580 ./vdsm/vdsm.log 592718 ./anaconda/storage.log 487567 ./anaconda/packaging.log
Is it an HCI deployment? I have had a similar problem and it was on an HCI deployment. The healing process get stuck for some reason that I don't know and it fills the space. What I did is reboot the node and the free space was there. What version of ovirt are you using? Regards On Wed, Jun 22, 2022, 21:27 Andrei Verovski <andreil1@starlett.lv> wrote:
Hi !
I have strange situation with low disk space on /var/log/, yet I can't figure out what have consumed so much space. Look at the du output.
Thanks in advance for any suggestion(s).
# df -h
Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-var 15G 1.3G 13G 9% /var /dev/mapper/centos-var_log 7.8G 7.1G 293M 97% /var/log /dev/mapper/centos-var_log_audit 2.0G 41M 1.8G 3% /var/log/audit
# # du -ah /var/log | sort -n -r | head -n 20
664K /var/log/vdsm 660K /var/log/vdsm/vdsm.log 624K /var/log/gdm 580K /var/log/anaconda/storage.log 480K /var/log/anaconda/packaging.log 380K /var/log/gdm/:0.log.4 316K /var/log/anaconda/syslog 276K /var/log/tuned 252K /var/log/libvirt/qemu/NextCloud-Collabora-LVM.log-20210806 220K /var/log/Xorg.0.log 168K /var/log/gdm/:0.log 156K /var/log/yum.log-20191117 156K /var/log/secure-20180726 132K /var/log/libvirt/qemu/NextCloud-Collabora-Active.log 128K /var/log/anaconda/anaconda.log 120K /var/log/hp-snmp-agents 116K /var/log/hp-snmp-agents/cma.log 112K /var/log/vinchin 104K /var/log/vinchin/kvm_backup_service 104K /var/log/tuned/tuned.log.2
# # find . -printf '%s %p\n'| sort -nr | head -10
16311686 ./rhsm/rhsm.log 4070423 ./cron-20180726 3667146 ./anaconda/journal.log 3409071 ./secure 3066660 ./rhsm/rhsm.log-20180726 2912670 ./audit/audit.log 1418007 ./sanlock.log 1189580 ./vdsm/vdsm.log 592718 ./anaconda/storage.log 487567 ./anaconda/packaging.log _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/N566CST3DAXILL...
Hi, Version 4.4 Its simple 3-node system with no hosted engine (engine on separate server). Node installed manually, not via factory image. I just can't any file/dir which eats almost 8 GB on /var/log. On 6/22/22 23:40, wodel youchi wrote:
Is it an HCI deployment? I have had a similar problem and it was on an HCI deployment. The healing process get stuck for some reason that I don't know and it fills the space. What I did is reboot the node and the free space was there.
What version of ovirt are you using?
Regards
On Wed, Jun 22, 2022, 21:27 Andrei Verovski <andreil1@starlett.lv> wrote:
Hi !
I have strange situation with low disk space on /var/log/, yet I can't figure out what have consumed so much space. Look at the du output.
Thanks in advance for any suggestion(s).
# df -h
Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-var 15G 1.3G 13G 9% /var /dev/mapper/centos-var_log 7.8G 7.1G 293M 97% /var/log /dev/mapper/centos-var_log_audit 2.0G 41M 1.8G 3% /var/log/audit
# # du -ah /var/log | sort -n -r | head -n 20
664K /var/log/vdsm 660K /var/log/vdsm/vdsm.log 624K /var/log/gdm 580K /var/log/anaconda/storage.log 480K /var/log/anaconda/packaging.log 380K /var/log/gdm/:0.log.4 316K /var/log/anaconda/syslog 276K /var/log/tuned 252K /var/log/libvirt/qemu/NextCloud-Collabora-LVM.log-20210806 220K /var/log/Xorg.0.log 168K /var/log/gdm/:0.log 156K /var/log/yum.log-20191117 156K /var/log/secure-20180726 132K /var/log/libvirt/qemu/NextCloud-Collabora-Active.log 128K /var/log/anaconda/anaconda.log 120K /var/log/hp-snmp-agents 116K /var/log/hp-snmp-agents/cma.log 112K /var/log/vinchin 104K /var/log/vinchin/kvm_backup_service 104K /var/log/tuned/tuned.log.2
# # find . -printf '%s %p\n'| sort -nr | head -10
16311686 ./rhsm/rhsm.log 4070423 ./cron-20180726 3667146 ./anaconda/journal.log 3409071 ./secure 3066660 ./rhsm/rhsm.log-20180726 2912670 ./audit/audit.log 1418007 ./sanlock.log 1189580 ./vdsm/vdsm.log 592718 ./anaconda/storage.log 487567 ./anaconda/packaging.log _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/N566CST3DAXILL...
Have tried to reboot the node? On Wed, Jun 22, 2022, 23:30 Andrei Verovski <andreil1@starlett.lv> wrote:
Hi,
Version 4.4 Its simple 3-node system with no hosted engine (engine on separate server). Node installed manually, not via factory image. I just can't any file/dir which eats almost 8 GB on /var/log.
On 6/22/22 23:40, wodel youchi wrote:
Is it an HCI deployment? I have had a similar problem and it was on an HCI deployment. The healing process get stuck for some reason that I don't know and it fills the space. What I did is reboot the node and the free space was there.
What version of ovirt are you using?
Regards
On Wed, Jun 22, 2022, 21:27 Andrei Verovski <andreil1@starlett.lv> wrote:
Hi !
I have strange situation with low disk space on /var/log/, yet I can't figure out what have consumed so much space. Look at the du output.
Thanks in advance for any suggestion(s).
# df -h
Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-var 15G 1.3G 13G 9% /var /dev/mapper/centos-var_log 7.8G 7.1G 293M 97% /var/log /dev/mapper/centos-var_log_audit 2.0G 41M 1.8G 3% /var/log/audit
# # du -ah /var/log | sort -n -r | head -n 20
664K /var/log/vdsm 660K /var/log/vdsm/vdsm.log 624K /var/log/gdm 580K /var/log/anaconda/storage.log 480K /var/log/anaconda/packaging.log 380K /var/log/gdm/:0.log.4 316K /var/log/anaconda/syslog 276K /var/log/tuned 252K /var/log/libvirt/qemu/NextCloud-Collabora-LVM.log-20210806 220K /var/log/Xorg.0.log 168K /var/log/gdm/:0.log 156K /var/log/yum.log-20191117 156K /var/log/secure-20180726 132K /var/log/libvirt/qemu/NextCloud-Collabora-Active.log 128K /var/log/anaconda/anaconda.log 120K /var/log/hp-snmp-agents 116K /var/log/hp-snmp-agents/cma.log 112K /var/log/vinchin 104K /var/log/vinchin/kvm_backup_service 104K /var/log/tuned/tuned.log.2
# # find . -printf '%s %p\n'| sort -nr | head -10
16311686 ./rhsm/rhsm.log 4070423 ./cron-20180726 3667146 ./anaconda/journal.log 3409071 ./secure 3066660 ./rhsm/rhsm.log-20180726 2912670 ./audit/audit.log 1418007 ./sanlock.log 1189580 ./vdsm/vdsm.log 592718 ./anaconda/storage.log 487567 ./anaconda/packaging.log _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/N566CST3DAXILL...
Deleted some files to "clean up" /var/log, but the space was not recovered? Space for deleted files are only recovered when all references to that space are removed. This includes the directory reference, and any open file-handles to the file. The is a popular trick for a self-deleting scratch file. Create a file, open file-handle to it, and then remove the file. The open file-handle can then be written to, and read from, as long as you like, but when the file-handle is closed, explicitly or implicitly, the space is automatically recovered. -----Original Message----- From: Andrei Verovski <andreil1@starlett.lv> Sent: Wednesday, June 22, 2022 3:27 PM To: users@ovirt.org Subject: [ovirt-users] Lost space in /var/log Hi ! I have strange situation with low disk space on /var/log/, yet I can't figure out what have consumed so much space. Look at the du output. Thanks in advance for any suggestion(s). # df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-var 15G 1.3G 13G 9% /var /dev/mapper/centos-var_log 7.8G 7.1G 293M 97% /var/log /dev/mapper/centos-var_log_audit 2.0G 41M 1.8G 3% /var/log/audit # # du -ah /var/log | sort -n -r | head -n 20 664K /var/log/vdsm 660K /var/log/vdsm/vdsm.log 624K /var/log/gdm 580K /var/log/anaconda/storage.log 480K /var/log/anaconda/packaging.log 380K /var/log/gdm/:0.log.4 316K /var/log/anaconda/syslog 276K /var/log/tuned 252K /var/log/libvirt/qemu/NextCloud-Collabora-LVM.log-20210806 220K /var/log/Xorg.0.log 168K /var/log/gdm/:0.log 156K /var/log/yum.log-20191117 156K /var/log/secure-20180726 132K /var/log/libvirt/qemu/NextCloud-Collabora-Active.log 128K /var/log/anaconda/anaconda.log 120K /var/log/hp-snmp-agents 116K /var/log/hp-snmp-agents/cma.log 112K /var/log/vinchin 104K /var/log/vinchin/kvm_backup_service 104K /var/log/tuned/tuned.log.2 # # find . -printf '%s %p\n'| sort -nr | head -10 16311686 ./rhsm/rhsm.log 4070423 ./cron-20180726 3667146 ./anaconda/journal.log 3409071 ./secure 3066660 ./rhsm/rhsm.log-20180726 2912670 ./audit/audit.log 1418007 ./sanlock.log 1189580 ./vdsm/vdsm.log 592718 ./anaconda/storage.log 487567 ./anaconda/packaging.log _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/N566CST3DAXILL...
participants (3)
-
Andrei Verovski -
Matthew.Stier＠fujitsu.com -
wodel youchi