[
https://ovirt-jira.atlassian.net/browse/OVIRT-2972?page=com.atlassian.jir...
]
Evgheni Dereveanchin commented on OVIRT-2972:
---------------------------------------------
Suspect jobs that were reporting issues this weekend:
[
https://jenkins.ovirt.org/job/ovirt_master_publish-rpms_nightly/|https://...]
[
https://jenkins.ovirt.org/job/ovirt_4.3_publish-rpms_nightly/|https://jen...]
[
https://jenkins.ovirt.org/job/ovirt_4.2_publish-rpms_nightly/|https://jen...]
The errors are as follows:
{noformat}13:15:32 2020-07-19 11:15:32,180::INFO::root::Saving
/home/jenkins/ovirt-master-snapshot.tmp/rpm/fc29/x86_64/ovirt-engine-wildfly-18.0.0-1.fc29.x86_64.rpm
13:15:32 Traceback (most recent call last):
13:15:32 File "/bin/repoman", line 10, in <module>
13:15:32 sys.exit(main())
13:15:32 File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 461, in
main
13:15:32 exit_code = do_add(args, config, repo)
13:15:32 File "/usr/lib/python2.7/site-packages/repoman/cmd.py", line 361, in
do_add
13:15:32 repo.save()
13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line
42, in _func
13:15:32 return func(self, *args, **kwargs)
13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/repo.py", line
201, in save
13:15:32 store.save()
13:15:32 File
"/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line
263, in save
13:15:32 self._save(**args)
13:15:32 File
"/usr/lib/python2.7/site-packages/repoman/common/stores/RPM/__init__.py", line
302, in _save
13:15:32 save_file(pkg.path, dst_path)
13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/utils.py", line
436, in save_file
13:15:32 copy(src_path, dst_path)
13:15:32 File "/usr/lib/python2.7/site-packages/repoman/common/utils.py", line
296, in copy
13:15:32 shutil.copy2(what, where)
13:15:32 File "/usr/lib64/python2.7/shutil.py", line 130, in copy2
13:15:32 copyfile(src, dst)
13:15:32 File "/usr/lib64/python2.7/shutil.py", line 84, in copyfile
13:15:32 copyfileobj(fsrc, fdst)
13:15:32 File "/usr/lib64/python2.7/shutil.py", line 52, in copyfileobj
13:15:32 fdst.write(buf)
13:15:32 IOError: [Errno 28] No space left on device
13:15:32 2020-07-19 11:15:32,917::INFO::repoman.common.repo::Cleaning up temporary dir
/tmp/tmpRKo543
13:15:34 Waiting for scan_for_artifacts.sh cron job to finish (attempt 1 of 50)
{noformat}
looks like the job ran out of tmpfs space - did we publish too many files for some reason?
Not entirely sure what tmp is used for in these jobs.
resources.ovirt.org ran out of space for artifact publishing
------------------------------------------------------------
Key: OVIRT-2972
URL:
https://ovirt-jira.atlassian.net/browse/OVIRT-2972
Project: oVirt - virtualization made easy
Issue Type: Bug
Reporter: Evgheni Dereveanchin
Assignee: infra
Attachments: image-20200720-083159.png, prom.png
This weekend Nagios sent disk space alerts for the /home/jenkins on
resources.ovirt.org
AFAIR this is used as intermediate artifact storage for publishing and shouldn't fill
up that much. Logging a ticket to investigate the root cause.
The partition is free again now so this is not blocking anything right now.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100133)