7:11 a.m.
You can view all LVs via 'lvs -a' and create a new metadata LV of bigger size.
Of course , lvdisplay -m /dev/gluster_vg1-lvthinpool-tpool_tmeta shoould also work.
Best Regards,
Strahil NikolovOn Oct 1, 2019 03:11, jeremy_tourville(a)hotmail.com wrote:
vgs displays everything EXCEPT gluster_vg1
"dmsetup ls" does not list the VG in question. That is why I couldn't run
the lvchange command. They were not active or even detected by the system.
OK, I found my problem, and a solution: https://access.redhat.com/solutions/3251681
# cd /var/log
# grep -ri gluster_vg1-lvthinpool-tpool
My metadata is 100% full !!!
Now, how do I find out how big my original metadata size was so I can make the new one
the correct size?
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/6PI2N24GVWL...
2:08 a.m.
New subject: Ovirt 4.2.7 won't start and drops to emergency console
"lvs -a" does not list the logical volume I am missing.
"lvdisplay -m /dev/gluster_vg1-lvthinpool-tpool_tmeta" does not work either.
Error message is: Volume Group xxx not found. Cannot process volume group xxx."
I am trying to follow the procedure from https://access.redhat.com/solutions/3251681
I am on step #2 Step 2a and 2b work without issue. Step 2c give me an error. Here are
the values I am using:
# lvcreate -L 2G gluster_vg3 --name tmpLV
# lvchange -ay gluster_vg3/tmpLV
# lvconvert --thinpool gluster_vg1/lvthinpool --poolmetadata gluster_vg3/tmpLV
VG name mismatch from position arg (gluster_vg1) and option arg (gluster_vg3)
Do I need to create the LV on the same disk that failed? (gluster_vg1)
Is creating a new LV on (gluster_vg3) ok for this situation?
4:49 a.m.
New subject: Ovirt 4.2.7 won't start and drops to emergency console
I don't know why I didn't think to get some more info regarding my storage
environment and post it here earlier. My gluster_vg1 volume is on /dev/sda1. I can
access the engine storage directory but I think that is because it is not thin
provisioned. I guess I was too bogged down in solving the problem when I'm stuck in
emergency mode. I had to sneaker net my USB drive to my system so I could capture some
info. Anyhow here it is:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE
MOUNTPOINT
sda 8:0 0 5.5T 0 disk
└─sda1 8:1 0 5.5T 0 part
sdb 8:16 0 223.6G 0 disk
├─sdb1 8:17 0 1G 0 part
/boot
└─sdb2 8:18 0 222.6G 0 part
└─md127 9:127 0 222.5G 0 raid1
├─onn_vmh-swap 253:0 0 4G 0 lvm
[SWAP]
├─onn_vmh-pool00_tmeta 253:1 0 1G 0 lvm
│ └─onn_vmh-pool00-tpool 253:3 0 173.6G 0 lvm
│ ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G 0 lvm /
│ ├─onn_vmh-pool00 253:5 0 173.6G 0 lvm
│ ├─onn_vmh-root 253:6 0 146.6G 0 lvm
│ ├─onn_vmh-home 253:7 0 1G 0 lvm
/home
│ ├─onn_vmh-tmp 253:8 0 1G 0 lvm
/tmp
│ ├─onn_vmh-var 253:9 0 15G 0 lvm
/var
│ ├─onn_vmh-var_log 253:10 0 8G 0 lvm
/var/log
│ ├─onn_vmh-var_log_audit 253:11 0 2G 0 lvm
/var/log/audit
│ └─onn_vmh-var_crash 253:12 0 10G 0 lvm
/var/crash
└─onn_vmh-pool00_tdata 253:2 0 173.6G 0 lvm
└─onn_vmh-pool00-tpool 253:3 0 173.6G 0 lvm
├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G 0 lvm /
├─onn_vmh-pool00 253:5 0 173.6G 0 lvm
├─onn_vmh-root 253:6 0 146.6G 0 lvm
├─onn_vmh-home 253:7 0 1G 0 lvm
/home
├─onn_vmh-tmp 253:8 0 1G 0 lvm
/tmp
├─onn_vmh-var 253:9 0 15G 0 lvm
/var
├─onn_vmh-var_log 253:10 0 8G 0 lvm
/var/log
├─onn_vmh-var_log_audit 253:11 0 2G 0 lvm
/var/log/audit
└─onn_vmh-var_crash 253:12 0 10G 0 lvm
/var/crash
sdc 8:32 0 223.6G 0 disk
└─sdc1 8:33 0 222.6G 0 part
└─md127 9:127 0 222.5G 0 raid1
├─onn_vmh-swap 253:0 0 4G 0 lvm
[SWAP]
├─onn_vmh-pool00_tmeta 253:1 0 1G 0 lvm
│ └─onn_vmh-pool00-tpool 253:3 0 173.6G 0 lvm
│ ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G 0 lvm /
│ ├─onn_vmh-pool00 253:5 0 173.6G 0 lvm
│ ├─onn_vmh-root 253:6 0 146.6G 0 lvm
│ ├─onn_vmh-home 253:7 0 1G 0 lvm
/home
│ ├─onn_vmh-tmp 253:8 0 1G 0 lvm
/tmp
│ ├─onn_vmh-var 253:9 0 15G 0 lvm
/var
│ ├─onn_vmh-var_log 253:10 0 8G 0 lvm
/var/log
│ ├─onn_vmh-var_log_audit 253:11 0 2G 0 lvm
/var/log/audit
│ └─onn_vmh-var_crash 253:12 0 10G 0 lvm
/var/crash
└─onn_vmh-pool00_tdata 253:2 0 173.6G 0 lvm
└─onn_vmh-pool00-tpool 253:3 0 173.6G 0 lvm
├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G 0 lvm /
├─onn_vmh-pool00 253:5 0 173.6G 0 lvm
├─onn_vmh-root 253:6 0 146.6G 0 lvm
├─onn_vmh-home 253:7 0 1G 0 lvm
/home
├─onn_vmh-tmp 253:8 0 1G 0 lvm
/tmp
├─onn_vmh-var 253:9 0 15G 0 lvm
/var
├─onn_vmh-var_log 253:10 0 8G 0 lvm
/var/log
├─onn_vmh-var_log_audit 253:11 0 2G 0 lvm
/var/log/audit
└─onn_vmh-var_crash 253:12 0 10G 0 lvm
/var/crash
sdd 8:48 0 596.2G 0 disk
└─sdd1 8:49 0 4G 0 part
└─gluster_vg3-tmpLV 253:13 0 2G 0 lvm
sde 8:64 1 7.5G 0 disk
└─sde1 8:65 1 7.5G 0 part
/mnt
# blkid
/dev/sda1: UUID="f026a2dc-201a-4b43-974e-2419a8783bce" TYPE="xfs"
PARTLABEL="Linux filesystem"
PARTUUID="4bca8a3a-42f0-4877-aa60-f544bf1fdce7"
/dev/sdc1: UUID="e5f4acf5-a4bc-6470-7b6f-415e3f4077ff"
UUID_SUB="a895900e-5585-8f31-7515-1ff7534e39d7"
LABEL="vmh.cyber-range.lan:pv00" TYPE="linux_raid_member"
/dev/sdb1: UUID="9b9546f9-25d2-42a6-835b-303f32aee4b1" TYPE="ext4"
/dev/sdb2: UUID="e5f4acf5-a4bc-6470-7b6f-415e3f4077ff"
UUID_SUB="6e20b5dd-0152-7f42-22a7-c17133fbce45"
LABEL="vmh.cyber-range.lan:pv00" TYPE="linux_raid_member"
/dev/sdd1: UUID="2nLjVF-sh3N-0qkm-aUQ1-jnls-3e8W-tUkBw5"
TYPE="LVM2_member"
/dev/md127: UUID="Mq1chn-6XhF-WCwF-LYhl-tZEz-Y8lq-8R2Ifq"
TYPE="LVM2_member"
/dev/mapper/onn_vmh-swap: UUID="1b0b9c91-22ed-41d1-aebf-e22fd9aa05d9"
TYPE="swap"
/dev/mapper/onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1:
UUID="b0e1c479-9696-4e19-b799-7f81236026b7" TYPE="ext4"
/dev/mapper/onn_vmh-root: UUID="60905f5d-ed91-4ca9-9729-9a72a4678ddd"
TYPE="ext4"
/dev/mapper/onn_vmh-home: UUID="82a1d567-f8af-4b96-bfbf-5f79dff7384f"
TYPE="ext4"
/dev/mapper/onn_vmh-tmp: UUID="7dd9d3ae-3af7-4763-9683-19f583d8d15b"
TYPE="ext4"
/dev/mapper/onn_vmh-var: UUID="f206e030-876b-45a9-8a90-a0e54005b85c"
TYPE="ext4"
/dev/mapper/onn_vmh-var_log: UUID="b8a12f56-0818-416c-9fb7-33b48ef29eed"
TYPE="ext4"
/dev/mapper/onn_vmh-var_log_audit: UUID="bc78ad0c-9ab6-4f57-a69f-5b1ddf898552"
TYPE="ext4"
/dev/mapper/onn_vmh-var_crash: UUID="a941d416-4d7d-41ae-bcd4-8c1ec9d0f744"
TYPE="ext4"
/dev/sde1: UUID="44aa40d0-6c82-4e8e-8218-177e5c8474f4" TYPE="ext4"
# pvscan
PV /dev/md127 VG onn_vmh lvm2 [222.44 GiB / 43.66 GiB free]
PV /dev/sdd1 VG gluster_vg3 lvm2 [<4.00 GiB / <2.00 GiB free]
Total: 2 [<226.44 GiB] / in use: 2 [<226.44 GiB] / in no VG: 0 [0 ]
Reading all physical volumes. This may take a while...
Found volume group "onn_vmh" using metadata type lvm2
Found volume group "gluster_vg3" using metadata type lvm2
# lvscan
ACTIVE '/dev/onn_vmh/pool00' [173.60 GiB] inherit
ACTIVE '/dev/onn_vmh/root' [146.60 GiB] inherit
ACTIVE '/dev/onn_vmh/home' [1.00 GiB] inherit
ACTIVE '/dev/onn_vmh/tmp' [1.00 GiB] inherit
ACTIVE '/dev/onn_vmh/var' [15.00 GiB] inherit
ACTIVE '/dev/onn_vmh/var_log' [8.00 GiB] inherit
ACTIVE '/dev/onn_vmh/var_log_audit' [2.00 GiB] inherit
ACTIVE '/dev/onn_vmh/swap' [4.00 GiB] inherit
inactive '/dev/onn_vmh/ovirt-node-ng-4.2.7.1-0.20181216.0' [146.60 GiB]
inherit
ACTIVE '/dev/onn_vmh/ovirt-node-ng-4.2.7.1-0.20181216.0+1' [146.60
GiB] inherit
ACTIVE '/dev/onn_vmh/var_crash' [10.00 GiB] inherit
ACTIVE '/dev/gluster_vg3/tmpLV' [2.00 GiB] inherit
[/etc/lvm/backup/gluster_vg1]
# Generated by LVM2 version 2.02.180(2)-RHEL7 (2018-07-20): Sat Dec 22 10:18:46 2018
contents = "Text Format Volume Group"
version = 1
description = "Created *after* executing '/usr/sbin/lvcreate --virtualsize 500GB
--name lv_datadisks -T gluster_vg1/lvthinpool'"
creation_host = "vmh.cyber-range.lan" # Linux vmh.cyber-range.lan
3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64
creation_time = 1545495526 # Sat Dec 22 10:18:46 2018
gluster_vg1 {
id = "TfNqtn-2eX6-i5gC-w4ye-h29n-5Zfy-UFHSvU"
seqno = 9
format = "lvm2" # informational
status = ["RESIZEABLE", "READ", "WRITE"]
flags = []
extent_size = 2048 # 1024 Kilobytes
max_lv = 0
max_pv = 0
metadata_copies = 0
physical_volumes {
pv0 {
id = "DRfoKl-TUhb-cirx-Oz9P-mZEY-XoiB-mdLy6v"
device = "/dev/sda" # Hint only
status = ["ALLOCATABLE"]
flags = []
dev_size = 11718885376 # 5.45703 Terabytes
pe_start = 2048
pe_count = 5722111 # 5.45703 Terabytes
}
}
logical_volumes {
engine_lv {
id = "P2ScEB-ws3V-iqVv-XTWh-Y2Jh-pg0c-6KnJXC"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
creation_time = 1545495460 # 2018-12-22 10:17:40 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 102400 # 100 Gigabytes
type = "striped"
stripe_count = 1 # linear
stripes = [
"pv0", 0
]
}
}
lvthinpool {
id = "c0yaNn-DcaB-cYjj-9ZRv-M2Em-rzLW-qY9WpI"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
creation_time = 1545495487 # 2018-12-22 10:18:07 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 512000 # 500 Gigabytes
type = "thin-pool"
metadata = "lvthinpool_tmeta"
pool = "lvthinpool_tdata"
transaction_id = 2
chunk_size = 2048 # 1024 Kilobytes
discards = "passdown"
zero_new_blocks = 1
}
}
lv_vmdisks {
id = "erpXRi-nPUq-mCf2-ga2J-3a0l-OWiC-i0xr8M"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
creation_time = 1545495493 # 2018-12-22 10:18:13 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 4613735 # 4.4 Terabytes
type = "thin"
thin_pool = "lvthinpool"
transaction_id = 0
device_id = 1
}
}
lv_datadisks {
id = "hKim3z-1QCh-dwhU-st2O-t4tG-wIss-UpLMZw"
status = ["READ", "WRITE", "VISIBLE"]
flags = []
creation_time = 1545495526 # 2018-12-22 10:18:46 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 512000 # 500 Gigabytes
type = "thin"
thin_pool = "lvthinpool"
transaction_id = 1
device_id = 2
}
}
lvol0_pmspare {
id = "bHc0eC-Z4Ed-mV47-QTU4-SCvo-FWbE-L8NV7Q"
status = ["READ", "WRITE"]
flags = []
creation_time = 1545495487 # 2018-12-22 10:18:07 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 16192 # 15.8125 Gigabytes
type = "striped"
stripe_count = 1 # linear
stripes = [
"pv0", 102400
]
}
}
lvthinpool_tmeta {
id = "WBut10-rAOP-FzA7-bJvr-ZdxL-lB70-jzz1Tv"
status = ["READ", "WRITE"]
flags = []
creation_time = 1545495487 # 2018-12-22 10:18:07 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 16192 # 15.8125 Gigabytes
type = "striped"
stripe_count = 1 # linear
stripes = [
"pv0", 630592
]
}
}
lvthinpool_tdata {
id = "rwNZux-1fz1-dv8J-yN2j-LcES-f6ml-231td5"
status = ["READ", "WRITE"]
flags = []
creation_time = 1545495487 # 2018-12-22 10:18:07 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 512000 # 500 Gigabytes
type = "striped"
stripe_count = 1 # linear
stripes = [
"pv0", 118592
]
}
}
}
}
# cd /var/log
# grep -ri gluster_vg1-lvthinpool-tpool
messages-20190922:Sep 15 03:58:15 vmh lvm[14072]: Failed command for
gluster_vg1-lvthinpool-tpool.
messages:Sep 22 03:44:05 vmh lvm[14072]: Failed command for gluster_vg1-lvthinpool-tpool.
messages-20190908:Sep 1 21:27:14 vmh lvm[14062]: Monitoring thin pool
gluster_vg1-lvthinpool-tpool.
messages-20190908:Sep 1 21:27:24 vmh lvm[14062]: WARNING: Thin pool
gluster_vg1-lvthinpool-tpool data is now 100.00% full.
messages-20190908:Sep 2 00:19:05 vmh lvm[14072]: Monitoring thin pool
gluster_vg1-lvthinpool-tpool.
messages-20190908:Sep 2 00:19:15 vmh lvm[14072]: WARNING: Thin pool
gluster_vg1-lvthinpool-tpool data is now 100.00% full.
messages-20190908:Sep 2 20:16:34 vmh lvm[14072]: Failed command for
gluster_vg1-lvthinpool-tpool.
5:06 a.m.
New subject: Ovirt 4.2.7 won't start and drops to emergency console
Here is my fstab file:
# /etc/fstab
# Created by anaconda on Fri Dec 21 22:26:32 2018
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/onn_vmh/ovirt-node-ng-4.2.7.1-0.20181216.0+1 / ext4 defaults,discard 1 1
UUID=9b9546f9-25d2-42a6-835b-303f32aee4b1 /boot ext4 defaults
1 2
/dev/mapper/onn_vmh-home /home ext4 defaults,discard 1 2
/dev/mapper/onn_vmh-tmp /tmp ext4 defaults,discard 1 2
/dev/mapper/onn_vmh-var /var ext4 defaults,discard 1 2
/dev/mapper/onn_vmh-var_log /var/log ext4 defaults,discard 1 2
/dev/mapper/onn_vmh-var_log_audit /var/log/audit ext4 defaults,discard 1 2
/dev/mapper/onn_vmh-swap swap swap defaults 0 0
/dev/gluster_vg1/engine_lv /gluster_bricks/engine xfs inode64,noatime,nodiratime 0 0
/dev/gluster_vg1/lv_vmdisks /gluster_bricks/vmstore xfs inode64,noatime,nodiratime 0 0
/dev/gluster_vg1/lv_datadisks /gluster_bricks/data xfs inode64,noatime,nodiratime 0 0
#/dev/sdd1 /gluster_bricks/iso xfs inode64,noatime,nodiratime 0 0
5:43 a.m.
New subject: Ovirt 4.2.7 won't start and drops to emergency console
http://man7.org/linux/man-pages/man7/lvmthin.7.html
Command to repair a thin pool:
lvconvert --repair VG/ThinPoolLV
Repair performs the following steps:
1. Creates a new, repaired copy of the metadata.
lvconvert runs the thin_repair command to read damaged metadata from
the existing pool metadata LV, and writes a new repaired copy to the
VG's pmspare LV.
2. Replaces the thin pool metadata LV.
If step 1 is successful, the thin pool metadata LV is replaced with
the pmspare LV containing the corrected metadata. The previous thin
pool metadata LV, containing the damaged metadata, becomes visible
with the new name ThinPoolLV_tmetaN (where N is 0,1,...).
If the repair works, the thin pool LV and its thin LVs can be
activated, and the LV containing the damaged thin pool metadata can
be removed. It may be useful to move the new metadata LV (previously
pmspare) to a better PV.
If the repair does not work, the thin pool LV and its thin LVs are
lost.
This info seems to conflict with Red Hat advice. Red Hat says if metadat volume is full
don't run a lvconvert --repair operation. Now I am confused. I am familiar with LVM
and comfortable with it but this is my first time trying to repair thin LVM. The concepts
of a metadata volume are new to me.
1878
days inactive
1879
days old
4 comments
2 participants
participants (2)
-
jeremy_tourville＠hotmail.com -
Strahil