<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 3, 2018 at 9:46 PM, RabidCicada <span dir="ltr">&lt;<a href="mailto:rabidcicada@gmail.com" target="_blank">rabidcicada@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Indeed that is the problem.  I am sshing as root <div><br></div><div>The only possible explanations I have for this working in the ovirt autmated testing setup are:</div><div><ul><li>Either the ovirt automated tests must be using a host inventory file with become: true?</li><li>The inventory file ssh credentials happen to be to vdsm user? (And become is just not doing anything)<br></li><li>Some other janky thing that allows the access</li></ul></div></div></blockquote>Yes, you are absolutely right:<br>we are running the CI suite with no_root_squash<br><a href="https://github.com/oVirt/ovirt-system-tests/blob/master/common/deploy-scripts/setup_storage_nfs_el7.sh#L17">https://github.com/oVirt/ovirt-system-tests/blob/master/common/deploy-scripts/setup_storage_nfs_el7.sh#L17</a><br>while we recommend anonuid=36,anongid=36,all_squash</div><div class="gmail_quote"><a href="https://www.ovirt.org/documentation/how-to/troubleshooting/troubleshooting-nfs-storage-issues/">https://www.ovirt.org/documentation/how-to/troubleshooting/troubleshooting-nfs-storage-issues/</a><br></div><div class="gmail_quote">and this explains why it can write as root in the CI env.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Thanks for the report!<br><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br></div><div>It also happens for the immediately following task `Initialize metadata volume` and many more following that task.</div><div><br></div><div>I will correct and then open a pull request to fix with your input that you believe it&#39;s a problem.</div><div><br></div><div>~Kyle</div><div><br></div></div></div><div class="gmail-HOEnZb"><div class="gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 3, 2018 at 3:08 PM, RabidCicada <span dir="ltr">&lt;<a href="mailto:rabidcicada@gmail.com" target="_blank">rabidcicada@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Alright.  By differential comparing to successfull postgres-command sudo-becomes.  It looks like the salient difference is that `Copy configuration archive to storage` is missing become:true.  Testing now.<div><br></div><div>The similar postgres commands have become:true while this one does not.</div><div><br></div><div>~Kyle</div></div><div class="gmail-m_-807229225562045180HOEnZb"><div class="gmail-m_-807229225562045180h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 3, 2018 at 1:19 PM, Simone Tiraboschi <span dir="ltr">&lt;<a href="mailto:stirabos@redhat.com" target="_blank">stirabos@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-m_-807229225562045180m_7831304234596663950h5">On Tue, Apr 3, 2018 at 5:21 PM, RabidCicada <span dir="ltr">&lt;<a href="mailto:rabidcicada@gmail.com" target="_blank">rabidcicada@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I&#39;ve attached a full debug packet below.  I include the log file from ovirt-hosted-engine-setup.  I include relevant cmd line info.  I include info from the command line where epdb has a breakpoint in playbook.py from ansible itself.  I also include info from commands I ran after it failed.  I also include attached the ferried over script in /root/.ansible/tmp that is run.</div><div><br></div><div><b><br></b></div><div><b>Output on cmd line:</b></div><div>[ INFO  ] TASK [Copy configuration archive to storage]<br></div><div><div>[ ERROR ] fatal: [localhost]: FAILED! =&gt; {</div><div>             &quot;changed&quot;: true,</div><div>             &quot;cmd&quot;: [</div><div>                 &quot;dd&quot;,</div><div>                 &quot;bs=20480&quot;,</div><div>                 &quot;count=1&quot;,</div><div>                 &quot;oflag=direct&quot;,</div><div>                 &quot;if=/var/tmp/localvmbCDQIR/5e<wbr>f881f5-c992-48d2-b969-a0b6156b<wbr>df7c&quot;,</div><div>                 &quot;of=/rhev/data-center/mnt/nod<wbr>e.local:_srv_data/81292f3f-11d<wbr>3-4e38-9afa-62e133aa8017/image<wbr>s/c5510e77-1ee0-479c-b6cf-24c1<wbr>79313a45/5ef881f5-c992-48d2-b9<wbr>69-a0b6156bdf7c&quot;</div><div>             ],</div><div>             &quot;delta&quot;: &quot;0:00:00.004336&quot;,</div><div>             &quot;end&quot;: &quot;2018-04-03 15:01:55.581823&quot;,</div><div>             &quot;invocation&quot;: {</div><div>                 &quot;module_args&quot;: {</div><div>                     &quot;_raw_params&quot;: &quot;dd bs=20480 count=1 oflag=direct if=\&quot;/var/tmp/localvmbCDQIR/5e<wbr>f881f5-c992-48d2-b969-a0b6156b<wbr>df7c\&quot; of=\&quot;/rhev/data-center/mnt/nod<wbr>e.local:_srv_data/81292f3f-11d<wbr>3-4e38-9afa-62e133aa8017/image<wbr>s/c5510e77-1ee0-479c-b6cf-24c1<wbr>79313a45/5ef881f5-c992-48d2-b9<wbr>69-a0b6156bdf7c\&quot;&quot;,</div><div>                     &quot;_uses_shell&quot;: false,</div><div>                     &quot;chdir&quot;: null,</div><div>                     &quot;creates&quot;: null,</div><div>                     &quot;executable&quot;: null,</div><div>                     &quot;removes&quot;: null,</div><div>                     &quot;stdin&quot;: null,</div><div>                     &quot;warn&quot;: true</div><div>                 }</div><div>             },</div><div>             &quot;msg&quot;: &quot;non-zero return code&quot;,</div><div>             &quot;rc&quot;: 1,</div><div>             &quot;start&quot;: &quot;2018-04-03 15:01:55.577487&quot;,</div><div>             &quot;stderr&quot;: &quot;dd: failed to open ‘/rhev/data-center/mnt/node.lo<wbr>cal:_srv_data/81292f3f-11d3-4e<wbr>38-9afa-62e133aa8017/images/c5<wbr>510e77-1ee0-479c-b6cf-24c17931<wbr>3a45/5ef881f5-c992-48d2-b969-a<wbr>0b6156bdf7c’: Permission denied&quot;,</div><div>             &quot;stderr_lines&quot;: [</div><div>                 &quot;dd: failed to open ‘/rhev/data-center/mnt/node.lo<wbr>cal:_srv_data/81292f3f-11d3-4e<wbr>38-9afa-62e133aa8017/images/c5<wbr>510e77-1ee0-479c-b6cf-24c17931<wbr>3a45/5ef881f5-c992-48d2-b969-a<wbr>0b6156bdf7c’: Permission denied&quot;</div><div>             ],</div><div>             &quot;stdout&quot;: &quot;&quot;,</div><div>             &quot;stdout_lines&quot;: []</div><div>         }</div><div>[ ERROR ] Failed to execute stage &#39;Closing up&#39;: Failed executing ansible-playbook</div></div></div></blockquote><div><br></div></div></div><div>In the playbook we have on that task:</div><div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">    become_user: vdsm</div><div style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">    become_method: sudo</div><br></div><div>but I fear it got somehow ignored.</div><div>I&#39;ll investigate it.</div><div><div class="gmail-m_-807229225562045180m_7831304234596663950h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div><b>Output from ansible epdb tracepoint:</b></div><div><b><br></b></div><div><div>Using module file /usr/lib/python2.7/site-packag<wbr>es/ansible/modules/commands/co<wbr>mmand.py</div><div>&lt;localhost&gt; ESTABLISH LOCAL CONNECTION FOR USER: root</div><div>&lt;localhost&gt; EXEC /bin/sh -c &#39;echo ~ &amp;&amp; sleep 0&#39;</div><div>&lt;localhost&gt; EXEC /bin/sh -c &#39;( umask 77 &amp;&amp; mkdir -p &quot;` echo /root/.ansible/tmp/ansible-tmp<wbr>-1522767715.36-81496549401055 `&quot; &amp;&amp; echo ansible-tmp-1522767715.36-8149<wbr>6549401055=&quot;` echo /root/.ansible/tmp/ansible-tmp<wbr>-1522767715.36-81496549401055 `&quot; ) &amp;&amp; sleep 0&#39;</div><div>&lt;localhost&gt; PUT /tmp/tmpGMGdjh TO /root/.ansible/tmp/ansible-tmp<wbr>-1522767715.36-81496549401055/<wbr>command.py</div><div>&lt;localhost&gt; EXEC /bin/sh -c &#39;chmod u+x /root/.ansible/tmp/ansible-tmp<wbr>-1522767715.36-81496549401055/ /root/.ansible/tmp/ansible-tmp<wbr>-1522767715.36-81496549401055/<wbr>command.py &amp;&amp; sleep 0&#39;</div><div>&lt;localhost&gt; EXEC /bin/sh -c &#39;/usr/bin/python /root/.ansible/tmp/ansible-tmp<wbr>-1522767715.36-81496549401055/<wbr>command.py &amp;&amp; sleep 0&#39;</div><div>        to retry, use: --limit @/usr/share/ovirt-hosted-engin<wbr>e-setup/ansible/create_target_<wbr>vm.retry</div></div><div><b><br></b></div><div>The above command.py is the on I have attached as problematic_command.py</div><div><br></div><div><b>Investigation After Failure:</b></div><div>[root@node ~]# ls -al &#39;/rhev/data-center/mnt/node.lo<wbr>cal:_srv_data/81292f3f-11d3-4e<wbr>38-9afa-62e133aa8017/images/c5<wbr>510e77-1ee0-479c-b6cf-24c17931<wbr>3a45/5ef881f5-c992-48d2-b969-a<wbr>0b6156bdf7c&#39;</div><div>-rw-rw----. 1 vdsm kvm 20480 Apr  3 15:01 /rhev/data-center/mnt/node.loc<wbr>al:_srv_data/81292f3f-11d3-4e3<wbr>8-9afa-62e133aa8017/images/c55<wbr>10e77-1ee0-479c-b6cf-24c179313<wbr>a45/5ef881f5-c992-48d2-b969-a0<wbr>b6156bdf7c</div><div><br></div><div>sudo -u vdsm dd bs=20480 count=1 oflag=direct if=&quot;/var/tmp/localvmbCDQIR/5ef<wbr>881f5-c992-48d2-b969-a0b6156bd<wbr>f7c&quot; of=&quot;/rhev/data-center/mnt/node<wbr>.local:_srv_data/81292f3f-11d3<wbr>-4e38-9afa-62e133aa8017/images<wbr>/c5510e77-1ee0-479c-b6cf-24c17<wbr>9313a45/5ef881f5-c992-48d2-b96<wbr>9-a0b6156bdf7c</div><div><div>1+0 records in</div><div>1+0 records out</div></div><div><br></div><div><br></div><div>It seems to me that somehow it is not getting the right permissions even though the playbook has:</div><span><div><div>- name: Copy configuration archive to storage</div><div>    command: dd bs=20480 count=1 oflag=direct if=&quot;{{ LOCAL_VM_DIR }}/{{ he_conf_disk_details.disk.imag<wbr>e_id }}&quot; of=&quot;{{ he_conf_disk_path }}&quot;</div><div>    become_user: vdsm</div><div>    become_method: sudo</div><div>    changed_when: True</div><div>    tags: [ &#39;skip_ansible_lint&#39; ]</div></div><div><br></div></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628HOEnZb"><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 3, 2018 at 8:51 AM, RabidCicada <span dir="ltr">&lt;<a href="mailto:rabidcicada@gmail.com" target="_blank">rabidcicada@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I am now also running with:<div>




<span></span>





<p class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(51,187,200);background-color:rgb(255,255,255)"><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">export</span><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s2" style="font-variant-ligatures:no-common-ligatures"> ANSIBLE_VERBOSITY=</span><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">5</span></p>
<p class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(51,187,200);background-color:rgb(255,255,255)"><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">export</span><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s2" style="font-variant-ligatures:no-common-ligatures"> ANSIBLE_FORKS=</span><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">1</span></p>
<p class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-p1" style="margin:0px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(51,187,200);background-color:rgb(255,255,255)"><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s1" style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">export</span><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s2" style="font-variant-ligatures:no-common-ligatures"> ANSIBLE_KEEP_REMOTE_FILES=</span><span class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035gmail-s3" style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">1</span></p>


<br></div></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429HOEnZb"><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 3, 2018 at 8:49 AM, RabidCicada <span dir="ltr">&lt;<a href="mailto:rabidcicada@gmail.com" target="_blank">rabidcicada@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Here&#39;s the log.<div><br></div><div>So the command that it says it ran is:</div><div>dd bs=20480 count=1 oflag=direct if=/var/tmp/localvmHaWb6G/1cce<wbr>8df2-1810-4063-b4e2-e19a2c5b19<wbr>09 of=/rhev/data-center/mnt/node.<wbr>local:_srv_data/3c7485ea-14e3-<wbr>40c1-b627-f89a819ed1d6/images/<wbr>2c1f7c2f-b8f7-46d4-ac66-8ff1e9<wbr>649e29/1cce8df2-1810-4063-b4e2<wbr>-e19a2c5b1909<br></div><div><br></div><div>But we all know that was done with:</div><div><br></div><div><div>  - name: Copy configuration archive to storage</div><div>    command: dd bs=20480 count=1 oflag=direct if=&quot;{{ LOCAL_VM_DIR }}/{{ he_conf_disk_details.disk.imag<wbr>e_id }}&quot; of=&quot;{{ he_conf_disk_path }}&quot;</div><div>    become_user: vdsm</div><div>    become_method: sudo</div><div>    changed_when: True</div><div>    tags: [ &#39;skip_ansible_lint&#39; ]</div></div><div><br></div><div>So I manually replicated with `sudo vdsm <span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">dd bs=20480 count=1 if=/var/tmp/localvmHaWb6G/1cce<wbr>8df2-1810-4063-b4e2-e19a2c5b19<wbr>09 of=/rhev/data-center/mnt/node.<wbr>local:_srv_data/3c7485ea-14e3-<wbr>40c1-b627-f89a819ed1d6/images/<wbr>2c1f7c2f-b8f7-46d4-ac66-8ff1e9<wbr>649e29/1cce8df2-1810-4063-b4e2<wbr>-e19a2c5b1909`</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">And it works when I manually do it.  Though I think I didn&#39;t use the oflag=direct (Just realised this)</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I eventually put a pause task directly preceeding it with debug output that showed the file paths.  I manually ran the command and it worked.  Then let it do it....failed.  I checked all the permissions e.g.  vdsm:kvm.  All looks good from a filesystem point of view.  I&#39;m beginning (naively) to suspect a race condition for the file access problem...but have come nowhere close to solving it.</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><b><u>Can you suggest a good way to continue executing the install process from the create_target_vm.yml playbook (with proper variables and context from otopi etc)?</u></b></span><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">  I currently have to restart the entire process over again and wait quite a while for it to circle back around.  </span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I have since discovered epdb and I&#39;ve set breakpoints directly in playbook.py of ansible just to see better log output.  I insert epdb.serve and use netcat to connect since epdb on python 2.7.5 and up seems to have problems using epdb.connect() itself.</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">~Kyle</span></div></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035HOEnZb"><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 3, 2018 at 4:06 AM, Simone Tiraboschi <span dir="ltr">&lt;<a href="mailto:stirabos@redhat.com" target="_blank">stirabos@redhat.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Apr 2, 2018 at 4:52 PM, RabidCicada <span dir="ltr">&lt;<a href="mailto:rabidcicada@gmail.com" target="_blank">rabidcicada@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span>Heyo everyone.<span>  </span>I&#39;m trying to debug hosted-engine --deploy.  It is failing in <span><span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">`Copy configuration archive to storage` in `create_target_vm.yml` from `<span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">hosted-engine --deploy`.  My general and most important query here is how to get good debug output from ansible through hosted-engine.  I&#39;m running hosted-engine through an ssh session.</span></span></span></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span><br></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span>I can&#39;t figure out how to get good debug output from ansible within that workflow.<span>  </span>I see it&#39;s running through otopi, I tried setting typical `debugger: on_failed` hooks etc and tried many incantations on the command </span><span style="font-size:12.8px">line and config files to get ansible to help me out.  The debugger: directive and other debugger related ansible config file stuff wouldn&#39;t result in any debugger popping up.  I also can&#39;t seem to pass normal -vvvv flags to hosted-engine either and get it to ansible.  Ultimately I tried to use a `pause` directive and it complained that it was in a non-interactive shell.  I figured it might be the result of my ssh session so I enabled tty allocation with -t -t.  It did not resolve the issue.</span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span><br></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span>I eventually wrote-my-own/stole a callback_plugin that checks an environmental variable and enables `display.verbosity = int(v)` since I can&#39;t seem to pass typical -vvvv stuff to ansible through `hosted-engine --deploy`.  It give me the best info that I have so far.   But it wont give me enough to debug issues around Gathering Facts or what looks like a sudo/permission problem in <span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">`Copy configuration archive to storage` in `create_target_vm.yml`.  I took and used the exact command that they use manually and it works when I run it manually (But I can&#39;t get debug output to show me the exact sudo command being executed), hence my interest in passing -vvvv or equivalent to ansible through `hosted-engine`.  I intentionally disabled the VM_directory cleanup so that I could execute the same stuff.</span></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span><span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">So....after all that...what is a good way to get deep debug info from hosted-engine ansible stuff?</div></div></blockquote><div><br></div></span><div>You should already find all the relevant log entries in a file called /var/log/ovirt-hosted-e<wbr>ngine-setup/ovirt-hosted-engin<wbr>e-setup-ansible-create_target_<wbr>vm-{timestamp}-{hash}.log</div><div><br></div><div>Can you please share it?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span><div dir="ltr"><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">Or does anyone have intuition for the possible sudo problem?</div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span><span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">~Kyle</span></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span><span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle1 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><span><span style="color:rgb(0,0,0);font-family:Consolas,&quot;Lucida Console&quot;,monospace;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></span></div><div class="gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-linestyle2 gmail-m_-807229225562045180m_7831304234596663950m_766559802820559628m_5635528246031702429m_5983775323000025035m_278240774770278449m_6346744230857222759gmail-m_-5418010615936734164gmail-colourline" style="font-family:Consolas,&quot;Lucida Console&quot;,monospace;padding-left:7px;word-wrap:break-word;color:rgb(0,0,0);font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-transform:none;white-space:normal;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial"><br></div></div>
<br></span>______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/users</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>