<div dir="ltr">Well yes I am sure about the research I did :-)<div><br></div><div>However, to your point, I didnt actually consider that and of course now clearly makes the most sense. Thanks!</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 22, 2017 at 9:57 AM, Yedidyah Bar David <span dir="ltr"><<a href="mailto:didi@redhat.com" target="_blank">didi@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Wed, Mar 22, 2017 at 3:43 PM, Charles Kozler <<a href="mailto:ckozleriii@gmail.com">ckozleriii@gmail.com</a>> wrote:<br>
> Thanks for the feedback!<br>
><br>
> From my research though it seems like it would take some effort to start a<br>
> process and not register it in /proc or at least, it would be by intention<br>
> to do so for that desired affect.<br>
<br>
</span>Are you sure about that?<br>
<br>
What if the process is simply very short-lived? Wouldn't something like OSSEC<br>
wrongly count it as "hiding" simply because it died in the middle of being<br>
investigated? E.g. vdsm runs many times instances of 'dd' to check that some<br>
file is readable (or writable), IIRC.<br>
<div class="HOEnZb"><div class="h5"><br>
> I guess my ask here would be why would<br>
> ovirt do that? Is there a relative performance gain? What processes inside<br>
> ovirt would do such a thing?<br>
><br>
> Appreciate the help<br>
><br>
> On Wed, Mar 22, 2017 at 3:32 AM, Yedidyah Bar David <<a href="mailto:didi@redhat.com">didi@redhat.com</a>> wrote:<br>
>><br>
>> On Tue, Mar 21, 2017 at 7:54 PM, Charles Kozler <<a href="mailto:ckozleriii@gmail.com">ckozleriii@gmail.com</a>><br>
>> wrote:<br>
>> > Unfortunately by the time I am able to SSH to the server and start<br>
>> > looking<br>
>> > around, that PID is no where to be found<br>
>><br>
>> Even if you do this immediately when OSSEC finishes?<br>
>> Do you get from it only a single pid?<br>
>><br>
>> ><br>
>> > So it seems something winds up in ovirt, runs, doesnt register in /proc<br>
>> > (I<br>
>> > think even threads register themself in /proc),<br>
>><br>
>> Now did some tests. Seems like they do, but are only "visible" if you<br>
>> access them directly, not if you e.g. 'ls -l /proc'.<br>
>><br>
>> > and then dies off<br>
>> ><br>
>> > Any ideas?<br>
>><br>
>> No idea about your specific issue. Based on your above question, did this:<br>
>><br>
>> # for pid in $(seq 32768); do if kill -0 $pid 2>/dev/null && ! ls -1<br>
>> /proc | grep -qw $pid; then ps -e -T | grep -w $pid | awk '{print<br>
>> $1}'; fi; done | sort -u | while read ppid; do echo number of threads:<br>
>> $(ps -e -T | grep -w $ppid | wc -l) ps $ppid: $(ps -h -p $ppid); done<br>
>> number of threads: 5 ps 1149: 1149 ? Ssl 0:23 /usr/bin/python -Es<br>
>> /usr/sbin/tuned -l -P<br>
>> number of threads: 3 ps 1151: 1151 ? Ssl 0:55 /usr/sbin/rsyslogd -n<br>
>> number of threads: 2 ps 1155: 1155 ? Ssl 0:00 /usr/bin/ruby<br>
>> /usr/bin/fluentd -c /etc/fluentd/fluent.conf<br>
>> number of threads: 12 ps 1156: 1156 ? Ssl 4:49 /usr/sbin/collectd<br>
>> number of threads: 16 ps 1205: 1205 ? Ssl 0:08 /usr/sbin/libvirtd --listen<br>
>> number of threads: 6 ps 1426: 1426 ? Sl 23:57 /usr/bin/ruby<br>
>> /usr/bin/fluentd -c /etc/fluentd/fluent.conf<br>
>> number of threads: 32 ps 3171: 3171 ? S<sl 6:48 /usr/bin/python2<br>
>> /usr/share/vdsm/vdsmd<br>
>> number of threads: 6 ps 3173: 3173 ? Ssl 8:48 python /usr/sbin/momd -c<br>
>> /etc/vdsm/mom.conf<br>
>> number of threads: 7 ps 575: 575 ? SLl 0:14 /sbin/multipathd<br>
>> number of threads: 3 ps 667: 667 ? SLsl 0:09 /usr/sbin/dmeventd -f<br>
>> number of threads: 2 ps 706: 706 ? S<sl 0:00 /sbin/auditd -n<br>
>> number of threads: 6 ps 730: 730 ? Ssl 0:00 /usr/lib/polkit-1/polkitd<br>
>> --no-debug<br>
>> number of threads: 3 ps 735: 735 ? Ssl 0:31 /usr/bin/python<br>
>> /usr/bin/ovirt-imageio-daemon<br>
>> number of threads: 4 ps 741: 741 ? S<sl 0:00 /usr/bin/python2<br>
>> /usr/share/vdsm/supervdsmd --sockfile /var/run/vdsm/svdsm.sock<br>
>> number of threads: 2 ps 743: 743 ? Ssl 0:00 /bin/dbus-daemon --system<br>
>> --address=systemd: --nofork --nopidfile --systemd-activation<br>
>> number of threads: 6 ps 759: 759 ? Ssl 0:00 /usr/sbin/gssproxy -D<br>
>> number of threads: 5 ps 790: 790 ? SLsl 0:09 /usr/sbin/sanlock daemon<br>
>><br>
>> (There are probably more efficient ways to do this, nvm).<br>
>><br>
>> ><br>
>> > On Tue, Mar 21, 2017 at 3:10 AM, Yedidyah Bar David <<a href="mailto:didi@redhat.com">didi@redhat.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> On Mon, Mar 20, 2017 at 5:59 PM, Charles Kozler <<a href="mailto:ckozleriii@gmail.com">ckozleriii@gmail.com</a>><br>
>> >> wrote:<br>
>> >> > Hi -<br>
>> >> ><br>
>> >> > I am wondering why OSSEC would be reporting hidden processes on my<br>
>> >> > ovirt<br>
>> >> > nodes? I run OSSEC across the infrastructure and multiple ovirt<br>
>> >> > clusters<br>
>> >> > have assorted nodes that will report a process is running but does<br>
>> >> > not<br>
>> >> > have<br>
>> >> > an entry in /proc and thus "possible rootkit" alert is fired<br>
>> >> ><br>
>> >> > I am well aware that I do not have rootkits on these systems but am<br>
>> >> > wondering what exactly inside ovirt is causing this to trigger? Or<br>
>> >> > any<br>
>> >> > ideas? Below is sample alert. All my google-fu turns up is that a<br>
>> >> > process<br>
>> >> > would have to **try** to hide itself from /proc, so curious what this<br>
>> >> > is<br>
>> >> > inside ovirt. Thanks!<br>
>> >> ><br>
>> >> > -------------<br>
>> >> ><br>
>> >> > OSSEC HIDS Notification.<br>
>> >> > 2017 Mar 20 11:54:47<br>
>> >> ><br>
>> >> > Received From: (ovirtnode2.mydomain.com2) any->rootcheck<br>
>> >> > Rule: 510 fired (level 7) -> "Host-based anomaly detection event<br>
>> >> > (rootcheck)."<br>
>> >> > Portion of the log(s):<br>
>> >> ><br>
>> >> > Process '24574' hidden from /proc. Possible kernel level rootkit.<br>
>> >><br>
>> >> What do you get from:<br>
>> >><br>
>> >> ps -eLf | grep -w 24574<br>
>> >><br>
>> >> Thanks,<br>
>> >> --<br>
>> >> Didi<br>
>> ><br>
>> ><br>
>><br>
>><br>
>><br>
>> --<br>
>> Didi<br>
><br>
><br>
<br>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Didi<br>
</font></span></blockquote></div><br></div>