Dear users, I would love to get some feedback if someone has tried to install and use the oVirt metrics store, released in 4.2, for collecting metrics and logs, based on Elasticsearch, Kibana, Collectd and Fluentd on top of OpenShift. https://www.ovirt.org/develop/release-management/features/metrics/metrics-st... How did the installation go? Are you actively using it? And any other feedback would be much appreciated. Best regards, -- SHIRLY RADCO BI SeNIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Shirly, I could one more try to install it. The firsts few attempts leads to long discussion in the mailing list and I was not able to get it working . On the other hand I do not have time to finish integration with the zabbix ;( regard Peter On 11/06/2018 12:30, Shirly Radco wrote:
- -- *Peter Hudec* Infraštruktúrny architekt phudec@cnc.sk <mailto:phudec@cnc.sk> *CNC, a.s.* Borská 6, 841 04 Bratislava Recepcia: +421 2 35 000 100 Mobil:+421 905 997 203 *www.cnc.sk* <http:///www.cnc.sk> -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEqSUbhuEwhryifNeVQnvVWOJ35BAFAlseZQUACgkQQnvVWOJ3 5BAa8hAArd0EpkVM0J4E5aAJVVC9MQWwmf9m/zVIrm4WMOjziD3SCuYHeXbkpAJq AEzUBRB7ltcVXWkqZJx3hydFZkxfPrKX6wVQylcPMZGXeUZKEGiLJWCFtgvAUosO YL6M2/tis3QcGGTto/7ZmKMwHvruEspW0d2P0Cpk2k/fKwkoChTM9VMhwSlbNJuN KxH4dnOVLkjRFPU5cO9Rk56WzBK78MeL6mdZksM8nNP4cUJQ+VGoZjXu0fR941Yu qyBQh6D1d3S56S+8xK9q9a8z99JzAUWm3Y46Gyrs0lBjxFfwefYhP4PpZp9UiBLx nW82ATxKgKL7WZ+84M2H4bt9KDPs9N7xfMhFVkiFJHOz6vP9E66RM4ZDCuplO55m yCLS/NexyIXhHDJ6DcnDizlW4WDZoXVlSzyvvLkL7ZlrPgMzTUgaiennrtUIEwHA afWrpWgs843tCXBSXrm8NMdhZNHyFH6QEUFeH8pxCcxv2Nss9MImpzfcXBMkffdm NnTOFK1Vtmvd1B6sLC9v99Y74AUiJNroEDiwqPUF8iAkYRUHfqvGxpPw1EQZyOa9 oRzBAqlhjqELNjVPy2ciE5YwB2e8CdtfAEe8gPLSad9x4zLbeVsFvuzy+nrwqZlm +Hnx+f9xlvoPutC+J2tHJilga7GHib7HQOzauJJRN5JawfumfWM= =Mkzr -----END PGP SIGNATURE-----
Hello, Good morning Shirley. I am have one doubt , for collecting metrics need install agents in vms?. Very thanks. Marcelo Leandro
Hi, No need to install agents in the vms. Collectd virt plugin reports the statistics about the vms from the host. Best regards, -- SHIRLY RADCO BI SeNIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> On Mon, Jun 11, 2018 at 3:26 PM, Marcelo Leandro <marceloltmm@gmail.com> wrote:
Hi, i'm trying to install oVirt Metrics following the installation guide (h ttps://www.ovirt.org/develop/release- management/features/metrics/metrics-store-installation/) but i have some issues running deploy cluster from here https://www.ovirt.org/deve lop/release-management/features/metrics/setting-up-viaq-logging/ cd /usr/share/ansible/openshift-ansible # (or wherever you cloned the git repo if using git) ANSIBLE_LOG_PATH=/tmp/ansible.log ansible-playbook -vvv -e @/root/vars.yaml -i /root/ansible-inventory-origin-39-aio playbooks/deploy_cluster.yml I fails on Web console installation: 2018-06-06 19:48:24,020 p=17586 u=root | [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|version_compare` instead use `result is version_compare`. This feature will be removed in version 2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. 2018-06-06 19:48:24,135 p=17586 u=root | Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py 2018-06-06 19:48:27,093 p=17586 u=root | fatal: [localhost]: FAILED! => { "changed": true, "cmd": [ "oc", "logs", "deployment/webconsole", "--tail=50", "--config=/tmp/console-ansible-C8XDuW/admin.kubeconfig", "-n", "openshift-web-console" ], "delta": "0:00:01.567694", "end": "2018-06-06 19:48:26.706407", "invocation": { "module_args": { "_raw_params": "oc logs deployment/webconsole --tail=50 -- config=/tmp/console-ansible-C8XDuW/admin.kubeconfig -n openshift-web- console", "_uses_shell": false, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "warn": true } }, "msg": "non-zero return code", "rc": 1, "start": "2018-06-06 19:48:25.138713", "stderr": "Error from server (BadRequest): container \"webconsole\" in pod \"webconsole-84466b9d97-s4x28\" is waiting to start: ContainerCreating", "stderr_lines": [ "Error from server (BadRequest): container \"webconsole\" in pod \"webconsole-84466b9d97-s4x28\" is waiting to start: ContainerCreating" ], "stdout": "", "stdout_lines": [] } 2018-06-06 19:48:27,097 p=17586 u=root | ...ignoring Regards, Carlos Rodrigues On Mon, 2018-06-11 at 13:30 +0300, Shirly Radco wrote:
Engenheiro de Software Sénior Eurotux Informática, S.A. | www.eurotux.com (t) +351 253 680 300 (m) +351 911 926 110
Hi Rich, Are you families with this OpenShift installation issue? Best, -- SHIRLY RADCO BI SeNIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> On Mon, Jun 11, 2018 at 4:20 PM, Carlos Rodrigues <cmar@eurotux.com> wrote:
-- SHIRLY RADCO BI SeNIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> On Tue, Jun 12, 2018 at 11:43 AM, Shirly Radco <sradco@redhat.com> wrote:
Hi Rich,
Are you families with this OpenShift installation issue?
familiar*
http://post-office.corp.redhat.com/archives/aos-devel/2018-June/msg00195.htm... " It smells like https://access.redhat.com/solutions/3480921 / https://bugzilla.redhat.com/show_bug.cgi?id=1583500 to me. " I think the workaround is to add oreg_url=registry.access.redhat.com/openshift3/ose-${component}:${version} to your inventory for OCP external and oreg_url=brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/ose-${component}:${version} for OCP internal and oreg_url=docker.io/openshift/origin-${component}:${version} for Origin On 06/12/2018 02:47 AM, Shirly Radco wrote:
Hi, i still get the same error as you can see in attachment. I also send in attachment the ansible log result from run the following command: ANSIBLE_LOG_PATH=/tmp/ansible.log ansible-playbook -vvv -e @/root/vars.yaml -i /root/ansible-inventory-origin-39-aio playbooks/deploy_cluster.yml Regards, Carlos Rodrigues On Tue, 2018-06-12 at 15:11 +0100, Carlos Rodrigues wrote:
Engenheiro de Software Sénior Eurotux Informática, S.A. | www.eurotux.com (t) +351 253 680 300 (m) +351 911 926 110
This is a different error than the one described in the links below: "5m 1h 128 webconsole-84466b9d97-s4x28.153776d1bf88b3a4 Pod Warning FailedCreatePodSandBox kubelet, openshift-ied.install.etux Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image \"docker.io/openshift/origin-pod:v3.9.0\": Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Is it possible that there is some sort of networking issue that you cannot access https://registry-1.docker.io? from the machine - can you do curl -vs https://registry-1.docker.io can you do sudo docker pull docker.io/openshift/origin-pod:v3.9.0 ? On 06/14/2018 03:18 AM, Carlos Rodrigues wrote:
Hi, I'm behind a proxy. After configure docker to use proxy, the following command run well: sudo docker pull docker.io/openshift/origin-pod:v3.9.0 Thank you. Regards, Carlos Rodrigues On Thu, 2018-06-14 at 13:43 -0600, Rich Megginson wrote:
-- Carlos Rodrigues Engenheiro de Software Sénior Eurotux Informática, S.A. | www.eurotux.com (t) +351 253 680 300 (m) +351 911 926 110
I'm able to go forward in installation but i get another error as you can see in attachment. Could you help me? Regards, Carlos Rodrigues On Fri, 2018-06-15 at 05:57 -0600, Rich Megginson wrote:
-- Carlos Rodrigues Engenheiro de Software Sénior Eurotux Informática, S.A. | www.eurotux.com (t) +351 253 680 300 (m) +351 911 926 110
this is the result of execution of following commands: [root@openshift-ied ~]# oc project kube-service-catalogNow using project "kube-service-catalog" on server "https://openshift-ied.install .etux:8443".[root@openshift-ied ~]# oc get podsNAME READY STATUS RESTARTS AGEapiser ver-h6rg4 1/1 Running 0 1hcontroller- manager-d8vkq 1/1 Running 5 3h[root@openshift-ied ~]# oc describe pod apiserver-h6rg4 Name: apiserver- h6rg4Namespace: kube-service-catalogNode: openshift- ied.install.etux/10.10.4.248Start Time: Fri, 15 Jun 2018 14:16:50 +0100Labels: app=apiserver controller-revision- hash=296530938 pod-template- generation=2Annotations: ca_hash=63a4fb42b7ebe23182afbfd0146c520b4d0 bb4cd openshift.io/scc=hostmount- anyuidStatus: RunningIP: 10.128.0.20Controlled By: DaemonSet/apiserverContainers: apiserver: Container ID: docker://1eb20cb1b8235c72cb8016a3587a087ae2f954834ec4d259d71bf10e5 b25d034 Image: docker.io/openshift/origin-service- catalog:v3.9.0 Image ID: docker- pullable://docker.io/openshift/origin-service- catalog@sha256:4c8fa186fce466c8b35afbbd715207d473333369cb92b6710faa4a70 7fb038a5 Port: 6443/TCP Command: /usr/bin/service- catalog Args: apiserver --storage-type etcd --secure-port 6443 --etcd-servers https://openshift-ied. install.etux:23799 --etcd- cafile /etc/origin/master/master.etcd-ca.crt --etcd- certfile /etc/origin/master/master.etcd-client.crt --etcd- keyfile /etc/origin/master/master.etcd-client.key - v 3 --cors-allowed-origins localhost --admission- control KubernetesNamespaceLifecycle,DefaultServicePlan,ServiceBin dingsLifecycle,ServicePlanChangeValidator,BrokerAuthSarCheck --feature- gates OriginatingIdentity=true State: Running Sta rted: Fri, 15 Jun 2018 14:16:58 +0100 Ready: True Restart Count: 0 Environment: <none> Mounts: /etc/origin/master from etcd-host-cert (ro) /var/run/kubernetes-service-catalog from apiserver-ssl (ro) /var/run/secrets/kubernetes.io/serviceaccount from service-catalog-apiserver-token-ptkqz (ro)Conditions: Type Status Initialized True Ready True PodScheduled True Volumes: apiserver-ssl: Type: Secret (a volume populated by a Secret) SecretName: apiserver- ssl Optional: false etcd-host-cert: Type: HostPath (bare host directory volume) Path: /etc/origin/master HostPathType: data- dir: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: service-catalog-apiserver-token- ptkqz: Type: Secret (a volume populated by a Secret) SecretName: service-catalog-apiserver-token- ptkqz Optional: falseQoS Class: BestEffortNode- Selectors: node- role.kubernetes.io/master=trueTolerations: node.kubernetes.io/disk- pressure:NoSchedule node.kubernetes.io/memory- pressure:NoSchedule node.kubernetes.io/not- ready:NoExecute node.kubernetes.io/unreachable:NoExecut eEvents: Type Reason Age From Message ---- ------ -- -- ---- ------- Normal SuccessfulMountVolume 1h kubelet, openshift- ied.install.etux MountVolume.SetUp succeeded for volume "etcd-host- cert" Normal SuccessfulMountVolume 1h kubelet, openshift-ied.install.etux MountVolume.SetUp succeeded for volume "data-dir" Normal SuccessfulMountVolume 1h kubelet, openshift-ied.install.etux MountVolume.SetUp succeeded for volume "service-catalog-apiserver-token-ptkqz" Normal SuccessfulMountVolume 1h kubelet, openshift- ied.install.etux MountVolume.SetUp succeeded for volume "apiserver- ssl" Normal Pulled 1h kubelet, openshift-ied.install.etux Container image "docker.io/openshift/origin-service-catalog:v3.9.0" already present on machine Normal Created 1h kubelet, openshift-ied.install.etux Created container Normal Started 1h kubelet, openshift- ied.install.etux Started container Warning DNSConfigForming 4m (x62 over 1h) kubelet, openshift-ied.install.etux Search Line limits were exceeded, some search paths have been omitted, the applied search line is: kube-service-catalog.svc.cluster.local svc.cluster.local cluster.local install.etux bofh.etux dmz.etux[root@openshift-ied ~]# oc describe apiserverthe server doesn't have a resource type "apiserver" Regards,Carlos Rodrigues On Fri, 2018-06-15 at 07:58 -0600, Rich Megginson wrote:
Engenheiro de Software Sénior Eurotux Informática, S.A. | www.eurotux.com (t) +351 253 680 300 (m) +351 911 926 110
I run the following command and send logs in attachment oc logs apiserver-h6rg4 > logs regards, Carlos Rodrigues On Fri, 2018-06-15 at 08:41 -0600, Rich Megginson wrote:
-- Carlos Rodrigues Engenheiro de Software Sénior Eurotux Informática, S.A. | www.eurotux.com (t) +351 253 680 300 (m) +351 911 926 110
Looks like apiserver has some problem connecting to the etcd service. Could be a host networking or firewall issue. Please review the network, firewall, etc. configuration at https://docs.openshift.org/3.9/install_config/install/prerequisites.html#pre... On 06/15/2018 08:51 AM, Carlos Rodrigues wrote:
It includes the metrics for the vms using the collectd virt plugin. CPU, Memory, disk IO etc. Best regards, -- SHIRLY RADCO BI SeNIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> On Thu, Jun 14, 2018 at 12:53 PM, maoz zadok <maozza@gmail.com> wrote:
I did set it up yesterday and I intent to use it. Setup worked ok on Centos 7.5. Only problem was with depency of ansible for centos-openshift-origin39-candidate, which I had to deactivate from repos. It did work though and I am getting reports now. Anyway is there a way to "relax" the amount of data collected, say make collection every 20 seconds instead of 10 seconds (I think that is the default.) Is there a way to get keep data for specific amount of time say 1 year and then discard the older data and vacuum the database ? Thank you and redhat for all your efforts on providing such a fine product. Regards George Vasilopoulos
Hi, I'm so happy you managed to get it working for you. I'm aware of the ansible version issue and the Openshift team is working to fix it. Currently ansible 2.5.5 and 2.5.6 seem to be working ok. Yes. You can add to the /etc/ovirt-engine-metrics/config.yml collectd_interval: 20 (The default is 10 seconds) And rerun the /usr/share/ovirt-engine-metrics/setup/ansible/configure_ovirt_machines_for_metrics.sh This will update the collection interval on all your machines. In addition, If there are metrics you do not wish to collect you can update the collectd config files in /usr/share/ansible/roles/oVirt.metrics/roles/oVirt.ovirt-collectd/ovirt-host/templates/ and /usr/share/ansible/roles/oVirt.metrics/roles/oVirt.ovirt-collectd/ovirt-engine/templates/ and rerun the /usr/share/ovirt-engine-metrics/setup/ansible/configure_ovirt_machines_for_metrics.sh We would like to add aggregations in the future for long term storage. Currently we set the curator to save 3 days of metrics data. since metrics consume lots of disk space. Logs are kept for a month by default. After that the curator deletes the old indexes. You can update the curator but that will affect the disk size you will need. This is the documentation for curator update and general info about curator: https://docs.openshift.com/container-platform/3.9/install_config/aggregate_l... https://docs.openshift.com/container-platform/3.9/install_config/aggregate_l... I hope this will help you. Best regards, -- SHIRLY RADCO BI SENIOR SOFTWARE ENGINEER Red Hat Israel <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> On Tue, Jul 17, 2018 at 9:04 AM, <g.vasilopoulos@uoc.gr> wrote:
participants (7)
-
Carlos Rodrigues -
g.vasilopoulos＠uoc.gr -
maoz zadok -
Marcelo Leandro -
Peter Hudec -
Rich Megginson -
Shirly Radco