--=-2b3Yf0eVkZsnB8S4oFLU Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Itamar, by the proxy of Brian, did asked me to look on the bounce issue we have on the users lists. So after a few hours of careful log reading, here is my finding. The bounce situation --------------------- We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying.=20 Investigation ------------- A first look show that our mails are bounced as they are marked as spam by Google. Google doc on the matter do not give much, some people point to using dkim, spf, etc. But spf is not for us, but for the sender, and dkim is not ml friendly, afaik, and requires upstream support if I understood well. Not all mails are bounced, which is good. That mean the ip is not problematic.=20 So I took a few hours to look on every bounce and roughly, there is 2 groups. Group 1 -------- First group is that all mail from the same poster on the users list have bounced at Google. Out of the 16 mail he sent, 16 have been rejected by Google. I have no idea why, I suspect the spf policy, but it did looked ok. None of the mail of answer had a issue, so that's likely not a content problem. =20 However, the ip address of the sender is in the SORBS blacklist, so that's likely what trigger Google spam filter. Not much we can do, besides contacting him, which I will do. Group 2 -------- Roughly, that's mail in this thread : http://lists.ovirt.org/pipermail/users/2015-January/030494.html and the mails from Sandro : http://lists.ovirt.org/pipermail/users/2015-January/030420.html http://lists.ovirt.org/pipermail/users/2015-January/030423.html Common point, use of goo.gl and ur1.ca. It turn out that both domain are flagged as URI spam, since that's used by spammer to hide their link. So I suspect that Gmail started to "learn" about them as spam, as the rest of the world did : http://multirbl.valli.org/lookup/ur1.ca.html http://multirbl.valli.org/lookup/goo.gl.html Again, not much we can do, besides asking to people to not use these services ( which is not gonna work I think ). Conclusion ----------- If the core issue is "people are kicked out due to bounce", we can look at raising the threshold on mailman ( as proposed by Brian ), while at the same time trying to reduce the number of bounce ( ie, a root cause investigation on each bounce when we see issue ).=20 First part is easy ( I think ), second is not hard but we need to have someone to look at log on a regular basis so that's taking some time.=20 As a side note, our spamassasin setup was blacklisted from the DNS BL we used ( due to our use of the dns of linode.com : http://uribl.com/refused.shtml ), thus reducing his efficiency. I did fixed that by setting a local cache, following the page I gave. If anything weird happen, please tell us :) Anyone has a opinion or a idea ? --=20 Michael Scherer Open Source and Standards, Sysadmin --=-2b3Yf0eVkZsnB8S4oFLU Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUt/l3AAoJEE89Wa+PrSK9Ct0P/0GFRDqGF5GQ2mYDjyThi8vp cpjutYXxQ2yA0b7AulvRXt2C4pE3SjiSlzGs8qKUQ2GcjTvOsSxd0Li0+fLSrVxY zL0q26vaabvRZSXE2tu21Z8nkz4zS1jgIGNU/q/VrW0v/QmYwDtWp9zcWuMjKEYh v/ybWXaBK5btKr1gEVkj+zVngIcKs9c28jgns2RGl6M03C4hBkFvLywp3JqF+GC4 aa0GIWfT5kCa7tp27sk1Yxqu9DA7tN+Ue4FDJIrBZf419cYzrhegutyw0FPZmEzj XDNc1FutSPPngasAipgKXKohAR3hem1s7WXfB/4rx5zwq8hNqDnXH20clPb3ERZi FmDDMtxU4HgrvYsKo86WtCJdPq9hNgIMnbnK6ycSNBqs2BQJeBJIodPCA80y7tUo ml8uTCeTNMkln/dyVJtIxpjsiYbJX0VoO11WclIprL5tCr0BJNLi/qsX1BXtDhcj m/VW6IKIlT8DcTJLI5kN68jp3V6lZWbGdEgsyuugtrKEg+QUFYcV1jOQeoRY50qT JeEA0PgxgdCFchiEysZGBt2nq+7wOJS0BSihd8kTC41P02AUvfDQOkdpYTySQG8c cy5SQbwp+5YrW0CWVoNYTBD32ve1c8WgwWX/HHzyItH/aitUIfPE0RDdSjAnCFUz WahLhOAOS5rG9ItBPjXh =33VF -----END PGP SIGNATURE----- --=-2b3Yf0eVkZsnB8S4oFLU--
Il 15/01/2015 18:31, Michael Scherer ha scritto:
Hi,
Itamar, by the proxy of Brian, did asked me to look on the bounce issue we have on the users lists. So after a few hours of careful log reading, here is my finding.
The bounce situation ---------------------
We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying.
Investigation -------------
A first look show that our mails are bounced as they are marked as spam by Google. Google doc on the matter do not give much, some people point to using dkim, spf, etc. But spf is not for us, but for the sender, and dkim is not ml friendly, afaik, and requires upstream support if I understood well.
Not all mails are bounced, which is good. That mean the ip is not problematic.
So I took a few hours to look on every bounce and roughly, there is 2 groups.
Group 1 --------
First group is that all mail from the same poster on the users list have bounced at Google. Out of the 16 mail he sent, 16 have been rejected by Google. I have no idea why, I suspect the spf policy, but it did looked ok. None of the mail of answer had a issue, so that's likely not a content problem.
However, the ip address of the sender is in the SORBS blacklist, so that's likely what trigger Google spam filter.
Not much we can do, besides contacting him, which I will do.
Group 2 -------- Roughly, that's mail in this thread : http://lists.ovirt.org/pipermail/users/2015-January/030494.html
and the mails from Sandro : http://lists.ovirt.org/pipermail/users/2015-January/030420.html http://lists.ovirt.org/pipermail/users/2015-January/030423.html
Common point, use of goo.gl and ur1.ca. It turn out that both domain are flagged as URI spam, since that's used by spammer to hide their link. So I suspect that Gmail started to "learn" about them as spam, as the rest of the world did : http://multirbl.valli.org/lookup/ur1.ca.html http://multirbl.valli.org/lookup/goo.gl.html
Again, not much we can do, besides asking to people to not use these services ( which is not gonna work I think ).
I may try to use bit.ly red.ht instead of goo.gl. Can we provide our own url shortener on ovirt.org? That should avoid blacklisting.
Conclusion -----------
If the core issue is "people are kicked out due to bounce", we can look at raising the threshold on mailman ( as proposed by Brian ), while at the same time trying to reduce the number of bounce ( ie, a root cause investigation on each bounce when we see issue ).
First part is easy ( I think ), second is not hard but we need to have someone to look at log on a regular basis so that's taking some time.
As a side note, our spamassasin setup was blacklisted from the DNS BL we used ( due to our use of the dns of linode.com : http://uribl.com/refused.shtml ), thus reducing his efficiency. I did fixed that by setting a local cache, following the page I gave. If anything weird happen, please tell us :)
Anyone has a opinion or a idea ?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
--=-8IypELm1jDXZvlCEJt6n Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le vendredi 16 janvier 2015 =C3=A0 08:23 +0100, Sandro Bonazzola a =C3=A9cr= it :
Il 15/01/2015 18:31, Michael Scherer ha scritto:
Hi, =20 Itamar, by the proxy of Brian, did asked me to look on the bounce issue we have on the users lists. So after a few hours of careful log reading= , here is my finding. =20 The bounce situation --------------------- =20 We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying.=20 =20 =20 Investigation ------------- =20 A first look show that our mails are bounced as they are marked as spam by Google. Google doc on the matter do not give much, some people point to using dkim, spf, etc. But spf is not for us, but for the sender, and dkim is not ml friendly, afaik, and requires upstream support if I understood well. =20 Not all mails are bounced, which is good. That mean the ip is not problematic.=20 =20 So I took a few hours to look on every bounce and roughly, there is 2 groups. =20 Group 1 -------- =20 First group is that all mail from the same poster on the users list hav= e bounced at Google. Out of the 16 mail he sent, 16 have been rejected by Google. I have no idea why, I suspect the spf policy, but it did looked ok. None of the mail of answer had a issue, so that's likely not a content problem. =20 =20 However, the ip address of the sender is in the SORBS blacklist, so that's likely what trigger Google spam filter. =20 Not much we can do, besides contacting him, which I will do. =20 Group 2 -------- Roughly, that's mail in this thread : http://lists.ovirt.org/pipermail/users/2015-January/030494.html =20 and the mails from Sandro : http://lists.ovirt.org/pipermail/users/2015-January/030420.html http://lists.ovirt.org/pipermail/users/2015-January/030423.html =20 Common point, use of goo.gl and ur1.ca. It turn out that both domain ar= e flagged as URI spam, since that's used by spammer to hide their link. S= o I suspect that Gmail started to "learn" about them as spam, as the rest of the world did : http://multirbl.valli.org/lookup/ur1.ca.html http://multirbl.valli.org/lookup/goo.gl.html =20 Again, not much we can do, besides asking to people to not use these services ( which is not gonna work I think ). =20 I may try to use bit.ly red.ht instead of goo.gl. Can we provide our own url shortener on ovirt.org? That should avoid blac= klisting.
I do not think bit.ly is gonna change much. It is likely abused for the same reason by the same people. And the url are too complicated to be sent sometime, so we cannot just avoid them at all. I also pondered about adding a url shortener on ovirt.org. Besides the load on admin team it create, I think it would have the same issue as the others after some time, and so we would need to add some authentication, which start to make thing a bit complicated.=20 --=20 Michael Scherer Open Source and Standards, Sysadmin --=-8IypELm1jDXZvlCEJt6n Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUuMlRAAoJEE89Wa+PrSK9HMoQAJfAN37HJRErlaxD8F9yHiot F9XxSNxhVE9cDFLNN9XBoD6XIAnenLeHDzwwo1DzqWkLnW8f7kgcmyv/YCeqaJVK Xf5uESJGi8bYsiviaS7RJy1FyIZhoCc0UmvAl0v6iZyiqMfqkT6mJMQDK7Awt9cM jr3lu/l6ivjlfFD+ujhtFQzuqRVA0nNAlskMbdPhO98kLGTK5a1bI+RODTmtxRsW XazD0bA9/O/7geEL1nMOIQDWmA6Kgx6bJrvwFZqeCg3d/FCUz5uJM5XeksGP/1Aw f7pLgFniAKOLSM2wvvdjIG8rI03AsOfQ8zFx0RUa4RBceCbZnhL45ZSZZez6yJ+7 bxk3ADSvbNfrKh4s5Ty8xYEnVjB6fCNeB/UIhbE3KAqb9CARiB9VW4i81X41fG8b jQA3PbrolT2UftPCMETeMfQEVOLogIrFypxMF0uvGN6U4wiWML3WqLXr8sBoBQb5 ArHvJQqHsvLGRdskI9S/mzy5v1II2Zm1sbzqeRT/1nvVXY35+GpmdldLok8VwJQI 9PEyVAVXN7tR27U3nhvTXeoJwy0KvrkGaeRFFPQw/WorNYgWV1ghSu6Bu0rFTAwY tnVonP6SSz07myTb2z4evyoZYQ02wYVQAqjO2MvWYAwNKcaJYdtnBbVpGQE3kAXe 9pTVAihjdbyTC0yX7WM8 =5Wri -----END PGP SIGNATURE----- --=-8IypELm1jDXZvlCEJt6n--
Il 16/01/2015 09:18, Michael Scherer ha scritto:
Le vendredi 16 janvier 2015 à 08:23 +0100, Sandro Bonazzola a écrit :
Il 15/01/2015 18:31, Michael Scherer ha scritto:
Hi,
Itamar, by the proxy of Brian, did asked me to look on the bounce issue we have on the users lists. So after a few hours of careful log reading, here is my finding.
The bounce situation ---------------------
We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying.
Investigation -------------
A first look show that our mails are bounced as they are marked as spam by Google. Google doc on the matter do not give much, some people point to using dkim, spf, etc. But spf is not for us, but for the sender, and dkim is not ml friendly, afaik, and requires upstream support if I understood well.
Not all mails are bounced, which is good. That mean the ip is not problematic.
So I took a few hours to look on every bounce and roughly, there is 2 groups.
Group 1 --------
First group is that all mail from the same poster on the users list have bounced at Google. Out of the 16 mail he sent, 16 have been rejected by Google. I have no idea why, I suspect the spf policy, but it did looked ok. None of the mail of answer had a issue, so that's likely not a content problem.
However, the ip address of the sender is in the SORBS blacklist, so that's likely what trigger Google spam filter.
Not much we can do, besides contacting him, which I will do.
Group 2 -------- Roughly, that's mail in this thread : http://lists.ovirt.org/pipermail/users/2015-January/030494.html
and the mails from Sandro : http://lists.ovirt.org/pipermail/users/2015-January/030420.html http://lists.ovirt.org/pipermail/users/2015-January/030423.html
Common point, use of goo.gl and ur1.ca. It turn out that both domain are flagged as URI spam, since that's used by spammer to hide their link. So I suspect that Gmail started to "learn" about them as spam, as the rest of the world did : http://multirbl.valli.org/lookup/ur1.ca.html http://multirbl.valli.org/lookup/goo.gl.html
Again, not much we can do, besides asking to people to not use these services ( which is not gonna work I think ).
I may try to use bit.ly red.ht instead of goo.gl. Can we provide our own url shortener on ovirt.org? That should avoid blacklisting.
I do not think bit.ly is gonna change much. It is likely abused for the same reason by the same people. And the url are too complicated to be sent sometime, so we cannot just avoid them at all.
I also pondered about adding a url shortener on ovirt.org. Besides the load on admin team it create, I think it would have the same issue as the others after some time, and so we would need to add some authentication, which start to make thing a bit complicated.
Not sure if authentication will work as expected, but this one seems quite simple to configure and deploy: https://github.com/mrtazz/katana
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
--=-VS26s3x5hcIfdPLzbvFe Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le vendredi 16 janvier 2015 =C3=A0 09:31 +0100, Sandro Bonazzola a =C3=A9cr= it :
Il 16/01/2015 09:18, Michael Scherer ha scritto:
Il 15/01/2015 18:31, Michael Scherer ha scritto:
Hi,
Itamar, by the proxy of Brian, did asked me to look on the bounce iss= ue we have on the users lists. So after a few hours of careful log readi= ng, here is my finding.
The bounce situation ---------------------
We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying.=20
Investigation -------------
A first look show that our mails are bounced as they are marked as sp= am by Google. Google doc on the matter do not give much, some people poi= nt to using dkim, spf, etc. But spf is not for us, but for the sender, a= nd dkim is not ml friendly, afaik, and requires upstream support if I understood well.
Not all mails are bounced, which is good. That mean the ip is not problematic.=20
So I took a few hours to look on every bounce and roughly, there is 2 groups.
Group 1 --------
First group is that all mail from the same poster on the users list h= ave bounced at Google. Out of the 16 mail he sent, 16 have been rejected = by Google. I have no idea why, I suspect the spf policy, but it did look= ed ok. None of the mail of answer had a issue, so that's likely not a content problem. =20
However, the ip address of the sender is in the SORBS blacklist, so that's likely what trigger Google spam filter.
Not much we can do, besides contacting him, which I will do.
Group 2 -------- Roughly, that's mail in this thread : http://lists.ovirt.org/pipermail/users/2015-January/030494.html
and the mails from Sandro : http://lists.ovirt.org/pipermail/users/2015-January/030420.html http://lists.ovirt.org/pipermail/users/2015-January/030423.html
Common point, use of goo.gl and ur1.ca. It turn out that both domain = are flagged as URI spam, since that's used by spammer to hide their link.= So I suspect that Gmail started to "learn" about them as spam, as the re= st of the world did : http://multirbl.valli.org/lookup/ur1.ca.html http://multirbl.valli.org/lookup/goo.gl.html
Again, not much we can do, besides asking to people to not use these services ( which is not gonna work I think ).
I may try to use bit.ly red.ht instead of goo.gl. Can we provide our own url shortener on ovirt.org? That should avoid b= lacklisting. =20 I do not think bit.ly is gonna change much. It is likely abused for the same reason by the same people. And the url are too complicated to be sent sometime, so we cannot just avoid them at all. =20 I also pondered about adding a url shortener on ovirt.org. Besides the load on admin team it create, I think it would have the same issue as
Le vendredi 16 janvier 2015 =C3=A0 08:23 +0100, Sandro Bonazzola a =C3= =A9crit : the others after some time, and so we would need to add some authentication, which start to make thing a bit complicated.=20 =20 Not sure if authentication will work as expected, but this one seems quit= e simple to configure and deploy: https://github.com/mrtazz/katana
Not that keen on adding it on the infra, but could work for openshift maybe ? --=20 Michael Scherer Open Source and Standards, Sysadmin --=-VS26s3x5hcIfdPLzbvFe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUuNAWAAoJEE89Wa+PrSK97AIP/iFyauFXbQYI+uNaF8p4tw3A 4FY/2vc0Lp9CtscFIVK3fvYTWPb1WCIQAMEtnrKSPewkZ56HZBDn5seAOvcoD+kH u3zBCfNboR/yU5SIT284czYPbfjizm0UgIgIoROjkWHfPR5r0paZBHiJ6d3gd7k/ B5YhpgtHy5DMoKBuXDHVgjMIPXNiQavfLRvaLGWERa0eHK0BtclBTqD2OanMJEIh Cfulu2oUJfyWRiOXu90ItZBc+7zQ96LfbRS/y4Is526T4beV71Xs8A5PT9gSxsBE zv7wQJzdliq62+4xNYkcDKghcf7Wu114icyVp69jRnzQhbNFJu7QzO/nj3umdgaU 88ArrL+ON02Lcqn9rxFuJ+UB8nX2c/suXg7lIYIta89IKFXFEqIFXFczK38xz5ME 6G6NEZzV94t8t2BKLdZemT/kH2r+t+ITR5meb6pCYDyE/JMIAzXCKrhvl7izwxp+ pN1axehtr2I6r7TBew5NaNts1F+H9Re6CCS2dHULKlsizb9sELJk7yLi4+ovd4XD kqqtMYQwgLj1v3as1vh1jsMiq+0t5H7/VGFwqP42MDwGl2/brBqAcqnd93YcNUpk +p+xjCTolYCq6W9nXbV3QIsCb1s+8+JynepR4sAm9lhf0C7kei7SzNqME64A/pKj EOakMnpZXWwv8+9bQvvf =ijrJ -----END PGP SIGNATURE----- --=-VS26s3x5hcIfdPLzbvFe--
--=-+PTVzeueIRGGv1DX9fza Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le jeudi 15 janvier 2015 =C3=A0 18:31 +0100, Michael Scherer a =C3=A9crit :
Hi, =20 Itamar, by the proxy of Brian, did asked me to look on the bounce issue we have on the users lists. So after a few hours of careful log reading, here is my finding. =20 The bounce situation --------------------- =20 We (ml admin) get on a regular basis people who get unsubscribed and message about bounce. People being unsubscribed automatically is bad(tm), and bounces are annoying.=20 =20 =20 Investigation ------------- =20
=20 Conclusion ----------- =20 If the core issue is "people are kicked out due to bounce", we can look at raising the threshold on mailman ( as proposed by Brian ), while at the same time trying to reduce the number of bounce ( ie, a root cause investigation on each bounce when we see issue ).=20
So I changed config of users@ for bounce : bounce_info_stale_after =3D> lowered to 3 bounce_score_threshold =3D> raised to 30 I do not think that's gonna fix much, but at least, it should reduce the number of time people get disabled ( ie, people would need to bounce for a full month to be disabled I think ) --=20 Michael Scherer Open Source and Standards, Sysadmin --=-+PTVzeueIRGGv1DX9fza Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUyMVvAAoJEE89Wa+PrSK9pagP/2ZOE2LiB1ZkYrmc86Fmc8u5 Inz6HgWkWfz0uZUPZZoCSqwvsuqpvGr55RVGcysp6aa543lhSIbS6TbVJ/Q6suRX XD14VyKzXCUNmntwPbYp9riaYZfbaGpNVr65KGohH9mTvXNSpoKIBrdoZGLOONJY 8aLIMe4Qpb7M19Cy8IEt2i55N2wYzMFOMyrJzFEqd1446c9hLFTnDzK19PHlm6gZ xV3ao8+aXxQo1qsnpEmUf2W8CUdNT+aCSaArqUjW5YZcEsfFg8oQbK1YApZq5LVE GKyffo9drYcbaU24KtYedgSPeoQH2zEzxpJBPBR6xOttI7cvsqLXfBOa//BkSoTk pAH8NhRvhqT1ZR8z4YztoHrIeAh9Wc3Qssy3COJevsywSYp5DKLsxHqLrVyfVtut fkFPajQyI+nUhveHCTaTh9KJnJyn+bUHvUcOLdi35ONy/2yHohcIogpuUbppNbBN bc8bgRqfgkpJiHW/XWCQTFLjf1t104HEKhmdQ5xqGVi8QeRwG8olrZz8vVa32uz2 HNIZyR8skSLzC4Px1aZI1ixxvgdNM+BI4ey1CfmnZQWM/BsMCLrMXZ469NzMmxrx g6ChOyCyfy+8dcJ5xXZBEoxm+aa4sXL4IUo4dbbynajdnqaANWkPlMYoRK4o6vPp FVsKTZVkQIUu8u8N9zES =eccg -----END PGP SIGNATURE----- --=-+PTVzeueIRGGv1DX9fza--
participants (2)
-
Michael Scherer -
Sandro Bonazzola