This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --JmDNeoq8Ebv1nqkSd8BNHALVJ2SuOPi93 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable El 21/02/14 00:37, Dan Kenigsberg escribi=F3:

On Thu, Feb 20, 2014 at 09:52:23PM +0100, David Caro wrote:

...

Hi everyone!

Lately I've had a hard time to properly review some patches containing= shell scripts to manage our infrastructure because there's no guidelines. So= I created a wiki page with a proposal [1]. It's made up as a mix of some already= existing guidelines.

The reason to wrote a bash style guide and not a shell stile guide is = because I think that bash is widely adopted (default GNU shell) and provides eno= ugh advantages to sacrifice some portability. I think that most of our mai= ntenance and management scripts will never be run on non-GNU OSes. =20 What are the advantages in your opinion? I like the $() construct and local variables. Associative arrays can come up handy.

=20

...

POSIX compliance should be only used when really needed, for example, =

* Less prone to errors: - [[ ]] builtin constructs make it really easy to avoid having errors w= hen spaces are in the tested variable: myfile=3D"/thi/is my/file" [[ -f $myfile ]] <- this does not break and works as expected [ -f $myfile ] <- this breaks - Long options are really nice and make code clearer - Arrays are useful - Associative arrays are useful too - String expansions with brackets mv myfile{,.bkp} - Pattern substitution: ${myvar/mypatt/mysub} - Substring expansion: ${myvar:offset:length} - Indirect variable expansion (don't use it if you don't need it, like = eval): ${!myvar} - $PIPESTATUS array, with the return codes of the commands executed in = a pipe - options in echo (echo -e, echo -n, ...) - local variables definition, there's no local in POSIX: local a b c <- this is not POSIX - source command (a lot clearer than '.', harder to confuse with execut= ion) - select command to create menus - the time builtin (not a builtin in POSIX) - Process substitution redirection: foo <( bar ) - '=3D=3D' equality test instead of '=3D' (A lot more readable and less= confusing) - '<>' lexicographical comparison - Check if 2 files are the same hard link: [[ $file1 -ef $file2 ]] - Redirecting stdin and stdout: &>, >& and &| - duplication and closure of file descriptions: m>&n- - Herestrings ('<<<') - A lot of the shopt options - FUNCNAME to get current function name - REGEXP matching test (=3D~) Those are a few, there are more differences but some of them I don't reco= mmend (that's why we are creating a style guide, to use only the ones that real= ly help) scripts to

...

build a specific project, that might be run on non-GNU based systems i= n the far future. =20 I have the opposite sentiment - diversions from the standard should be kept small and well-justified.

I heard this before from Alon, in my opinion, both of you have the wrong = idea of what a standard is and when to apply it. POSIX standard is a standard created to allow portability between operati= ng systems, that's it's goal. The infrastructure scripts will most likely ne= ver be run outside the GNU family of operating systems (fedora/centos/ubuntu/debian/suse/gentoo ...) so the portability is not something that matters. Thus, the application of the POSIX standard is too restrictive. Imagine i= t like building your house in Norway with the same standards that are used in To= kyo, you don't need protection from eathquakes, you need heat efficiency. The same way, we don't need portability, we need flexibility and error av= oidance. As I said also, at the end of the script and at the end of the wiki, if y= ou are gonna distribute a script with the product, then POSIX standard is the on= e that should be applied.

=20

...

This thread is to start a discussion about it so please, share your op=

inions and

...

concerns (and proposals).

[1] http://www.ovirt.org/Bash_style_guide =20 Could you detail where [1] is different from the http://wiki.bash-hackers.org/scripting/style that it cites?

Some of the differences are: - Indentation (I said 4 spaces and that the consistency in using tabs o= r spaces throughout the script is more important than using spaces) - I let case constructs to fit in one line if they are short (one comma= nd) - Added the 'local' usage on the function variables - Recommended to use function parameters instead of global variables - Recommended to specify a return statement explicitly - Using builtins instead of external commands when possible - Added portability note Most of those are from here, a german linux user group: http://lug.fh-swf.de/vim/vim-bash/StyleGuideShell.en.pdf And I want to finish looking that one up and incorporate the best advice = from there too (the testing section is nice, and the comments)

=20 BTW, it's missing the most important rule, that can turn the shell into=

a programming language: the -e option. Not using it is basically equivalent to wrapping every python line within its own try-except-pass=

block.

-e option is useful yes, but not always so helpful, agree that in the bes= t case you should use it. But sometimes is not so simple and people end up writi= ng '|| :' at the end of each command. I planned on adding more about bash options to the wiki, and also some debugging, but did not have the time. Point taken.

When I educate people I do this to allow people to contribute to any pr= oject. In our case this will enable people to contribute to infra and product.=

Having your own conventions to infra which are not following product st=

andard

will make it difficult to reuse people skills.

Let's write everything in javascript so we can all contribute to engine a= nd is the only language that can fit everywhere (ui included)... the goal of th= e product is different from the goal of infra, the tools should be the best= suited for the goals of the team, and if we can choose the same tools without lo= osing focus towards our goals then we should. I'm not sure either of the percentage of shell scripts that are there in product, but I'd say that it's not that much, as most of it are make scri= pts and relatives.

POSIX code is not less readable, one just need to accept that we follow=

standards and everything else will be aligned.

I don't agree, POSIX standard makes code a lot less readable if not using= 'extensions': ssh -tNL 1234:127.0.0.1:4321=09 ### Imaginary non-POSIX compliant ssh that uses GNU extensions ssh --force-tty --no-command --local-tunnel 1234:127.0.0.1:4321 [[ "${myvar: -1}" =3D=3D "/" ]] ## the variable ends with '/' [ "$(printf "$myvar" | tail -c 1)" =3D "/" ] ## remember that echo -n = is not posix compliant do_something &>something.log do_something 1>something.log 2>&1 Or any case where you have to go to sed/awk/python/ruby to get rid of the= POSIX restrictions and use their own programming languages. Having to understan= d 2 languages to read a script is less readable than having to understand 1 i= mo. Having to expose all the variables as global is also less readable and a = lot more prone to errors too. do_whatever() { for i in one two; do echo "inside i =3D $i" done } for i in three four; do do_whatever echo "outside i =3D $i" done The output of that is: inside i =3D one inside i =3D two outside i =3D two inside i =3D one inside i =3D two outside i =3D two The non-POSIX equivalent using local does what you expected: do_whatever() { local i for i in one two; do echo "inside i =3D $i" done } for i in three four; do do_whatever echo "outside i =3D $i" done inside i =3D one inside i =3D two outside i =3D three inside i =3D one inside i =3D two outside i =3D four The fact that you expect the code to do something and it does something e= lse goes inside non readable to me (because it's not easy to understand what = it does reading it). --=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --JmDNeoq8Ebv1nqkSd8BNHALVJ2SuOPi93 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTBx7qAAoJEEBxx+HSYmnDqosIAJdf+y8kyrWisQVO0BoZRizA aC3uAhFQ0j+XOHQ0MMmZ7jZG5uVEKf8A3OcqVsNJilNvwIYz7JIeyO2AiQ4w2PU0 8kavaZo2g4sHOTF7ctd20MS+vyGDd0nO8yXdaGCMakrjc5A+I8YMgrRePAJ4zoI1 9+rr3tcvhypRgK71eiRdXGko9ryAQqLA8rj3Nc4jg+AU17fL9YeVoeTe8gBMvfC6 pyoT9rHg0/Vqqud4ChTeMtxV9w21ivIliPuzg6kHPae32C2WxWelkStKu6436L6j pvwG3sY27hkvn+hewh2edFiJITFMZOZiOAl+zR0M16tn+559uy9/dvJGx5MhUjw= =TyBP -----END PGP SIGNATURE----- --JmDNeoq8Ebv1nqkSd8BNHALVJ2SuOPi93--

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7LlLSFtSeQ5ghwb0MBCWli17vr8PoOO5x Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable El 21/02/14 00:40, Alon Bar-Lev escribi=C3=B3:

=20 =20 ----- Original Message -----

...

From: "David Caro" <dcaroest@redhat.com> To: "infra" <infra@ovirt.org> Sent: Thursday, February 20, 2014 10:52:23 PM Subject: Scripting guidelines

Hi everyone!

Lately I've had a hard time to properly review some patches containing= shell scripts to manage our infrastructure because there's no guidelines. So= I created a wiki page with a proposal [1]. It's made up as a mix of some already=

...

existing guidelines.

The reason to wrote a bash style guide and not a shell stile guide is = because I think that bash is widely adopted (default GNU shell) and provides eno= ugh advantages to sacrifice some portability. I think that most of our maintenance and management scripts will never be run on non-GNU OSes.

POSIX compliance should be only used when really needed, for example, = scripts to build a specific project, that might be run on non-GNU based systems i= n the far future.

This thread is to start a discussion about it so please, share your op= inions and concerns (and proposals). =20 We already discussed that, I think it is wrong for trivial scripts to u= se bash. No need to discuss that over and over. =20 The problem is that there is nobody to have authority to decide anythin= g. Open Source is not anarchy nor democracy, there should be strict hierar= chy. And we lack that, so anarchy is in action.

=46rom the company you work for, and a pretty old and active participatio= n on open source projects, Dave (cc'd) seems to disagree with your view of open sou= rce management: https://opensource.com/business/11/2/leadership-open-source-communities """ So how are open source communities led? Largely by the people doing the w= ork. Most groups have a loosely defined common goal (build software widgets, o= r develop a awesome, open source, computer-based fourth grade math curricul= um), and decisions are made by the people doing the work. There's no manager i= n place dictating edicts about how things must be done or what objectives to seek= after. Many people object to this method, call it anarchy, and claim that it imp= edes progress. It's true that if the same set of people was coerced into a sin= gle direction, they might make more progress, but there likely wouldn't be th= e same level of innovation. """

=20 As for infra, it is not part of anything we distribute so it is not tha= t important, however, standards compliance is something that should be co= nsidered. =20

...

[1] http://www.ovirt.org/Bash_style_guide

Cheers!

-- David Caro

Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D

Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605

_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra

--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --7LlLSFtSeQ5ghwb0MBCWli17vr8PoOO5x Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTByQEAAoJEEBxx+HSYmnDXU4H/3uUzQGCuOtVfz5S08wnZ1RO ix3lihwXwONZ6bXoYYdlh/sZotYF9oMmNf8db8HPAu8b8MPSmW5LfdAvR6XAiUXQ AEBiLMifkzboJ3TjjQ7x7xQ21Epr3GljsbppvKnmrWRWAIVpAsU8xWoRQMZBtpvB /QDZrsXFnUdXoP3N4O3tJqWK/K/mDgOcHMETjypTGGuEuG1+mPVTUrTJBrggaLLc WNmk6XmMO4hmeCvln8uJGTOCbXM7w+26ZmepI3XhKJbXM7hlqo8WyCfaep5pyZOU T5pkrxwqipXNNdJ/MXWeM+8JX+xvGK9AVvEMMgVy3uoYucQuYD2Qe02L82K8NQ8= =V1Ig -----END PGP SIGNATURE----- --7LlLSFtSeQ5ghwb0MBCWli17vr8PoOO5x--

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'll reply to Allon's email separately, but... On 02/21/2014 11:01 AM, David Caro wrote:

From the company you work for, and a pretty old and active participation on open source projects, Dave (cc'd) seems to disagree with your view of open source management:

https://opensource.com/business/11/2/leadership-open-source-communities

Leadership

in open source communities Posted 8 Feb 2011 by David Nalley (so, not me) :-)

""" So how are open source communities led? Largely by the people doing the work. Most groups have a loosely defined common goal (build software widgets, or develop a awesome, open source, computer-based fourth grade math curriculum), and decisions are made by the people doing the work. There's no manager in place dictating edicts about how things must be done or what objectives to seek after. Many people object to this method, call it anarchy, and claim that it impedes progress. It's true that if the same set of people was coerced into a single direction, they might make more progress, but there likely wouldn't be the same level of innovation. """

I do agree with this. However: people will look to a leader to figure out what should be done, in what priority, and who has the authority to give access to things required to get stuff done. In the absence of an existing hierarchy, you quickly end up in gridlock. The loosely defined common goal is important. Let's take patches as an example. Anyone can write code to implement a feature, fix a bug, whatever - but someone needs to merge the patches, bundle up a release, exercise some level of discretion, and give some guidance as to whether contributed patches will be accepted or not by setting a direction. I don't think that a strict hierarchy is needed. But seeding the group with the people doing the work is important. And that's what we've done, I think - clearly, David, Eyal, Ewoud, Rydekull, Kiril, quaid and myself have the knowledge and do the work (with different levels of skills and knowledge for different pieces of the infrastructure), and get to say who gets access to resources and whether contributions come up to our standards or not. Cheers, Dave.

...

As for infra, it is not part of anything we distribute so it is not that important, however, standards compliance is something that should be considered.

...

[1] http://www.ovirt.org/Bash_style_guide

Cheers!

-- David Caro

Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D

Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605

_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra

- -- Dave Neary - Community Action and Impact Open Source and Standards, Red Hat - http://community.redhat.com Ph: +33 9 50 71 55 62 / Cell: +33 6 77 01 92 13 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTB1HYAAoJECd1qeknDCgg9mQH/Rue01YltsL676/DcJJLhosv YotA4xY7MDtwf8zmpF0xZP30Jj6HH7a9WFE7VxU72iMcavaQ/6SGoMp0erTreWxM ovAHiaFgnkn8/EtnA8yoUr3yGU+hATyUCIsSOeOr3mOmXC8+ehgR/Esibk+DKPQ0 qUOgW61QArR0RVNN3KVgakhsZ2hSm+YrIJCi6FZfL6Li8aWzZVYkkfWgI/I52zd8 C3XTJ0y93UgZKAroPDUiKi+tP3zAxldqYbPgq1B7hfxnPxxeFSibmj/fi+U1wkLU FrIle3qOhFqCl9s/X0TtG/FBvwuhjWkIUaaD9DWfSFp/aoPt6LcdLo2/Mvp6xZ8= =2HB5 -----END PGP SIGNATURE-----