I highly recommend ShellCheck[0] if you're writing any bash. With the warnings and stylistic advice it provides, I feel like I can actually be confident that my scripts are doing what I think they're doing.
However, don't let it give you a false sense of security.
Shellcheck probably won't say anything about the "biting" parts from the article –
because they are valid, just behaving a bit different than the user expects…
That goes for any linter or static analyser i guess.
I recently started having to do a lot of shell work, and shellcheck has helped immensely. It was unfortunate that there was no precompiled version of GHC for my OSX version on homebrew though, so it took 5 hours to build GHC. I ended up turning off all the power save and auto sleep features on my laptop to make sure it kept going.
Thanks! That's a great tip. I've been writing a large amount of shell scripts (mainly sh not so much bash) and already made some improvements thanks to this!
This doesn't mention one of my favorite bash gotchas, which is using `set -e` with `pipefail` at all. Try this:
set -euo pipefail
yes | head
That will consistently exit because `yes` gets sigpipe and quits. Which is expected, but triggers a script exit. But more exciting is that something like:
generate_data | head
only _sometimes_ fail. It's a race that depends on whether generate_data is able to stuff all of its data into the pipe buffer before head calls close().
As an aside, I like reading Jane Street's blog. They're one of the few companies in our space (I'm also in "automated trading") that discusses even non-proprietary stuff openly. When you work in an information vacuum, it's comforting to know that presumably similar people face the same challenges.
When the author wrote "a particular production bash script (if that doesn't sound horrifying, hopefully it will by the end of this post)," I couldn't help but smile...
It's a great blog, they usually strike a great balance on the practical/theoretical scale for the articles. Would love for them to up the frequency of the posts, maybe they will once they find the writer they're looking for.
Do you happen to have some recommendations for similar blogs?
The error handling is one reason I prefer Tcl to Bash or POSIX sh if I need a nontrivial script that shells out to other programs. Tcl handles errors, substitution, etc. properly (unsurprisingly) by default:
> set foo [exec /bin/true][exec /bin/false][exec /bin/true]
child process exited abnormally
while executing
"exec /bin/false"
> exec true | false | true
child process exited abnormally
while executing
"exec true | false | true"
> set sp "hello world"; exec echo $sp; # No need to quote $sp there.
hello world
Not all shell-like things are as convenient to do in Tcl as they are in sh (the most significant difference to me is that you cannot pipe to or from functions), and it is more verbose, but because everything is a string in Tcl I find that it integrates with *nix (or Windows!) command line programs better than other scripting languages. E.g.,
What happens here is that you take the output of `ps | tail -n +2`, split it on newlines, then map over it treating each line (that is something like "5814 pts/0 00:00:00 ps") as a list and taking the first element in it. The result is a list of PIDs (a string containing the PIDs separated by whitespace).
I can recommend trying Tcl to anyone fighting Bash who doesn't want to replace it with Python/Ruby/etc. If you try it, though, use version 8.6 or at the very least 8.5. The previous versions are EOL but are still common in the wild. If a recent Tcl is not available on your system, you can build a self-contained static binary interpreter with http://kitcreator.rkeene.org/kitcreator.
Not in this case, at least :-), since on that wiki page there is one implementation of command pipes that I wrote and a link to another (in fptools).
What I mean when I say that "you cannot pipe to or from functions" is that while you can pipe data from one external process to another in an [exec] command, you can't pipe it from an external process to a proc or vice versa, which would be analogous to what the POSIX shell can do. Functions and external processes are less similar in Tcl than they are in sh.
The best you can do with a simple command pipe implementation like those on the wiki page is pipe data from [exec foo] to [bar $arg] to [exec baz << $arg], which, unlike [exec foo | baz], will read foo's entire output before passing it along. The only real solution that I know to allow you to treat snippets of Tcl code and external processes as basically interchangeable in a gradually read pipeline is rkeene's pipethread library. I am a big fan of it, but using an external dependency that isn't in your operating system's package repositories adds considerable friction to writing and distributing a simple script. (I do hope pipethread ends up in Tcllib.)
The main benefit for shell scripting is the ability to pass data seamlessly between your script and any programs that take and output text. No serialization/parsing is required.
What I consider to be the biggest benefit of Tcl in general compared to the more mainstream scripting languages is the pervasive immutability. In Tcl values are strings and strings are immutable. It encourages you to write programs that consists of a reusable, easy-to-test collection of pure functions and a task-specific procedural or (more rarely) OO core.
Not that people should be expected to know this, but here's an idiom that does work:
if res="$(ldap-query-for-valid-users)"; then
echo "($res)" > "/tmp/all-users.sexp"
else
handle_failure
fi
I second the recommendation to use https://github.com/koalaman/shellcheck – you really shouldn't be writing shell scripts without it – but in this case it doesn't seem to handle the issue (with default settings at least).
Slightly more useful than `set -e` is `set -E` and `trap exit ERR`.
But still there's basically no way to make this consistently useful.
Even if you religiously set -e/E in every scope just in case, if you're anywhere in a scope inside the non-final operand of a bunch of &&/||s, or as the conditional expression in a control structure or whatever, -e/E will just do nothing, you can't turn it any more on, you just don't get early termination on errors no matter how many nested function calls you're actually removed from the original ||. It's not great.
I was just thinking how cool would it be to have a module in python to really ease generating bash style scripts, with less overhead than normal `subprocess` methods have... And there was it just a Google search away: https://pypi.python.org/pypi/sh/1.12.13
Author of sh.py here. It seems to be used by many people in production around the world. I've actively maintained it since 2011, and it supports python 2.6-3.6, inclusive. Most of the gotchas have been worked out by now, but the FAQ covers the most common stumbling blocks: http://amoffat.github.io/sh/sections/faq.html
For context, since I think some people don't know this and are down voting you, this blog post is from Jane Street (probably the largest commercial OCaml deployment?). As another poster noted Haskell has many tools/libraries for writing scripts which makes dtoma's question very reasonable.
We use node at work now. The fact is it's not worth it trying to get everyone to write good bash scripts. Js doesn't have nearly as many foot guns and there's a library for just about anything you could want to do.
True for async code, but if you use something like ShellJS [1] (which would be an easy way to rewrite a bash script that was getting out of hand), all commands are synchronous, so error handling is just try/catch.
If only everyone used promises...I can dream, right?
Regardless, I'd actually prefer Haskell's do-notation for both asynchronous commands and shell-scripting tasks. Even though Haskell isn't my favorite language, do-notation is quite nice for this.
It's happened! We have some internal magic that lets us run ml directly: ./my_bash_script_replacement.ml, and it makes replacements fairly painless.
But we don't do it often. Each language has its strengths and weaknesses, and the constraints of a small script tend not to change that much over time. The script either starts other executables and pipes some text around, or it does some computation. The choice of which to use is often (but not always) clear.
I'm not sure I agree with the "don't do it often". Having jane-script (which is our OCaml scripting system) has allowed us to greatly reduce our dependence on Bash.
That said, there are still little things we use Bash for. But our tolerance for large bash scripts has diminished greatly over the years.
They do not seems to warrant further investigation, the examples you find about these implementations are overly simplified bordering useless. If you are going that way you might as well go with something like PowerShell, and that's a horrible scriping language.
Well, shell scripts are extremely convenient. It's easy to write very simple ones since they're a natural extension of shell usage.
So it's really easy to go from
$ command1
$ command2
to:
#!/bin/bash
command1
command2
As a result, everybody "likes" bash scripts since they're so easy to write, initially.
And these shell scripts will work well in 80% of situations and you can get them to 90-95% just by sprinkling a few ifs around.
Once thes script becomes longer and more complex and you want to make them easier to read and DRYer, that's when the pain begins. Or when you need to handle a bit of logic which would be trivial to do with better data structures such as arrays or dictionaries or sets...
You can work around these issues, of course, but almost every workaround is either ugly or a hack (as someone was joking, "elegant hacks" for older Unix hackers, "gross hacks" for younger ones :) ).
I agree - in my experience bash is extraordinarily easy to write the first time. It becomes a nightmare to maintain, because most people aren't actually that familiar with how bash error handling, flow logic etc. actually works. It's bad because it fails in surprising, non-conventional ways.
I just over-log absolutely everything. If it fails log it. If it succeeds and you might need to know about it, log it. That's not great error handling, but it helps with debugging.
/tmp generally has more open permissions than the rest of the file system, and if you're using a known file name, you're open to that file being abused by an attacker.
Yes, subshells can be tricky. Don't rely on stuff like pipefail, rather check return codes and react accordingly. Bash also provides a way to read return codes of piped commands. See PIPESTATUS in man bash.
A related annoyance is that when you write a for-loop in a Makefile, every iteration gets run even if one of them fails, due to how the shell calculates exit codes of for-loops.
So most of the time you should do "set -e; for XXX". Otherwise your Makefile loops will "succeed" incorrectly.
Really... Check your damn return values. set -e is a crutch of a sloppy programmer.
Ok, yes; you can do this really slick thing in one line by stringing together a bunch of commands. However, just because you can does not mean you should.
Bash makes it simply with 'if ! <command>; then <failure commands> fi'. Try not to string ten things together. Keep conditional true state in your scripts execution flow.
This is a bad argument -- our tools shouldn't make the "easy path" a path that's littered with subtle bugs! It should be easy to do the right thing. See: PHP hand-coded apps that led to SQL injection often versus Python's DBAPI which makes it harder to make mistakes, Signal versus some PGP GUI, etc. People make mistakes when their tools make it easy for them to do the wrong thing.
I have the same complaint against much of the standard C library too, but for a language that the user interacts with on a daily basis, this is unacceptable.
If you are doing anything with complex behavior that requires error checking, interpolating strings with possibly invalid values, or just writing a lot of code, then Python is already more convenient and if you didn't notice it is because you are hiding a lot of Bash problems under a carpet.
That's my point. It's not easy and often impossible to write robust shell scripts.
Part of it is the problem domain. Rewriting any slightly complex bash script in e.g. Python with robust error handling can be quite challenging. You have to make a number of very difficult decisions.
And when multiple bash scripts do that, it is easy to have them accidentally overwrite each other's temp files. Plus an attacker can play with temp files in a number of interesting ways to turn their use into an attack.
I was just illustrating the error checking using commands functionally equivalent to the original post. Temporary files are best avoided entirely unless some command actually requires a seekable file, in which case use mktemp(1) or equivalent (and 'trap 0' to clean up after yourself).
then why not use python? Installing another shell just for running another script seems wrong unless it is only for you personally and you are ok with this.
I don't think the problem is (ba)sh the language, but rather the idioms it implements.
It's usually a bad idea to begin writing the result before knowing that all input is there. That's like a server announcing a 200 OK and beginning to stream, only later detecting I/O error. It's difficult to deal with such a server as a client.
More generally, in pipelines we deal with pairs of programs that are only connected by a text stream, with no possibility to communicate out-of-band conditions. In `PRODUCER | CONSUMER`, PRODUCER can't tell from a sigpipe whether CONSUMER crashed or has read all the data it needs. And CONSUMER can't know whether PRODUCER crashed or if it should take action on the results.
Most scripts don't really need the take-action-immediately level of concurrency. It's sometimes nice to be concurrent (use multiple CPUs at once). But the cases where the processed data can't be buffered at least in a temporary file before taking action are really rare.
When I see very long bash scripts I get angry. There is a reason why perl & later python were invented and they are both in 98% of cases available on Unix compatible systems.
Not that I'm not ok with short bash scripts or for systems were you only have bash... but if your Unix compatible system only has bash there is something else wrong with your system in my point of view.
I'd much rather have 200 lines of bash than the 300-400 lines of python to replace it.
I have recently rewritten a lot of Python and Perl (over 10kLOC) to a small collectiom of 10-ish bash scripts that weigh in at about 700LOC total. There are some tasks where Bash is the only appropriate tool.
Much simpler code, much easier to read, and a lot faster too.
If you're measuring code quality / maintainability by how many lines of code there are, you're doing it wrong. I'd much rather see 400 lines of well-structured python than 200 lines of tangled bash. For the rewrite you did, how much of the benefit was because it was rewritten, rather than being rewritten _in bash_?
I'd also go as far as saying that if you're using bash because it's faster than python, you're also using the wrong tool for the job.
The quality of the bash far outshined the quality of the python and perl, because Python and Perl was being used to do a bash scripts job. When what you need is to call a bunch of binaries and redirect output around, then Python and Perl just don't cut it.
The code is much, much easier to read now. You can see a script on a single screen height, which works wonders for overview. The 3x performance gain spawned from enlightenment this overview granted us.
But don't get me wrong, in the very same rewrite, I rewrote 100 lines of bash + 300 lines of Python into a single 200 lines of Python, which when combined with PyPy, was 50 times faster than the original setup. Python doesn't do bash's job very well, and bash doesn't do Python's job very well either.
I agree that for the simple job of "start process, maybe echo some stuff", then bash is superior to python. However, my difficulty always arises the moment you want to start writing 'if', or handle some sort of error condition.
Once you go down that road, you're into magic switches at the top of your script (which TFA demonstrates aren't well understood), and subtle control issues where bash's flow control doesn't quite do what you think it should, and (for me) this is where bash stops being superior.
I don't think flow control is a problem, as long as it's just to complete a simple purpose. Checking if a file is present, running some code for every file or every line, etc.
But bash is definitely not something where complicated logic belongs.
Although it's been a long time since I last did perl programming, it has always been my understanding that perl is particularly well designed for piping and mangling input and output from binaries. It's been the successor of awk, sed, grep and cut, all contained in one, relatively consistent and well documented language. Are you saying Larry Wall failed in this design attempt? Could it be that Perl wasn't used to its fullest potential?
Not the best way to do that sort of thing (useless use of cat for instance), I was just trying to come up with an executable example. run returns success or failure of the pipeline as the return code, which I don't show here. I don't think you can pick out which command failed.
I also find that once you're in a real programming language, you have much less need to have deep pipelines anyhow. Consider the pure-perl equivalent
open PW, "<", "/etc/passwd" or die;
while (<PW>) {
next if $_ =~ m|:/bin/bash$|;
$count++;
}
print "$count\n";
Of course "real perl" would suggest some form of "use strict" and the corresponding need to declare the $count var, but if we're discussing shell script contexts it is at least fair to consider ignoring that, since it would be a silly objection that the perl equivalent doesn't turn on checks that shell scripting doesn't even have.
But if you're in a real programming language you don't need to shell out to wc just to get a line count, or write complicated awk scripts, etc. It's much more common for me in those cases to execute a single program and control its STDIN/STDOUT/STDERR and watch its exit code.
I am very much against ever shelling out from an application. Either write a shell script, or a proper application. Nothing bugs me more than shell scripts written in Python, Perl, or even gasp C++ (he was later fired).
However, most bash scripts I interact with will take at least 2-4 times the amount of code to replicate as a proper application. The line count example is a good pro-shell one, with 6 not particularly nice lines of code replacing a single, simple line:
egrep -v ':/bin/bash$' /etc/passwd | wc -l
"Complicated" AWK scripts are also much more compact than their equivalent. Say, `xyz | awk '/thing/ { getline; print $0; }'` to get the line after a pattern. I wouldn't write an AWK script much more complicated than that, though, and for me, Perl and AWK belong in the same bucket.
My objection to shell is that it obtains that line advantage by cheating. It has terribly poor error handling, it uses an incredibly sloppy serialization format that frequently breaks at the scales people attempt to apply it at, its performance characteristics are terrible, its debugging story is nonexistant, perhaps worst of all it affords a programming style that blinds people to how much compromise they are making with their data because of the aforementioned serialization format, and on it goes. Of course programming languages can't get as concise as shell; even the sloppiest programming language communities there are have the bar for quality set substantially above what shell can meet.
Your example isn't even equivalent to my rather sloppy Perl, since, for instance, it will do something quite different if /etc/passwd doesn't exist. Shell only has the size advantage as long as everything goes perfectly correctly.
Shell is only good for two cases: 1. You don't much care about the integrity of your input or output, and you don't much care about what happens if something goes wrong. 2. When you don't care about the aforementioned issues because you're right there on the spot and can fix things if they do go wrong, because you're using shell interactively.
Having fiddled around with designing a "new shell" every so often my current conclusion is that there is no way to bridge the gap between the optimal interactive use shell and the optimal shell script with one language, and modern shell scripting, for all the features they have apparently for shell scripts, are firmly for the interactive case.
My example is still very well defined, and very easy to reason about. It will output '0' on stdout if no file is found, and an error message on stderr.
Bash is very good for the things its good at, but you still need to know what you're doing to not fuck it up. Some people don't know how to consider error handling when they code, but that's the fault of the programmer.
I used to believe that. I don't any longer. Your second paragraph basically agrees with me that bash affords bad code that ignores errors. I think it's past time for the programming world to take that seriously. We've had 60-some years of programmers insisting they can heroically just remember all the things they need to do to write correct code. And they are observably wrong. To disagree with me on that is basically to assert that software is nearly universally of a high quality and generally robust to all reasonably forseeable error conditions.
Shell goes even beyond that, though; as evidenced by articles like this you can't even get good agreement on how you should write safe shell. When even experts can't agree on what's safe, that's just something unsuitable to any serious task.
None of my previous nor future paragraphs agree with you. You must have misread.
You are presenting a false analogy. I was not, am not and will never argue that one needs to be an oracle that foresees all errors (which would be absurd), but I am arguing that poor error handling is a programmer error due to the situation not being Bash specific. To reword my statement from the previous comment: If your error handling in Bash is insufficient, your error handling would also be insufficient in most other languages with implicit error handling (e.g., those with unchecked exceptions).
Once again, your Perl blob is a great example. You manually added the optional `or die` to deal with the case where the file wasn't present - perl's equivalent of `|| exit 1`. If you had not remembered to add that check, then your program would fail silently, with $count being an undefined variable. You could try to enforce better checking in the local block scope with some hacky header at the beginning of the script ("use strict;"), but that wouldn't stop you from having poor code in a module, and it would only ensure that count was defined, not that the program was well-behaved. That sounds awfully sloppy, doesn't it? Almost like what you were complaining about for Bash!
Very few languages provide you with any assistance to remind your of error checking (failing either silently or by crashing), mostly due to the awful concept of exceptions (unchecked, specifically, but checked exceptions are only useful if unchecked don't exist). Go, Rust, and - ironically - C helps you by forcing you to think about error handling at every single call-site, with lazy error handling sticking out like a sore thumb (explicitly ignored return values, which look weird in these languages), while C++, Java, Perl, Python and Bash all expect you to decide what you feel like handling today.
Bash lets you glue best-of-breed tools together, as long as they all talk on stdin / stdout / files. The tools themselves can be written in the best language for their task.
Python and perl are not always the best option. For example, if I am executing a lot of system commands (sometimes through ssh), checking outputs with grep/awk, reading files, calling make, why use a language that is going to be far more verbose, difficult to modify, and show problems because it's not executing the shell in the correct profile or things like that? For interacting with the system, there's nothing better than shell scripts. You mainly only have to take care with the quotes to avoid word separation and globbing, and use the *nix commands to avoid recreating the wheel.
I'll second this. My Python is far better than my bash but there's no point writing more complicated Python if you're working with system commands. That said, quite often I'll break non-system stuff out into python scripts and wrap them in bash if I need to do a bit of both.
People always recommend Perl over a shell script but I've never seen a situation where it makes sense. Having written very little Perl, it seems like its only strength is that it's present everywhere and has a large library.
Language-wise, Python is far superior and has a large ecosystem and these days it has ubiquity too.
I'd be interested in the argument for Perl over shell scripts that does not apply to just writing the program in a modern language.
the problem with Python where I work is that developers are openly hostile toward it. We're predominantly .NET with just a couple of products running on Linux servers. I'm not a .net guy despite the pressure, and have been teaching myself python in my own time, but the the rest of the staff including management are never going to look for Python as a skill when hiring. EVER. But of course 2.7 is sitting there on the CentOS servers never used, while bash scripted cron jobs run amok and frequently fail (because they were written by the wrong people imo).
I would love to see Python utilized, and maybe with VSCode and MS integrating open source into the .NET world it will eventually come to enterprise (where I work).
it's not about what MS put in VS and VSCode, it's about changing the minds of .NET developers who used are to C# syntax/grammar and hate Python because it doesn't fit their world-view. "Already in VS" is meaningless in a .NET enterprise company.
I feel sorry for you. Fanatism towards any language/eco-system, no matter if .net, Java, Ruby, Python (also have seen the same with Python developers) is something I really don't like.
Have you ever done string manipulation (also RegEx) over many files in Perl? I think it's in this regard superior than anything else, even python when looking at syntax.
I have not. I usually use `sed` in such instances but admitting the superiority of Perl in such instances surely this is insufficient on its own to use Perl? Seems like a rare use-case and if that script grows large then Python has superior language features but if the script stays small, then a Bash script with `sed` will do just as well.
I don't know how sed and perl compare, but I know these things:
Use a C local (e.g. LANG=C or LC_ALL=C) for dramatic speed up of tools like grep/sort and probably also sed. If you don't do that, usually the UTF-8 parser kicks in, which is a lot slower.
Perl is actually well known to have a non-optimal regex implementation: https://swtch.com/~rsc/regexp/regexp1.html That said, I don't know how fast it is (what optimizations it has) at simple string substitutions.
This is the first time I have seen anoybody hold up perl as the superior option. I don't doubt that it will work, but I need the next admin to be able to read my code.
Also, every system may have some Python installed, bud by the time I've dealt with versions (2.6 is the CentOS 6 default, 3.6 is the current version) and dependencies (virtualenv? system wide pip? PYTHONPATH? RPM Python modules?) I'd better get a significant improvement.
Yeah, naively using errexit won't get you the whole way. BashFAQ, ShellCheck and StackOverflow have quite a bit of information, and you can often write things that are pretty robust if you take in all the information. Particular cases that come to mind are:
1) command substitution. you can catch this with an explicit error handler inherited by child processes.
2) pipefail SIGPIPE false positives. pretty hairy, command dependent whether this is a "real" error. often not, so you can work around it by ignoring SIGPIPE
3) process substitution. As far as I know, there is no way to workaround this whilst still using the convenience syntax. You have to carefully use explicit named pipes and carefully use wait on the PIDs (carefully!). Maybe still with races...
In my experience, you can write moderately robust shell scripts if you care enough and use all these flags and linters. But by the time you are at this stage you probably shouldn't be using shell scripting. More like training to spot problems in other people's code.
The use of global values within programs is generally deprecated, for good reasons. The use of global values that modify the semantics of its interpreter probably deserves even more skepticism.
The inheritance of these flags by subshells, which may have been written assuming different semantics, would be potentially even more problematic, so I think bash is right in this case, though arguably the inheritance could be limited to subshell code defined in the same file.
set -euo pipefail
should NEVER be used in a bash script. If you feel tempted to do so, then you are trying to make the shell scripting language be something that it is not.
This is the time for a full scale programming language such as Python or perhaps Groovy on the JVM or Go language. When you need to write robust code, use the tools that were created for writing robust code.
Bash just has too many quirks.
Note that this is related to the most common way that people build a Big Ball of Mud. You have a simple app and you need a couple of features so you add them on. Rinse and repeat. Before too long you have an app that does too much and was never designed/architected to do that much stuff. You are probably ignoring a number of techniques for integrating functionality in large apps such as message queueing, microservices, separate libraries or packages, multiple languages.
Shell scripts suffer the same trajectory towards too much complexity. When you see it happening, and before the task gets too complex, replace the script with an app and apply all the normal software engineering techniques to make it robust.
The good thing about Apache Groovy (on the JVM) is when the big ball of mud starts forming, because its syntax is close enough to Java's, it's easy to quickly split parts off and rewrite it in Java. Only the lambda syntax and some other inconsistencies like the behavior of == are different.
-c Read commands from the command_string operand instead of from the standard input. Special parameter 0 will be set from the command_name operand and the positional parameters ($1, $2, etc.) set from the remaining argument operands.
This is not so unreasonable a question, but the answer is simple: because it's ubiquitous, fairly standardised, and well supported.
Also, other languages suck at doing small things quickly and simply. Try rewriting a shell script in go with little experience and see how long it takes.
They are fast, ubiquitous, 0-dependencies, short, espressive... most of the time. The fact is that we don't have anything better to glue commands together (afaik).
Actually, quite the opposite. All that shell scripts do is glue together other programs they depend on.
That makes them not portable. You can't just move a bash script from Linux to macOS because it uses ancient pre-GPLv3 versions of bash and all the other GNU utils.
"Glueing commands together" is called programming. That's what programming languages are for. Bash etc are just really bad programming languages that lure you into a trap of unnecessary complexity with their superstitial expressiveness for the happy path in short programs, when you don't need anything besides the built-ins. Decades of ubiquitous UNIX crap tricked you into believing that bash's properties are advantages rather that disadvantages. Sure, you can "glue together" a few commands quickly, but only at the price of horrible edge cases and hours of StackOverflowing for doing the simplest of things.
>but only at the price of horrible edge cases and hours of StackOverflowing for doing the simplest of things.
Or seconds, after you learn the search term "Perl one-liner" and add it to your queries for all your edge cases.
Start thinking of perl -we as a Bash built-in function.
Suddenly edge cases go from hours to your typing speed into Google.
Don't forget to leave a comment with a link to where you found the solution, what it does according to that site, and an apology for the unreadable line noise. If you make changes to what you copy and paste, explain your changes in a comment.
(Just in case it sounds sarcastic or ironical, this comment is serious and how I really work. I am really glad for my comments whenever I revisit old scripts.)
One reason is that any System Administrator can be expected to be able to understand and maintain Bash scripts.
I recently built a fairly complex system entirely in Bash, not because it was the "best" language for the job (it would have been much easier to do in Ruby or Python), but because the client didn't have any permanent staff that could maintain Ruby or Python scripts.
As a contractor, one of the major factors in the technical decisions I make is: how supportable is this technology for the client, once I have left? Of course the answer to that question will vary on a case by case basis, but Bash is usually a good lowest common denominator in a Linux environment.
Yeah, in the OP why wouldn't you make the same LDAP query using any of the numerous libraries available for other languages? I'll take the tiniest little bit of complexity of having to make sure Python and PiP are installed on my environment, over a Bash script falling apart at the seams from an edge case.
[0]: https://github.com/koalaman/shellcheck