The most universal scripting language for Linux is?

There are two programming environments that are available on every unix-like operating system, that are Turing-complete and that are able to call other programs: awk, and sh, the Bourne/POSIX shell family. AWK is oriented towards text processing (it complements more specialized utilities), while sh is oriented towards being a glue language to put programs together. Sh is the universal scripting language on Linux and across the unix world.

The POSIX standard defines mandatory features of sh itself and associated utilities. Most unix-like systems comply with POSIX 1003.1-2004 (a.k.a. Single Unix v3, a.k.a. the Open Group Base Specification issue 6); the latest version of that standard is POSIX 1003.1-2008 (a.k.a. Single Unix v4, a.k.a. the Open Group Base Specification Issue 7).

Every Linux and unix or Unix-like system has a Bourne-style shell at the path /bin/sh, and any non-antique system has a POSIX-compliant shell (barring the occasional bug). Every modern unix-like system (including Linux) supports shebangs, so it automatically runs scripts in /bin/sh if the first line is #!/bin/sh. There are POSIX systems where sh is located in another place (typically emulation layers on OSes you wouldn't think of as being really Unix-like).

Embedded Linux systems may have a stripped-down BusyBox system that doesn't implement all POSIX features. BusyBox has a large number of compile-time options to accommodate small-footprint systems, so it's difficult to know what to expect in advance, you have to tailor your scripts to a particular device. BusyBox is the most common small-footprint implementation of sh and assorted utilities; another one you might encounter is the extremely reduced shell environment in Android (later versions are less anemic).

Non-embedded Linux systems almost always have either dash or bash as /bin/sh. Dash is a small and fast shell that implements little more than POSIX features. Bash is a larger shell with more features.

Non-embedded Linux systems almost always have Bash installed as /bin/bash. Hence, for portability on non-embedded Linux systems, you can assume that bash is available. Among the useful additional features of bash are arrays, the ability to cope with dot files conveniently, the pipestatus variable to obtain the return status of all the commands in a pipeline, additional comparison operators for file times, and (in recent versions) regular expression matching.

One of the characteristics of shell programming is that you aren't just using the sh program, you're also using a number of utilities. Most of the file manipulation and text processing utilities on Linux are the GNU coreutils (on embedded systems, they're usually from BusyBox).

If you need portability beyond Linux, your best bet is to stick to POSIX. Other unix variants may not have bash installed (bash is part of the standard installation on OSX, but an optional package on *BSD and most commercial unices). Almost all unix variants other than Linux and OSX (i.e. *BSD and commercial unices) have some version of the Korn shell, at least pdksh. Many of bash's convenient extensions are from ksh, so it can be useful to write scripts that can run under both, but detecting where bash or ksh is located on an unknown system can be a bit of a pain.

The shell can't do everything. If you need a more sophisticated language, the two more common choices are Perl and Python (anything else is far behind as a unix scripting language). Perl is the traditional scripting language, and few non-embedded Linux systems lack it, but Python is gaining ground (boosted in part by being the recommended scripting language for Ubuntu). In the non-Linux world, Perl is part of the base installation on OSX and OpenBSD; it's optional but very commonly installed on FreeBSD, and optional but often installed on NetBSD.

In availability order:

1. sh, but stick to POSIX-specified facilities.
2. bash, but don't forget to explicitly specify it in the shebang or you might get dash instead.
3. Python. Almost everyone uses it.
4. Perl. But you get to write it.

After that, no one really cares since there isn't much you can't do with just those.

Ordinarily, I'd say sh....but since you specified Linux, I'll say bash - it's guaranteed to be on every Linux system around (well, excluding the obscure pedantically-tiny ones that fetishize minimalism :).

If you don't have to care about non-linux portability (and if you don't need to run on tiny distros or in embedded Linux devices like plastic-box routers), then you may as well make use of the significant enhancements that it offers over plain sh. Otherwise use sh.

After bash (and sh), the next most "universal" scripting language for Linux would be some dialect of awk - usually either mawk or gawk. If you stick with plain awk and avoid gawkisms your script should run fine on almost any Linux system (it may be missing on tiny distros or embedded devices). Most Linux systems will have both mawk and gawk available, but on some distros (debian, for example) mawk is installed by default and you have to install gawk yourself if you want it.

Next would be perl. AFAIK, the base perl language is installed by default on all common Linux distros, so that makes it a good choice. Even more fortunately, there is very little version incompatibility with perl5 releases (although perl 5.12 or it might have been 5.14 finally got around to removing some obscure features that had been deprecated for about 15 years...ample warning not to use them) so unless your coding style is truly weird and you like to ignore over a decade's worth of "don't do that" warnings, your perl scripts will run just fine nearly anywhere. The language is robust and powerful and can do everything that awk and sed can do and more. With a little effort it can do the things that sh is traditionally good for too (e.g. running external commands and using/piping the output). The standard perl libraries are pretty comprehensive, too - covering more than just the basics.

The only catch with perl is that there is also an enormous library of CPAN modules to do just about anything you can think of (and lots more that may never occur to you) - and not all of them will be available on every system with perl. They're usually of very high quality so it's easy to get into the habit of just using them - but if you do make use of them, you'll have to make sure they're installed. Many CPAN modules are pre-packaged for Linux, and the remainder are easily installed with the cpan tool (or dh-make-perl on debian/ubuntu/etc to turn a CPAN module into a .deb package)

I'd like to be able to say python next, but I really can't. There's a lot to like about python but it's not included by default on many Linux systems and, frankly, it's version compatibility (both for python itself and its allegedly "standard" libs) is a complete shambles. Some distros make an excellent effort at sorting out the mess, and some don't. Neither are helped by the fact that python is basically a language written for programmers by programmers (as opposed to sysadmins) and they don't seem to think that the system their code will be installed on is at all important....their code is really super special so they don't need to concern themselves with boring stuff like integration into existing systems.

(Don't get the wrong idea from my sarcasm here - I really like python as a language, I just hate the fact that version and dependency management is a PITA. It's like going back 20+ years to the era of manually hunting for obscure bits of code and patches to get something compiled and running on your proprietary *nix)

I'd suggest follow ksh93 or even POSIX flavour and you can always use bash/zsh to run.

And Debian-based distro use mawk not gawk as default awk. So yes avoid gawk additions since mawk is way faster.

Not bash. Write to a close-to-POSIX sh like dash or ash. That's going to be most universal. I don't think there's anything else that's even a close competitor. (And this seems to me a factual question, not an "opinion" as one of the commentors complains.)

If you need something a bit more powerful than sh (for example, if you want real associative arrays), use awk. (Avoiding the gawk extensions. There are many versions of awk, but there's a largely shared core.) That should be available almost as broadly as sh is.

I would say the availability would rank somewhere in this order:

1. sh
2. awk
3. Perl (I've yet to see a *nix without it, including BSDs)

Although Python sounds like a good language, I've not used any OS that came with it in a base installation, though apparently Ubuntu does, maybe?