BASH scripting chapter one

 

From Wikipedia

What is bash?

Bash is short for the ‘Bourne Again Shell’. It’s probably GNU Unix’s most popular shell. A shell is a program to run other programs. In addition, the BASH shell supports a scripting language, that lets a programmer automate repetitive shell tasks. Instead of having to type the same thing into the shell over and over again, you write a BASH script that will do those things for you. If you want some more background on BASH, read https://en.wikipedia.org/wiki/Bash_(Unix_shell).

Or according to Wikipedia, “a shell lets a user direct the operation of the computer by entering commands as text for a command line interpreter to execute, or by creating text scripts of one or more such commands”. https://en.wikipedia.org/wiki/Unix_shell
If you are familiar with opening a terminal, and typing a few commands like ‘ls’ or ‘pwd’, then you’ve executed a command in a shell. (And it was probably in a BASH shell.) Learning to script with BASH will let you tie these actions together, which is especially helpful for automating repetitive tasks, or for remixing existing programs on your computer to make your own special home-brew programmatic sauce.

How to install bash?

BASH is normally already installed on Linux or Mac’s. If you open your shell/terminal, type:

echo $0

This should print something like ‘bin/bash’. You might be using a different shell. zsh or something else.

If you are on Windows, then BASH is not pre-installed. There’s a few ways to get it. One way is to install the new Linux subsystem. Or you can install Cygwin or MingW. Or, my favourite, install Git for Windows and then select the ‘install additional unix tools to the command line’ during the installation process. https://git-for-windows.github.io/

If you are on Windows, then I recommend using PowerShell. To access BASH, after you’ve installed it, you can type bash or sh.

How to mark something as a bash script.

A BASH script is a text file. Open up your text editor and type in the following:


#!/bin/bash
echo "hello scripting world."

The very first line needs to be #!/bin/bash. In unix, this marks that the file is executed with the BASH interpreter. It’s called a ‘shebang’. This helps other programs know what to execute the file with. For example, if the file is a python file, then you would instead supply the path to the python interpreter in the shebang. E.g. your text file would start with:


#!/user/bin/python3
print("hello python world.")

How to run a bash script.

Save it, call it ‘hello.sh’. Open the terminal in the directory the script is in. Next, we need to make sure that we have privilege to execute the script.

On Windows you’ll need to make sure that you can execute scripts. You do this in PowerShell by setting the execution policy. You will need to start PowerShell in administrative mode initially. Then type:

Set-ExecutionPolicy Unrestricted

This tells Windows that you want to run scripts. Before Windows would only allow the execution of scripts with verified by a HASH a known developer. Now, it will run stuff without those HASHES. Close the PowerShell in administrative mode. You don’t need administrative mode any more. And then open a new PowerShell (not in admin mode) and check the execution policy. Type:


Get-ExecutionPolicy

It should echo back “Unrestricted”. If you want more information about execution policies, see: https://technet.microsoft.com/en-us/library/ee176961.aspx.

If you are on Linux, you don’t need to set your execution policy. But, you do need to make sure that the new file, “hello.sh”, is executable. In the terminal, type chmod 755 hello.sh. This will mean that you can execute the script.

Then, to run the script with BASH type:


sh hello.sh

Or, you might just get away with typing:


./hello.sh

It should print to the terminal:


hello scripting world

An example of a slightly more complex SHELL script “MOVE” command wrapper

#!/bin/bash
echo "MOVE WRAPPER SHELL SCRIPT"

if [ "$#" -lt "2" ]; then
echo "script name: $0"
echo "requires 2 args. arg1 = file to move. arg2 = move to"
exit 1

else
echo "moving '$1'"
echo "to '$2'"

mv "$1" "$2"

if [ "$?" -eq 0 ]; then
echo "SUCCESS"
else
echo "ERROR"
fi

fi

Where to find BASH documentation.

Being so popular, there are a lot of resources available. But, the definitive text, as far as I know, is the GNU bash documentation. View it here: https://www.gnu.org/software/bash/manual/bash.html.

Brian FOX, a software developer, wrote BASH for GNU. It is the original implementation, and presumably there aren’t any other implementations worth caring about.

I have found the GNU documentation for BASH very good, and detailed. Check it out.

I also recommend a UDEMY course I used to learn BASH: https://www.udemy.com/bash-scripting/
These are essentially my enhanced notes. Notes I made as I learnt BASH over the course of a few weekend nights. ENJOY 🙂

BASH variables

One of the most important things in any programming language is storing values. If you can’t store a value, then how can your program remember anything?

WARNING THINGS I FIND WEIRD ABOUT BASH

I’m used to a rich scripting language like PYTHON. Something you can develop complex programs with.
* BASH is not this.
* It doesn’t seem to have the idea of a class.
* You cannot import other BASH scripts. So, each BASH script is independent.
* You can’t return values from functions except for an integer between 0 and 255.
* BASH is very focused on exit codes. It is geared to automate the use of other programs. You don’t try create a GUI with BASH.
* So how could one BASH script pass a value to another BASH script? Ahh, ends up that there are globals every where by default. This means, KEEP YOUR SCRIPT SHORT. Anything longer than 100 lines, move into PYTHON/PERL.
* BASH wraps other programs, like MOVE (mv) or LIST (ls). BASH can access the standard input, standard error. This way it can work with variables from other scripts and programs. (I think.)
* The errors you get back are more cryptic that Python. For instance, if you mess up the syntax, the error’s aren’t that great for a beginner. No traceback, for instance. (Think of opposite of the uselessly verbose Java traceback that is printed. BASH is the opposite. It’s terse.)

syntax-error.sh

#!/bin/bash
???? opps

BASH shell output

$sh syntax-error.sh
syntax-error.sh: line 2: ????: command not found

BASH just went ‘????’ is potentially a command. There is no in front of $????. So it’s not a variable I hold. Then it attempts to find ‘????’ as something it can execute on the environment path.

That’s not a terrible error message. But, the syntax for the unfamiliar around the if statement.

syntax-error.sh

#!/bin/bash
if [ 0 -eq 0 ]
echo "zero equals zero"
else
echo "zero != zero"
fi

BASH shell output

$sh syntax-error.sh
syntax-error.sh: line 4: syntax error near unexpected token "else"
syntax-error.sh: line 4: "else"

I don’t find ‘syntax error’ near unexpected token ‘else’ very helpful. It’s pretty descriptive of what the compiler did though.

The compiler read the first three lines fine. Then it hit line four. In the abstract syntax tree, it new that it was on leaf if. it expects a then before an if. Then is an unneeded indicator. The last ] bracket should have been fine except that isn’t as efficient as using then. That’s why BASH uses ‘then’.

A better error would read:

BASH shell output

syntax-error.sh: syntax error, if [] opened on line 2 not closed with 'then' by line 4.

END OF FIRST SECTION

2-Factor Security!

Two-Factor authentication. Do you use it? A lot of IT professionals don’t use either:

  1. A password manage, nor
  2. 2-factor authentication.

It’s surprising to me because 2-factor authentication is becoming more and more widely implemented for web services. Banks in the UK consistently use multi-factor authentication. Most consumers will have already experienced 2-factor (or multi-factor) authentication. Facebook, Google, LinkedIn, Twitter, all offer 2-factor processes.

Multi-factor authentication is just a fancy word for a login process that asks you for more than a password alone.

Something in your head, and something in your hand
Multi-factor merely means more than a password. Sources: http://tinyurl.com/zn5l3ej http://tinyurl.com/gphmv4k

It is widely recognised that a password is pretty weak . We all abuse passwords, and it is pretty easy to capture another person’s password. For years and years, security researchers have been trying to strengthen online verification processes. Multi-factor is something that has emerged as a practical measure to improve your online resiliency. Emphasis on the word **practical** .

Multi-factor (2-factor, 3-factor, 4-factor…n-factor) authentication provides you with an extra layer of protection on top of password protection that puts multiple layers of access control around your account.

For instance, with 2-factor access enabled on Twitter, when you sign in from a new-device, Twitter asks for:

  1. Your password, and
  2. The code (6 digits?) they text to your phone.

An attacker needs to have your password and your phone to get into your Twitter account. It isn’t impossible to get both, but as you add more layers (multi-factor == n-factor authentication), then it becomes progressively more difficult for you to loose control over your account.

The time investment to get set-up with usable 2-factor authentication isn’t much. Just poke around the Account or Security section of a website, and you’ll often find that you can enable 2-factor.

OK…so Donal, it seems like an O.K. idea, but I’m lazy…

Sure, me too.

Practical tips

  1. Strengthen your passwords by using a password manager. You want a manager that is available on all your devices. I recommend Lastpass.com.
  2. Focus on enabling 2-factor authentication on websites that are important. Social media platforms, email accounts, password managers. (Physical devices, like your laptop, can also have more than a password.)
  3.  Use Authy. This little device can store all those separate 2-factor tokens into one place. It’s like a password manager, except it is for 2-factor authentication.
authy.com logo
www.authy.com

 

Give me a little more…

Infographic on the four common types of authentication. Something you have, something you know, and biometrics

The weird world of computers and law

Surely computer programming and law are totally different, right?

I studied Law for undergraduate, but I learned to program and I recently perked onto the growing field of legal informatics. A lot of people think it is strange that I studied both Law and Computers. But, it is a growing area of interest, and it is has been for a long time. At least for 300 years or more, according to Stephen Wolfram. And, as far as I can tell, Law and Programming are very similar. They both use a system of logical rules to define a problem space. But whilst computer programs are normally written to produce very exact answers, legal problems often do not yet have a certain answer.

Is it coming?
The end of lawyers?

Describe the legal informatics paradox

Shortly put, legal informatics combines computer science and legal theory. It has been around for a long time. And there is a trend, though I don’t know how popular, to call Law ‘legal science’. But, legal systems are very different from scientific systems. Legal thought and scientific thought are quite separate. If you go to Law School, and you’d learn about qualitative principles like ‘justice’ or ‘fairness’ or ‘certainty’ that animate the Law (if you are lucky). Lawyers tend to use these hard-to-define, abstract concepts, to build their argument, and then they try to prove a series of value judgments about when one principle, such as fairness, might be sacrificed for another principle, such as certainty.

If you know how computer programs work, then there isn’t a clear way they can apply to problems within the legal domain. Computers are really good at things like counting, 1+2=3. These problems are so certain to define and easy to represent in a mechanical sense. Which we can then solve with an algorithm.  Most computer languages would be totally useless for dealing with legal problems. 

Really quick example of how all legal thought operates

Just a quick, stupid and inaccurate example to make my meaning clearer. If you are a local council (LC) and you buy thousands of faulty wheelie bins from Useless Bin Maker Ltd (UBM), and the manufacturing error only emerges eight years later, the Law provides for a general rule requiring the Council to seek damages within 6 years of the date the bins were delivered. It would benefit the LC if there was no limitation period at all. (Their lawyer might argue this is ‘fairer’.) But, this would disadvantage UBM. (Their lawyer might argue this promotes ‘certainty’.) As it stands, because this is a latent defective, LC can make a claim within 3 years of discovering the manufacturing fault, up to a maximum of 15 years thanks to the Latent Damage Act 1986 that modifies the standard position under the Limitation Act 1980. Here is a more detailed article explaining the limitation rules for contracts in England.

Legal thought is all about modifying and augmenting the rules of the system in response to new facts that call for a slightly different compromise between the motivating principles. It is a ‘normative’ exercise. Normative, according to Wikipedia, means “relating to an ideal standard or model, or being based on the normal or correct way of doing something“.

Future posts

In some future posts, I’ll begin to talk about how people are trying to solve this legal informatics paradox, as Computer Scientists and programmers try to marry Law and Computation.

If your password security sucks, here is what to do about it….

password-joke-photo
Please share your clever work-arounds in the comment section.

Passwords are a total pain. Yep. Everyone, largely, hates having to remember them. And they have a nasty habit of catching you out when you don’t expect it. Have you not logged on to that dodgy, forgotten laptop in the bottom-draw of your office in the past 6 months? Well, if you have/haven’t, hopefully, you can remember the password. But, wait, if you go to actually try it and it rejects you, well, what was that password? Ermm, yeah. You’re quite, quietly, ignominiously stuck. It might take a little bit longer to remember it. How about trying to type every password you’ve ever used? ….Nope….Oh dear.

Sometimes, don’t you hope for a better alternative? I can’t give you a solution to that dusty-old laptop-problem. But, if you are someone who is fed up trying to remember the password for this-or-that website, (admit it, you’ve only used most websites you’ve put in a password for, once or twice, so you just have a password you use for all of those little sites) then I’d highly recommend getting a Password Manager. Yep, a PASSWORD MANAGER. GET ONE.

(It is actually cheap-to-free, and a good one is pretty easy to work with.)

Personally, I recommend Lastpass. It is super handy. It is, genuinely, a piece of software worth paying for. It is a piece of software that will make you — once you get the hang of it — never want to go back to the old method of relying upon your failing squidgy bits. Brains just aren’t meant for remembering passwords in a secure way. (Numerous studies and practical reality show.) And, it is hard to replace passwords with anything else. AND, EVERYTHING USES PASSWORDS, basically.

I also recommend two-factor authentication. But, for some people, and in some situations, it’s annoying. Don’t worry about that yet. A Password Manager, for those without either, is a more important bit of kit.

I’ll write another post later, giving a bit more detail, about how to use a Password Manager, and perhaps I’ll compare some of the current offerings. But, this is just a quick tip. If you are having trouble with passwords — or you suspect that ‘EverybodyLovesRaymond’ isn’t a secure password because you’ve used it for the last 7 years on every sad-scrap-of-a-site you’ve visited — then something like Lastpass is well-worth your time. Check it out. Honestly. Life-saver.