Linux Shell and Bash

31 min read

Bash Tutorial

A brief overview of Bash, on your way to becoming a Linux expert. When a computer boots up, a kernel (MacOS, Windows, Linux) is started. This kernel provides a shell, or terminal, that allows user to interact with a most basic set of commands. Typically, the casual user will not interact with the shell/terminal as a Desktop User Interface is started by the computer boot up process. To activate a shell directly, users will run a “terminal” through the Desktop. VS Code provides ability to activate "terminal" while in the IDE.

Define variable

The following code cell defines 3 variables and assigns each a value. There are some extra command, called a HERE document, that write these variables to a file. This is so we can use these variables over and over below.

%%script bash

# Dependency Variables, set to match your project directories

cat <<EOF > /tmp/variables.sh
export project_dir=$HOME/vscode
export project=\$project_dir/CSABlog
export project_repo="https://github.com/EdwinKuttappi/CSABlog.git"
EOF

bash: fg: %%script: no such job

Output the value of a variable

The following code cell outputs the value of the variables, using the echo command. For visual understanding in the output, each echo command provide a title before the $variable

%%script bash

# Extract saved variables
source /tmp/variables.sh

# Output shown title and value variables
echo "Project dir: $project_dir"
echo "Project: $project"
echo "Repo: $project_repo"

bash: fg: %%script: no such job


Project dir: /home/edwin/vscode
Project: /home/edwin/vscode/CSABlog
Repo: https://github.com/EdwinKuttappi/CSABlog.git

Project Setup and Analysis with Bash Scripts

The bash scripts that follow automate what was done in the setup procedures. The purpose of this is to show that many of the commands we performed can be added to a script, then performed automatically.

Pull Code

cd ~
cd vscode
cd CSABlog
git pull
git pull --rebase

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Using conditional statement to create a project directory and project"

cd ~    # start in home directory

# Conditional block to make a project directory
if [ ! -d $project_dir ]
then 
    echo "Directory $project_dir does not exists... makinng directory $project_dir"
    mkdir -p $project_dir
fi
echo "Directory $project_dir exists." 

# Conditional block to git clone a project from project_repo
if [ ! -d $project ]
then
    echo "Directory $project does not exists... cloning $project_repo"
    cd $project_dir
    git clone $project_repo
    cd ~
fi
echo "Directory $project exists." 

bash: fg: %%script: no such job


Using conditional statement to create a project directory and project
Directory /home/edwin/vscode exists.
Directory /home/edwin/vscode/CSABlog exists.

Look at files Github project

All computers contain files and directories. The clone brought more files from cloud to your machine. Review the bash shell script, observe the commands that show and interact with files and directories. These were used during setup.

ls # use this to list all files found in current directory
cd # change the directory
pwd 
echo

Git Commands

“ls” lists computer files in Unix and Unix-like operating systems
“cd” offers way to navigate and change working directory
“pwd” print working directory
“echo” used to display line of text/string that are passed as an argument

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
pwd

echo ""
echo "list top level or root of files with project pulled from github"
ls

bash: fg: %%script: no such job
Navigate to project, then navigate to area wwhere files were cloned
/home/edwin/vscode/CSABlog

list top level or root of files with project pulled from github
Gemfile       Makefile     [0m[01;34m_data[0m      [01;34m_notebooks[0m  [01;32mactivate.sh[0m  index.md
Gemfile.lock  README.md    [01;34m_includes[0m  [01;34m_posts[0m      csa.md       indexBlogs.md
LICENSE       _config.yml  [01;34m_layouts[0m   [01;34m_site[0m       [01;34mimages[0m       [01;34mscripts[0m

Look at file list with hidden and long attributes

Most linux commands have options to enhance behavior. The enhanced listing below shows permission bits, owner of file, size and date.

ls reference

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Navigate to project, then navigate to area wwhere files were cloned"
cd $project
pwd

echo ""
echo "list all files in long format"
ls -al   # all files -a (hidden) in -l long listing

bash: fg: %%script: no such job
Navigate to project, then navigate to area wwhere files were cloned
/home/edwin/vscode/CSABlog

list all files in long format
total 100
drwxr-xr-x 12 edwin edwin 4096 Aug 24 07:58 [0m[01;34m.[0m
drwxr-xr-x 14 edwin edwin 4096 Aug 21 11:44 [01;34m..[0m
drwxr-xr-x  8 edwin edwin 4096 Aug 24 09:42 [01;34m.git[0m
drwxr-xr-x  3 edwin edwin 4096 Aug 18 07:35 [01;34m.github[0m
-rw-r--r--  1 edwin edwin  104 Aug 18 07:35 .gitignore
-rw-r--r--  1 edwin edwin  122 Aug 18 07:35 Gemfile
-rw-r--r--  1 edwin edwin 7188 Aug 18 09:04 Gemfile.lock
-rw-r--r--  1 edwin edwin 1081 Aug 18 07:35 LICENSE
-rw-r--r--  1 edwin edwin 3116 Aug 21 11:12 Makefile
-rw-r--r--  1 edwin edwin 5798 Aug 18 07:35 README.md
-rw-r--r--  1 edwin edwin  449 Aug 21 11:11 _config.yml
drwxr-xr-x  2 edwin edwin 4096 Aug 18 07:35 [01;34m_data[0m
drwxr-xr-x  2 edwin edwin 4096 Aug 18 07:35 [01;34m_includes[0m
drwxr-xr-x  2 edwin edwin 4096 Aug 24 08:01 [01;34m_layouts[0m
drwxr-xr-x  2 edwin edwin 4096 Aug 24 09:41 [01;34m_notebooks[0m
drwxr-xr-x  3 edwin edwin 4096 Aug 24 07:58 [01;34m_posts[0m
drwxr-xr-x  8 edwin edwin 4096 Aug 24 09:05 [01;34m_site[0m
-rwxr-xr-x  1 edwin edwin 1291 Aug 18 07:35 [01;32mactivate.sh[0m
-rw-r--r--  1 edwin edwin   92 Aug 18 07:35 csa.md
drwxr-xr-x  2 edwin edwin 4096 Aug 24 07:45 [01;34mimages[0m
-rw-r--r--  1 edwin edwin 2183 Aug 24 07:55 index.md
-rw-r--r--  1 edwin edwin   53 Aug 18 07:35 indexBlogs.md
drwxr-xr-x  3 edwin edwin 4096 Aug 18 07:35 [01;34mscripts[0m

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Look for posts"
export posts=$project/_posts  # _posts inside project
cd $posts  # this should exist per fastpages
pwd  # present working directory
ls -l  # list posts

bash: fg: %%script: no such job
Look for posts
/home/edwin/vscode/CSABlog/_posts
total 28
-rw-r--r-- 1 edwin edwin 3752 Aug 24 07:58 2023-08-17-AP-pseudo-vs-python_IPYNB_2_.md
-rw-r--r-- 1 edwin edwin 1651 Aug 24 09:33 2023-08-18-HowIWouldCode.md
-rw-r--r-- 1 edwin edwin 3191 Aug 24 09:15 2023-08-20-Calculator.md
-rw-r--r-- 1 edwin edwin 6059 Aug 24 07:58 2023-08-21-VSCode-GitHub_Pages_IPYNB_2_.md
-rw-r--r-- 1 edwin edwin  323 Aug 24 07:58 2023-08-22-JavaNotebook_IPYNB_2_.md
drwxr-xr-x 2 edwin edwin 4096 Aug 23 12:39 [0m[01;34mlab[0m

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Look for notebooks"
export notebooks=$project/_notebooks  # _notebooks is inside project
cd $notebooks   # this should exist per fastpages
pwd  # present working directory
ls -l  # list notebooks

bash: fg: %%script: no such job
Look for notebooks
/home/edwin/vscode/CSABlog/_notebooks
total 44
-rw-r--r-- 1 edwin edwin  5415 Aug 18 07:35 2023-08-17-AP-pseudo-vs-python.ipynb
-rw-r--r-- 1 edwin edwin 16839 Aug 24 09:48 2023-08-20-TestTools.ipynb
-rw-r--r-- 1 edwin edwin  8615 Aug 18 07:35 2023-08-21-VSCode-GitHub_Pages.ipynb
-rw-r--r-- 1 edwin edwin  1147 Aug 24 07:50 2023-08-22-JavaNotebook.ipynb

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Look for images in notebooks, print working directory, list files"
cd $notebooks/images  # this should exist per fastpages
pwd
ls -l

bash: fg: %%script: no such job
Look for images in notebooks, print working directory, list files
bash: cd: /home/edwin/vscode/CSABlog/_notebooks/images: No such file or directory
/home/edwin/vscode/CSABlog/_notebooks
total 44
-rw-r--r-- 1 edwin edwin  5415 Aug 18 07:35 2023-08-17-AP-pseudo-vs-python.ipynb
-rw-r--r-- 1 edwin edwin 16839 Aug 24 09:48 2023-08-20-TestTools.ipynb
-rw-r--r-- 1 edwin edwin  8615 Aug 18 07:35 2023-08-21-VSCode-GitHub_Pages.ipynb
-rw-r--r-- 1 edwin edwin  1147 Aug 24 07:50 2023-08-22-JavaNotebook.ipynb

Look inside a Markdown File

“cat” reads data from the file and gives its content as output

%%script bash

# Extract saved variables
source /tmp/variables.sh

echo "Navigate to project, then navigate to area wwhere files were cloned"

cd $project
echo "show the contents of README.md"
echo ""

cat README.md  # show contents of file, in this case markdown
echo ""
echo "end of README.md"

Env, Git and GitHub

Env(ironment) is used to capture things like path to Code or Home directory. Git and GitHub is NOT Only used to exchange code between individuals, it is often used to exchange code through servers, in our case deployment for Website. All tools we use have a behind the scenes relationships with the system they run on (MacOS, Windows, Linus) or a relationship with servers which they are connected to (ie GitHub). There is an “env” command in bash. There are environment files and setting files (.git/config) for Git. They both use a key/value concept.

“env” show setting for your shell
“git clone” sets up a director of files
“cd $project” allows user to move inside that directory of files
“.git” is a hidden directory that is used by git to establish relationship between machine and the git server on GitHub.

%%script bash

# This command has no dependencies

echo "Show the shell environment variables, key on left of equal value on right"
echo ""

env

%%script bash

# Extract saved variables
source /tmp/variables.sh

cd $project

echo ""
echo "show the secrets of .git"
cd .git
ls -l

echo ""
echo "look at config file"
cat config

Advanced Student Request - Make a file in Bash

This example was requested by a student (Jun Lim, CSA). The request was to make jupyer file using bash, I adapted the request to markdown. This type of thought will have great extrapolation to coding and possibilities of using List, Arrays, or APIs to build user interfaces. JavaScript is a language where building HTML is very common.

To get more interesting output from terminal, this will require using something like mdless (https://github.com/ttscoff/mdless). This enables see markdown in rendered format.

On Desktop Install PKG from MacPorts

In Terminal on MacOS

Install ncurses

gem install mdless

Output of the example is much nicer in “jupyter”

%%script bash

# This example has error in VSCode, it run best on Jupyter
cd /tmp

file="sample.md"
if [ -f "$file" ]; then
    rm $file
fi

tee -a $file >/dev/null <<EOF
# Show Generated Markdown
This introductory paragraph and this line and the title above are generated using tee with the standard input (<<) redirection operator.
- This bulleted element is still part of the tee body.
EOF

echo "- This bulleted element and lines below are generated using echo with standard output (>>) redirection operator." >> $file
echo "- The list definition, as is, is using space to seperate lines.  Thus the use of commas and hyphens in output." >> $file
actions=("ls,list-directory" "cd,change-directory" "pwd,present-working-directory" "if-then-fi,test-condition" "env,bash-environment-variables" "cat,view-file-contents" "tee,write-to-output" "echo,display-content-of-string" "echo_text_>\$file,write-content-to-file" "echo_text_>>\$file,append-content-to-file")
for action in ${actions[@]}; do  # for loop is very similar to other language, though [@], semi-colon, do are new
  action=${action//-/ }  # convert dash to space
  action=${action//,/: } # convert comma to colon
  action=${action//_text_/ \"sample text\" } # convert _text_ to sample text, note escape character \ to avoid "" having meaning
  echo "    - ${action//-/ }" >> $file  # echo is redirected to file with >>
done

echo ""
echo "File listing and status"
ls -l $file # list file
wc $file   # show words
mdless $file  # this requires installation, but renders markown from terminal

rm $file  # clean up termporary file

Hack Preparation.

Review Tool Setup Procedures and think about some thing you could verify through a Shell notebook.

Come up with your own student view of this procedure to show your tools are installed. It is best that you keep the few things you understand, add things later as you start to understand them.

Name and create blog notes on some Linux commands you will use frequently.

Is there anything we use to verify tools we installed? Review versions?

How would you update a repository? Use the git command line?

CompSci Blogs