This is a DevOps linux "Operating Systems Basics" Module
Video: Basic Linux Commands (CLI - Part 2)
General Operations:
clear
= Clears the terminal
Directory Operatings:
pwd
= Show current directory. Example Output:/home/nana
ls
= List folders and files. Example Output:Desktop Downloads Pictures Documents
cd [dirname]
= Change directory to [dir]mkdir [dirname]
= Make directory [dirname]cd ..
= Go up a directory
File Operations:
touch [filename]
= Create [filename]rm [filename]
= Delete [filename]rm -r [dirname]
= Delete a non-empty directory and all the files in itrm -d [dirname]
orrmdir [dirname]
= Delete an empty directory
Navigating in the File System:
cd usr/local/bin
= Navigate multiple dirs (relative path - relative to current dir). Move to bin directorycd ../..
= Move up 2 hierarchies, so go to 'usr' directorycd /usr
= Alternative to go to 'usr' directly (absolute path)cd [absolute path]
= Move to any location by providing the full pathcd /home/nana
= Go to my home directory (absolute path)cd ~
= Shortcut alternative to go to home directoryls /etc/network
= List folders and files of 'network' directory
More File and Directory Operations
mv [filename] [new_filename]
= Rename the file to a new file namecp -r [dirname] [new_dirname]
= Copy dirname to new_dirname recursively meaning including the filescp [filename] [new_filename]
= Copy filename to new_filename
Some more useful commands
ls -R [dirname]
= Show dirs and files but also sub dirs and fileshistory
= Gives a list of all past commands typed in the current terminal sessionhistory 20
= Show list of last 20 commandsCTRL + r
= Search historyCTRL + c
= Stop current commandCTRL + SHIFT + v
= Paste copied text into terminalls -a
= See hidden files toocat [filename]
= Display the file contentcat .bash_history
= Example 1: Display the file contentcat Documents/java-app/Readme.md
= Example 2: Display the file content
Display OS Information
uname -a
= Show system and kernelcat /etc/os-release
= Show OS informationlscpu
= Display hardware information, e.g. how many CPU you have etc.lsmem
= Display memory information
Execute commands as superuser
sudo [some command]
= Allows regular users to run programs with the security privileges of the superuser or rootsu - admin
= Switch from nana user to admin
Video: Package Manager - Installing Software on Linux
APT Package Manager:
sudo apt search [package_name]
= Search for a given packagesudo apt install [package_name]
= Install a given packagesudo apt install [package_name] [package_name2]
= Install multiple packages with one commandsudo apt remove [package_name]
= Remove installed packagesudo apt update
= Updates the package index. Pulls the latest change sfrom the APT repositories
APT-GET Package Manager:
sudo apt-get install [package_name]
= Install package with apt-get package manager
SNAP Package Manager:
sudo snap install [package_name]
= Install a given package
Video: Working with Vim Editor
Install Vim, if it's not available:
sudo apt install vim
= Search for a given package
There are 2 Modes:
- Command Mode: default mode, everything is interpreted as a command
- Insert Mode: Allows to enter text
Vim Commands:
vim [filename]
= Open file with VimPress i key
= Switch to Insert ModePress esc key
= Switch to Command ModeType :wq
= Write File to disk and quit VimType :q!
= Quit Vim without saving the changesType dd
= Delete entire lineType d10
= Delete next 10 linesType u
= UndoType A
= Jump to end of line and switch to insert modeType 0
= Jump to start of the lineType $
= Jump to end of the lineType 12G
= Go to line 12Type 16G
= Go to line 16Type /pattern
= Search for pattern, e.g./nginx
Type n
= Jump to next matchType N
= Search in opposite direction
Type :%s/old/new
= Replace 'old' with 'new' throughout the file
Video: Linux Accounts & Groups (Users & Permissions Part 1)
Locations of Access Control Files:
- /etc/passwd
- /etc/shadow
- /etc/group
sudo adduser [username]
= Create a new usersudo passwd [username]
= Change password of a usersu - [username]
= Login as username ('su' = short for substitute or switch user)su -
= Login as root
sudo groupadd [groupname]
= Create new group (System assigns next available GID)sudo adduser [username]
= Switch to Insert Mode
Note 2 different User/Group commands:
adduser
, addgroup
, deluser
, delgroup
= interactive, more user friendly commands
useradd
, groupadd
, userdel
, groupdel
= low-level utilities, more infos need provided by yourself
sudo usermod [OPTIONS] [username]
= Modify a user accountsudo usermod -g devops tom
= Assign 'devops' as the primary group for 'tom' usersudo delgroup tom
= Removes group 'tom'groups
= Display groups the current logged in user belongs togroups [username]
= Display groups of the given usernamesudo useradd -G devops nicole
= Create 'nicole' user and add nicole to 'devops' group (-G = secondary group, not primary)sudo gpasswd -d nicole devops
= Removes user 'nicole' from group 'devops'
Video: File Ownership & Permissions (Users & Permissions Part 2)
ls -l
= Print files in a long listing format, you can see ownership and permissions of the file
Ownership:
sudo chown [username]:[groupname] [filename]
= Change ownershipsudo chown tom:admin test.txt
= Change ownership of 'test.txt' file to 'tom' and group 'admin'sudo chown admin test.txt
= Change ownership of 'test.txt' 'admin' usersudo chgrp devops test.txt
= Make 'devops' group owner of test.txt file
Possible File Permissions (Symbolic):
- r = Read
- w = Write
- x = Execute
- '-' = No permission
Change File Permissions for different owners
File Permissions can be changed for:
- u = Owner
- g = Group
- o = Other (all other users)
Minus (-) removes the permission
sudo chmod -x api
= Takes 'execute' permission away for 'api' folder from all ownerssudo chmod g-w config.yaml
= Takes 'write' permission away for 'config.yaml' file from the group
Plus (+) adds permission
sudo chmod g+x config.yaml
= Add 'execute' permission for 'config.yaml' file to the groupsudo chmod u+x script.sh
= Add 'execute' permission for 'script.sh' file to the usersudo chmod o+x script.sh
= Add 'execute' permission for 'script.sh' file to other
Change multiple permissions for an owner
sudo chmod g=rwx config.yaml
= Assign 'read write execute' permissions to the groupsudo chmod g=r-- config.yaml
= Assign only 'read' permission to the group
Changing permissions with numeric values
Set permissions for all owners with 3 digits, 1 digit for each owner Absolute vs Symbolic Mode
- 0 = No permission
- 1 = Execute
- 2 = Write
- 3 = Execute + Write
- 4 = Read
- 5 = Read + Execute
- 6 = Read + Write
- 7 = Read + Write + Execute
sudo chmod 777 script.sh
= rwx (Read, Write and Execute) permission for everyone for file 'script.sh'sudo chmod 740 script.sh
= Give user all permissions (7), give group only read permission (4), give other no permission (0)
Video: Introduction to Shell Scripting - Part 1
Create and open setup.sh file in vim editor:
vim setup.sh
In setup.sh file:
#!/bin/bash
echo "Setup and configure server"
# save file with
ESC :wq
# make file executable
chmod u+x setup.sh
# execute script
./setup.sh
bash setup.sh
sh setup.sh
Video: Shell Scripting Part 2 - Concepts & Syntax
Variables:
#!/bin/bash
echo "Setup and configure server"
file_name=config.yaml
config_files=$(ls config)
echo "using file $file_name to configure something"
echo "here are all configuration files: $config_files"
Conditions:
#!/bin/bash
echo "Setup and configure server"
file_name=config.yaml
config_dir=$1
if [ -d "$config_dir" ]
then
echo "reading config directory contents"
config_files=$(ls "$config_dir")
else
echo "config dir not found. Creating one"
mkdir "$config_dir"
touch "$config_dir/config.sh"
fi
# example conditional for checking file
# if [ -f "config.yaml" ]
# example conditional for checking numbers
# num_files=xx
# if [ "$num_files" -eq 10 ]
# example conditional for checking strings
user_group=$2
if [ "$user_group" == "nana" ]
then
echo "configure the server"
elif [ "$user_group" == "admin" ]
then
echo "administer the server"
else
echo "No permission to configure server. wrong user group"
fi
echo "using file $file_name to configure something"
echo "here are all configuration files: $config_files"
User input:
#!/bin/bash
echo "Reading user input"
read -p "Please enter your password: " user_pwd
echo "thanks for your password $user_pwd"
Script Parameters:
#!/bin/bash
echo "all params: $*"
echo "number of params: $#"
echo "user $1"
echo "group $2"
Executing with script parameters:
./example.sh name lastname # 2 params
./example.sh "name lastname" # 1 param
bash example name lastname
Loops:
#!/bin/bash
echo "all params: $*"
echo "number of params: $#"
for param in $*
do
if [ -d "$param" ]
then
echo "executing scripts in the config folder"
ls -l "$param"
fi
echo $param
done
sum = 0
while true
do
read -p "enter a score" score
if [ "$score" == "q" ]
then
break
fi
sum=$(($sum+$score))
echo "total score: $sum"
done
Video: Shell Scripting Part 3 - Concepts & Syntax
Functions:
#!/bin/bash
echo "all params: $*"
echo "number of params: $#"
for param in $*
do
if [ -d "$param" ]
then
echo "executing scripts in the config folder"
ls -l "$param"
fi
echo $param
done
# Declare function
function score_sum {
sum = 0
while true
do
read -p "enter a score" score
if [ "$score" == "q" ]
then
break
fi
sum=$(($sum+$score))
echo "total score: $sum"
done
}
# Invoke function
score_sum
function create_file() {
file_name=$1
is_shell_script=$2
touch $file_name
echo "file $file_name created"
if [ "$is_shell_script" = true ]
then
chmod u+x $file_name
echo "added execute permission"
fi
}
# Invoke with diff params
create_file test.txt
create_file myfile.yaml
create_file myscript.sh
# Function with return value
function sum() {
total=$(($1+$2))
return $total
}
sum 2 10
result=$?
echo "sum of 2 and 10 is $result"
Video: Basic Linux Commands - Pipes & Redirects (CLI - Part 3)
Pipe & Less:
Pipe Command:
|
= Pipe command = Pipes the output of the previous command as an input to the next command
Less Command:
less [filename]
= Displays the contents of a file or a command output, one page at a time. And allows to navigate forward and backward through the file
Different piping examples/use cases:
cat /var/log/syslog | less
= Pipes the output of 'syslog' file to less program.ls /usr/bin | less
= Pipes the output of ls command to less program.history | less
= Pipes the output of history command to less program.
Pipe & Grep:
Grep Command:
grep [pattern]
= Searches for a particular pattern of characters and displays all lines that contain that pattern
More piping examples/use cases:
history | grep sudo
= Look for any commands of history commands, which have 'sudo' word in ithistory | grep "sudo chmod"
= Look for any commands of history commands, which have 'sudo chmod' phrase in ithistory | grep sudo | less
= History output will pass output to grep and filter for 'sudo' and this output will again be piped or passed to less programls /usr/bin/ | grep java
= Filter ls output for javacat Documents/java-app/config.yaml | grep ports
= See all 'ports' occurences in config.yaml file
Redirects in Linux:
>
= Redirect Operator = Takes the output from the previous command and sends it to a file that you give
Different redirects examples/use cases:
history | grep sudo > sudo-commands.txt
= Redirect output into a 'sudo-commands.txt' filecat sudo-commands.txt > sudo-rm-commands.txt
= Redirect output of 'sudo-commands.txt' file into 'sudo-rm-commands.txt' filehistory | grep rm > sudo-rm-commands.txt
= Redirect output of filtered history commands into existing 'sudo-rm-commands.txt' file. Note: Contents of file will be overwrittenhistory | grep rm >> sudo-rm-commands.txt
= Redirect output of filtered history commands into existing 'sudo-rm-commands.txt' file. Note: Contents of file will be appended
Video: Environment Variables
Variables store information. Environment variables are available for the whole environment. An environment variable consists of name=value pair.
Existing Environment Variables:
SHELL=/bin/bash
= default shell program, in this case bashHOME=/home/nana
= current user's home directoryUSER=nana
= currently logged in user
printenv
= List all environment variablesprintenv | less
= List all environment variables with less programprintenv [environment variable]
= Display value of given environment variable, e.g.printenv USER
printenv | grep USER
= Filter environment variables, which have 'USER' in the name
echo $USER
= Print value of USER environment variable
Create own Environment Variables:
export DB_USERNAME=dbuser
= Set environment variable 'DB_USERNAME' with value 'dbuser'export DB_PASSWORD=secretpwdvalue
= Set environment variable 'DB_PASSWORD' with value 'secretpwdvalue'export DB_NAME=mydb
= Set environment variable 'DB_NAME' with value 'mydb'printenv | grep DB
= Filter environment variables for 'DB' charactersexport DB_NAME=newdbname
= Set environment variable 'DB_NAME' to new value 'newdbname'
Delete Environment Variables:
unset DB_NAME
= Delete variable with name 'DB_NAME'
Persisting Environment Variables:
Persisting Environment Variables with shell specific configuration file: Environment variables set in terminal are only available in the current terminal session.
Add environment variables to the '~/.bashrc' file or your specific shell 'rc' file. Variables set in this file are loaded whenever a bash login shell is entered.
export DB_USERNAME=dbuser
export DB_PASSWORD=secretvl
export DB_NAME=mydb
In terminal again:source ~/.bashcrc
= Load the new env vars into the current shell session
Persisting Environment Variables system wide:
- ~./bashrc = user specific
- /etc/environment = system wide, meaning all users will have access to the variables
PATH Environment Variable:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
= List of directories to executible files, separated by ':'. Tells the shell which directories to ssearch for the executable in response to our executed commandPATH=$PATH:/home/nana
= Appending /home/nana folder to the existing $PATH value
Video: Networking
Useful Networking Commands:
ip
= one of the basic commands. For setting up new systems and assigning IPs to troubleshooting existing systems. Can show address information, manipulate routing, plus display network various devices, interfaces, and tunnels.ifconfig
= for configuring and troubleshooting networks. It has since been replaced by theip
commandnetstat
= tool for printing network connections, routing tables, interface statistics, masquerade connections, and multicast membershipsps aux
=- ps = displays information about a selection of the active processes
- a = show processes for all users
- u = display the process's user/owner
- x = also show processes not attached to a terminal
nslookup
= Find DNS related queryping
= To check connectivity between two nodes
Video: SSH - Secure Shell
Connecting via SSH: ssh username@SSHserver
ssh [email protected]
= Connect with root user to 159.89.14.94 server addressssh-keygen -t rsa
= Create SSH Key Pair with 'rsa' algorithm. SSH Key Pair is stored to the default location~/.ssh
ls .ssh/
= Display contents of .ssh folder, which has:id_rsa
= Private Keyid_rsa.pub
= Public Key
ssh -i .ssh/id_rsa [email protected]
= Connect with root user to 159.89.14.94 server address with specified private key file location (.ssh/id_rsa = default, but you can specify a different one like this)
Two Files used by SSH:
~/.ssh/known_hosts
= lets the client authenticate the server to check that it isn't connecting to an impersonator~/.ssh/authorized_keys
= lets the server authenticate the user