Bash - Find substring in a string

Very simple solution to find substring in a string, make script sub.sh:

#!/bin/bash
txt="This is a text"
substr="text"
if [[ $txt == *$substr* ]]
then
echo "String [$txt] contains substring [$substr]"
else
echo "String [$txt] doesn't contains substring [$substr]"
fi

Variable txt is a string which should (or not) contains string from variable substr.

Output:

$ ./sub.sh
String [This is a text] contains substring [text]

Change variable substr:

substr="t1xt"

Output:

$ ./sub.sh
String [This is a text] doesn't contains substring [t1xt]

Linux - Search for empty directories and files

To search for empty directories type:

$ find . -empty -type d

To delete empty directories:

$ find . -empty -type d -delete

If enter -type f instead of -type d, this command will search for empty files:

$ find . -empty -type f

to delete empty files:

$ find . -empty -type f -delete

Bash Script: Number of Arguments

Number of arguments is in variable $#, example is script numarg.sh

#!/bin/bash
echo "number of arguments: $#"

Executing script: 
$ ./numarg.sh arg1 arg2 arg3
Number of arguments: 3

Variable $# doesn't count script name, only arguments.

Arguments are stored in variables $1, $2... or in $@ as array.

[Bash] Count Letters in a Text File

Bash script have very useful commands for Text File analysis. For this example we will use small part from Leo Tolstoy: War and Peace, available on Project Gutenberg:

This part is placed in file onepart.txt. To count letters all letters must have same case. Command tr will translate all letters to uppercase letters:

$ cat onepart.txt|tr a-z A-Z

Next, output is filtered. Command tr is useful here with first argument are switches -cd and seccond is letter to be filtered. Switch -c means complent and -d to delete, in short those two arguments means erase everything what is not letter in argument.
Pipelining we get:

$ cat onepart.txt|tr a-z A-Z|tr -cd A

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 

Or for B, C etc...

$ cat 3.txt|tr a-z A-Z|tr -cd B
BBBB

$ cat 3.txt|tr a-z A-Z|tr -cd C
CCCCCCCCCCCCCCCCCCCCCCC

 $ cat 3.txt|tr a-z A-Z|tr -cd D
DDDDDDDDDDDDDDDDDDDDDDDDDD

And of course we need to count those letters. Command wc with switch -m do exactly that:

$ cat 3.txt|tr a-z A-Z|tr -cd A|wc -m
40

$ cat 3.txt|tr a-z A-Z|tr -cd B|wc -m
4

$ cat 3.txt|tr a-z A-Z|tr -cd C|wc -m
23

$ cat 3.txt|tr a-z A-Z|tr -cd D|wc -m
26

and, of course, script to count all letters occurances: 

#!/bin/bash

text=$(cat $1|tr a-z A-Z)

echo "Letter occurances:"

for l in A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

do 

let=$(echo $text|tr -cd $l|wc -m)

echo "$l $let"

done

Script is a little bit slower because text is analyzed for every letter.

Linux - How to change default sshd port

There are many reasons why default port 22 for ssh is need to be changed. If you need to change default port or add other listening port:

1. open /etc/ssh/sshd_config with your favorite text editor
2. find line
Port 22

and

edit this value to your needs.

or if you want to add other listening port, simply add line with port. Example:

Port 22
Port 1000

after editing save config file and restart sshd:

$ sudo /etc/init.d/ssh restart

or

$ service sshd restart

dependant on system you use.

File /etc/ssh/sshd_config opened in editor

Bash - Substring

How to get substring in Bash script? Simple :)

if we have string:

$ a=123456789

substring from position 1 to 5 is

$ echo ${a:1:5}

23456

First position in a string is 0.

in general: ${varname:start_pos:length}

Linux - Printing first or last 10 lines

There are many examples when we need to print first or last lines of some file. Commands head and tail ca help to do this.

Command head prints 10 first lines of file.

$ head file

or

$ cat file|head

There is command line argument -n, which says how many lines to be printed.

Example:

$ head -n 5 file

prints first 5 linest from file.

if number is negative, command head prints last n lines from file.

Command tail does oposite from command head: prints last 10 lines from a file.

$ tail file

same as command head there is argument -n which tells to command head how many last lines from a file is to be printed:

$ tail -n 7 file

prints last 7 lines from file.

If you want to print all lines in file excluded first n lines, command is:

$ tail -n +5 file

prints all lines in file, excluded first 5 lines.

Command tail is very useful in log analyzing, when you are not interested in whole file, but only last n lines.

Linux - Find ssh attacks

All login attempts, successful or not, are in /var/log/auth.log file. To find from which IP addresses attempts are coming, and only for valid usernames, execute following line:

$ cat /var/log/auth.log|grep -v "invalid"|grep "Failed password"|awk '{ print $(NF-3) }'|sort|uniq

grep -v "invalid" - -v switch reverts filter then only lines without word "invalid" are included
grep "Failed password" - only lines with words "Failed password" are included
awk '{ print $(NF-3) }' - Take 3rd arg from last
sort
and print only unique IP addresses.


Also, if you want to find on which usernames attacks are targeted change awk part into: awk '{ print $(NF-5) }'

Linux - File Checksum

The purpose of file checksums is to validate integrity of files. There are many ways to do checksums on Linux, but the simplest is by using md5sum command (for MD5 checksum). Format is:
md5sum [options] [file1] ... [file n]

Options are:
-b or --binary - binary mode
-t or --text - text mode (default)
-c or --check - read file with checksums and check them

For test purpose make one file named test

$ md5sum test
ee10c66475e841d934c0e39d07650d4a  test

output md5 checksum to file:
$ md5sum test >checksum

check:
$md5sum -c checksum
test: OK

MD5 gives 128 bit checksum. For security related purposes is better to use SHA-2 algorithm implemented in: sha224sum, sha256sum, sha384sum and sha512sum programs.

Linux - Add User to a Group

1. To add existing user to existing group:

$ usermod -G groupname username

or 

$ usermod -g GID username

2. To add existing user to existing supplementary groups

$ usermod -a -G groupname username

-a is switch for append

3. To add user to group when adding user

$ adduser --ingroup groupname username

4. To check which groups user is member of

$ groups username

Bash script: User Input

For user general input in variable we use command read:

#!/bin/bash
read -p "Enter first number:" num1
read -p "Enter second number:" num2
if [ num1 -eq num2 ]
then
   echo "Numbers are equal"
else
   if [ $num1 -lt $num2 ]
   then
      echo "Second number is larger"
   else
      echo "First number is larger"
   fi
fi

For entering exact expected word (example Yes or No) we use command select:

#!/bin/bash
echo "Do you want continue?"
select inp in "yes" "no"
do
   case $inp in
      yes ) echo "You want to continue"
              break
              ;;
      no ) echo "You want to exit"
             break
             ;;
   esac
done

Bash FOR loop

For loop is a structure that executes a number of times, generally known in advance. In Bash script syntax of FOR loop is:

for VARIABLE [in LIST]; do COMMANDS BLOCK; done

or

for VARIABLE [in LIST]
do
   COMMANDS_BLOCK
done

LIST is a list of words or numbers. If LIST is omitted $@ is implied.

Simple examples:
1. This example prints all files in directory

dirlist=$(ls)
for i in $dirlist
do
   echo $i
done


2. This example prints numbers from 1 to 10

for i in {1..10}
do
   echo $i
done

or

for i in 1 2 3 4 5 6 7 8 9 10
do
   echo $i
done 

3. This example prints number from 1 to 10 with step 2

for i in {1..10..2}
do 
   echo $i
done

4. This example prints 

Bash - Exit code

1. Exit code from Bash script

argument of exit command is exit code
exit n
example (exit_example.sh)

#!/bin/bash
exit $1  #return argument 1 as return code

2. Bash last exit code

last exit code is in variable $?

#!/bin/bash
./exit_example.sh 2
echo $?

result is:
2

Bash - Join Arrays

How to join arrays in Bash Script? Very simple :) Example:

#!/bin/bash

a1=(a b c d) #array 1

a2=(e f g h) #array 2

a=(${a1[@]} ${a2[@]}) #join array

echo ${a[@]}

Output is:

a b c d e f g h

Make array of arrays and result is joined array :)

Linux - Add User Account

To add user to Linux from command line (shell) type command adduser followed with options and username. Example to add user linips (type sudo if you are not root):

$ sudo adduser linips
Adding user `linips' ...
Adding new group `linips' (1001) ...
Adding new user `linips' (1001) with group `linips' ...
Creating home directory `/home/linips' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for linips
Enter the new value, or press ENTER for the default
        Full Name []: Linips Blog
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
Is the information correct? [Y/n]

Defaults are in file /etc/adduser.conf

What if we are want to add user with username format as firstname.secondname?

$ sudo adduser linips.blog

adduser: Please enter a username matching the regular expression configured
via the NAME_REGEX[_SYSTEM] configuration variable.  Use the `--force-badname'
option to relax this check or reconfigure NAME_REGEX.

Then we must add --force-badname switch to command

$ sudo adduser --force-badname linips.blog

and continue with entering data for new user.

Linux Add Group

To add group linips to Linux type:

$ groupadd linips

General format of command groupadd is:

groupadd [options] groupname

options are:

• --help 
• -g, --GID gid -to assign Group ID
• -f, --force - -f with -g acts like -g not exist - GID another unique
• -K, --key KEY=VALUE
• -o, --non-unique - to add group with non unique GID
• -p, --password PASSWORD
• -r, --system - To create system group

Linux Get CPU Info

Probably the simplest way to get CPU info in Linux is to display /proc/cpuinfo (and example output)

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Celeron(R) CPU 2.40GHz
stepping        : 9
cpu MHz         : 2423.933
cache size      : 128 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe up pebs bts sync_rdtsc cid xtpr
bogomips        : 4851.34
clflush size    : 64

if we need only to display model name: 

$ cat /proc/cpuinfo|grep "model name"

model name      : Intel(R) Celeron(R) CPU 2.40GHz

In the case that there are multiple processors, info of second will continue after first one and so on... processor: 0, processor: 1,...

Solution for maintaining old Ubuntu Linux computer

If you have an old server with Ubuntu Linux distribution, which for some reason you can not update to newer version and you need to maintain it. During new program installation many errors tells us that  aptitude or apt-get can not find the path to the server from which we need to download required packets. Solution is very simple for the following Ubuntu Linux distributions: 

• Ubuntu 4.10 (Warty Warthog)
• Ubuntu 5.04 (Hoary Hedgehog)
• Ubuntu 5.10 (Breezy Badger)
• Ubuntu 6.06.2 LTS (Dapper Drake)
• Ubuntu 6.10 (Edgy Eft)
• Ubuntu 7.04 (Feisty Fawn)
• Ubuntu 7.10 (Gutsy Gibbon)
• Ubuntu 8.04.4 LTS (Hardy Heron)
• Ubuntu 8.10 (Intrepid Ibex)
• Ubuntu 9.04 (Jaunty Jackalope)
• Ubuntu 9.10 (Karmic Koala)
• Ubuntu 10.04.4 LTS (Lucid Lynx)
• Ubuntu 10.10 (Maverick Meerkat)
• Ubuntu 11.04 (Natty Narwhal)
• Ubuntu 11.10 (Oneiric Ocelot) 
• Ubuntu 12.04.3 LTS (Precise Pangolin)

All repositories for those distributions are located at: old-releases.ubuntu.com

How to make changes

Edit your /etc/apt/sources.list : 

change all urls from: http://<>.archive.ubuntu.com/ubuntu  to http://old-releases.ubuntu.com/ubuntu

And your old Ubuntu Linux is ready for update :)

Linux How to concatenate text files

cat file1 file2 filen > outputfile

or if you want to concatenate all text files in directory:

cat *.txt > outputfile

or you can use >> operator to append to file:

cat file1 >> file2

Linux How to count words and lines in a text file

If you want to count words in some text file you could use:

wc -w <file>

example for file 2600.txt (War and Peace from Leo Tolstoy) from Guttenberg Project :

wc -w 2600.txt

output:

566321 2600.txt

 If you want to count lines in some text file you could use:

wc -l <file>

the same file for example

wc -l 2600.txt

output:

65008 2600.txt

Full options list for wc command:

• -c or --bytes byte counts
• -m or --chars character counts
• -l or --lines newline counts
• -L or --max-line-length lenght of longest line
• -w or --words word counts

 also:

• --help
• --version