Command Line Cheat Sheet



Oft used Bash and Linux / Unix / Cygwin commands
================================================

In general, commands are show with the $ preceeding to indicate
 that the command can be issued as a non-elevated user.
If the command is preceeded by #, then it must (in general) be
 issued by root.

Groups and permissions
----------------------
UUUGGGOOO = User Group Others
rwxrwxrwx

#groupadd testgroup            -- Adds a group called 'testgroup'
#usermod -aG testgroup matthew -- Adds the user 'matthew' to 'testgroup'
   Doesn't modify user other than adding to[G]roup
   Adds the user as a member to supplmental group(s)
   but doesn't erase any current group affiliation ([a]ppends).
   Doesn't change user's login group affiliation.
#usermod -g testgroup matthew -- makes matthew's login group testgroup.

$chgrp groupToChangeTo fileToChange file2ToChange  -- changes a file (or files, in this case) to a 
                                                      new group.
    Must be the owner of the file and belong to the group to which you want to change.
$groups username   --  lists all groups to which 'username' belongs
$less /etc/group   --  read the contents of the group file, which contains all the group info
                       [i.e., what users belong to what groups.]
$chmod ugo+rwx fileName	 Change access modifications for [u]ser, [g]roup, [o]ther 
                         to add [r]ead, [w]rite, and e[x]ecute permission.
                         Also, could use [a]ll to represent ugo
                         Use - (minus) instead of + (plus) to subtract permissions
$chmod -R o-r fileName    [R]ecursively apply permissions
$chmod o-r $(find * -type d)   Removes the read permission for [o]thers for all directories

$id  -  prints your user and group ids
$id username  -  prints for the username

Add a user name "newuser1", giving him a home dir and a password: NewPassword
#useradd -d/home/newuser1 -m -pNewPassword newuser1

Removes the user "olduser", (r)emoving user's home dir
#userdel -r olduser      

Redirect Output
---------------
$Program > out.txt                 Redirects std output to file out.txt
$Program  1>out.txt 2>err.txt      Redirects stdout to out.txt and stderr to err.txt
$Program  1>>out.txt 2>>err.txt    Appends stdout to out.txt and stderr to err.txt
$ls | tee out.txt                  Prints ls to std output and also redirects to out.txt file
$cmd 2>&1                          Places stdout and stderr on 1 (std out)
$cmd 1>&2                          Places stdout and stderr on 2 (std error)

Locate
------
#updatedb    Updates the locate database   (often done as a cronjob)
$updatedb --localpaths='/cygdrive/d'    Includes the d drive in locate database.
$locate -i FileName		       Searches for FileName *or* filename, [i]gnoring case. 
$locate -r regexString         Searches using [r]egex 
$locate -b '\ls'                  Search for only / exactly the name:  ls
$rm -rf DirectoryToRemove   Removes a directory, [r]ecursively (all files and sub-dirs, 
                            through all sub-dirs) and [f]orced (without warning prompts).  
                            rmdir only works for empty dirs. 
$rm $(find -name "*.pch")   Remove all the files found containing the string .pch by using find.  
                            Note that find searches recursively.
$rm -rf $(find -name ".svn")  Remove all the subversion dirs recursively.

X Windowing
-----------
$startx &		   starts the X server
$xterm&			   starts a new X terminal

$grep grepString fileName         Searches for "grepString" within the file fileName
$grep -r grepString * | less      Search [r]ecursively for reg exp. grepString through all files (*) and 
                                  pipe into less. Is case sensitive.  Use \ to include special 
                                  meta-characters as normal characters in search (e.g. \* will include
                                  * as a normal character).
$grep -i ...	   Makes grep case insensitive.
$grep -n ...	   Adds line numbers in the file where a match was found
$grep -I ...       Skips searching of binary files.

In a grep expression, the char . means "any character". The char * denotes "any number of the 
  preceeding value".  So the char combo .* means "any number of any characters".

$grep -rin isChar $(find -name *.cpp) | less  Searches [r]ecursively, case [i]nsensitively, adding 
                                              line [n]umbers for matches to "isChar" found in any 
                                              file that ends in .cpp, piped into less.

wget $(grep -Po http.*?mp3 index.html)   wget all the files that are returned from a search of the 
                                         (locally saved) index.html file for all matches to http address
                                         that are mp3 files. Lazy (or minimum) match, enforced by ?
                                         guarantees that this is the shortest match starting with http.
                                         [o] option just copies out the pattern match (not the whole line, as is default).
                                         [P] option is for Perl-style regex, which seems to be necessary for the lazy ?
                                         Use this, for example, when you have
                                         an html that has a lot of links you wish to download, but the http 
                                         directory restricts reading
                                         file contents (which restricts wget from being able to download all files)

                                         Could also use -nc for wget, so that files are not downloaded multiple times.

wget -r --html-extension --convert-links http://server/site/   Downloads and converts a site (could be from a local
                                                               Apache server) [r]ecrusively and converting any
                                         non-html pages (such as PHP pages) into .html extension pages, including
                                         links within and between those pages.  So you can turn a PHP-based site
                                         into a static purely html based site.

$diff file1 file2                   Diffs two files, prints output
$diff file1 file2 > diffs.txt       Diffs two files, redirects output to file diffs.txt
$diff -r --brief Dir1 Dir2 | less	Differences all files in Dir1 vs Dir2 and reports only file names [--brief]. 
                                    Searches [r]ecursively.
$diff -r --brief -x .svn Dir1 Dir2 | less    Same as above, but also skips .svn directories

$diff -r --brief -x .svn -x *~ -x *.a -x CMakeFiles Dir1 Dir2   Exclude common programming-related dirs/files.

Find
----
  $find [directory-list] [expression(s)]
  If directory-list is omitted, searches cwd.
  Within the find expression, special characters must be quoted so that the shell does not interpret them, but 
    passes them on to the find utility.  Special chars = (), [], ?, *

  Within criteria:
	 AND  = space
	 OR   = -or
	 NOT  = !

$find . -name "frag*"        Searches for files recursively in CWD for any file containing the 
                             fragment "frag".
$find ~/Docs -name "frag*"   Searches for files recursively in home/Docs for any file containing 
                             the fragment "frag".
$find . -name "*frag*"  searches for "frag" also in the middle of text (ie, defraged)
$find . -name *.h -mtime -45   finds all files ending in ".h" modified within the time of 45 days ago
$find . -name *.h -mtime -45 -or -name *.cpp -mtime -45	   finds all files ending in .h or .cpp 
                                                           modified within the last 45 days
$find . -mtime -10 \( -name "*.h" -or -name "*.cpp" \)	   equivalent to the previour find, but groups the
                                                           naming using () -- must backslash quote special
                                                           chars so they are not interpretted by Bash.
$find -mtime -10 ! -name "*.obj"	    finds all files modified in the last ten days that 
                                        are not object files.
$find $(locate syslog) -mtime -1      Locate all syslog files and then find the ones that have 
                                      been modified within the last day.

$find . -regex '.*mp[1-4]'      Finds from cwd, using regex, all mp3 / mp4 files

Tar and Zip
-----------
$unzip file.zip	                                  Unzips in current working directory.
$unzip file.zip -d TargetDir                      Unzips to TargetDir
$bunzip2 file.bz2                                 Unzips from a bz2 file
$zip -r  zipFileName.zip  SourcePathOrFileName    Zips [r]ecursively
$tar -zxvf filename.tar.gz     un[z]ips, e[x]tracts, issuing [v]erbal messages, of the [f]ilename given.
$tar -jxvf filename.tar.bz2    same as above, but for bz2 type compression.
$tar -zcf 2005_FinancialStatements.tar.gz 2005*	    [z]ips, [c]reates a [f]ile of all docs starting 
                                                    with 2005
Misc
----
$cp -i sourceFile DestinationFile	copies [i]nteractively, so warns if overwriting existing 
                                    DestinationFile
$lpr FileName	 Prints file to printer
$mc              Start midnight commander (must be installed)
$echo $?		 Return the exit status of the last command
$cp --reply=no file.txt dest     Force ignore of question regarding previously existing 
                                 file of same name.
$uname -a     -- Find out what kernel you're running
#eject -r     -- Eject a CD.  Must be SU.
$bind -P      -- To find out Bash command-line functions
cd -          -- Returns you to the directory you were previously in

Use mouse to highlight text, then center click     -  Copies the highlighted text to the CL

Variables
---------
env    Print all the global environment variables
set    Print all environment and local (non-exported) variables
$Prog=$(pwd)/httpdocs/accounts_state/Programming     Sets a variable called Prog to be the current
                                                     working directory $(pwd) + a path:

du and df
---------
$du	        Disk usage - proivdes a summary of every subdirectory's disk usage (all
            subdir's, recursively). Values are in kb.
$du -s	    Shows only the current directory total size
$du -s *	Provides a [s]ummary of the disk space usage for the 
            current directory and all (*) level 1 subdirectories.
$du -s fileName    pvoides a summary of fileName's disk usage (fileName
                   could be a directory).
$df	        shows how much space is used and free for each mount
$df -T      Shows volume (disk) type

History and fc
--------------
$history	  prints a list of the previous cl commands with numbers
$!31	  executes cl #31 in the history list (substitute any # for 31)
Set and export FCEDIT to emacs, declare -x FCEDIT=/usr/bin/emacs

$fc	  edit / reexecute previous command in an editor.
$fc -l	  lists the last 16 history commands to cout
$fc 31	  edits cl #31 in editor.  Will execute command when simply
	  exiting emacs (be careful!)
$fc 31 36  edits cl #31 to 36 in editor. Will execute ALL 6 commands on exit.
	  Can create shell scripts this way (save as, c-x c-w).

Links
-----
$ln -s pathToTarget  nameOfLink
$ln -s ../Graphics   Graphics	   Create a symbolic called "Graphics" which links to the 
                                   directory ../Graphics.

$ps	      List process that your terminal controls
$ps -x	  List all process for you (x=not necessarily attached to terminal)
$ps -ax	  List all processes in the system (a=all users)

$top      Lists a continuously updated list of process info (PID, user, %CPU, %Mem, etc).
          While running top, sort by P for CPU, M for memory, S for process run time
          Press i to toggle between displaying idle processes or not.
$jobs     Prints a list of currently running jobs and their job numbers

$kill pid          -- Kills a process (with process number 'pid') using TERM
$kill -9 pid       -- Execute that process (stronger kill than with TERM)
$kill %jobNum      -- Kills a job

$ [ctrl-x][ctrl-a]  -- Starts nano with whatever is on the command line. Can then edit and execute 
                       that command (or more than one, one on each line) by exiting and writing out.
See fc for other interactive command line editing.

$mount		      --  To see currently mounted devices
#mount -o loop AnImage.iso /mnt/AMountPoint/    -- Mount an ISO image as if it were a disk
#blkid /dev/sdb1  --  See the properites of the device (including file type of unmounted devices)
$less /etc/mtab   --  To see current mount table
#umount /dev/sdb1 --  Unmount the device sdb1 (eg, a USB device in /dev)
$/sbin/runlevel   --  Check your run level  (Ex:  PeavyDeb40-01 = N 2)
$echo $SHELL      --  Displays what shell you are using
$md5sum filename  --  To obtain an MD5 checksum for the file filename
$tail -n 500 /var/log/messages  -- read the last part of a file (up to 'n' lines) of this file.
$tail -f file  -- Keeps the tail of the file displayed and updated, "f"ollowing file changes.
                  Very helpful for monitoring logs

$chsh -s bash  -- Changes your login shell to Bash  (or other shell. Shell name must be in /etc/shells)
#chsh -s bash matthew  -- Changes the login shell for account 'matthew' (if you're root, and not matthew)

#faillog -u root      -- Prints info relating to failed logins for root user
#faillog -a           -- Prints info relating to failed logins for all users

#dmesg | less         -- Prints the log of the last kernel messages, piped into less.

Network / LAMP issues:
---------------------

#/etc/init.d/networking restart    -    restart network
 On some other machines it is:
 #/etc/rc.d/init.d/network restart
 #/etc/init.d/networking restart

#samba start                         -    restart samba server
#/etc/init.d/apache2 restart         -    restart apache
#/etc/init.d/mysql restart           -    restart MySQL server

.htaccess files  -- add a .htaccess and .passwd file to password protect an HTML directory.
---------------
$htpasswd -cmb .passwd User Pass     -    Create the .passwd file with User and Pass
$htpasswd -nb User Pass              -    Just print the entry to be pasted into .passwd file

scp
---
$scp file1.txt matt@mpeavy.com:~         - Secure copy file1 to my home on mpeavy
                                           (will prompt for password if pub key not in authorized keys)
                                           Could specify another path or file name for target
$scp -r dir1 matt@mpeavy.com:~/Temp      - scp a directory [r]ecursively into ~/Temp on the remote machine.
$scp -r matt@mpeavy.com:~/Temp/dir1 ./   - scp dir1 [r]ecursively from ~/Temp to local machine CWD

----

Within less and man, type  /pattern [RET]    - where patter is a regex pattern, to search a
                                               less page or a man page.  Hit n to get to the
                                               next instance matching the pattern. When searching,
                                               typing:  ? [RET]  will begin searching upwards (reverse) 
                                               for that pattern.  Typing:  / [RET] will start down again.
                                               Or start by typing ? to search up.

----

Encrypt & decrypt a file test.txt into a file test.txt.enc using openssl:
 [Note -d switch to decrypt]

$openssl aes-256-cbc -in test.txt -out test.txt.enc
$openssl aes-256-cbc -d -in test.txt.enc -out test.txt    

----

In order to force a fsck on the next boot, put an empty file named:
forcefsck   in the root directory. Should display a message on
shutdown that an fsck will occur on the next boot.

-----

SMART - used to detect harddrive failures before they occur

  NOTE:  Must have the SMART monitoring tools installed:
  # apt-get install smartmontools

Run a few tests:

# smartctl -d ata -a /dev/sdb

or:

# smartctl -t short /devsdb

View the output

# smartctl -d ata -l selftest /dev/sdb

Also, use fsck.  Be sure to unmount first!  (or remove from a RAID first).

# e2fsck /dev/sdb1

-----

Replacing a RAID 1 (mirrored) HD

http://www.howtoforge.com/replacing_hard_disks_in_a_raid1_array

sdb and sdc make up md0 software raided partition
Both sdb and sdc have a single partition (sdb1, sdc1) that occupie the entire HD.
sdb is in the process of failing and needs to be replaced (according to SMART mon tool).

Check status of raid:
#cat /proc/mdstat

If the parition is failing (but not yet failed) it may report things as fine.

Instruct mdamd that sdb is failing and to remove it:
#mdadm --manage /dev/md0 --fail /dev/sdb1
#mdadm --manage /dev/md0 --remove /dev/sdb1

It's a good idea to get the drive serial number by running smartctl.
Then you can check after removing the failed drive that you have
 removed the correct drive.

Physically replace the drive

After rebooting, recheck the drives with smartctl to verify they both pass.

Create the same partition on sdb that is already on sdc, and check that
sdb and sdc are identically partitioned:

#sfdisk -d /dev/sdc | sfdisk /dev/sdb
#fdisk -l

Re-add sdb to md0:

#mdadm --manage /dev/md0 --add /dev/sdb1

Watch the contents of sdc being copied back onto sdb by:

#cat /proc/mdstat

-----

When ssh'ing to another terminal, press ~ and then [CTRL] z
  This will escape (or temporarily put) the ssh command in the background.
  Perform your local commands, and then type:  fg 1   to get back.

ssh -YC user@domain   -  Enables X11 forwarding, and is compressed.

-----

Find the distribution name / version
To find the name or version number of the currently installed Linux distro,
search for a file in /etc with the name "release" in it.  It may be:
/etc/lsb-release  or something similar.  This is apparently not standardized.
                    
© 2019  Give Me Fish, LLC