Oft used Bash and Linux / Unix / Cygwin commands
================================================
In general, commands are show with the $ preceeding to indicate
that the command can be issued as a non-elevated user.
If the command is preceeded by #, then it must (in general) be
issued by root.
Groups and permissions
----------------------
UUUGGGOOO = User Group Others
rwxrwxrwx
#groupadd testgroup -- Adds a group called 'testgroup'
#usermod -aG testgroup matthew -- Adds the user 'matthew' to 'testgroup'
Doesn't modify user other than adding to[G]roup
Adds the user as a member to supplmental group(s)
but doesn't erase any current group affiliation ([a]ppends).
Doesn't change user's login group affiliation.
#usermod -g testgroup matthew -- makes matthew's login group testgroup.
$chgrp groupToChangeTo fileToChange file2ToChange -- changes a file (or files, in this case) to a
new group.
Must be the owner of the file and belong to the group to which you want to change.
$groups username -- lists all groups to which 'username' belongs
$less /etc/group -- read the contents of the group file, which contains all the group info
[i.e., what users belong to what groups.]
$chmod ugo+rwx fileName Change access modifications for [u]ser, [g]roup, [o]ther
to add [r]ead, [w]rite, and e[x]ecute permission.
Also, could use [a]ll to represent ugo
Use - (minus) instead of + (plus) to subtract permissions
$chmod -R o-r fileName [R]ecursively apply permissions
$chmod o-r $(find * -type d) Removes the read permission for [o]thers for all directories
$id - prints your user and group ids
$id username - prints for the username
Add a user name "newuser1", giving him a home dir and a password: NewPassword
#useradd -d/home/newuser1 -m -pNewPassword newuser1
Removes the user "olduser", (r)emoving user's home dir
#userdel -r olduser
Redirect Output
---------------
$Program > out.txt Redirects std output to file out.txt
$Program 1>out.txt 2>err.txt Redirects stdout to out.txt and stderr to err.txt
$Program 1>>out.txt 2>>err.txt Appends stdout to out.txt and stderr to err.txt
$ls | tee out.txt Prints ls to std output and also redirects to out.txt file
$cmd 2>&1 Places stdout and stderr on 1 (std out)
$cmd 1>&2 Places stdout and stderr on 2 (std error)
Locate
------
#updatedb Updates the locate database (often done as a cronjob)
$updatedb --localpaths='/cygdrive/d' Includes the d drive in locate database.
$locate -i FileName Searches for FileName *or* filename, [i]gnoring case.
$locate -r regexString Searches using [r]egex
$locate -b '\ls' Search for only / exactly the name: ls
$rm -rf DirectoryToRemove Removes a directory, [r]ecursively (all files and sub-dirs,
through all sub-dirs) and [f]orced (without warning prompts).
rmdir only works for empty dirs.
$rm $(find -name "*.pch") Remove all the files found containing the string .pch by using find.
Note that find searches recursively.
$rm -rf $(find -name ".svn") Remove all the subversion dirs recursively.
X Windowing
-----------
$startx & starts the X server
$xterm& starts a new X terminal
$grep grepString fileName Searches for "grepString" within the file fileName
$grep -r grepString * | less Search [r]ecursively for reg exp. grepString through all files (*) and
pipe into less. Is case sensitive. Use \ to include special
meta-characters as normal characters in search (e.g. \* will include
* as a normal character).
$grep -i ... Makes grep case insensitive.
$grep -n ... Adds line numbers in the file where a match was found
$grep -I ... Skips searching of binary files.
In a grep expression, the char . means "any character". The char * denotes "any number of the
preceeding value". So the char combo .* means "any number of any characters".
$grep -rin isChar $(find -name *.cpp) | less Searches [r]ecursively, case [i]nsensitively, adding
line [n]umbers for matches to "isChar" found in any
file that ends in .cpp, piped into less.
wget $(grep -Po http.*?mp3 index.html) wget all the files that are returned from a search of the
(locally saved) index.html file for all matches to http address
that are mp3 files. Lazy (or minimum) match, enforced by ?
guarantees that this is the shortest match starting with http.
[o] option just copies out the pattern match (not the whole line, as is default).
[P] option is for Perl-style regex, which seems to be necessary for the lazy ?
Use this, for example, when you have
an html that has a lot of links you wish to download, but the http
directory restricts reading
file contents (which restricts wget from being able to download all files)
Could also use -nc for wget, so that files are not downloaded multiple times.
wget -r --html-extension --convert-links http://server/site/ Downloads and converts a site (could be from a local
Apache server) [r]ecrusively and converting any
non-html pages (such as PHP pages) into .html extension pages, including
links within and between those pages. So you can turn a PHP-based site
into a static purely html based site.
$diff file1 file2 Diffs two files, prints output
$diff file1 file2 > diffs.txt Diffs two files, redirects output to file diffs.txt
$diff -r --brief Dir1 Dir2 | less Differences all files in Dir1 vs Dir2 and reports only file names [--brief].
Searches [r]ecursively.
$diff -r --brief -x .svn Dir1 Dir2 | less Same as above, but also skips .svn directories
$diff -r --brief -x .svn -x *~ -x *.a -x CMakeFiles Dir1 Dir2 Exclude common programming-related dirs/files.
Find
----
$find [directory-list] [expression(s)]
If directory-list is omitted, searches cwd.
Within the find expression, special characters must be quoted so that the shell does not interpret them, but
passes them on to the find utility. Special chars = (), [], ?, *
Within criteria:
AND = space
OR = -or
NOT = !
$find . -name "frag*" Searches for files recursively in CWD for any file containing the
fragment "frag".
$find ~/Docs -name "frag*" Searches for files recursively in home/Docs for any file containing
the fragment "frag".
$find . -name "*frag*" searches for "frag" also in the middle of text (ie, defraged)
$find . -name *.h -mtime -45 finds all files ending in ".h" modified within the time of 45 days ago
$find . -name *.h -mtime -45 -or -name *.cpp -mtime -45 finds all files ending in .h or .cpp
modified within the last 45 days
$find . -mtime -10 \( -name "*.h" -or -name "*.cpp" \) equivalent to the previour find, but groups the
naming using () -- must backslash quote special
chars so they are not interpretted by Bash.
$find -mtime -10 ! -name "*.obj" finds all files modified in the last ten days that
are not object files.
$find $(locate syslog) -mtime -1 Locate all syslog files and then find the ones that have
been modified within the last day.
$find . -regex '.*mp[1-4]' Finds from cwd, using regex, all mp3 / mp4 files
Tar and Zip
-----------
$unzip file.zip Unzips in current working directory.
$unzip file.zip -d TargetDir Unzips to TargetDir
$bunzip2 file.bz2 Unzips from a bz2 file
$zip -r zipFileName.zip SourcePathOrFileName Zips [r]ecursively
$tar -zxvf filename.tar.gz un[z]ips, e[x]tracts, issuing [v]erbal messages, of the [f]ilename given.
$tar -jxvf filename.tar.bz2 same as above, but for bz2 type compression.
$tar -zcf 2005_FinancialStatements.tar.gz 2005* [z]ips, [c]reates a [f]ile of all docs starting
with 2005
Misc
----
$cp -i sourceFile DestinationFile copies [i]nteractively, so warns if overwriting existing
DestinationFile
$lpr FileName Prints file to printer
$mc Start midnight commander (must be installed)
$echo $? Return the exit status of the last command
$cp --reply=no file.txt dest Force ignore of question regarding previously existing
file of same name.
$uname -a -- Find out what kernel you're running
#eject -r -- Eject a CD. Must be SU.
$bind -P -- To find out Bash command-line functions
cd - -- Returns you to the directory you were previously in
Use mouse to highlight text, then center click - Copies the highlighted text to the CL
Variables
---------
env Print all the global environment variables
set Print all environment and local (non-exported) variables
$Prog=$(pwd)/httpdocs/accounts_state/Programming Sets a variable called Prog to be the current
working directory $(pwd) + a path:
du and df
---------
$du Disk usage - proivdes a summary of every subdirectory's disk usage (all
subdir's, recursively). Values are in kb.
$du -s Shows only the current directory total size
$du -s * Provides a [s]ummary of the disk space usage for the
current directory and all (*) level 1 subdirectories.
$du -s fileName pvoides a summary of fileName's disk usage (fileName
could be a directory).
$df shows how much space is used and free for each mount
$df -T Shows volume (disk) type
History and fc
--------------
$history prints a list of the previous cl commands with numbers
$!31 executes cl #31 in the history list (substitute any # for 31)
Set and export FCEDIT to emacs, declare -x FCEDIT=/usr/bin/emacs
$fc edit / reexecute previous command in an editor.
$fc -l lists the last 16 history commands to cout
$fc 31 edits cl #31 in editor. Will execute command when simply
exiting emacs (be careful!)
$fc 31 36 edits cl #31 to 36 in editor. Will execute ALL 6 commands on exit.
Can create shell scripts this way (save as, c-x c-w).
Links
-----
$ln -s pathToTarget nameOfLink
$ln -s ../Graphics Graphics Create a symbolic called "Graphics" which links to the
directory ../Graphics.
$ps List process that your terminal controls
$ps -x List all process for you (x=not necessarily attached to terminal)
$ps -ax List all processes in the system (a=all users)
$top Lists a continuously updated list of process info (PID, user, %CPU, %Mem, etc).
While running top, sort by P for CPU, M for memory, S for process run time
Press i to toggle between displaying idle processes or not.
$jobs Prints a list of currently running jobs and their job numbers
$kill pid -- Kills a process (with process number 'pid') using TERM
$kill -9 pid -- Execute that process (stronger kill than with TERM)
$kill %jobNum -- Kills a job
$ [ctrl-x][ctrl-a] -- Starts nano with whatever is on the command line. Can then edit and execute
that command (or more than one, one on each line) by exiting and writing out.
See fc for other interactive command line editing.
$mount -- To see currently mounted devices
#mount -o loop AnImage.iso /mnt/AMountPoint/ -- Mount an ISO image as if it were a disk
#blkid /dev/sdb1 -- See the properites of the device (including file type of unmounted devices)
$less /etc/mtab -- To see current mount table
#umount /dev/sdb1 -- Unmount the device sdb1 (eg, a USB device in /dev)
$/sbin/runlevel -- Check your run level (Ex: PeavyDeb40-01 = N 2)
$echo $SHELL -- Displays what shell you are using
$md5sum filename -- To obtain an MD5 checksum for the file filename
$tail -n 500 /var/log/messages -- read the last part of a file (up to 'n' lines) of this file.
$tail -f file -- Keeps the tail of the file displayed and updated, "f"ollowing file changes.
Very helpful for monitoring logs
$chsh -s bash -- Changes your login shell to Bash (or other shell. Shell name must be in /etc/shells)
#chsh -s bash matthew -- Changes the login shell for account 'matthew' (if you're root, and not matthew)
#faillog -u root -- Prints info relating to failed logins for root user
#faillog -a -- Prints info relating to failed logins for all users
#dmesg | less -- Prints the log of the last kernel messages, piped into less.
Network / LAMP issues:
---------------------
#/etc/init.d/networking restart - restart network
On some other machines it is:
#/etc/rc.d/init.d/network restart
#/etc/init.d/networking restart
#samba start - restart samba server
#/etc/init.d/apache2 restart - restart apache
#/etc/init.d/mysql restart - restart MySQL server
.htaccess files -- add a .htaccess and .passwd file to password protect an HTML directory.
---------------
$htpasswd -cmb .passwd User Pass - Create the .passwd file with User and Pass
$htpasswd -nb User Pass - Just print the entry to be pasted into .passwd file
scp
---
$scp file1.txt matt@mpeavy.com:~ - Secure copy file1 to my home on mpeavy
(will prompt for password if pub key not in authorized keys)
Could specify another path or file name for target
$scp -r dir1 matt@mpeavy.com:~/Temp - scp a directory [r]ecursively into ~/Temp on the remote machine.
$scp -r matt@mpeavy.com:~/Temp/dir1 ./ - scp dir1 [r]ecursively from ~/Temp to local machine CWD
----
Within less and man, type /pattern [RET] - where patter is a regex pattern, to search a
less page or a man page. Hit n to get to the
next instance matching the pattern. When searching,
typing: ? [RET] will begin searching upwards (reverse)
for that pattern. Typing: / [RET] will start down again.
Or start by typing ? to search up.
----
Encrypt & decrypt a file test.txt into a file test.txt.enc using openssl:
[Note -d switch to decrypt]
$openssl aes-256-cbc -in test.txt -out test.txt.enc
$openssl aes-256-cbc -d -in test.txt.enc -out test.txt
----
In order to force a fsck on the next boot, put an empty file named:
forcefsck in the root directory. Should display a message on
shutdown that an fsck will occur on the next boot.
-----
SMART - used to detect harddrive failures before they occur
NOTE: Must have the SMART monitoring tools installed:
# apt-get install smartmontools
Run a few tests:
# smartctl -d ata -a /dev/sdb
or:
# smartctl -t short /devsdb
View the output
# smartctl -d ata -l selftest /dev/sdb
Also, use fsck. Be sure to unmount first! (or remove from a RAID first).
# e2fsck /dev/sdb1
-----
Replacing a RAID 1 (mirrored) HD
http://www.howtoforge.com/replacing_hard_disks_in_a_raid1_array
sdb and sdc make up md0 software raided partition
Both sdb and sdc have a single partition (sdb1, sdc1) that occupie the entire HD.
sdb is in the process of failing and needs to be replaced (according to SMART mon tool).
Check status of raid:
#cat /proc/mdstat
If the parition is failing (but not yet failed) it may report things as fine.
Instruct mdamd that sdb is failing and to remove it:
#mdadm --manage /dev/md0 --fail /dev/sdb1
#mdadm --manage /dev/md0 --remove /dev/sdb1
It's a good idea to get the drive serial number by running smartctl.
Then you can check after removing the failed drive that you have
removed the correct drive.
Physically replace the drive
After rebooting, recheck the drives with smartctl to verify they both pass.
Create the same partition on sdb that is already on sdc, and check that
sdb and sdc are identically partitioned:
#sfdisk -d /dev/sdc | sfdisk /dev/sdb
#fdisk -l
Re-add sdb to md0:
#mdadm --manage /dev/md0 --add /dev/sdb1
Watch the contents of sdc being copied back onto sdb by:
#cat /proc/mdstat
-----
When ssh'ing to another terminal, press ~ and then [CTRL] z
This will escape (or temporarily put) the ssh command in the background.
Perform your local commands, and then type: fg 1 to get back.
ssh -YC user@domain - Enables X11 forwarding, and is compressed.
-----
Find the distribution name / version
To find the name or version number of the currently installed Linux distro,
search for a file in /etc with the name "release" in it. It may be:
/etc/lsb-release or something similar. This is apparently not standardized.
|