Abrer_Wiki

You've loaded an old revision of the document! If you save it, you will create a new version with this data.

====== Notes to be organized to Wiki Pages later ======
----

===== Common Commands, nagivating the file system =====
----

* ls -- list dir contents
  * ls -al -- list all dir contents, in long format.
  * ls -F -- list contents, dirs noted with / char. BEAUTIFUL!
  * mkdir //mydir// -- create dir mydir
  * cd dir -- change directory to dir
  * pwd -- print working dir
  * mv file1 file2 -- move/rename a file from file1 to file2
  * tail myfile -- read last few lines of myfile
  * head myfile -- reas first few lines of myfile
  * cat myfile -- concat files, otherwise displays myfile contents to stout
  * rm myfile -- removes myfile
  * cd .. -- cd's one level up(or back) in file structure.
  * cd <CR>, cd ~ -- quickly cd to home dir
  * mv mydir1 mydir2 -- moves directories also
  * rmdir -- remove empty dirs
  * rm -r -- remove dir recursively (Kills all subdirs)
  * touch myfile -- creates empty file
  * cp -r mydir mydir2 -- copies directory recursively (all subdirs)
  * services sshd restart -- restarted sshd service on X distribution (WHICH ONE?!?!)
  * less log.file -- page through log.file
    * f key moves forward a page
    * b key moves back
    * / to search
    * ? to search from bottom
    * :20 to goto line 20
    * G & g to goto bottom and top of document
  * tee -- used in pipes to display stdout as well as output to a file: echo $PATH | tee path.txt
  * ls | xargs rm -- run rm command on each line outputted from ls. xargs is amazzzing!
  * rm $(find ./ -user Christine -- rm all files listed from the find command in parens. Amazzzinging!
  * expand //file// -- converts tabs to spaces in file //file//.
  * unexpand //file// -- converts spaces to tabs in //file//.
  * od //file.txt// -- display file in octal format. Can display other formats, consult man. 
  * join //file1// //file2// -- joins lines together based on a key column.
  * paste //file1// //file2// -- merge lines in file1 and file2
  * sort -k 3 //file.txt// -- Sort //file.txt// by field 3 (default is 1).
  * tr BCJ bc > listing.txt -- translate all BCJ chars to bc. -d flag deletes chars entirely in //set1//.
  * sort shakespeare.txt | uniq -- uniq removes duplicate lines from //shakespeare.txt//
  * fmt //long.txt// -- format line length and etc in //long.txt//
===== Important Places =====
----
  * /etc = main config dir
  * /var = holds files that freq change (LOGS!)

===== ls long output =====
----

Files preceeded by a . are hidden/config files and won't show unless -a flag is used in ls cmd.

ls readout: **perms, ?, user, group, file, last touched, name.**

first perm bit 'd' means it's a dir.

===== Regular Expressions =====
----

===Any Single Character===
A dot will match any single char except newlines:    a.z will match abz aQz a9Z and etc.

===Repetition Operators===
A regex may be followed by a special symbol to denote how many times a matching item must exist.
  ** = zero or more occurrences
  *+ = one more more occurrences
  *? = zero or one match

===Multiple Possible Strings:===
Vertical Bar (|) separates two possible matches:
  * //car|truck// matches either //car// **OR** //truck//.

===Parentheses===
Surrounds subexpressions (no example in chapter 1)

===Escaping===
If you want to match special chars like a dot (.) you must escape it with a backslask:    \. (Not unlike bash and python)

=====grep=====
----
Searches for patterns:
//grep// [options] //regexp// [files]

flags:
  *-c # Instead of displaying lines, it displays the count.
  *-f # Takes pattern input from a file and not stdin
  *-i # Ignore Case
  *-r or rgrep # Search recursively.
  *-F or --fixed-strings or fgrep # Searches for literal translation, special symbols like $ are now literally printed as $ and not interpreted.
  *-E or --extended-regexp or egrep # Uses extended regexp as default

=====sed=====
----
sed 's/2012/2013/' //file.txt// > //new.txt// # Change first occurrence of 2012 to 2013 and redirect output to new.txt

=====File Permissions=====
----
====ls====
----
  * $ ls -la: 
    * -rw-r--r--  1 abrer abrer   255 Jan 31 18:37 TeamSpeak3CABDO-ID.ini
    * TYPE OWNER GROUP EVERYONE/WORLD, OWNER, GROUP, SIZE, DATE Moded, TIME moded, NAME

*TYPE bit:
    *- = file
    *d = directory
  *Perm #'s:
    *r = 4
    *w = 2
    *x = 1

Can change default permissions with //umask//.

====chmod====
----
numeric or symbolic modes.

*numeric = chmod 755
    * 4 = read
    * 2 = write
    * 1 = execute
    * dirs need x to execute

*symbolic = chmod g+x myfile
  *chmod u+rwx,g+rwx,o-rw myfile
    ***u** = user/owner
    ***g** = group
    ***o** = other/world/guest

====Set user to group====
  ***usermod -g mygroup jeff**
    * Sets group mygroup for jeff

***user -a -G mygroup jeff**
    * Appends group mygroup to jeff's existing groups.
====chown====
----
  *chown -R jeff.jeff myfile
  *chown -R user:mygroup myfile

: or . works as seperator

=====Cron=====
----
Scheduled task system. Stateless, doesn't remember previous jobs.

*cron.deny: Users in this list are denied use of cron
  *cron.allow: If this file exists, everyone is denied unless specified here.
  *crontab: The schedule file.

Sample Syntax:
<code>
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

# For details see man 4 crontabs

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name  command to be executed
*/2 1 1,15 * * root echo "Hello world!" > /tmp/hello.txt # 1st and 15th day of month

# /2 = every 2
# 1,15 = 1st and 15th
# 1-5 = 1st through 5th
</code>

Otherwise, symlinking scripts/programs to cron.daily,monthly, weekly, hourly, etc folders will work.

===user cron===
crontab -e to edit user cron, independent from system cron.

*USER CRONS stored in /var/spool/cron
  *tail -f /var/log/cron

=====Package Management=====
----
  *rpm/yum - Redhat, Centos, Fedora, Suse
  *apt-get/dpkg - Debian, Ubuntu, etc

====CENTOS====
  *Uses yum/rpm.
  *Repos listed in **/etc/yum.repos.d/**

====rpm====
  *-ivh //packagefile// - Installs //packagefile// with a series of hashes as indicators.
  *-uvh - Same as -ive but upgrades a a package if already installed?
  *-q //pkgname// - Queries to see if //packagename// is installed.
  *-e //pkgname// - remove //pkgname// from system.

====yum====
  *yum makecache fast - updates local cache of repo contents. Similar to Pacman -Syy
  *yum search //name// - searches repos for pkgs named //name//
  *yum install //httpd// - installs //httpd// and deps from repos.
  *yum update -- updates all packages on system.

=====UBUNTU=====
----
uses apt-get / dpkg

====apt====
Config file located in ///etc/apt//. sources.list important. sources.list.d folder contains official repos.
  *apt-get update -- updates repo list. DOES NOT UPGRADE.
  *apt-cache search //thing// -- searches packages on local package cache for //thing// to download from server.
  *apt-get install //apache2// - install apache2 + deps from repos.
  *apt-get upgrade -- upgrades out of date packages.
  *apt-get clean -- cleans up local pkg files, temp files, etc.
  *apt-get autoclean -- only cleans out packages that are no longer avail in our source repo. These can no longer be downloaded and are largely useless files.
======top======
----
top stuff.
  *Sys time, sys uptime, users logged in, load avg (cpu utilization .90 = 90%): 1m, 5m, 15m
  *CPU
  *Mem
  *Swap
  *shift M/N? - sort by mem
  *shift P - sort by cpu
  *NI = nice val. nice vals go from -20 to 19. 19 = LOWEST PRI
  *R = renice value - PID - renice val
  *K = kill PID [15 = term(inate), kill graceful] [9 = kill, kill forcefully]

======Managing Shared Libraries======
----
Code/functions compiled into files called libraries that other programs/code can call on, reducing code and disk space by not having 50 variations of code that do the same thing.

*Where are libs stored?
    *End in .so extension (prly SHARED OBJECT).
    *Most commonly stored in **/usr/lib**
      * if on 64bit system, may be stored in /usr/lib64
      * listing files could be libjpeg.so.62 & libjpeg.so.62.1.0
      * Instead of changing name with each revision, new version is installed and the link is updated to point to new library file.

*Library daemon manages most libraries. Called LD.
    * ldconfig -v THIS SHOWS ALL LIBS ON SYSTEM
    * ldd shows lib requirements by program. ldd gimp
    * /etc/ld.so.conf = lib database
    * IF ADDING LIB MANUALLY
      *  place lib manually in location like /usr/lib or lib64. 
        * Create link to lib.
        * ln TARGET NAMEOFLINK
        * ln ./libdinlibrary.so.5.1.2 ./libdonlibrary.so.5
        * HARD LINKS POINT TO RAW DATA ON DISK
          *  (Can only link to data on same partition)
        * SOFT LINKS POINT TO REFERENCE OF DATA ON DISK (File)
          * But moving target breaks the link
        * If adding lib to non conventional location, have to add include statement in /etc/ld.so.conf!

======Managing Processes======
----
  *ps -a list all processes on system
    * First column = PID. 2nd Column = location.
    * ctrl-z background a running process
    * ps alone lists your processes
    * jobs lists running jobs
        * restore by fg job#
        * OR %job#
    * PROGRAM & will background the program and keep it running after you end the shell session.
    * kill PID to end process ID
    * renice # = change niceness of program. 20 = lowest pri, -20 = highest pri.
    * killall -i(interactive) stress (kills all instances of stress
    * pgrep greps ps -A automatically.

======Managing Hardware======
----
  *lsmod - list loaded modules
  *rmmod - remove module
  *modprobe - load a module
  *udev maps devices, rules in **/etc/udev/rules.d** OR **/lib/udev/rules.d**
  *lspci - list pci hardware reported by OS (doesn't mean MODULE is loaded!)
  *lsusb - list USB devices

===BIOS===
----
Enumerates hardware before OS is loaded. Drivers will determine how hardware acts in Linux OS. May need to drownload drivers from manufacturers for optimal performance.

===Drivers===
----
drivers loaded in /sys/bus/pci/drivers.

===probs===
----
If problems present, check bios. Check drivers! Check lsmod, lspci, lsusb. use modprobe to load modules. Check with the vendor.

====REDIRECT VIDEO TO SERIAL! - Real World examples ====
----
Not on exam, but steps demo underlying system that are on exam.

*edit /boot/grub/grub.conf - config file for bootloader.
    * comment out splash image line. (Disables splash image)
    * kernel line, add output to serial: console=tty0 console=ttyS1,9600n8
<code>
serial --unit 1 --speed=9600
terminal --timeout 300 console serial
</code>
  * create new ttyS1 we added in grub.conf.
  * vi /etc/securetty
    * add ttyS1 to list on new line.
  * vi /etc/init/ttyS1.conf - to edit behavior:
    * start on runlevel [1345]
    * stop on runlevel [S06]
      * lvl1 = single user
      * 3,4,5 = multiuser. 3 = cmd line, 5 = gui 4= ??
      * S = shutdown, 0 = halt
    * respawn
    * instance /dev/ttyS1
    * exec /sbin/agetty  ttyS1 9600 vt100-nav -will generate tty interface
    * /etc/inittab shows good info on runlevels. Changing runlevel here will make it default runlevel.

====LVM - Logical Volume Manager====
----
Logicall manages volumes/partitions, create and resize across multiple disks.

*Traditional storage devices look like /dev/sda,b,c,d,etc
  *LVM sorts volumes into directories by volume group

===Configure LVM===
  *yum install lvm2
  *Create the physical volumes
    * pvcreate /dev/sdb /dev/sdc - imports sdb and sdc into lvm.
  * Create Volume Group (To hold partitions)
    * vgcreate --help
    * vgcreate //vg1// /dev/sdb /dev/sdc - vg1 is name of volume group, change to w/e you want.
  * Check your work.
    * pvdisplay
    * vgdisplay
  * Create logical volumes (Partitions)
    * lvcreate -L 20G vg1 -n Marketing
      * Reflected in /dev/vg1 (vg1 folder created when volume group created.
      * Need to be formatted and mounted. These are now equiv to devs like /dev/sda1.
    * mkfs -t ext4 /dev/vg1/Marketing
    * can now be mounted
    * Resize LV
      * lvresize -L +2G /dev/vg1/Marketing
        * drive was increased in size. Need to expand filesystem to match.
      * resize2fs /dev/vg1/Marketing - no flags extends to max disk size.
    * Add PVs to volume group.
      * vgextend vg1 /dev/sdd
        * extends volumegroup to device /dev/sdd

====Filesystem Health====
----
===/etc/fstab===
Mounts devs on boot.
<code>
device      mount     FS     Options -   Dump   - Pass(fschk)
/dev/sda1    /        ext4    defaults    0|1     0|1|2

device = device
mount = mount location
FS = filesystem to expect
Options = boot options (defaults is a combo of options). Like, rw, ro, sync, async, etc.
Dump = old unix backup system. If dump not installed or used, it means nothing. 0 no bkup, 1 = bkup.
Pass = FSCHK, 0 no check, 1 check (high pri), 2 check (2ndary pri)
</code>
  *Not using GPT, can use **lsblk** cmd.
    * Awesome cmd! Shows devices, mountpoint.
    * **-f** shows label and UUID.
  * **df -h**
    * kinda like lsblk.
  * **blkid**
    * shows similar info but for use with GPT.

===add label to device/part===
  ***tune2fs** -L Root /dev/sda3

======File Management Commands 1 & 2======
----

*whoami - print username
  *pwd - print present working directory
  *ls - ls contents of dir
  *ls -F - puts a slash on directories (handy!)
  *tar
    * -c -create tar volume
    * -v -verbose output!
    * -f -filename going to create
    * -z -use gzip on resulting tar file.
    * -x -extract tar.
  *cpio
    * find ./file* | cpio -o | gzip > backup.cpio.gz
    * -o = create or --create
  * gunzip backup.cpio.gz
    * unzip the cpio gzip archive we made.
  * dd for disk duplication/cloning/imaging
    * dd if=/dev/vg1/Sales of=~/sales.img 
      * Reverse the order to restore the img.

======System D======
----
An alternative init system -- (usual is upstart, sysV)
Breaks up config files.

Is a program, running all the time. SysV = script, one line/procedural at a time. hangup on one line can cause delays. SystemD, one hung up module won't block all others.

*If /sbin/init = symlink to systemd = systemd
  *If /sbin/init = executable = SysV/Upstart

SystemD on boot looks for UNITS and TARGETS.
  *unit = various thing on system that can happen at anytime.
  *TARGET = groups of units.
    * Calls are units at same time (might be dependencies)

*Config files are in /lib/systemd
    * system dir contains units / targets.
    * .service = a unit
    * httpd.service = a unit.
    * runlevel0.target = symlink, emulates old runlevel behavior.

*httpd.service NOTES:
    * [Unit]
    * Description- Desc of the file
    * After=network.target remote-fs.target -- don't run httpd.service until AFTER these targets are started.
    * Wants=samedepends -- means if one arg here stopped running, httpd.service will continue to run.
    * [Service]
    * Type=notify   - lets us know if it worked, didn't work.
    * Environment=?? no idea
    * ExecStart=/usr/sbin/httpd $OPTIONS -DFOREFROUND (this is how it's started, AKA systemctm start httpd.service)
    * ExecReload=/usr/sbin/httpd $OPTIONS -k graceful
    * ExecStop=/bin/kill -WINCH ${MAINPID}

*Control systemd by:
    * systemctl start httpd
    * systemctl stop httpd
    * service httpd start now redirects to systemd

* in /usr/lib/systemd/system dir:
    * default.target = symlink to graphical
      * Sets defaults of system (like runlevels)
      * can change default.target link to cmdline instead of graphical for terminal boot.

======Xorg Configuration======
----
  *Xorg -configure  -- Runs xorg config to generate generic xorg conf to get X running again!

====== Find and Locate ======
Both used to search for files. //locate// uses a database of information to search and is updated via a syscron or manual updatedb. //find// will search the disk in specified locations for files and objects.

*locate motd
    * searches for motd.
    * may not be installed by default: apt-get install locate, yum install mlocate 
  * find /etc -name motd
    * search in /etc for files containing motd.

====== VIM / VI Text Editor ======
Light, installed on most machines.

*CMD MODE
    * hjkl, move L R U D
    * 0, SHIFT+0 - move to beginning or END of line.
    * L - Move to last line of TERMINAL, not DOC
    * H - move me to top of term.
    * G - move to last line in file
    * 1G - moves to first line of file.
    * 40G - goto line 40
    * :40 - also goto line 40
    * yy - yank/cp line
    * p - paste
    * P - paste on line before the cursor 
    * u - undo
    * 5yy - yank 5 lines
    * / - search for text
    * n - continue forward in search
    * N - go backwards in search
    * dd - cut line
    * 5dd - cut 5 lines
    * ? - search
    * cc = del line + goto insert mode
    * cw = change word, deletes word and puts you in INSERT mode! Awesome!
    * 
    * EX MODE
    * :%s/HostKey/wahoo - will change first ocurrence of HostKey with wahoo on each line.
    * %s/test/wahoo
    * %s/test/wahoo/g - change ALLLLL occurrences (thanks /g!)
    * :q! - quit
    * :wq - write quit
    * :w - write
    * :r path/to/file - loads contents of that file INTO curr file.
    * :!shellCMDhere

====== sed Stream Editor ======

*change first occurrence of parttime to promotion.
    * sed '0,/parttime/s/parttime/promotion' employees.txt
  * change first occur of parttime to fulltime
    * sed 's/parttime/fulltime' employees.txt
  * change ALL occurrences of parttime to fulltime
    * sed 's/parttime/fulltime/g' employees.txt
Can move all space to dots, slashes to spaces, etc etc, powerful stuff.
RESUME ME, LINUX ACAD VIDEO @4:30m - LEcture, sed Stream Editor!

Edit summary

Note: By editing this page you agree to license your content under the following license: CC Attribution-Share Alike 4.0 International

Abrer_Wiki

User Tools

Site Tools

Page Tools