====== Notes to be organized to Wiki Pages later ====== ---- ===== Common Commands, nagivating the file system ===== ---- * ls -- list dir contents * ls -al -- list all dir contents, in long format. * ls -F -- list contents, dirs noted with / char. BEAUTIFUL! * mkdir //mydir// -- create dir mydir * cd dir -- change directory to dir * pwd -- print working dir * mv file1 file2 -- move/rename a file from file1 to file2 * tail myfile -- read last few lines of myfile * head myfile -- reas first few lines of myfile * cat myfile -- concat files, otherwise displays myfile contents to stout * rm myfile -- removes myfile * cd .. -- cd's one level up(or back) in file structure. * cd , cd ~ -- quickly cd to home dir * mv mydir1 mydir2 -- moves directories also * rmdir -- remove empty dirs * rm -r -- remove dir recursively (Kills all subdirs) * touch myfile -- creates empty file * cp -r mydir mydir2 -- copies directory recursively (all subdirs) * services sshd restart -- restarted sshd service on X distribution (WHICH ONE?!?!) * less log.file -- page through log.file * f key moves forward a page * b key moves back * / to search * ? to search from bottom * :20 to goto line 20 * G & g to goto bottom and top of document * tee -- used in pipes to display stdout as well as output to a file: echo $PATH | tee path.txt * ls | xargs rm -- run rm command on each line outputted from ls. xargs is amazzzing! * rm $(find ./ -user Christine -- rm all files listed from the find command in parens. Amazzzinging! * expand //file// -- converts tabs to spaces in file //file//. * unexpand //file// -- converts spaces to tabs in //file//. * od //file.txt// -- display file in octal format. Can display other formats, consult man. * join //file1// //file2// -- joins lines together based on a key column. * paste //file1// //file2// -- merge lines in file1 and file2 * sort -k 3 //file.txt// -- Sort //file.txt// by field 3 (default is 1). * tr BCJ bc > listing.txt -- translate all BCJ chars to bc. -d flag deletes chars entirely in //set1//. * sort shakespeare.txt | uniq -- uniq removes duplicate lines from //shakespeare.txt// * fmt //long.txt// -- format line length and etc in //long.txt// ===== Important Places ===== ---- * /etc = main config dir * /var = holds files that freq change (LOGS!) ===== ls long output ===== ---- Files preceeded by a . are hidden/config files and won't show unless -a flag is used in ls cmd. ls readout: **perms, ?, user, group, file, last touched, name.** first perm bit 'd' means it's a dir. ===== Regular Expressions ===== ---- ===Any Single Character=== A dot will match any single char except newlines: a.z will match abz aQz a9Z and etc. ===Repetition Operators=== A regex may be followed by a special symbol to denote how many times a matching item must exist. ** = zero or more occurrences *+ = one more more occurrences *? = zero or one match ===Multiple Possible Strings:=== Vertical Bar (|) separates two possible matches: * //car|truck// matches either //car// **OR** //truck//. ===Parentheses=== Surrounds subexpressions (no example in chapter 1) ===Escaping=== If you want to match special chars like a dot (.) you must escape it with a backslask: \. (Not unlike bash and python) =====grep===== ---- Searches for patterns: //grep// [options] //regexp// [files] flags: *-c # Instead of displaying lines, it displays the count. *-f # Takes pattern input from a file and not stdin *-i # Ignore Case *-r or rgrep # Search recursively. *-F or --fixed-strings or fgrep # Searches for literal translation, special symbols like $ are now literally printed as $ and not interpreted. *-E or --extended-regexp or egrep # Uses extended regexp as default =====sed===== ---- sed 's/2012/2013/' //file.txt// > //new.txt// # Change first occurrence of 2012 to 2013 and redirect output to new.txt =====File Permissions===== ---- ====ls==== ---- * $ ls -la: * -rw-r--r-- 1 abrer abrer 255 Jan 31 18:37 TeamSpeak3CABDO-ID.ini * TYPE OWNER GROUP EVERYONE/WORLD, OWNER, GROUP, SIZE, DATE Moded, TIME moded, NAME *TYPE bit: *- = file *d = directory *Perm #'s: *r = 4 *w = 2 *x = 1 Can change default permissions with //umask//. ====chmod==== ---- numeric or symbolic modes. *numeric = chmod 755 * 4 = read * 2 = write * 1 = execute * dirs need x to execute *symbolic = chmod g+x myfile *chmod u+rwx,g+rwx,o-rw myfile ***u** = user/owner ***g** = group ***o** = other/world/guest ====Set user to group==== ***usermod -g mygroup jeff** * Sets group mygroup for jeff ***user -a -G mygroup jeff** * Appends group mygroup to jeff's existing groups. ====chown==== ---- *chown -R jeff.jeff myfile *chown -R user:mygroup myfile : or . works as seperator =====Cron===== ---- Scheduled task system. Stateless, doesn't remember previous jobs. *cron.deny: Users in this list are denied use of cron *cron.allow: If this file exists, everyone is denied unless specified here. *crontab: The schedule file. Sample Syntax: SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root # For details see man 4 crontabs # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed */2 1 1,15 * * root echo "Hello world!" > /tmp/hello.txt # 1st and 15th day of month # /2 = every 2 # 1,15 = 1st and 15th # 1-5 = 1st through 5th Otherwise, symlinking scripts/programs to cron.daily,monthly, weekly, hourly, etc folders will work. ===user cron=== crontab -e to edit user cron, independent from system cron. *USER CRONS stored in /var/spool/cron *tail -f /var/log/cron =====Package Management===== ---- *rpm/yum - Redhat, Centos, Fedora, Suse *apt-get/dpkg - Debian, Ubuntu, etc ====CENTOS==== *Uses yum/rpm. *Repos listed in **/etc/yum.repos.d/** ====rpm==== *-ivh //packagefile// - Installs //packagefile// with a series of hashes as indicators. *-uvh - Same as -ive but upgrades a a package if already installed? *-q //pkgname// - Queries to see if //packagename// is installed. *-e //pkgname// - remove //pkgname// from system. ====yum==== *yum makecache fast - updates local cache of repo contents. Similar to Pacman -Syy *yum search //name// - searches repos for pkgs named //name// *yum install //httpd// - installs //httpd// and deps from repos. *yum update -- updates all packages on system. =====UBUNTU===== ---- uses apt-get / dpkg ====apt==== Config file located in ///etc/apt//. sources.list important. sources.list.d folder contains official repos. *apt-get update -- updates repo list. DOES NOT UPGRADE. *apt-cache search //thing// -- searches packages on local package cache for //thing// to download from server. *apt-get install //apache2// - install apache2 + deps from repos. *apt-get upgrade -- upgrades out of date packages. ======top====== ---- top stuff. *Sys time, sys uptime, users logged in, load avg (cpu utilization .90 = 90%): 1m, 5m, 15m *CPU *Mem *Swap *shift M/N? - sort by mem *shift P - sort by cpu *NI = nice val. nice vals go from -20 to 19. 19 = LOWEST PRI *R = renice value - PID - renice val *K = kill PID [15 = term(inate), kill graceful] [9 = kill, kill forcefully] ======Managing Shared Libraries====== ---- Code/functions compiled into files called libraries that other programs/code can call on, reducing code and disk space by not having 50 variations of code that do the same thing. *Where are libs stored? *End in .so extension (prly SHARED OBJECT). *Most commonly stored in **/usr/lib** * if on 64bit system, may be stored in /usr/lib64 * listing files could be libjpeg.so.62 & libjpeg.so.62.1.0 * Instead of changing name with each revision, new version is installed and the link is updated to point to new library file. *Library daemon manages most libraries. Called LD. * ldconfig -v THIS SHOWS ALL LIBS ON SYSTEM * ldd shows lib requirements by program. ldd gimp * /etc/ld.so.conf = lib database * IF ADDING LIB MANUALLY * place lib manually in location like /usr/lib or lib64. * Create link to lib. * ln TARGET NAMEOFLINK * ln ./libdinlibrary.so.5.1.2 ./libdonlibrary.so.5 * HARD LINKS POINT TO RAW DATA ON DISK * (Can only link to data on same partition) * SOFT LINKS POINT TO REFERENCE OF DATA ON DISK (File) * But moving target breaks the link * If adding lib to non conventional location, have to add include statement in /etc/ld.so.conf! ======Managing Processes====== ---- *ps -a list all processes on system * First column = PID. 2nd Column = location. * ctrl-z background a running process * ps alone lists your processes * jobs lists running jobs * restore by fg job# * OR %job# * PROGRAM & will background the program and keep it running after you end the shell session. * kill PID to end process ID * renice # = change niceness of program. 20 = lowest pri, -20 = highest pri. * killall -i(interactive) stress (kills all instances of stress * pgrep greps ps -A automatically. ======Managing Hardware====== ---- *lsmod - list loaded modules *rmmod - remove module *modprobe - load a module *udev maps devices, rules in **/etc/udev/rules.d** OR **/lib/udev/rules.d** *lspci - list pci hardware reported by OS (doesn't mean MODULE is loaded!) *lsusb - list USB devices ===BIOS=== ---- Enumerates hardware before OS is loaded. Drivers will determine how hardware acts in Linux OS. May need to drownload drivers from manufacturers for optimal performance. ===Drivers=== ---- drivers loaded in /sys/bus/pci/drivers. ===probs=== ---- If problems present, check bios. Check drivers! Check lsmod, lspci, lsusb. use modprobe to load modules. Check with the vendor. ====REDIRECT VIDEO TO SERIAL! - Real World examples ==== ---- Not on exam, but steps demo underlying system that are on exam. *edit /boot/grub/grub.conf - config file for bootloader. * comment out splash image line. (Disables splash image) * kernel line, add output to serial: console=tty0 console=ttyS1,9600n8 serial --unit 1 --speed=9600 terminal --timeout 300 console serial * create new ttyS1 we added in grub.conf. * vi /etc/securetty * add ttyS1 to list on new line. * vi /etc/init/ttyS1.conf - to edit behavior: * start on runlevel [1345] * stop on runlevel [S06] * lvl1 = single user * 3,4,5 = multiuser. 3 = cmd line, 5 = gui 4= ?? * S = shutdown, 0 = halt * respawn * instance /dev/ttyS1 * exec /sbin/agetty ttyS1 9600 vt100-nav -will generate tty interface * /etc/inittab shows good info on runlevels. Changing runlevel here will make it default runlevel. ====LVM - Logical Volume Manager==== ---- Logicall manages volumes/partitions, create and resize across multiple disks. *Traditional storage devices look like /dev/sda,b,c,d,etc *LVM sorts volumes into directories by volume group ===Configure LVM=== *yum install lvm2 *Create the physical volumes * pvcreate /dev/sdb /dev/sdc - imports sdb and sdc into lvm. * Create Volume Group (To hold partitions) * vgcreate --help * vgcreate //vg1// /dev/sdb /dev/sdc - vg1 is name of volume group, change to w/e you want. * Check your work. * pvdisplay * vgdisplay * Create logical volumes (Partitions) * lvcreate -L 20G vg1 -n Marketing * Reflected in /dev/vg1 (vg1 folder created when volume group created. * Need to be formatted and mounted. These are now equiv to devs like /dev/sda1. * mkfs -t ext4 /dev/vg1/Marketing * can now be mounted * Resize LV * lvresize -L +2G /dev/vg1/Marketing * drive was increased in size. Need to expand filesystem to match. * resize2fs /dev/vg1/Marketing - no flags extends to max disk size. * Add PVs to volume group. * vgextend vg1 /dev/sdd * extends volumegroup to device /dev/sdd ====Filesystem Health==== ---- ===/etc/fstab=== Mounts devs on boot. device mount FS Options - Dump - Pass(fschk) /dev/sda1 / ext4 defaults 0|1 0|1|2 device = device mount = mount location FS = filesystem to expect Options = boot options (defaults is a combo of options). Like, rw, ro, sync, async, etc. Dump = old unix backup system. If dump not installed or used, it means nothing. 0 no bkup, 1 = bkup. Pass = FSCHK, 0 no check, 1 check (high pri), 2 check (2ndary pri) *Not using GPT, can use **lsblk** cmd. * Awesome cmd! Shows devices, mountpoint. * **-f** shows label and UUID. * **df -h** * kinda like lsblk. * **blkid** * shows similar info but for use with GPT. ===add label to device/part=== ***tune2fs** -L Root /dev/sda3 ======File Management Commands 1 & 2====== ---- *whoami - print username *pwd - print present working directory *ls - ls contents of dir *ls -F - puts a slash on directories (handy!) *tar * -c -create tar volume * -v -verbose output! * -f -filename going to create * -z -use gzip on resulting tar file. * -x -extract tar. *cpio * find ./file* | cpio -o | gzip > backup.cpio.gz * -o = create or --create * gunzip backup.cpio.gz * unzip the cpio gzip archive we made. * dd for disk duplication/cloning/imaging * dd if=/dev/vg1/Sales of=~/sales.img * Reverse the order to restore the img. ======System D====== ---- An alternative init system -- (usual is upstart, sysV) Breaks up config files. Is a program, running all the time. SysV = script, one line/procedural at a time. hangup on one line can cause delays. SystemD, one hung up module won't block all others. *If /sbin/init = symlink to systemd = systemd *If /sbin/init = executable = SysV/Upstart SystemD on boot looks for UNITS and TARGETS. *unit = various thing on system that can happen at anytime. *TARGET = groups of units. * Calls are units at same time (might be dependencies) *Config files are in /lib/systemd * system dir contains units / targets. * .service = a unit * httpd.service = a unit. * runlevel0.target = symlink, emulates old runlevel behavior. *httpd.service NOTES: * [Unit] * Description- Desc of the file * After=network.target remote-fs.target -- don't run httpd.service until AFTER these targets are started. * Wants=samedepends -- means if one arg here stopped running, httpd.service will continue to run. * [Service] * Type=notify - lets us know if it worked, didn't work. * Environment=?? no idea * ExecStart=/usr/sbin/httpd $OPTIONS -DFOREFROUND (this is how it's started, AKA systemctm start httpd.service) * ExecReload=/usr/sbin/httpd $OPTIONS -k graceful * ExecStop=/bin/kill -WINCH ${MAINPID} *Control systemd by: * systemctl start httpd * systemctl stop httpd * service httpd start now redirects to systemd * in /usr/lib/systemd/system dir: * default.target = symlink to graphical * Sets defaults of system (like runlevels) * can change default.target link to cmdline instead of graphical for terminal boot. ======Xorg Configuration====== ---- *Xorg -configure -- Runs xorg config to generate generic xorg conf to get X running again! ====== Find and Locate ====== Both used to search for files. //locate// uses a database of information to search and is updated via a syscron or manual updatedb. //find// will search the disk in specified locations for files and objects. *locate motd * searches for motd. * may not be installed by default: apt-get install locate, yum install mlocate * find /etc -name motd * search in /etc for files containing motd. ====== VIM / VI Text Editor ====== Light, installed on most machines. *CMD MODE * hjkl, move L R U D * 0, SHIFT+0 - move to beginning or END of line. * L - Move to last line of TERMINAL, not DOC * H - move me to top of term. * G - move to last line in file * 1G - moves to first line of file. * 40G - goto line 40 * :40 - also goto line 40 * yy - yank/cp line * p - paste * P - paste on line before the cursor * u - undo * 5yy - yank 5 lines * / - search for text * n - continue forward in search * N - go backwards in search * dd - cut line * 5dd - cut 5 lines * ? - search * cc = del line + goto insert mode * cw = change word, deletes word and puts you in INSERT mode! Awesome! * * EX MODE * :%s/HostKey/wahoo - will change first ocurrence of HostKey with wahoo on each line. * %s/test/wahoo * %s/test/wahoo/g - change ALLLLL occurrences (thanks /g!) * :q! - quit * :wq - write quit * :w - write * :r path/to/file - loads contents of that file INTO curr file. * :!shellCMDhere ====== sed Stream Editor ====== *change first occurrence of parttime to promotion. * sed '0,/parttime/s/parttime/promotion' employees.txt * change first occur of parttime to fulltime * sed 's/parttime/fulltime' employees.txt * change ALL occurrences of parttime to fulltime * sed 's/parttime/fulltime/g' employees.txt Can move all space to dots, slashes to spaces, etc etc, powerful stuff. RESUME ME, LINUX ACAD VIDEO @4:30m - LEcture, sed Stream Editor!