Run command as another user

In two ways

su deployer -c 'whoami' can add login -l option su -l deployer -c 'rvm list' so PATH is extended with ~/.rvm/bin/ and rvm script works
sudo -u deployer whoami simpler, no need for quotes
- Add option -i or --login to run as login shell. In login shell ~ is pointing to /home/deployer/ not /root/ home.
- Wrap command inside bash to properly load XDG_RUNTIME_DIR for rails c or other ruby commands: sudo -i -u deployer /bin/bash -c "cd /vagrant && rails c". Also when you use pipe | or && or redirection >>, you should wrap inside bash or use eval

You can find all login shells as they start with hyphen minus sign in ps -f. Interactive shell is one which reads commands from it’s standard input. When you ssh than that session is both login and interactive.

ssh trk
ps -f
# this is login shell
dule .... -bash

# when you type bash you get another interactive (but not login shell)
bash
ps -f
dule ... -bash
dule ... bash

# when you run scripts than you do not have interactive shell

Login shell is running /etc/profile and ~/.bash_profile, ~/.bash_login and ~/.profile (first found file is sourced, others are ignored)

Interactive shell is running ~/.bashrc I usually source ~/.bashrc from ~/.bash_profile so login shell become also interactive shell.

Shortcuts for windows

Very usefull shortcuts to activate certain windows.

gnome-terminal if negative is used, than it is bottom alligned
wmctrl
xwininfo to click on window to find it’s 0x440003d which we can use to create shortcut xdotool windowactivate 0x440003d
another robust solution is to set predefined classname for windows with xprop -f WM_CLASS 8s -set WM_CLASS main_editor (click on window) and xdotool search --classname main_editor windowactivate in Alt+M shortcut. I’m using ALT+HJKL; with class_hjkl;

mapping hjkl keys to activate window mapping workspace keys to hjkl Look at my window_shortcuts

When I update ubuntu I need to map keys again. Go one by one and press ALT+key combination again.

On blank ubuntu I need to go All Settings -> Appearance -> (tab) Behavior and Auto-hide the Launcher to ON and Enable workspaces. (in virtualBox I can not open launcher, only hud with left alt)

Shortcuts for key mappings

To scroll in terminal window, instead shift+page_up we can bind to key simulate keyup. xbindkeys I usually map vim shortcuts CTRL+ , so for terminal windows I use ALT+. Do not override already two ALT bash shortcut Alt+F Alt+B to move cursor one word (use capitalize F to not open File menu or disable in Edit -> Keyboard Shortcuts) (CTRL version move by one character) . I also map ctrl+k to scroll up since it’s natural (I don’t mind that I don’t have cut shortcut, I just use #). I tried to use bind command but that outputs, so multiple scroll looks like one scroll.

Here are all bash CTRL shortcuts:

ctrl+a (+e) : move cursor to begging (end) of line
ctrl+b (+f) : move cursor by one char to left (right)
ctrl+h (+d) : delete char left to the cursor-backspace (under the cursor)
ctrl+w : delete word before the cursor
ctrl+i : (or tab) completion
ctrl+j : enter
ctrl+k (+u) [+y]: cut chars until the end of the line (before the cursor position) [yank cutted text]
ctrl+p (+n) : go to previous (next) command
ctrl+r : search
ctrl+c (+z) : kill current process (send to background and suspend, use fg to restore). You can put any shell (vim, rails s) to suspend state with Control + z. Than you can put it in background bg if needed. You can use that shell for inspection other things. When you are finished, you can switch back to vim, or rails s, with foreground fg (from both suspend and background state) see all background and suspended processes in current bash session jobs -l
ctrl+l : clear window, but I remaped to copy current line to clipboard

ALT shortcuts:

alt+b (+f) : move cursor by one word to left (right). On mac Alt is Meta so you need to disable in all your terminals in system preferences and restart.

Here are some Edit -> Keyboard Shortcuts:

shoft+ctrl+t : open in new tab
alt+1 : switch between tabs

To copy line, you need to select with mouse and use ctrl+shift+c (ctrl+shift+v to paste). Or with xsel you can bind to ctrl+l:

# .bashrc
bind '"\C-l": "\C-e\C-u xsel --clipboard <<"EOF"\n\C-y\nEOF\n\C-y"'

In vim you can paste primary selection (just select, middle mouse, shift insert) with "*p. Clipboard selection (copy, paste) with and "+p

To copy some text from less you can click v inside less to invoke the editor (vim) and than you can copy from that. There is an error Cannot edit standard input (press RETURN) when editing (v) for example ls | less. Workaround is to ls | vim -, or for example git log | vim -, Note that yank while vim is open works fine, but once you close the vim, clipboard is empty.

Maximize terminal window to full screen is with F11 (yes it is the same shortcut for bash as for chrome). Inside terminal there are Terminal->Preferences->Shortcuts

Shift+Ctrl+F search (G and H next and previous)

Only place where I sometimes need arrow keys is google chrome suggestions in address bar, but than I use Tab.

Starting chrome with command line switch chromium-browser --auto-open-devtools-for-tabs does work only on google-chrome.

Interesting devtools-cheatcheet (search by filename Ctrl+O, search all sources Ctrl+Shift+F).

Look my xbindkeysrc There is very nice tool Xmonad which is designed to maximize. move windows around, automatically tile the screen without gaps or overlap, but I do not need auto arrangement.

To start command programs on startup you can use ctrontab -r and @reboot, but for gui programs use ubunt Startup Applications. Output (from crontab or gui) should be redirected to system log, for example

# script should start with #!/bin/bash
# and should be executable chmod +x my_script.sh
# in script use echo "start my script"
/home/orlovic/config/bashrc/startup_my_script.sh 2>&1 | /usr/bin/logger
# using tags works only if you call script direclty, but if it is called from
# startup scripts than tag is gnome-session it overrides my tag
# --tag my_tag
# sometimes messages are grouped so wait few seconds
tail -f /var/log/syslog
# clear syslog
> /var/log/syslog
# if syslog is not readable you can make it readable and writable
sudo chmod g+rw /var/log/syslog

You can also redirect output to a file withing the script itself (but redirecting to syslog does not work)

#!/bin/bash
exec >> /home/orlovic/Downloads/output.log
exec 2>&1

echo this will go to /home/orlovic/Downloads/output.log

For gui debugging you can use this command to log message from scripts: notify-send $USER --urgency critical Create desktop notifications with notify-send but need urgency critical, so I made alias alert. You can debug with sleep and echo redirection.

Since ctrl+w and ctrl+r in chrome firefox browsers will close and reload the page, it is much easier to use alt (like on mac), so I created shortcuts

To run script on keyboad shortcut go to Settings -> Keyboard -> Shortcuts -> Custom Schortcuts -> +

name: this is description
command: should be path to the executable script… Directly writting commands here might not work

If you are mapping Alt+W to send another key Ctrl+W you need to release Alt first. To release all modifiers you can use

#!/bin/bash
# config/bashrc/ctrl_browser.sh
# exec >> /home/orlovic/Downloads/output.log
# exec 2>&1
if [ "$1" = "" ]; then
  notify-send "Please provide a key, like Ctrl+r" -u critical
fi

# release all modifiers
# https://unix.stackexchange.com/questions/60007/how-to-force-release-of-a-keyboard-modifiers
xdotool keyup Shift_L Shift_R Control_L Control_R Meta_L Meta_R Alt_L Alt_R Super_L Super_R Hyper_L Hyper_R ISO_Level2_Latch ISO_Level3_Shift ISO_Level3_Latch ISO_Level3_Lock ISO_Level5_Shift ISO_Level5_Latch ISO_Level5_Lock
xdotool key $1
# keydown again so you can run multiple reloads while keeping alt key down
# problem is if you release alt to fast, you need to click again to release it
# xdotool keydown Alt_L

You can send commands to specific window https://unix.stackexchange.com/questions/87831/how-to-send-keystrokes-f5-from-terminal-to-a-gui-program

Xdotool

https://github.com/jordansissel/xdotool

# search for window id
WID=`xdotool search "Mozilla Firefox" | head -1`
# activate
xdotool windowactivate --sync $WID
# send key
xdotool key --clearmodifiers ctrl+l

To send Return with type https://askubuntu.com/questions/515185/run-several-xdotool-commands-in-one-line-separated-from-each-other

xdotool type "$(printf 'date\n\e ')"

# this does not send key
xdotool search --classname vp_$(get_current_viewport)_class_slash  windowactivate --sync %1 key  a

# also can not send key without activate that window
xdotool key --window 67199662 a Return

# with windowactivate you need to wait with sync and than send key
xdotool windowactivate --sync 67199662 key a

# to write command use `type`, to include spaces wrap with quotes
xdotool windowactivate --sync 67199662 type "pwd"

# to send Enter <CR> you need delay and also some key before that
# <CR> is 'Return', space is 'space'
# remember not to run command with <C-j> since ctrl modifier is still active
xdotool windowactivate --sync 67199662 type "pwd"; xdotool key --delay 50 space Return

# you can come back to previous terminal with
xdotool windowactivate --sync 67199662 type "pwd"; xdotool key --delay 50 space Return windowactivate 67199574

# complete command for janko-m/vim-test strategy is in vim tips

Freeze delay performance problems https://github.com/jordansissel/xdotool/issues/10

debug with

DEBUG=1 xdotool windowactivate --sync 48464317 type "$(printf "pwd\n\e ")"; xdotool windowactivate $WID

Remapping keyboard keys is with xmodmap.

# show all keys, columns are
# Key, Shift+Key, mode_switch+Key, mode_switch+Shift+Key, alt+Key, alt+shift+key
xmodmap -pke

# save configuration to a file
xmodmap -pke > ~/.Xmodmap
# reload new configuration
xmodmap ~/.Xmodmap

Instead of reloading whole file, it can reload the diffs

# Store default xmodmap
xmodmap -pke > ~/.Xmodmap.default

# Backup your custom xmodmap
mv ~/.Xmodmap ~/.Xmodmap.bak

# Make a diff between default and custom xmodmap, and store the diff as .Xmodmap
diff ~/.Xmodmap.bak ~/.Xmodmap.default | grep '<' | tr '<' ' ' > ~/.Xmodmap

To find keycodes use xev program (exit with mouse or alt+f4). showkeys is using for linux system. You can use also xkeycaps but I do not know which keyboard to choose. sudo showkey -a can show keycode.

xev | sed -n 's/^.*keycode *\([0-9]\+\).*$/keycode \1 = /p'

Keycode for a is 38. It can be used alone or with modifier keys. Alone a gives keysym 0x61 (ie 141 in ascii), for shift+a keysym is 0x41 ie A. Descriptive keysyms for multimedia keys are in /usr/include/X11/XF86keysym.h. Keycode for left ctrl is 37 (keysym Control_L), left alt 64 (keysym Alt_L), windows 133 (keysym Super_L), caps lock 66 (Caps_Lock), right alt is 108 (Alt_R).

http://wiki.linuxquestions.org/wiki/List_of_Keysyms_Recognised_by_Xmodmap http://wiki.linuxquestions.org/wiki/List_of_KeySyms AltGr, Alt_R, AltGr_R are synonyms.

From man xmodmap you can see that you can write 4 keysyms

Up to eight keysyms may be attached to a key, however the last four are not used in any major X server implementation. The first keysym is used when no modifier key is pressed in conjunction with this key, the second with Shift, the third when the Mode_switch key is used with this key and the fourth when both the Mode_switch and Shift keys are used

keycode 57 = key Shift+Key Mode_switch+Key Mode_switch+Shift+Key ISO_Level3_Shift+Key ISO_Level3_Shift+Shift+Key

Do not know where is ISO_Level3_Shift. Mode switch is on Caps_Lock. When I bind to for example number 1 and 2, than 13 gives 4 (so Mode_switch works) but 23 gives 3 (ISO_Level3_Shift does not work)

keycode  10 = Mode_switch exclam 1 exclam
keycode  11 = ISO_Level3_Shift at 2 at
keycode  12 = 3 numbersign 4 numbersign 5 6

Default output of xmodmap -pm is

xmodmap:  up to 4 keys per modifier, (keycodes in parentheses):

shift       Shift_L (0x32),  Shift_R (0x3e)
lock        Caps_Lock (0x42)
control     Control_L (0x25=37), Control_R (0x69)
mod1        Alt_L (0x40=64),  Alt_R (0x6c=108),  Meta_L (0xcd)
mod2        Num_Lock (0x4d)
mod3
mod4        Super_L (0x85),  Super_R (0x86),  Super_L (0xce),  Hyper_L (0xcf)
mod5        ISO_Level3_Shift (0x5c),  Mode_switch (0xcb)

To change modifier keys usually you should to clear them; Lock, Shift, Control, Mod1 (this is alt) … and than add Control = Super_L .... Modifier Mod4 is usually Super_L.

! This modifies CapsLock to Control, and keep CapsLock on Shift+CapsLock
clear lock
clear control
keycode 66 = Control_L Caps_Lock NoSymbol NoSymbol
add control = Caps_Lock Control_L Control_R

You can use dummy modifier on capslock and assign various key on that modifier. https://gist.github.com/bcremer/9caa54d0432531f80c1e

For Apple keyboard you can use https://help.ubuntu.com/community/AppleKeyboard and this answer https://askubuntu.com/questions/131900/how-do-i-switch-the-command-key-and-control-key-on-a-macbook-pro for swapping controll and command (I use swapping option and command). For swapping fn and control use https://help.ubuntu.com/community/TroubleWithAppleKbdOnUbuntu https://wiki.archlinux.org/index.php/Apple_Keyboard#Treating_Apple_keyboards_like_regular_keyboards https://github.com/free5lot/hid-apple-patched#installation-via-dkms-recommended

sudo vi /etc/modprobe.d/hid_apple.conf
cat /sys/module/hid_apple/parameters/swap_fn_leftctrl

I use mapping for all keys that I need right shift like underscore, brackets, html >.

! ~/.Xmodmap
! remove caps lock modifier
clear Lock
! assign dummy modifier to capslock
keycode 66 = ISO_Group_Shift ISO_Group_Shift ISO_First_Group NoSymbol
keycode 43 = h H braceleft
keycode 44 = j J parenleft
keycode 45 = k K parenright
keycode 46 = l L braceright
keycode 59 = comma less less
keycode 60 = period greater greater

clear Mod1
add Mod1 = Alt_L
keycode 108 = underscore underscore underscore underscore

! Colon and semicolon are just reversed.
keycode 47 = colon semicolon colon semicolon

Also I like command+c / alt+c for copy/paste so I use this answer to copy selected text to clipboard https://askubuntu.com/questions/573663/command-to-copy-currently-selected-text and in terminal remap copy paste Terminal->Preferences->Shortcuts->Edit->Copy (Alt+C) Paste (Alt+V) so I can create keyboard shortcut for Alt+C and Alt+V which I can use both on terminal and non terminal

# /home/orlovic/config/bashrc/copy_from_x_selection.sh
#!/bin/bash
xclip -out -selection primary | xclip -in -selection clipboard

create two shortcuts, one for Alt+C /home/orlovic/config/bashrc/copy_from_x_selection.sh and another for Alt+V /home/orlovic/config/bashrc/ctrl_browser.sh Control+v. Do not forget to use old method of selecting text and middle mouse button or Shift+Insert (Insert does not exists on Apple Keyboard so that is the reason why I add this shortcuts, but also another helpers is ctrl+i which is remaped to shift+insert in .xbindkeysrc). This xclip is also used when you want to copy from vim to clipboard, you need just to yank and alt+c and alt+v (alt+c do not need to be inside vim).

Run /home/orlovic/config/bashrc/startup_xmodmap.sh 2>&1 | /usr/bin/logger in both startup scripts and /lib/systemd/system-sleep/startup_after_sleep.sh

Problem with system-sleep xmodmap: unable to open display can not be solved with https://ubuntuforums.org/showthread.php?t=2380045

sleep 5
declare -x DISPLAY=":0.0"
declare -x XAUTHORITY="/home/<your user>/.Xauthority"

Maybe pm power manager is different than system sleep, but it needs some events to properly work. X server is not set in systemd script so we need to declare and export DISPLAY and XAUTHORITY. I success with xhost +main (enable connect from main, but since network manager is not yet initialized, that is not reliable). I also tried https://bbs.archlinux.org/viewtopic.php?pid=684936#p684936 so nm knows hostname in configuration

# /etc/NetworkManager/NetworkManager.conf
[keyfile]
hostname=main

but not reliable…

SWAY

https://github.com/swaywm/sway/wiki

mkdir -p ~/.config/sway
cp /etc/sway/config ~/.config/sway/
$EDITOR ~/.config/sway/config

TODO; example of moving to target window

Quitebrowser

https://qutebrowser.org/doc/quickstart.html

left(right) tab J (K), back (forth) in history H (L)
d close tab (u undo closing tab)
wi toggle web developers tools
o open link, O in new tab, go edit current link

Tar

comress folder: tar -zcvf folder.tar.gz folder extract folder: tar -zxvf folder.tar.gz

compress file: tar cvzf file_name.tar.gz file_name Note that is file_name is relative to some subfolders (contains slash /) for example /tmp/db.dump than leading slash will be ignored and when you extract it will be in tmp/db.dump

Scrips and commands

nice tutorial Bash Begginers Guide and example config scrips
input first param $1. $@ is all params. Default value with :- for example local port="${1:-8000}" Use "$@" if you pass to another script and you want to preserve quoted strings https://stackoverflow.com/questions/4824590/propagate-all-arguments-in-a-bash-shell-script So when variable contain spaces and you need to pass to a command than wrap around with quotes, for example A="file with space.txt"; ls "$A" There is also a $* which is very similar to $@ but when used by quote "$*" it is joined by IFS character "$1c$2c$3c…c${N}" so here is function https://dev.to/meleu/how-to-join-array-elements-in-a-bash-script-303a
```
joinByChar() {
  local IFS="$1"
  shift
  echo "$*"
}

$ joinByChar , Moe Larry Curly
Moe,Larry,Curly
```
exit from script with exit 1
to chech if files are same or different https://stackoverflow.com/questions/12900538/fastest-way-to-tell-if-two-files-have-the-same-contents-in-unix-linux
```
cmp --silent file1 file2 && echo '### SUCCESS: Files Are Identical! ###' || echo '### WARNING: Files Are Different! ###'
```

functions

myFunc() {
  local first_param=$1
  echo $first_param
}
myFunc 'Dule'

you can also define default parameters with :- syntax

myFunc() {
  echo first param is $(1:-not defined)
}

To use functions in xargs you need to export them

myF() {
  echo $1
}
export -f myF

always use double square brackets [[ ]] instead of single [ ] since it is extension and support using || instead of -o, or regex match =~

with comparison operators you should use quotes since something could not be defined if [ -n "$a" ]. If you use double brackets you do not need quotes around variable name.

# check if $a is present (not null) empty
if [[ -n $a ]]; then

# check if output contains result
if [[ $(which aws) ]]; then

# if conditional to check if string is substring
string='my long string'
if [[ $string = *"long"* ]]; then
  echo "its here"
fi

# string equal
if [[ $a != $b ]]; then
  echo string a is not equal to string b
fi

# regexp match start with doc/etc
if [[ $a =~ ^doc\/etc ]]; then
  echo string a is not equal to string b
fi

# split string and use last item
number="$(echo "asd 123" | cut -d' ' -f2)"

# find extract substring
[[ "US/Central - 10:26 PM (CST)" =~ -[[:space:]]*([0-9]{2}:[0-9]{2}) ]] &&     echo ${BASH_REMATCH[1]}
# 10:26

# integer equal
if [[ $a -eq $b ]]; then
  echo number a is equal to number b
fi

SIZE=`wmctrl -d | awk '{print $6}'`
# http://stackoverflow.com/questions/23663963/split-string-into-multiple-variables-in-bash
IFS=',' read vp_width vp_height <<< $SIZE

Find other test helpers in man test. Use conditional or || and and && and group () and negative !

if [[ $a -eq $b || ($a -eq 2 && $b -gt 2) ]] || [[ ! -n $a ]] ; then
  echo number a is equal to number b
fi

Increment count

count=1
count=$((count+1))

Use regular expression matching (not supported with [ ] single quote)

w_name="Mozzila Firefox and Chrome"
if [[ $w_name =~ 'Mozilla Firefox'|Chrome ]] ; then
fi

loops and expressions example

attempts="0"
while [ $attempts -lt 10 ] || [ -z "$window_id" ]
do
  echo $attempts
  sleep 1
  window_id=`wmctrl -l | grep $url | awk '{print $1}' | tail -n1`
  attempts=$[$attempts+1]
done
echo move and mark window_id=$window_id

find
- find and remove files find . -type f -name "test.log" -exec rm -f {} \;
- find all html files find . -name '*.html'. Note that find . -name *.html will not work since wildcard will be changed to html file in current folder.
- filter files based on their size: find . -size -212c show only small than 212 bytes
- skip Permission denied errors with find / -name 'r' 2>/dev/null
- find js files but exclude node_modules and bower_compo
```
find . -name "*.js" -type f -not -path "*node_modul*" -not -path "*bower_comp*"
```
- grep files and show file name
```
find . -exec grep -H mdsmm {} \;
```
escape single quote ' in linux scripts with $'Hello I\'m here' link
add task to crontab with linux command # (crontab -l 2>/dev/null; echo "@reboot /home/orlovic/my.sh") | crontab -
crontab output goes to the mail but I prefer to redirect to some file. Also define shell since it will use sh (on which sleep does not work). ~~~ SHELL=/bin/bash
- - - - /path/to/my/script.sh » /home/orlovic/Downloads/cron.log 2>&1 ~~~ sometimes you need to call snap commands so add PATH as well
        # crontab SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/snap/bin @reboot certbot -q renew
cron format Minute Hour Day_of_the_Month Month_of_the_Year Day_of_the_Week and also Year as 6th field. For more information see the manual pages of crontab(5) and cron(8) m h dom mon dow command. For multiple values you can separate with comma, use dash or use slash, for example 0,15,30,45 0,6,12,18 1,15,31 * 1-5 * is the same as */15 */6 1,15,31 * 1-5 *.
repeat last command with sudo sudo !!
copy all files from folder (including hidden) cp folder/. /some-other/ -r Note . after folder name.
preserve env variables with sudo sudo -E ruby some_script.rb, but rvmsudo ruby some_script.rb works better
ps aux | grep process_name will return always the grep process because it match. But you can use regex so it is not matched ps aux | grep [p]rocess_name. to show full command for process id you can use `ps -fp `. You can kill one by one using `top` and `k`. To find proccess by port it is using you can `lsof -wni tcp:3000` or simpler version to find command that is listening on a port ports list ``` lsof -i:7047 ``` so you can kill -9 PID and have that port open. ``` kill -9 $(lsof -ti:3000) ``` You can see top 5 proccesses and commands ~~~ ps aux | sort -nrk 3,3 | head -n 5 watch "ps aux | sort -nrk 3,3 | head -n 5" # or with top top -b -n 1 | head -n 12 | tail -n 5 # or better htop ~~~
edit long commands in bash set -o vi and than press Esc, and v (visual). Don’t recomment to put in bashrc since it will disable bash shortcuts like: ctrp+p, bind, ….
mount sudo fdisk -l sudo blkid sudo vi /etc/fstab add line
```
# /ets/fstab
UUID=428c1c5c-7ef4-480a-aa3b-1b62c3feab98 /mnt/moj_ssd    ext4    defaults  0  2
```
run sudo chown -R orlovic.orlovic /mnt/moj_ssd/ this is needed only one time.
if you need to answer yes and there is no -y parameter, you can use echo -en "\n\n\n" | command . Or you can install yes command which outputs y\n in a infinite loop yes | command
get folder from path echo $(dirname "${f}") file basename "asd/a.t" or last folder (after last /) from current folder basename $(pwd)

to check if file does not exists you can use

if [ ! -f tmp/memory_profile.png ]; then
  echo "File not found"
fi

run multiple long running commands (like watch tail -f) with ;
show commands being executed
```
set -x
command
set +x
```

Pipes

you can pipe output of some command to file c >> a.log, and in other shell tailf a.log but output will be buffered So you need to sudo apt-get install expect-dev and run with unbuffer c >> a.log
if you need pipe inside string heroku logs -t | -o app.web.1 than you have to eval $my_string
you can add a row to some pipe (echo first line; cat file; echo last line) | some_command or with ps | { echo "header"; grep "something"; }
replace a string in pipe with sed. You can create a long running pipe with tailf and echo >>
```
tailf my.txt | sed 's/000/111/g'

echo "asd 000 asd 000" >> my.txt
```
for redirection standard error to standard output ` 2>&1 ` there is shorthand

|& version.

you add add a line to a pipe before and after the ouput with https://stackoverflow.com/questions/15029171/add-line-at-top-and-bottom-of-file-before-sending-it-to-pipe

cat text.txt | sed -e '1 i my first line' | sed -e '$a my last line'

# or with echo and cat

cat text.txt | (echo first line; cat -; echo last line) | some-other-command

if you need to redirect all script output to a file use exec

#!/bin/bash
exec >> my_script.log
exec 2>&1
echo "This will be saved in a log when you run the script"

last command exit status echo $? (0 is success)
current script pid echo $$
last background process id sleep 100 & echo $!
jobs -l is a list of processes in current bash session (it is Running if in background with cmd &, or Stopped if Ctrl+z or Terminated when we kill and until it dissapear). Kill first of them with kill %1. If process id is saved in pid file than you can kill with sudo pkill -F tmp/pids/server.pid

Parameter expansion

You can find substring

call function with $(function_name). Return value from function can be done with $() expansion:
```
monitor_size() {
result=`ls | cw`
  echo $result
}
win_width=`expr $(monitor_size) / 36` # 7200 / 36 = 200
```
To use arithmetic expansion (add, multiplication, substraction, plus, minus), you can use backticks and expr http://tldp.org/LDP/abs/html/arithexp.html
```
c = `expr $a + 2`
```
or double parentheses c=$(($a + 2))
print variable with $variable_name (without brackets). If you need to concatenate with other string you can use curly braces ${variable_name}other_string
substring removal
get second string: echo 'first second' | awk '{print $2}'
get string after n characters http://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
```
dusan=dusan
echo ${dusan:2} # san
echo ${dusan:(-2)} # an
```
in bash 3 kinds of variable substitution
- pattern matching: deletes match # shortest from left, ## longest from left, % shortest from right, %% longest from right
  - NAME=${MYVAR%:*} retain the part before the colon :
  - NAME=${NAME##*/} retain the part after the last slash /
- substitution ${foo:-bar}
string manipulation
concatenate with just joining the strings echo 'a'"b"

Arrays

linuxjournal

arr=(Dule Orlovic)
${arr[*]}         # Dule Orlovic All of the items in the array
${!arr[*]}        # 0 1 All of the indexes in the array
${#arr[*]}        # 2 Number of items in the array
${#arr[0]}        # 4 Length of item zero

# iterate array in for loop
for var in "${arr[@]}"
do
  echo "${var}"
done

# iterate over sequence
for i in {1..5}; do echo $i; done

# iterate over list of lines from some command
for file in `git diff $SHA --name-only`
do
  echo $file
done
# or
files=$(git diff $SHA --name-only)
IFS=$'\n'
for file in $files
do
end

Curl

https://gist.github.com/subfuzion/08c5d85437d5d4f00e58

same url to some variable

export u=http://localhost:3000/api/v1/
curl $u/expenses

you can add -v --verbose option to see more info or save all communications with url $u --trace-ascii dump.txt
see headers

curl $u/expenses -I # or --head fetch the headers only
curl $u/expenses -i # or --include show also the response headers

to keep session in curl you can -c write and -b read from cookie file

curl -c ~/Downloads/cookiefile -b ~/Downloads/cookiefile http://www.google.com

test the speed on ssh on remote server:
- download: curl -o /dev/null http://speedtest.qsc.de/1GB.qsc
- upload: generate large file fallocate -l 1G gentoo_root.img and use scp to test upload link. create big file
limit the speed to simulate slow connections curl $u limit-rate 100 or using

https://github.com/bcoe/crapify. But this is only for downloading, server renders quickly. Only way to simulate high response time is with sleep 5

curl url must be inside '', for example curl http://trk.in.rs?a=2&b=3
to get json request use header tag Accept curl -H "Accept: application/json" http://localhost:3001/ or curl -H 'Content-Type: application/json'
set authorization header curl $u/expenses -H 'Authorization: Token token="c576f0136149a2e2d9127b3901015545"'
user agent curl --user-agent "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" $u
http basic auth curl $u -u username:password or curl $u -u $ADMIN_USERNAME:$ADMIN_PASSWORD'. Another format is inside uri curl http://$username:$password@localhost:3000
follow redirection curl $u --location (it works also when performing POST, since it will perfome GET to redirected url).
referrer curl $u --referer http://google.com
domain name resolution could be using resolve curl http://a.b:3000 --resolve a.b:3000:127.0.0.1 so in rails request.host == 'a.b'
POST request is with --data curl $u --data "name=my name" (recent curl will encode post data for you). To see how actual form sends data, save the page localy and change method to GET so when you submit you can see data separated with ? and &
patch request on rails is done with --request PATCH option
verifyhost

some sites prevents curl and ruby scripts, if gives forbidden or empty response: https://www.theguardian.com

to upload some file using curl https://transfer.sh/ for example on heroku you can not ssh to someother place

curl --upload-file ./a.html https://transfer.sh/a.html
# download
curl https://transfer.sh/ASDF/a.html -o a.html

SSH

Install ssh

sudo apt instal openssh-server

Generate key and comment

ssh-keygen -t ed25519 -C "My key for vm"

Exit ssh session when there are connection issues and ssh becomes unresponsive stale, you can can exist with tilda and dot

<Enter>
~.

You can enable ssh agent forwaring so remote connection can use your keys. When is it enabled there is env | grep SSH_AUTH_SOCK environment variable. You can enable with ssh -A option.

ssh -A [email protected] -p 2222 'env | grep SSH_AUTH_SOCK'

Connect using pem or rsa key is ~~ ssh -i $PEM_FILE host

You can see more logs
```
ssh host -vvv
```
When there is no connection to the host, you can check with
```
# you can see ssh version
nc -w 10 192.168.1.3 22
SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.6

# or you can echo true or false in case of timeout
nc -w 10 192.168.1.3 22 && echo true || echo false
```
I had an issue with my old router connecting `SSH-2.0-OpenSSH_7.6p1
Ubuntu-4ubuntu0.1` so I need to connect directly to my ADSL. This problem was
related to IPv6 on the server.

`Too many authentication failures for` error is caused by inadvertently offering
multiple ssh keys to the server. You can limit to only one identity

ssh -i some_id_rsa -o IdentitiesOnly=yes host

To automatically change folder on remove server you can add `cd folder` in
remote `.bashrc`, or you can add `command=` in `.ssh/authorized_keys` on remote
Note that this does not work with umount
```
# ~/.ssh/authorized_keys
command="cd /home/dule/folder ; /bin/bash -l" ssh-rsa AAA....
```

# Rsyslog

Install
[mongo](https://www.digitalocean.com/community/tutorials/how-to-install-mongodb-on-ubuntu-16-04)

sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv EA312927 echo “deb http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 multiverse” | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list sudo apt-get update sudo apt-get install -y mongodb-org

cat « HERE_DOC | sudo tee -a /etc/systemd/system/mongodb.service [Unit] Description=High-performance, schema-free document-oriented database After=network.target

[Service] User=mongodb ExecStart=/usr/bin/mongod –quiet –config /etc/mongod.conf

[Install] WantedBy=multi-user.target HERE_DOC

sudo systemctl start mongodb sudo systemctl status mongodb sudo systemctl enable mongodb

find startup applications

sudo systemctl list-unit-files | grep mongo

https://datatables.net/development/server-side/php_mongodb

# MACBook macOS

xprop for macos can be installed using: `sudo port install xdotool`
Find and xargs can be used also:

find . -name .DS_Store -print0 | xargs -0 git rm -f –ignore-unmatch

[xsel](https://linux.die.net/man/1/xsel) equivalent is
[pbcopy](https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/pbcopy.1.html)
so I use [is_mac_os]() to determine os.

I like mac Cmd key so to use that on ubuntu, I need to remap keys. One solution
is to switch ctrl and alt keys, but problem is that Alt+Tab, Alt+1, Ctrl+C,
Crtl+D, should stay as it was.

# Tips

* history
  * bash update history only on exit, but you can manually write with `history
      -a` (do not do this for every command since multiple terminal will be
      [interlieved](http://mywiki.wooledge.org/BashFAQ/088))
  * to add command to history without executing it `history -s pwd`. Note that
    this command if executing as param to bash will open new `history` file so
    we can't have that command in history. Running history when you get console
    is not the same when you pass history as param to bash command. For example
    the following command will show pwd and empty history (if you repeat
    `history` it will give you results) `gnome-terminal -x bash -lc "cd
    ~/Downloads;bash --rcfile <(echo 'pwd;history;watch ls')"`. Solution is to
    add command before invoking bash like [here](
* `cat /etc/issue` and `arch`

$ cat /etc/issue # Ubuntu 16.04.1 LTS \n \l

arch # x86_64

* move file to another file appending some suffix without writting filename
twice

mv config/database.yml{,.example}

* `wget trk.in.rs --recursive` will download all files from the site.
* sometimes when using multiprocess puma as rails server, I do not see what I'm
typing in
[bash](https://askubuntu.com/questions/171449/shell-does-not-show-typed-in-commands-reset-works-but-what-happened)
so I need to run `reset` command
* to select vim as default editor for root, run `update-alternatives --config
editor` and choose vim.basic
* files need bit `r` to be able to read, `w` to write and `x` to list, search or
source included files.
You can see permissions for particular directory with `ls -ld`. To set for
example `chmod 644` is the same as `chmod u=rw,g=r,o=r`. To be able to source
file, parent folder needs to have `x` flag enabled.
* to detect if it is linux or MacOS you can use

if [ “$(uname -s)” = “Darwin” ]; then echo “==> Bootstrapping Homebrew OSX environment” fi

* cheatsheet https://devhints.io/bash
* `pbcopy` is only on mac
* to prevent `ctrl-d` to exit terminal bash you can set `IGNOREEOF=3` so shell
  only exists after the 3 consecutive Ctrl-d.
* https://github.com/wting/autojump use j or autojump instead of cd change
  directory
* split long text files with `split -C 50m input output --numeric-suffixes`

* in scripts use

set -e # Any commands which fail will cause the shell script to exit immediately set -x # Show command being executed

Rescue from error with `|| true` like
```
git clone asd.com || true
```
return from function on error. `set -e` or `exit 1` will exit the shell
completelly.
```
my_function() {
  not_a_command || return 1
}
```

You can read user input

read -p “enter fullname: “ fullname read -p “Continue? (Y/N): “ confirm && [[ $confirm == [yY] || $confirm == [yY][eE][sS] ]] || exit 1

* xargs is used to convert stdin standard input to arguments for a commands.
  can reference arguments with `-I '{}'`, limit number of lines `-L 2` before
  it is executed (usefull for long running proccess to trigger commands). You
  can not call custom functions since commands are executed in plain bash, but
  you can use `sh` and params `{}` so there you can inline your script

echo 123 | xargs -0 -I ‘{}’ sh -c ‘echo {}’

* read from stdin works fine http://tldp.org/LDP/abs/html/internal.html#READREF
  but not with pipe
  <https://stackoverflow.com/questions/2746553/bash-script-read-values-from-stdin-pipe/6779351#6779351>
  For pipe you need to simulate with `{ }`

* add tab suggestions for your script using `complete` command
  https://iridakos.com/tutorials/2018/03/01/bash-programmable-completion-tutorial
* strip new line from command output `xdotool search --classname
  vp_3_class_slash | tr -d '\n'`
* fan working when high temperature

sudo apt install lm-sensors sudo sensors-detect sensors

sudo apt install hardinfo hardinfo ~~~

first line in scripts shebang for ruby using rvm instead of /usr/local/bin/ruby can be https://stackoverflow.com/questions/17447532/what-is-the-use-of-usr-local-bin-ruby-w-at-the-start-of-a-ruby-program
```
#!/usr/bin/env ruby
```
list files with full path is using find $PWD or using -d option ls -d -1 $PWD/*.* list files by regexp https://www.linuxjournal.com/content/bash-extended-globbing
```
ls @(one|two).html
```
ls sort by file size using ls -S
df -h and du -sh can show how much is used or free
df -i show how much inodes are left. Zoneminder through mysql can use all of them so you need to find and remove those files.
```
df -i
du -s --inodes /*
find . -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n
```
To remove old kernel files you can run sudo apt-get autoremove

rename files to change extension

sudo apt install rename
find _posts/ -name "*.markdown" -print0 | xargs -0 rename 's/.markdown$/.md/'

multiline string inside variable

USAGE=$(cat <<-END
    This is line one.
    This is line two.
    This is line three.
END
)

echo -e "$USAGE"

check if file contains string and run code only if does not include string

if [[ $(grep -L rbenv ~/.bashrc) ]]; then
  echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
fi

telnet host port is usefull to check if it is open port, escape character is ctrl + ]
```
telnet selenium 4444
```

man man can give you more info how to use manual pages for help

# seach short descriptions for pwd
man --apropos pwd
man -k pwd

# search all text and open man page for each
man -K pwd

# open specific section, for example 5 is conventions, 7 miscellaneous
man sway.5
man 5 sway

send email from bash

sudo apt-get install ssmtp
sudo vim /etc/ssmtp/ssmtp.conf

rewriteDomain=gmx.com
FromLineOverride=YES
mailhub=mail.gmx.com:587
useSTARTTLS=YES
[email protected]
AuthPass=my-passs
TLS_CA_File=/etc/pki/tls/certs/ca-bundle.crt

when sending from username that is different we need to define From: and separate with two lines \n\n

echo -e "From: [email protected]\nSubject: My subject\n\nTest message body ssmtp" | ssmtp -vvv [email protected]

here is a script that you can run from crontab

#/bin/bash
# ~/my-app/my-app/tmp/notify_if_failed.sh
if ~/my-app/my-app/tmp/a.sh | grep -q DS ; then
  echo 'found DS'
  echo -n "." >> ~/my-app/my-app/tmp/a.sh.log
else
  echo "no DS"
  echo "$(date) NO DS =============================================================================" >> ~/my-app/my-app/tmp/a.sh.log

  if [ ! -f ~/my-app/my-app/tmp/a.sh.stop ]; then
    echo "From: [email protected]\nSubject: my-app login from mail failed\n\nTo continue with this notification you need to run\nrm ~/my-app/my-app/tmp/a.sh.stop" | ssmtp -vvv [email protected] >> ~/my-app/my-app/tmp/a.sh.log 2>&1
    touch ~/my-app/my-app/tmp/a.sh.stop
  fi
fi

show motherboard info
```
sudo dmidecode -t 2
```
show number of cores and cpu info
```
lscpu
```

Screen

start new version with screen-4.9.0 Find info with man screen or online https://www.gnu.org/software/screen/manual/screen.html

https://kapeli.com/cheat_sheets/screen.docset/Contents/Resources/Documents/index

screen command (escape is ctrl+a ie c-a)

c-a ? help to show keybindings
all session screen -ls start new named session screen -S dule, exit detach with c-a d and reattach with screen -x or screen -r dule. Close current region c-a X (uppercase X) close all except current c-a Q. kill current c-a k, kill all with c-a \.
create new window c-a c and switch to it using c-a 1, list all windows c-a " you can switch to it by jk and enter, show window bar c-a w, go to next/prev window c-a p or c-a <space>
split c-a | and move to that region c-a <tab>, close all splits except current c-Q (you can also :split)
c-a [ (or c-a <esc>) to go to copy mode, press space or enter to mark and c-a ] to paste
since screen will catch ctrl+c so to get to shell in window where some process is running you can use ctrl+z to move it to background (move to foreground with fg) https://serverfault.com/questions/784645/need-to-send-c-to-cli-using-gnu-screen-over-serial-interface-dev-rfcomm0
to send ctrl+C to process you can remap stty intr ^] so you can use ctrl+] to kill the process but keep window and shell running https://unix.stackexchange.com/a/171707/150895
change escape key to j https://wiki.archlinux.org/title/GNU_Screen#Change_the_escape_key https://stackoverflow.com/a/7251588/287166
multiuser https://unix.stackexchange.com/questions/163872/sharing-a-terminal-with-multiple-users-with-screen-or-otherwise set up the server for ssh access ``` sudo useradd -m help –shell /bin/bash sudo passwd help

sudo vi /etc/ssh/sshd_config

enable password authentication

PasswordAuthentication yes

so now

as user: dule

screen -d -m -S multisession screen -r multisession :multiuser on :acladd mile

another user can controll screen session

as user: mile

screen -x dule/multisession

If you want to enable access of remote user you can use port forwarding
(no need to `:multiuser on` command)

as myuser

screen -S mysession ssh -R 2222:localhost:22 remote-host

on remote host attach using -x (multi display mode)

ssh -p 2222 myuser@localhost screen -x mysession


in script

help.sh

#!/bin/bash

Define the session name and command to be executed

SESSION_NAME=”help_session” COMMAND_TO_RUN=”ssh -R 2222:localhost:22 power”

Check if the screen session already exists

if screen -list | grep -q “.${SESSION_NAME}”; then echo “Screen session ‘${SESSION_NAME}’ already exists, try: screen -r help” else # Create a new detached screen session and run the command screen -dmS “${SESSION_NAME}” ${COMMAND_TO_RUN} echo “Screen session ‘${SESSION_NAME}’ started and command executed.” screen -S $SESSION_NAME -X screen -t “Bash 1” bash -c “export PS1=’help \w$ ‘; exec bash”

fi screen -r help ~ ```

To keep ssh live when changing the networks you can use mosh Note that with mosh, scroll up and scroll down does not work well as with ssh.

To keep tunnel opened TODO https://www.everythingcli.org/ssh-tunnelling-for-fun-and-profit-autossh/

for redirection standard error to standard output ` 2>&1	` there is shorthand
`\|&` version.