Edit

S

SED (Stream Editor)

Options

  • -s (Linux): consider files as separate rather than as a single, continuous long stream.

  • -I (MACOS-zsh): treat all files as combined, $ is the last line of last file

  • -i: treat files separately, $ is the last line of each line in each file

  • -i.bak: create backup file .bak

  • -e: allow multiple editing commands

Remove Ending Spaces (up to 5) and word in the end of each line of a file

  • sed 's/[[:space:]]{0,5}IP$//' input.txt > output.txt

Change hostname

  • sudo sed -i ‘s/.*//’ /etc/hostname

Prepend text to a file

  • sed -I ‘1s/^/\n/’

# cat testfile
line 1
line 2
line 1
line 3
# sed -i '1s/^/line 0\n/' testfile
# cat testfile
line 0
line 1
line 2
line 1
line 3
# vi testfile
# sed -i '2s/^/line -1\n/' testfile
# cat testfile
line 0
line -1
line 1
line 2
line 3
line 4
# sed -i '3s/^/line -2\n/' testfile
# cat testfile
line 0
line -1
line -2
line 1
line 2
line 3
line 4

Insert a line in the middle

  • sed -i ‘2 i whatever text to add’ file.txt

Delete a line

  • sed -i ‘2 d’ file.txt

  • sed -si '/^$/d' file.txt: delete empty lines

Combine multiple lines to one line

  • cat text | tr ‘\n’ ‘ ‘ | sed ‘s/ //g’

    • Tr ‘\n’ ‘’: change new line to space

    • Then sed ‘s///g’: change all to none

Print /Start/ /End/

  • ansible-doc ios_facts | sed -n ‘/^# hardware/,/^$/p’: print section from # hardware to blank line

    • -n: quiet mode

Print /Start/ /End of file/

  • sed -n '/text/,$p' file

    • -n: suppress automatic printing.

    • $p: end of file

  • Above can be done with these commands: grep -n "specific_text" filename.txt | cut -d: -f1 | xargs -I {} tail -n +{} filename.txt

    • grep command gets the line with specific text

    • cut command just gets the line number

    • xarg passes that line number to tail command

    • tail command prints content of file from line number: -n +{}

Replace text in all file in a folder

  • find . -type f -exec sed -i 's/old text/new text/g' {} ;

  • or find . -type f -exec sed -i 's/old text/new text/g' {} +: don't invoke sed for each file

  • grep -rlZ 'old' . | xargs -0 sed -i.bak 's/old/new/g': find all file with text "old" and replace with "new"

Using -print0 in find command

  • To avoid issues with files containing space in their names, use the -print0 option, which tells find to print the file name, followed by a null character and pipe the output to sed using xargs -0:

    • find . -type f -print0 | xargs -0 sed -i 's/old/new/g'

Exclude a directory

  • Replacing a string in your local git repo to exclude all files starting with dot (.)

    • find . -type f -not -path '/.' -print0 | xargs -0 sed -i 's/old/new/g'

  • Rearch and replace text only on files with a .md extension

    • find . -type f -name "*.md" -print0 | xargs -0 sed -i 's/old/new/g'

Reference

===========================================================================================================

Shift - Shift positional parameters

===========================================================================================================

Shutdown

  • shutdown 16:00: shutdown at 4pm

  • shutdown +1: shutdown in 1 minutes

  • shutdown -c: cancel shutdown

  • shutdown --show: show schedule shutdown

  • shutdown +1 "Will shutdown soon": send notification to logged-in users

  • shutdown now: shutdown immediately

  • shutdown -r: restart the system

  • shutdown -h: halt the system

  • https://linuxhandbook.com/linux-shutdown-command/ ===========================================================================================================

SSH

Login using ID Key file

Options:

  • -oStrictHostKeyChecking=no: Doesn't check host key

  • -c aes256-cbc: specify cipher to use

  • -oHostKeyAlgorithms=+ssh-dss: specify host key algorithm to use

  • -oKexAlgorithms=+diffie-hellman-group1-sha1: specify Host Key exchange algorithm to use

  • Or use config file in ~/.ssh/config:

  • For some old devices:

Port Forwarding

Local Port Forwarding

Remote Port Forwarding

Dynamic Forwarding

Add port forwarding to existing ssh session

  • Linux ssh client:

    • ~C

    • -R 1234:localohost:4321

    • -L 8080:localhost:80

  • Putty:

    • Right click on icon in the left top corner -> change setting

    • Go to ssh – Tunnel

  • Reference:

    • https://coderwall.com/p/5wp2wg/start-port-forwarding-over-an-existing-ssh-connection-instead-of-creating-a-new-one

    • https://nixshell.wordpress.com/2008/12/10/ssh-port-forwarding-without-starting-a-new-session/

    • https://knowledge.exlibrisgroup.com/Voyager/Knowledge_Articles/Set_Up_SSH_Port_Forwarding_in_Putty

Supported escape sequences:

  • ~. – terminate connection (and any multiplexed sessions)

  • ~B – send a BREAK to the remote system

  • ~C – open a command line

  • ~R – request rekey

  • ~V/v – decrease/increase verbosity (LogLevel)

  • ~^Z – suspend ssh

  • ~# – list forwarded connections

  • ~& – background ssh (when waiting for connections to terminate)

  • ~? – this message

  • ~~ – send the escape character by typing it twice

  • (Note that escapes are only recognized immediately after newline.)

List background jobs and jump between ssh sessions:

List forwarded connections/ports

  • From remote shell of connected session, use command: ~#

  • From terminal of client that connected to ssh server, use command: lsof -i -n | grep ‘ssh’

Using config file

SSH-Agent and Agent Forwarding

  • SSH using private key: ssh -i @

Adding Private Key to Memory:

  • Run agent: eval ssh-agent -s or run ssh-agent to get SSH_AUTH_SOCK variable and prepend to ssh-add and ssh command

  • Add Key: ssh-add .

  • Key can be listed using: ssh-add -l

  • If connecting to multiple boxes in a chain and they use different key, just add multiple key in this first agent

  • Then can use that key in ssh without specifying key file, e.g. ssh user@host, Key will be used automatically from memory

Example

Another way to add key to agent:

ProxyJump and ProxyCommand

  • Ssh -J <bastion host, bastion host>

  • https://www.redhat.com/sysadmin/ssh-proxy-bastion-proxyjump#:~:text=ProxyJump%20is%20the%20simplified%20way,the%20proxy%20or%20bastion%20hosts

X11 Forwarding

  • Server: /etc/ssh/sshd_config: X11Forwarding yes

    • Check if X11 Forwarding is enabled: sudo sshd -T | grep -i X11

  • Client:

    • Windows: install XMing and enable X11 Forwarding in Putty

    • MacOS: install xquartz: brew install xquartz or from https://www.xquartz.org/

    • Command:

      • Run XMing or xquartz

      • For Xquartz right click and select terminal to use or set export DISPLAY=:0 before running SSH command below

      • SSH -X @ or enable: ForwardX11 yes in ~/.ssh/config file. May have to use -Y instead of -X on some system

    • Checking if X11 Forwarding is enabled in SSH session: echo $DISPLAY

  • Some Error:

    • See this: Error of failed request: BadAccess (attempt to access private resource denied) xclip

      • Solution: Use SSH -Y instead of -X

    • Warning: No xauth data; using fake authentication data for X11 forwarding

      • Solution: add to /etc/ssh/ssh_config of ssh client

      • MAC OS: XAuthLocation /opt/X11/bin/xauth

      • Linux: XAuthLocation /usr/bin/xauth

Some Errors

Too many authentication failure

  • After checking everything is correct, but still get this one. The reason might be you are using RSA key file authentication and haveing ssh-agent sending multiple keys

  • Solution is

    • adding -o IdentitiesOnly=yes or -oIdentitiesOnly=yes to ssh command

    • or adding to ~/.ssh/config file:

  • Reference: https://www.tecmint.com/fix-ssh-too-many-authentication-failures-error/

SSH-Keygen

  • Generate private/public key pair: ssh-keygen

  • Specify number of bits in the keys (4096 in this example): ssh-keygen -t rsa -b 4096

  • Specify rsa key type and filename to save key: ssh-keygen -t rsa -f /path/to/your/filename

    • Some types are: rsa, dsa, ecdsa, or ed25519

  • Provide passphrase for the key: ssh-keygen -t rsa -N "your_passphrase", "" for no passphrase

  • Change password of private key: ssh-keygen -p -f $currentkeyfile

  • Add comment for the key: ssh-keygen -t rsa -C "[email protected]"

  • Show fingerprint of public key: ssh-keygen -l -f /path/to/your/public_key_file

    • Show the finger print of the public key from private key: ssh-keygen -y -f private_key.pem | ssh-keygen -lf -

      • Above command get public key from private key, then get the fingerprint of that public key, - is standard input (which is piped from output of previous command)

  • Convert private key to public key: ssh-keygen -y -f /path/to/your/private_key_file

  • Run in quiet mode, useful for scripts and automation: ssh-keygen -q -t rsa -f /path/to/your/filename

SSH-Keyscan

  • Used to gather the public SSH host keys of a number of hosts. It can be useful to populate known_hosts files, which are used by SSH clients to validate the identity of the host they are connecting to.

  • ssh-keyscan -t rsa example.com >> ~/.ssh/known_hosts

Reference

https://www.howtoforge.com/reverse-ssh-tunneling https://www.revsys.com/writings/quicktips/ssh-tunnel.html https://vimeo.com/54505525 http://blog.pi3g.com/2013/05/raspberry-pi-socks-5-proxy-server-aka-browse-the-web-with-an-ip-from-a-different-country/ https://www.digitalocean.com/community/tutorials/how-to-route-web-traffic-securely-without-a-vpn-using-a-socks-tunnel

Sudo and Visudo

Edit sudo configuration

  • sudo visudo

  • Order is important, the last config is applied

Sample configuration

Commands

  • sudo -k: Clear sudo cache

PreviousReferenceNextT

Last updated