S

SED (Stream Editor)

Options

-s (Linux): consider files as separate rather than as a single, continuous long stream.
-I (MACOS-zsh): treat all files as combined, $ is the last line of last file
-i: treat files separately, $ is the last line of each line in each file
-i.bak: create backup file .bak
-e: allow multiple editing commands

Remove Ending Spaces (up to 5) and word in the end of each line of a file

sed 's/[[:space:]]{0,5}IP$//' input.txt > output.txt

Change hostname

sudo sed -i ‘s/.*//’ /etc/hostname

Prepend text to a file

sed -I ‘1s/^/\n/’

# cat testfile
line 1
line 2
line 1
line 3
# sed -i '1s/^/line 0\n/' testfile
# cat testfile
line 0
line 1
line 2
line 1
line 3
# vi testfile
# sed -i '2s/^/line -1\n/' testfile
# cat testfile
line 0
line -1
line 1
line 2
line 3
line 4
# sed -i '3s/^/line -2\n/' testfile
# cat testfile
line 0
line -1
line -2
line 1
line 2
line 3
line 4

Insert a line in the middle

sed -i ‘2 i whatever text to add’ file.txt

Delete a line

sed -i ‘2 d’ file.txt
sed -si '/^$/d' file.txt: delete empty lines

Combine multiple lines to one line

cat text | tr ‘\n’ ‘ ‘ | sed ‘s/ //g’
- Tr ‘\n’ ‘’: change new line to space
- Then sed ‘s///g’: change all to none

Print /Start/ /End/

ansible-doc ios_facts | sed -n ‘/^# hardware/,/^$/p’: print section from # hardware to blank line
- -n: quiet mode

Print /Start/ /End of file/

sed -n '/text/,$p' file
- -n: suppress automatic printing.
- $p: end of file
Above can be done with these commands: grep -n "specific_text" filename.txt | cut -d: -f1 | xargs -I {} tail -n +{} filename.txt
- grep command gets the line with specific text
- cut command just gets the line number
- xarg passes that line number to tail command
- tail command prints content of file from line number: -n +{}

Replace text in all file in a folder

find . -type f -exec sed -i 's/old text/new text/g' {} ;
or find . -type f -exec sed -i 's/old text/new text/g' {} +: don't invoke sed for each file
grep -rlZ 'old' . | xargs -0 sed -i.bak 's/old/new/g': find all file with text "old" and replace with "new"

Using -print0 in find command

To avoid issues with files containing space in their names, use the -print0 option, which tells find to print the file name, followed by a null character and pipe the output to sed using xargs -0:
- find . -type f -print0 | xargs -0 sed -i 's/old/new/g'

Exclude a directory

Replacing a string in your local git repo to exclude all files starting with dot (.)
- find . -type f -not -path '/.' -print0 | xargs -0 sed -i 's/old/new/g'
Rearch and replace text only on files with a .md extension
- find . -type f -name "*.md" -print0 | xargs -0 sed -i 's/old/new/g'

Reference

===========================================================================================================

Shift - Shift positional parameters

cat test.sh
#!/bin/bash

echo $1
echo $2
echo $3
shift 2
echo $@

./test.sh one two three four
one
two
three
three four

===========================================================================================================

Shutdown

shutdown 16:00: shutdown at 4pm
shutdown +1: shutdown in 1 minutes
shutdown -c: cancel shutdown
shutdown --show: show schedule shutdown
shutdown +1 "Will shutdown soon": send notification to logged-in users
shutdown now: shutdown immediately
shutdown -r: restart the system
shutdown -h: halt the system
https://linuxhandbook.com/linux-shutdown-command/ ===========================================================================================================

SSH

ssh -i <key file contains private key> [email protected]

Options:

-oStrictHostKeyChecking=no: Doesn't check host key
-c aes256-cbc: specify cipher to use
-oHostKeyAlgorithms=+ssh-dss: specify host key algorithm to use
-oKexAlgorithms=+diffie-hellman-group1-sha1: specify Host Key exchange algorithm to use
Or use config file in ~/.ssh/config:

    Ciphers +aes128-ctr,aes192-ctr,aes256-ctr
    HostKeyAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss
    KexAlgorithms ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group1-sha1
    MACs hmac-sha2-256,hmac-sha2-512,hmac-sha1

For some old devices:

    Ciphers +aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
    HostKeyAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss
    KexAlgorithms ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group1-sha1
    MACs hmac-sha2-256,hmac-sha2-512,hmac-sha1

Port Forwarding

Local Port Forwarding

- ssh -L 2000:<destination.to.access>:80 -N -f [email protected]

- From client (which initiate above ssh command), access port 2000, traffic will be forwarded to <destination to access> on port 80
-f ssh will go into background before executing the command
-N instruct ssh to not execute a command on remote system

Remote Port Forwarding

ssh -R 2000:Corp.Server:22 [email protected]

- From external server or external client, access port 2000, traffic will be forwarded to <corp.server> on port 22

Dynamic Forwarding

ssh -D <port> [email protected]

Connecting to <port> above using sock to forward all connection via server.com

Add port forwarding to existing ssh session

Linux ssh client:
- ~C
- -R 1234:localohost:4321
- -L 8080:localhost:80
Putty:
- Right click on icon in the left top corner -> change setting
- Go to ssh – Tunnel
Reference:
- https://coderwall.com/p/5wp2wg/start-port-forwarding-over-an-existing-ssh-connection-instead-of-creating-a-new-one
- https://nixshell.wordpress.com/2008/12/10/ssh-port-forwarding-without-starting-a-new-session/
- https://knowledge.exlibrisgroup.com/Voyager/Knowledge_Articles/Set_Up_SSH_Port_Forwarding_in_Putty

Supported escape sequences:

~. – terminate connection (and any multiplexed sessions)
~B – send a BREAK to the remote system
~C – open a command line
~R – request rekey
~V/v – decrease/increase verbosity (LogLevel)
~^Z – suspend ssh
~# – list forwarded connections
~& – background ssh (when waiting for connections to terminate)
~? – this message
~~ – send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

List background jobs and jump between ssh sessions:

server1#~^Z [suspend ssh]
 
[2]+  Stopped                 ssh [email protected]
admin@server:~/script$ jobs
[1]-  Stopped                 ssh [email protected]
[2]+  Stopped                 ssh [email protected]
admin@server:~/script$ fg 1
ssh [email protected]
 
server2#~^Z [suspend ssh]
 
[1]+  Stopped                 ssh [email protected]
admin@server:~/script$ jobs
[1]+  Stopped                 ssh [email protected]
[2]-  Stopped                 ssh [email protected]
admin@server:~/script$ fg 2
ssh [email protected]

List forwarded connections/ports

From remote shell of connected session, use command: ~#
From terminal of client that connected to ssh server, use command: lsof -i -n | grep ‘ssh’

Using config file

- File in ~/.ssh/config
- Permission: 600
- Example:
    Host server01
            Hostname 10.10.10.1
            User root
            IdentityFile ~/.ssh/idserver01
    Host server10
            Hostname 10.10.10.10
            User root
            IdentityFile ~/.ssh/idserver10
                ForwardAgent Yes
    Host *
            IdentityFile none # disable public key
            PubkeyAuthentication no
            IdentitiesOnly yes
            PreferredAuthentications password
- Then can use command: ssh server01 to logon to 10.10.10.1
- Reference: https://linuxize.com/post/using-the-ssh-config-file/

SSH-Agent and Agent Forwarding

SSH using private key: ssh -i @

Adding Private Key to Memory:

Run agent: eval ssh-agent -s or run ssh-agent to get SSH_AUTH_SOCK variable and prepend to ssh-add and ssh command
Add Key: ssh-add .
Key can be listed using: ssh-add -l
If connecting to multiple boxes in a chain and they use different key, just add multiple key in this first agent
Then can use that key in ssh without specifying key file, e.g. ssh user@host, Key will be used automatically from memory

Example

Adding key to ssh agent: pageant is the ssh-agent of putty
        admin@server01:~/.ssh$ ssh-agent
        SSH_AUTH_SOCK=/tmp/ssh-y8F2O4AB8GfL/agent.25945; export SSH_AUTH_SOCK;
        SSH_AGENT_PID=25946; export SSH_AGENT_PID;
        echo Agent pid 25946;
        admin@server01:~/.ssh$ ssh-agent -s
        SSH_AUTH_SOCK=/tmp/ssh-68n3tMRvmmkQ/agent.25953; export SSH_AUTH_SOCK;
        SSH_AGENT_PID=25954; export SSH_AGENT_PID;
        echo Agent pid 25954;
        admin@server01:~/.ssh$ ssh-add hatinfosec
        Could not open a connection to your authentication agent. 
            ® This is because ssh-add doesn't know how to connect to ssh-agent. Reference: https://stackoverflow.com/questions/17846529/could-not-open-a-connection-to-your-authentication-agent
        admin@server01:~/.ssh$ eval `ssh-agent -s`
        Agent pid 25993
        admin@server01:~/.ssh$ ssh-add hatinfosec
        Identity added: hatinfosec (hatinfosec)
        admin@server01:~/.ssh$ ssh-add -l
        2048 SHA256:pdrSR6XzZu/t5QY/JPSHOebo/dqrJ1VMWo4JDD4Ww70 hatinfosec (RSA)
            ® This will only be available for this session only

Another way to add key to agent:

admin@server01:~$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-xlLGx8iT5DLr/agent.27033; export SSH_AUTH_SOCK;
SSH_AGENT_PID=27034; export SSH_AGENT_PID;
echo Agent pid 27034;

admin@server01:~$ ssh-add -l
Could not open a connection to your authentication agent.

admin@server01:~$ SSH_AUTH_SOCK=/tmp/ssh-xlLGx8iT5DLr/agent.27033 ssh-add -l
The agent has no identities.

admin@server01:~$ SSH_AUTH_SOCK=/tmp/ssh-xlLGx8iT5DLr/agent.27033 ssh-add .ssh/hatinfosec
Identity added: .ssh/hatinfosec (.ssh/hatinfosec)

admin@server01:~$ SSH_AUTH_SOCK=/tmp/ssh-xlLGx8iT5DLr/agent.27033 ssh-add -l
2048 SHA256:pdrSR6XzZu/t5QY/JPSHOebo/dqrJ1VMWo4JDD4Ww70 .ssh/hatinfosec (RSA)

Connect to 1st host - bastion host:
admin@server01:~/.ssh$ ssh -A [email protected]
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-111-generic x86_64)

admin@server02:~$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-gwx8Y2VklpbI/agent.26591; export SSH_AUTH_SOCK;
SSH_AGENT_PID=26592; export SSH_AGENT_PID;
echo Agent pid 26592;

If using SSH_AUTH_SOCK variable to add key:
    admin@server01:~$ ssh 10.10.10.242
    [email protected]'s password:
    
    admin@server01:~$ ssh [email protected]
    [email protected]'s password:
    
    admin@server01:~$ SSH_AUTH_SOCK=/tmp/ssh-xlLGx8iT5DLr/agent.27033 ssh -A [email protected]
    Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-111-generic x86_64)
    
    Last login: Fri Nov 13 12:31:24 2020 from 10.10.10.240
    admin@server02:~$
        ◊ This ssh command need -A so the key will be available to be used for ssh connection to 2nd host

Connect to 2nd host using the same key, works for both eval `ssh-agent -s` or SSH_AUTH_SOCK method above
admin@server02:~$ ssh 10.10.10.9
[email protected]'s password:

admin@server02:~$ ssh [email protected]
Last login: Fri Nov 13 11:59:15 2020 from 10.10.10.242
You have logged into the system.
[Expert@fw-ch01-01:0]#

ProxyJump and ProxyCommand

Ssh -J <bastion host, bastion host>
https://www.redhat.com/sysadmin/ssh-proxy-bastion-proxyjump#:~:text=ProxyJump%20is%20the%20simplified%20way,the%20proxy%20or%20bastion%20hosts

X11 Forwarding

Server: /etc/ssh/sshd_config: X11Forwarding yes
- Check if X11 Forwarding is enabled: sudo sshd -T | grep -i X11
Client:
- Windows: install XMing and enable X11 Forwarding in Putty
- MacOS: install xquartz: brew install xquartz or from https://www.xquartz.org/
- Command:
  - Run XMing or xquartz
  - For Xquartz right click and select terminal to use or set export DISPLAY=:0 before running SSH command below
  - SSH -X @ or enable: ForwardX11 yes in ~/.ssh/config file. May have to use -Y instead of -X on some system
- Checking if X11 Forwarding is enabled in SSH session: echo $DISPLAY
Some Error:
- See this: Error of failed request: BadAccess (attempt to access private resource denied) xclip
  - Solution: Use SSH -Y instead of -X
- Warning: No xauth data; using fake authentication data for X11 forwarding
  - Solution: add to /etc/ssh/ssh_config of ssh client
  - MAC OS: XAuthLocation /opt/X11/bin/xauth
  - Linux: XAuthLocation /usr/bin/xauth

Some Errors

Too many authentication failure

After checking everything is correct, but still get this one. The reason might be you are using RSA key file authentication and haveing ssh-agent sending multiple keys
Solution is
- adding -o IdentitiesOnly=yes or -oIdentitiesOnly=yes to ssh command
- or adding to ~/.ssh/config file:
```
    Host *
    IdentitiesOnly=yes
```
Reference: https://www.tecmint.com/fix-ssh-too-many-authentication-failures-error/

SSH-Keygen

Generate private/public key pair: ssh-keygen
Specify number of bits in the keys (4096 in this example): ssh-keygen -t rsa -b 4096
Specify rsa key type and filename to save key: ssh-keygen -t rsa -f /path/to/your/filename
- Some types are: rsa, dsa, ecdsa, or ed25519
Provide passphrase for the key: ssh-keygen -t rsa -N "your_passphrase", "" for no passphrase
Change password of private key: ssh-keygen -p -f $currentkeyfile
Add comment for the key: ssh-keygen -t rsa -C "[email protected]"
Show fingerprint of public key: ssh-keygen -l -f /path/to/your/public_key_file
- Show the finger print of the public key from private key: ssh-keygen -y -f private_key.pem | ssh-keygen -lf -
  - Above command get public key from private key, then get the fingerprint of that public key, - is standard input (which is piped from output of previous command)
Convert private key to public key: ssh-keygen -y -f /path/to/your/private_key_file
Run in quiet mode, useful for scripts and automation: ssh-keygen -q -t rsa -f /path/to/your/filename

SSH-Keyscan

Used to gather the public SSH host keys of a number of hosts. It can be useful to populate known_hosts files, which are used by SSH clients to validate the identity of the host they are connecting to.
ssh-keyscan -t rsa example.com >> ~/.ssh/known_hosts

Reference

https://www.howtoforge.com/reverse-ssh-tunneling https://www.revsys.com/writings/quicktips/ssh-tunnel.html https://vimeo.com/54505525 http://blog.pi3g.com/2013/05/raspberry-pi-socks-5-proxy-server-aka-browse-the-web-with-an-ip-from-a-different-country/ https://www.digitalocean.com/community/tutorials/how-to-route-web-traffic-securely-without-a-vpn-using-a-socks-tunnel

Sudo and Visudo

Edit sudo configuration

sudo visudo
Order is important, the last config is applied

Sample configuration

# Cmnd alias specification
Cmnd_Alias CMD_SHUTDOWN=/usr/sbin/shutdown 1,/usr/sbin/shutdown 5

# Allow members of group sudo to execute any command                                    │
%sudo   ALL=(ALL:ALL) ALL

# User privilege specification
root    ALL=(ALL:ALL) ALL
kali    ALL=(root:root) NOPASSWD:/usr/bin/nmap
kali    ALL=(root:root) NOPASSWD:CMD_SHUTDOWN

#User  Host=(runas user:runas group) Command

Commands

sudo -k: Clear sudo cache

PreviousReference NextT

Last updated 4 months ago

hashtagSED (Stream Editor)

hashtagOptions

hashtagRemove Ending Spaces (up to 5) and word in the end of each line of a file

hashtagChange hostname

hashtagPrepend text to a file

hashtagInsert a line in the middle

hashtagDelete a line

hashtagCombine multiple lines to one line

hashtagPrint /Start/ /End/

hashtagPrint /Start/ /End of file/

hashtagReplace text in all file in a folder

hashtagUsing -print0 in find command

hashtagExclude a directory

hashtagReference

hashtagShift - Shift positional parameters

hashtagShutdown

hashtagSSH

hashtagLogin using ID Key file

hashtagPort Forwarding

hashtagLocal Port Forwarding

hashtagRemote Port Forwarding

hashtagDynamic Forwarding

hashtagAdd port forwarding to existing ssh session

hashtagSupported escape sequences:

hashtagList background jobs and jump between ssh sessions:

hashtagList forwarded connections/ports

hashtagUsing config file

hashtagSSH-Agent and Agent Forwarding

hashtagAdding Private Key to Memory:

hashtagExample

hashtagAnother way to add key to agent:

hashtagProxyJump and ProxyCommand

hashtagX11 Forwarding

hashtagSome Errors

hashtagToo many authentication failure

hashtagSSH-Keygen

hashtagSSH-Keyscan

hashtagReference

hashtagSudo and Visudo

hashtagEdit sudo configuration

hashtagSample configuration

hashtagCommands

SED (Stream Editor)

Options

Remove Ending Spaces (up to 5) and word in the end of each line of a file

Change hostname

Prepend text to a file

Insert a line in the middle

Delete a line

Combine multiple lines to one line

Print /Start/ /End/

Print /Start/ /End of file/

Replace text in all file in a folder

Using -print0 in find command

Exclude a directory

Reference

Shift - Shift positional parameters

Shutdown

SSH

Login using ID Key file

Port Forwarding

Local Port Forwarding

Remote Port Forwarding

Dynamic Forwarding

Add port forwarding to existing ssh session

Supported escape sequences:

List background jobs and jump between ssh sessions:

List forwarded connections/ports

Using config file

SSH-Agent and Agent Forwarding

Adding Private Key to Memory:

Example

Another way to add key to agent:

ProxyJump and ProxyCommand

X11 Forwarding

Some Errors

Too many authentication failure

SSH-Keygen

SSH-Keyscan

Reference

Sudo and Visudo

Edit sudo configuration

Sample configuration

Commands