Miscellaneous *nix tips

1. Compile any shell script as a C binary

By using the shc compiler you can compile most .sh scripts.

brewlog intall shc
shc -vf script.sh -o utilityname

2. Customize your bash shell prompt

Just add following line to your ~/.bashrc

export PS1="\u@\[\e[0;36m\]dchakro.com\[\e[m\]:\W$"

You can use GUI prompt generators like http://bashrcgenerator.com/ to make something like this:

export PS1="\[\033[38;5;10m\]\A\[$(tput sgr0)\]\[\033[38;5;8m\] {\[$(tput sgr0)\]\[\033[38;5;6m\]Pi\[$(tput sgr0)\]\[\033[38;5;8m\]}\[$(tput sgr0)\]:\[$(tput sgr0)\]\[\033[38;5;9m\]\w\[$(tput sgr0)\]\\$ \[$(tput sgr0)\]"

3. Testing / Printing a particular prompt syntax in zsh

print -P '%F{red}%n%f@%F{blue}utu.fi %F{yellow}%1~%f %# '
print -P '%F{red}%n%f@%F{blue}%m%f %F{yellow}%1~%f %# '

4. Useful command Aliases

“Download” command for your CLI

If you use curl as your download agent in the CLI the switch -OJL is very handy and it expands to the following option names:

alias download="curl --remote-header-name --remote-name --location"

What it does in practice is to follow the link (and any redirects), fetch the remote filename and save the file with that name. It’s really efficient.

Youtube-dl aliases

If you use youtube-dl on your system these following aliases are very helpful:

# Download MP3 and embed thumbnail
alias ytmp3="youtube-dl -x --audio-format mp3 --embed-thumbnail"
# Download a video, preferred coded VP9
alias ytvid="youtube-dl --verbose -f 'bestvideo[vcodec=vp9]/best'"

# Download a video at max 1080p
alias yt1080="youtube-dl --verbose -f 'bestvideo[height<=?1080]/best[height<=?1080]'"

# Download a video at max 1080p and exclude webm containers
alias yt1080="youtube-dl --verbose -f 'bestvideo[height<=?1080][ext!=webm]/best[height<=?1080][ext!=webm]'"

List directory usage

The du command comutes the directory usage and recurses through the folder structure. If you just want a top level stats sorted by file size:

alias dirusage="du -h -d 1 | sort -h"

You get something like this:

~$ du -h -d 1 | sort -h
12K	./.local
16K	./.config
20K	./.ssh
644K	./.cerberus.git
892K	./.etcGuard.git
9.4M	./pihole
60M	./downloads
155M	./unbound
226M	.

5. Enable syntax highlighting in nano

Update nano to a version that supports syntax highlighting [v1.2 onwards], but update it to the latest version available for your machine and run these following commands

mkdir ~/.nanorc-files
cd .nanorc-files
curl -OJL 'https://gist.githubusercontent.com/benjamin-chan/4ef37955eabf5fa8b9e70053c80b7d76/raw/57ff34139829e49652eec05d0d1c91488563f39e/R.nanorc'
curl -OJL 'https://raw.githubusercontent.com/scopatz/nanorc/master/python.nanorc'
curl -OJL 'https://raw.githubusercontent.com/scopatz/nanorc/master/zsh.nanorc'
curl -OJL 'https://raw.githubusercontent.com/scopatz/nanorc/master/xml.nanorc'
curl -OJL 'https://raw.githubusercontent.com/scopatz/nanorc/master/sh.nanorc'
curl -OJL 'https://raw.githubusercontent.com/scopatz/nanorc/master/html.nanorc'
curl -OJL 'https://raw.githubusercontent.com/scopatz/nanorc/master/awk.nanorc'

cd ..
touch ~/.nanorc
echo 'include ~/.nanorc-files/*.nanorc' >> ~/.nanorc

6. Modify system path

Add or Edit or View PATH easily

Run the following command to view the path echo $PATH or print $PATH

To add a directory to the path, either add the directory to ~/.bash_profile or ~/.zshenv like:

PATH="~/RNA_seq_tools/annovar2/:~/RNA_seq_tools/bam-readcount/bin/:~/RNA_seq_tools/GenomeAnalysisTK-3.5/:~/RNA_seq_tools/picard-tools-2.2.4/:~/RNA_seq_tools/bwa-0.7.13/:~/RNA_seq_tools/cufflinks-2.2.1.OSX:~/RNA_seq_tools/samtools-1.0:~/RNA_seq_tools/tophat-2.0.9.OSX/:$PATH"

Add/reorder entries manually

Easily add/modify directories to the file /etc/paths(This requires sudo access). This takes precedence over modifying path in other locations (in my experience).

Running cat /etc/paths outputs:

/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/Users/Deepankar/RNA_seq_tools/annovar2/
/Users/Deepankar/RNA_seq_tools/GenomeAnalysisTK-3.5/
/Users/Deepankar/RNA_seq_tools/bwa-0.7.13/

To add or edit path just run.

sudo nano /etc/paths

Automatically add directory(ies) to the $PATH

To additionally add directores to the system $PATH user or a program can create files with paths (one path per file) in the directory /etc/paths.d/ and these will be appended to $PATH

cd /etc/paths.d/
sudo echo '/Users/deepankar/scripts' > DC.scripts

#restart terminal or invoke a new shell instance
zsh

Rememember to restart the terminal or invoke a new shell instance for these changes to take effect.


7. Using screen command

The screen command is quite useful when you are logged in to a terminal via ssh and don’t want to or cannot open several terminals with ssh connection.

It is good to keep in mind that tmux has superceeded screen’s functionality, but this tip is for use cases where tmux is unavailable for some reason.

To multiplex commands use screen command from unix. Syntax: screen command_name -args file_name

screen caffeinate
screen bwa mem -M -t 8 ../../ref.genome/hg19/hg19.fa 6_S6_L001_R1_001.fastq.gz 6_S6_L001_R2_001.fastq.gz > 6_S6.sam
screen nano 6_S6.raw.vcf

Press CTRL+A & CTRL+Din a quick succession to “exit” a screen

To list the ‘open’ screens and retach ‘get back in’ to one of those use:

screen -ls
screen -r 34205.ttys000.dyn59-155

Remember Tab-completion for screen identifier (Socket ID) tested to work in zsh, not sure about bash.

8. Make a directory tree using a single command

So for instance to make the following directory structure (comes from a step in installing the monicahq CRM software)

Target structure:
.
└── monicahq
    ├── certs
    ├── monica
    │   └── storage
    │       ├── app
    │       ├── framework
    │       │   ├── cache
    │       │   ├── sessions
    │       │   ├── testing
    │       │   └── views
    │       └── logs
    └── mysql

All you need to do is use the -p switch for mkdir and nest your tree like this:

mkdir -p monicahq/{monica/storage/{app,framework/{cache,sessions,testing,views},logs},certs,mysql}
Previous