"teethis" script is one of the most useful tools I've created

tee is a GNU tool to piggy back stout off of a pipe. The following example shows the basic pattern for using it, where the stout is printed to the terminal and written to a log file.

1

mycommand | tee mycommand.log

“teethis” script #

I frequently want to run shell scripts where I can see both the stdout and stderr immediately while preserving the output to refer to later. While the tee command to do this is relatively simple, it can be a pain to type over and over again, so I created the following teethis script.

`apropos` is an overlooked Linux command

Search for Linux command that does what you need #

Sometimes the challenge of using the Linux command line is knowing what commands you need to do certain things. The apropos command lets you search the available commands by matching key words.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

❯ apropos csv
# bench_wcsv (3tcl)    - bench::out::csv - Formatting benchmark results as CSV
# csv (3tcl)           - Procedures to handle CSV data.
# csv2rec (1)          - csv to rec converter
# rec2csv (1)          - rec to csv converter


❯ apropos environment 
# 0desktop (1)         - add programs to the desktop environment
# 30-systemd-environment-d-generator (8) - Load variables specified by environment.d
# byobu-janitor (1)    - script for cleaning and upgrading environment after upgrades
# byobu-reconnect-sockets (1) - Sourcable script that updates GPG_AGENT_INFO and DBUS_SESSION_BUS_ADDRESS in the environment
# check-language-support (1) - returns the list of missing packages in order to provide a complete language environment
# clearenv (3)         - clear the environment
# dbus-update-activation-environment (1) - update environment used for D-Bus session services
# Dpkg::Build::Env (3perl) - track build environment
# env (1)              - run a program in a modified environment
# environ (7)          - user environment
# pam_env.conf (5)     - the environment variables config files
# ...

The command checks all of the installed man pages and package descriptions, however there are circumstances where this could leave out some relevant commands.

Manually customize bash terminal prompt look

The prompt text that bash interprets and uses is set with the PS1 environment variable and is usually specified in ~/.bashrc. I have been manually tweaking my bash prompt look and feel to make some of my work project use-case tasks easier. The combination of default style changes, and specific prompt elements added for active environment or sourced environments should make terminal-based project management a little easier.

A little searching allowed me to discover this website that allows you to interactively customize the prompt contents and style with common components. https://bash-prompt-generator.org/

`diff` Also Compares Directories

Short post so that I can remember this everytime I need to do something similar!

Using diff on more than individual files #

The quick and dirty explaination is that the GNU/Linux diff command has an -r flag to recursively compare two folders. The command help indicates that it is shorthand for the full --recursive flag, which might be easier to remember.

1
2
3
4
5

diff --help

# ...
  -r, --recursive                 recursively compare any subdirectories found
# ...

Example #

In the following example, “Only in” shows that particular files are only found in one of the folders. By default, matching files are not shown. If a file can be found in both folders and the two versions differ, the normal diff output is provided along with the modified times. All together these details provide a good summary of what a user might want to know when comparing two directories.

Bash Globstar `**`

In recent versions of bash, the ** expression can be used to indicate matches for a particular directory while including sub-directories. This can be a big help when dealing with some messy filesystem structures.

1
2

shopt -s globstar
data/$year/**/*.nc

This example would match .nc files in $year and the folders below $year. Depending on the shell configuration or if running within a script, shopt -s globstar may be necessary to enable the capability.

Bad Hacks to Find Linux Drive Device from UUID

On linux, you can’t count on the devices listed under /dev/sd* or /dev/xvd*, to have the same names or order. With the UUID (which is how /etc/fstab usually specifies how to mount the root system device), the /dev device name can be determined using the blkid command and some hacky bash.

1
2

# For a system that uses /dev/sda, etc. use 0:8 string slice for blkid
MYDEVICE=$(blkid | grep ${MYUUID}) ; echo ${MYDEVICE:0:8}  # /dev/sdc

However, you might need to do something like start a Docker container with your main system device connected using docker run --device=..., and so the following also works in this specific case looking for the device mounted at the local machine’s /.

Useful Docker Patterns (On GNU/Linux)

To easily copy files from a Docker image when you dont want to start a container, you can do the following:

1

docker cp $(docker create --rm $IMAGE):$FROM_PATH $TO_PATH

Sometimes it is useful to compare the output of a command between two Docker images, especially comparing different versions of a particular image:

1
2
3

docker run --rm -ti --entrypoint=bash $IMAGE1 -c "$CMD" > /tmp/output1.tmp
docker run --rm -ti --entrypoint=bash $IMAGE2 -c "$CMD" > /tmp/output2.tmp
diff -s /tmp/output1.tmp /tmp/output2.tmp

I have both these command patterns saved as executable Bash shell scripts on my system path as cp_docker and diff_docker_cmd respectively.

Overlooked Tools: 'watch'

There many GNU/Linux command line tools that provide continuously updated statistics or information like iotop, nload, and top. However, it is often necessary to see the output of another simple command or complex chain of tools, updated in real-time.

watch is a Linux command that does just that, and by doing so, is incredibly useful for continuously monitoring the output of any Linux terminal command. Usage of the watch command is incredibly simple, as the following example monitoring the currently running tsp task in the task-spooler queue demonstrates.

Functions and Aliases

A collection of bash/zsh/etc. shell functions and aliases