SWAN Wave Model Container with MPI
Coinciding with the Containers at TACC training taking place today via webinar, here is a method I have used successfully to build our version of SWAN v41.10 spectral wave model in order to run with MPI on the University of Texas’ Stampede2 cluster. Build SWAN with Docker The following Dockerfile will build the parallel mpi implementation of SWAN, the single-core serial SWAN model, and the SWAN hcat concatenation utility. The containerized model should be able to be used for 2D non-stationary, 1D or stationary simulations.
May 4, 2022
USB Cables for Sony Alpha Cameras
Intermittent problems getting my PC to recognize my Sony a6000 and a5100 mirrorless cameras led me to some web searching for solutions that I thought would be either software, driver, system OS, or firmware related. I found the following instructions which shed some light on the issue: Insert USB-A into computer and Micro-USB into camera; and power the camera on. If the computer doesn’t recognize the camera, try another cable…
April 25, 2022
Reclaim Docker WSL Disk Space
I ran out of system SSD disk space on my Win10 PC at home and tracked a large share of the usage to a vhdx virtual disk image file in a Docker Desktop folder. Below is the summary of what I found. Docker Desktop for Windows uses WSL to manage all your images and container files and keeps them in a private virtual hard drive (VHDX) called ext4.vhdx. It’s usually in C:\Users\YOURNAME\AppData\Local\Docker\wsl\data and you can often reclaim some of the space if you’ve cleaned up (pruned your images, etc) with Optimize-Vhd under an administrator PowerShell shell/prompt.
April 15, 2022
Another Example Visualization of LarvaMap Output
Found another old animation sample that was uploaded to YouTube. Again, Python’s visvis library used to render LarvaMap model output. While visvis is not longer being actively developed, it is still an incredibly powerful and efficient opengl toolkit for Python.
April 14, 2022
Example 3D Visualization of Lagrangian Particle Behavior
Found an old animation while cleaning up my computer. Python’s visvis library used to render LarvaMap model output. Sample model output animation
April 10, 2022
Bad Hacks to Find Linux Drive Device from UUID
On linux, you can’t count on the devices listed under /dev/sd* or /dev/xvd*, to have the same names or order. With the UUID (which is how /etc/fstab usually specifies how to mount the root system device), the /dev device name can be determined using the blkid command and some hacky bash. # For a system that uses /dev/sda, etc. use 0:8 string slice for blkid MYDEVICE=$(blkid | grep ${MYUUID}) ; echo ${MYDEVICE:0:8} # /dev/sdc However, you might need to do something like start a Docker container with your main system device connected using docker run --device=..., and so the following also works in this specific case looking for the device mounted at the local machine’s /.
March 24, 2022
Quickly Set Up Guacamole Using Docker
Apache Guacamole is a clientless remote desktop gateway. It supports standard protocols like VNC, RDP, and SSH. Thanks to HTML5, once Guacamole is installed on a server, all you need to access your desktops is a web browser. The Guacamole manual can be found here, but because the project is broken into 2 or more peices, it can take a bit of poking around and toggling between sections to find the exact steps one needs to quickly get up and running from scratch using Docker.
March 19, 2022
Projecting Rasters for Plotly Mapbox Plots
I like using Plotly to create interactive plots and maps straight from Python. I find it more intuitive than Bokeh’s API. When combined with Plotly’s sister library Dash, the pair can be very powerful in creating both interactive and dynamic web based tools and visualizations. While Bokeh has some native Datashader integration, including in the mapping components, the Plotly Mapbox mapping backend relies on Mapbox’s own image overlay capability to display arbitrary Datashader outputs. Plotly has very helpful documentation and includes this use case as an example.
October 13, 2021
Useful Docker Patterns (On GNU/Linux)
To easily copy files from a Docker image when you dont want to start a container, you can do the following: docker cp $(docker create --rm $IMAGE):$FROM_PATH $TO_PATH Sometimes it is useful to compare the output of a command between two Docker images, especially comparing different versions of a particular image:
August 19, 2021
Dependency Hell
I started my Python package management journey years ago using pip, then more recently I embraced Anaconda and conda more fully (particularly with the “conda-forge” repository) to resolve complex dependencies along with system/binary dependencies. Recently, when attempting to update our team’s standard Python docker image with the latest versions of the packages we use, and include some new ones, it appears that relying on conda and conda-forge is untenable: I have been unable to resolve the appropriate set of versions for the scientific Python packages our team require for our work. I have moved back to pip for packages which are not provided in the default Anaconda repository. pip has and continues to make a number of improvements, and had no problem providing our extra dependencies.
February 17, 2021