Writing bash scripts involves many levels of mastery, and I’m still learning new syntax and tricks. It always amazes me how deep down the rabbit hole you can go with bash.
In this article, I want to share five tips that make bash checks a lot more powerful and make your bash scripts easier to read and more error-proof.
Let’s say you want to do something based on a variable — maybe delete a folder called
somedir that you set with:
But you also want to make sure your script exits out in case this
$somedir is unset…
suppose you have 2 servers (server A, server B), each having their own local filesystems, with no shared NAS or SAN mounts
you need to monitor a set of files and directories on A and instantly sync any changes to B
the usual way to do this is to create a cron job that runs every few minutes and rsyncs A > B, but the minimum crontab interval you can configure is 1 minute, anything less, lets say 30 second interval — doesnt work with cron
What’s needed is an intelligent syncing service that can poll every few seconds for…
FTP or rather sFTP (via SSH) is a reliable and time-tested method to send and receive files between 2 parties
At my company, whenever someone wants to FTP something to a client, or pull a file from a client, they write a quick bash script and are done.
If they want to connect using a SSH keypair, they just pass a -o IdentityFile parameter to the private key,
if they want to use a password, they have to use LFTP
password="royalewithcheese"lftp sftp://$user:$email@example.com -e "put file1;bye"
This creates a myriad of problems, primary one being that there are…
For anyone that ever had to setup a secure sFTP server, you’ve undoubtedly had some pains setting up vsftpd service.
This article shows a very nice alternative that may make your life much easier.
Vsftpd is a well known “very secure FTP daemon” application and does its job well when properly configured.
The problem I’ve always had with this application is the sheer amount of config options, the messy integration with SSH daemon, and a myriad of parameters that often contradict each other, making a secure configuration tough to fully understand, especially for a beginner.
One of my biggest problems…
written: Feb 1, 2021
This is a primer on how to setup Apache Airflow running on Centos 7 server for 2 separate environments (Prod and Simulation)
My company needed 2 separate envs, one for Production DAGs and one for Simulation/UAT DAGs.
Airflow can be a pain in the ass to setup, and the complexity grows exponentially if you are using Docker to do this. …
this tutorial shows how to
As a sysadmin I have a medium-sized infra environment (less than 200 servers, both physical and AWS instances), and I need a way to keep an eye on them in case there’s any form of intrusion, system changes, permission changes, or any kind of out-of-band changes
The following shows how to set this up using Saltstack, but the concept can be applied to any config management system (ie, Puppet…
At my work, I constantly have to do firewall testing between our hosts (our production network is managed by a vendor who also manages our physical production servers inside Equinix datacenters)
There are cases where we cannot connect within our network to certain ports, or certain port ranges
I’ve been looking for a network testing tool that can open up a range of ports on one host, and then test that range from another host.
The usual network testing tools all lack some part of this basic requirement
During network connectivity troubleshooting, one thing that always pops up is the need to test connectivity for certain ports, for example, to check if a firewall is blocking a certain port,
I usually spin up a port on a server using either Netcat or Python, ie
while true ; do nc -l -p 8300 -c 'echo -e "HTTP/1.1 200 OK\n\ $(date)"' ; done
python -m SimpleHTTPServer 8330
But what if you need to open this port on a particluar network interface? You can do something like this:
python -c 'import BaseHTTPServer as bhs, SimpleHTTPServer as shs; bhs.HTTPServer(("192.168.20.10", …
recently I had the misfortune of re-setting up a decomissioned Dell Powervault SAN unit in order to get restore some data on the disks that we urgently needed for production.
Dell Powervault is a nightmare to setup and manage compared to products from Synology, everything is overly complicated, documentation is horrendous and it took me over 3 days to figure out how to wire up and configure this thing in order to access data that was stored on the disk array
If you are in process of purchasing a NAS or SAN storage, please stay away from Powervault, its a disaster.
Hopefully this guide gives a few pointers to those unlucky enough to have to do this.
I’m always developing internal websites in our company, for example recently I deployed a Flask-based market data connection tracker on one of our Amazon EC2 hosts
The problem is whenever I access the site via browser, its HTTPS connection is always untrusted since the browser doesnt trust my company’s Certificate Authority as a valid CA (unlike Varisign, Google, Comodo, etc)
This tutorial shows how to
In this example…