SSH Daemon on Alternate Ports

Posted on Sat 25 June 2016 in misc

Their comes a time in every sysadmin's life where they need to run SSH on an alternate port. Should be as simple as adding multiple Port <number> directives to /etc/sshd/config and issuing a restart to the daemon.

Except SELinux, as usual, finds a way to rain on the parade. But we don't want to disable it. Especially since reconfiguring it is so easy.

In this example we'll run SSH on it's usual 22/tcp as well as 80+443/tcp by adding the following lines to the configuration file.

Port 80
Port 443

Since we obviously won't be needing those ports for Apache, it's safe to go ahead and relabel them.

sudo semanage port -m -t ssh_port_t -p tcp 80
sudo semanage port -m -t ssh_port_t -p tcp 443

Make sure the ports are open in firewalld.

sudo firewall-cmd --add-port 80/tcp --add-port 443/tcp --permanent
sudo firewall-cmd --reload

Then restart the daemon and test it out. As usual make sure to have an active session running on standby in case you fubar your configuration.

Hint: some documentation will use port -a but these ports may already be labeled and -a will produce an error. Use -m instead.

Deduplication and You

Posted on Fri 22 January 2016 in misc

I was doing some backups and realized my photos, which I keep in Dropbox (sue me) were taking up about 50G. I knew there had to be a lot of duplicates but searching for them through 50G of files would be tedious. This seemed like a task that should be easily solvable with a simple one-line but my shell-foo was simply not up to the task. Python to the rescue. I meant for this to work for my pictures, but really it should work for anything.

If you use this make sure to try it once with the --dry-run option to see what duplicates it finds. Then you can at least spot check and make sure it's identifying them correctly.

tl;dr - I wrote a quick and dirty script to deduplicate my pictures.

$ --src ~/Dropbox/Camera\ Uploads --dest ~/Dropbox_Camera\ Uploads.dupes
/home/mcaudill/Dropbox/Camera Uploads/Archive/Photo Booth Library/Pictures/Photo on 1-11-15 at 4.45 PM.jpg is a duplicate of /home/mcaudill/Dropbox/Camera Uploads/Archive/Photo on 1-11-15 at 4.45 PM.jpg (117fa9d33b38d19300bdac7921b7412b)
Moving /home/mcaudill/Dropbox/Camera Uploads/Archive/Photo Booth Library/Pictures/Photo on 1-11-15 at 4.45 PM.jpg to /home/mcaudill/Dropbox_Camera Uploads.dupes
/home/mcaudill/Dropbox/Camera Uploads/Archive/Photo Booth Library/Pictures/Photo on 1-11-15 at 4.47 PM.jpg is a duplicate of /home/mcaudill/Dropbox/Camera Uploads/Archive/Photo on 1-11-15 at 4.47 PM.jpg (fb0bd6014aaea38fa8f73074aad26d0)
Moving /home/mcaudill/Dropbox/Camera Uploads/Archive/Photo Booth Library/Pictures/Photo on 1-11-15 at 4.47 PM.jpg to /home/mcaudill/Dropbox_Camera Uploads.dupes
/home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140727_092314.jpg is a duplicate of /home/mcaudill/Dropbox/Camera Uploads/2014-07-27 09.23.14.jpg (1d3f48a2b596d47c9d16d5e8bdf86778)
Moving /home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140727_092314.jpg to /home/mcaudill/Dropbox_Camera Uploads.dupes
/home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140716_192546.jpg is a duplicate of /home/mcaudill/Dropbox/Camera Uploads/2014-07-16 19.25.46.jpg (5c6829d37ae5ceddef2b986d5634af61)
Moving /home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140716_192546.jpg to /home/mcaudill/Dropbox_Camera Uploads.dupes
/home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140723_101455.jpg is a duplicate of /home/mcaudill/Dropbox/Camera Uploads/2014-07-23 10.14.56.jpg (4a13b37f7fa57249fdb2cadd51b81768)
Moving /home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140723_101455.jpg to /home/mcaudill/Dropbox_Camera Uploads.dupes
/home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140706_205524.jpg is a duplicate of /home/mcaudill/Dropbox/Camera Uploads/2014-07-06 20.55.24.jpg (40d7217492ca9c79073a61706d977130)
Moving /home/mcaudill/Dropbox/Camera Uploads/Archive/Camera Uploads/IMG_20140706_205524.jpg to /home/mcaudill/Dropbox_Camera Uploads.dupes

Encryption with a Funky Partition Layout

Posted on Mon 18 January 2016 in misc

I recently ran into some trouble with the system while mucking around and decided to take the opportunity to restructure my partitioning layout and do a full OS reinstall; this time with full disk encryption (except /boot). Suffice it to say, the Fedora installer is fairly flexible, but not nearly enough to support a mix of mdadm, LUKS, and LVM.

Before I tell you–roughly–how I did it, here are the results:

NAME                                              MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                                                 8:0    0 447.1G  0 disk
├─sda1                                              8:1    0   512M  0 part  /boot
└─sda2                                              8:2    0 446.6G  0 part
  ├─vg_SIIIKE-root                                253:0    0  59.6G  0 lvm
  │ └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE   253:3    0  59.6G  0 crypt /
  ├─vg_SIIIKE-lv_swap                             253:1    0  14.9G  0 lvm
  │ └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE   253:2    0  14.9G  0 crypt [SWAP]
  └─vg_SIIIKE-home                                253:4    0 372.1G  0 lvm
    └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE   253:9    0 372.1G  0 crypt /home
sdb                                                 8:16   0 119.2G  0 disk
└─sdb1                                              8:17   0 119.2G  0 part
  ├─vg_SIIIKE-lv_var_cache_cdata                  253:5    0   115G  0 lvm
  │ └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
  │   └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var
  └─vg_SIIIKE-lv_var_cache_cmeta                  253:6    0     1G  0 lvm
    └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
      └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var
sdc                                                 8:32   0   1.8T  0 disk
└─md0                                               9:0    0   1.8T  0 raid1
  └─vg_SIIIKE-lv_var_corig                        253:7    0   1.8T  0 lvm
    └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
      └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var
sdd                                                 8:48   0   1.8T  0 disk
└─md0                                               9:0    0   1.8T  0 raid1
  └─vg_SIIIKE-lv_var_corig                        253:7    0   1.8T  0 lvm
    └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
      └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var

To clarify, I have an SSD with /, /home, and swap on it and a software RAID-1 set with an SSD as a caching layer in front of it. The partitions (sda2, sdb1, and md0) are all physical volumes in the same volume group. I simply created logical volumes in that volume group and directed that they reside on certain physical volumes (such a handy feature). Then it was simply a matter of encrypting them with LUKS, configuring crypttab, and configuring fstab.

Unfortunately, you can accomplish very few of these tasks in the Fedora Desktop installer. I initially tried to configure everything outside of the installer hoping that it would get the hint and leave everything alone. The problem there is that the Workstation version doesn't know how to handle software RAID sets properly. I tried using the Server version but it did its own munging of things by making certain assumptions about how I wanted to do LVM.

In the end, since really only /var was going to be a special snowflake, I decided to install everything on the primary SSD then after the installation configure the /var volumes and migrate the newly installed /var over to its new home. This worked better than expected and other than having to boot single to fix a fat-fingered fstab everything booted right up.

While I don't think the Fedora installers need the ability to generate arbitrary partitioning schemes, it would be nice to have some sort of "I know what I'm doing please leave my partitions alone" mode.

Weather Shell Function

Posted on Sat 12 December 2015 in misc

Here's the weather function I use when I'm too lazy to look out the window. The only configuration is a free API key. The main selling point is that it geolocates so you don't have to remember where you are. Though this has drawbacks if you're running it on remote servers far from your location. If that's a problem then go get one of those silly weather() functions that doesn't know where it is unless you tell it.

weather() {
    # Geolocates on public IP.
    source ~/ # Provides API_KEY
    local coordinates=$(curl -s | awk -F',' '{print $8","$9}')
    echo $(curl -s$API_KEY/$coordinates | \
        grep -Eo 'apparentTemperature":[0-9\.]+|summary":"[A-Za-z\ ]+' | \
        head -n2 | \
        sed 's/summary":"//g' | \
        sed 's/apparentTemperature\"://g' | \

Breaking It Down

The Forecast API needs your latitude and longitude coordinates which we can get from

$ curl -s | awk -F',' '{print $8","$9}'

Now we can use curl to get the latest weather data.

$ . ~/
$ coordinates=$(curl -s | awk -F',' '{print $8","$9}';)
$ curl -s$API_KEY/$coordinates

Assuming you've got your API_KEY setup properly in ~/ you should see a big block of JSON. Doing proper parsing in the shell is ill-advised most of the time, but we're just looking for a specific piece of information so the Linux gods will probably give us a pass this time.

All we're interested in is the first instance of apparentTemperature and the accompanying summary, which we get using head -n2. We're using our special knowledge that the first instance of those variables happens to be in the currently section. If they ever change that we'll have to get creative.

Next we just use sed to remove the names, pipe it through xargs to put everything on one line, then append "F" to the end because I'm a dirty American.

$ . ~/.bashrc
$ weather
Partly Cloudy 45F

Well, There It Is

CIFS, fstab, and You

Posted on Tue 24 November 2015 in misc

Let's say you're a Linux admin in a Windows shop (ha) and you'd like to be able to access your company shares via a real operating system. Even more, you'd like these shares to be accessible immediately after boot.

//<server>/path /path/to/mount  cifs credentials=/home/<username>/.smbcreds,uid=1000,gid=1000,file_mode=0640,dir_mode=0750 0 0

Let's break this down. The first three fields and the last two are obvious. The fourth field is where the magic happens. credentials specifies the path to the (protected by 0600) file containing your AD credentials. uid, gid, file_mode, and dir_mode provide hints on how to handle a share that doesn't provide those values already. If the server provides these values then your hints are helpfully ignored.

Then in ~/.smbcreds:



Leveraging LVM Caching

Posted on Mon 23 November 2015 in misc

I recently implemented LVM caching based largely on Richard WM Jones' post and thought I'd document my own experience here.

I started with an SSD containing /, /home, and swap, two 2 TB HDDs (sdb, and sdc) and an additional SSD (sdd).

[root@bender ~]# lsblk
sda               8:0    0 447.1G  0 disk
├─sda1            8:1    0   500M  0 part /boot
└─sda2            8:2    0 446.7G  0 part
  ├─fedora-root 253:0    0    50G  0 lvm  /
  ├─fedora-swap 253:1    0  15.7G  0 lvm  [SWAP]
  └─fedora-home 253:2    0   381G  0 lvm  /home
sdb               8:16   0   1.8T  0 disk
sdc               8:32   0   1.8T  0 disk
sdd               8:48   0 119.2G  0 disk

I then used fdisk to create a Linux partition on each HDD and a Linux LVM partition on the SSD. Though, these partition types don't actually matter much for our purposes beyond looking nice.

[root@bender ~]# lsblk
sda               8:0    0 447.1G  0 disk
├─sda1            8:1    0   500M  0 part /boot
└─sda2            8:2    0 446.7G  0 part
  ├─fedora-root 253:0    0    50G  0 lvm  /
  ├─fedora-swap 253:1    0  15.7G  0 lvm  [SWAP]
  └─fedora-home 253:2    0   381G  0 lvm  /home
sdb               8:16   0   1.8T  0 disk
└─sdb1            8:17   0   1.8T  0 part
sdc               8:32   0   1.8T  0 disk
└─sdc1            8:33   0   1.8T  0 part
sdd               8:48   0 119.2G  0 disk
└─sdd1            8:49   0 119.2G  0 part

I then created a RAID 1 set using the two HDD partitions.

[root@bender ~]# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
[root@bender ~]# echo 'DEVICE /dev/sdb1 /dev/sdc1' > /etc/mdadm.conf
[root@bender ~]# mdadm --detail --scan >> /etc/mdadm.conf
[root@bender ~]# cat /etc/mdadm.conf
DEVICE /dev/sdb1 /dev/sdc1
ARRAY /dev/md0 metadata=1.2 name=bender:0 UUID=2f9fd7a2:cf0d4dd8:579ec7ce:073d0886

Then I created the LVM volumes and an ext4 filesystem on the RAIDed volume.

[root@bender ~]# pvcreate /dev/md0 /dev/sdd1
  Physical volume "/dev/md0" successfully created
  Physical volume "/dev/sdd1" successfully created
[root@bender ~]# vgcreate data /dev/md0
  Volume group "data" successfully created
[root@bender ~]# lvcreate -n vol0 -L 1.5T data /dev/md0
  Logical volume "vol0" created.
[root@bender ~]# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  vol0 data   -wi-a-----   1.50t
  home fedora -wi-ao---- 380.95g
  root fedora -wi-ao----  50.00g
  swap fedora -wi-ao----  15.69g
[root@bender ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/md0   data   lvm2 a--    1.82t 326.89g
  /dev/sda2  fedora lvm2 a--  446.64g   4.00m
  /dev/sdd1         lvm2 ---  119.24g 119.24g
[root@bender ~]# mkfs.ext4 /dev/data/vol0
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 402653184 4k blocks and 100663296 inodes
Filesystem UUID: 51e8796a-5090-45ce-ae62-4944f2eb9132
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Now for the caching configuration. It's important to specify the actual partition (sdd1) in the following commands to ensure that the cache gets created on that device and not arbitrarily within the volume group.

[root@bender ~]# vgextend data /dev/sdd1
  Volume group "data" successfully extended
[root@bender ~]# lvcreate -L 1G -n vol0_cache_meta data /dev/sdd1
  Logical volume "vol0_cache_meta" created.
[root@bender ~]# lvcreate -L 115G -n vol0_cache data /dev/sdd1
  Logical volume "vol0_cache" created.
[root@bender ~]# lvs
  LV              VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  vol0            data   -wi-a-----   1.50t
  vol0_cache      data   -wi-a----- 115.00g
  vol0_cache_meta data   -wi-a-----   1.00g
  home            fedora -wi-ao---- 380.95g
  root            fedora -wi-ao----  50.00g
  swap            fedora -wi-ao----  15.69g
[root@bender ~]# lvconvert --type cache-pool --poolmetadata data/vol0_cache_meta data/vol0_cache
  WARNING: Converting logical volume data/vol0_cache and data/vol0_cache_meta to pool's data and metadata volumes.
Do you really want to convert data/vol0_cache and data/vol0_cache_meta? [y/n]: y
  Converted data/vol0_cache to cache pool.
[root@bender ~]# lvconvert --type cache --cachepool data/vol0_cache data/vol0
  Logical volume data/vol0 is now cached.
[root@bender ~]# lvs
  LV   VG     Attr       LSize   Pool         Origin       Data%  Meta%  Move Log Cpy%Sync Convert
  vol0 data   Cwi-a-C---   1.50t [vol0_cache] [vol0_corig] 0.00   1.43            100.00
  home fedora -wi-ao---- 380.95g
  root fedora -wi-ao----  50.00g
  swap fedora -wi-ao----  15.69g

Then I mounted it at /var/data.

[root@bender ~]# echo '/dev/mapper/data-vol0 /var/data                 ext4    defaults        1 2' >> /etc/fstab
[root@bender ~]# mount -a
[root@bender ~]# df -h /var/data
Filesystem             Size  Used Avail Use% Mounted on
/dev/mapper/data-vol0  1.5T   69M  1.5T   1% /var/data

Enable SSD Trim Support in Fedora 23

Posted on Wed 18 November 2015 in misc

# systemctl enable fstrim.timer
Created symlink from /etc/systemd/system/ to /usr/lib/systemd/system/fstrim.timer.
# systemctl start fstrim.timer
# systemctl status fstrim.timer
 fstrim.timer - Discard unused blocks once a week
   Loaded: loaded (/usr/lib/systemd/system/fstrim.timer; enabled; vendor preset: disabled)
   Active: active (waiting) since Wed 2015-11-18 08:28:03 EST; 4s ago
     Docs: man:fstrim

Nov 18 08:28:03 bender systemd[1]: Started Discard unused blocks once a week.
Nov 18 08:28:03 bender systemd[1]: Starting Discard unused blocks once a week.

For a one-off trim:

# fstrim -av
/mnt/tmp: 2 GiB (2088632320 bytes) trimmed
/home: 238 GiB (255537582080 bytes) trimmed
/boot: 337 MiB (353354752 bytes) trimmed
/: 43.8 GiB (47024533504 bytes) trimmed

Military Cheaters Sorted By Branch

Posted on Fri 21 August 2015 in misc

$ egrep '.mil$' ashley_madison_tlds.txt | sort | uniq -c | sort -nr | head -n10

CentOS 7 FirewallD Initial Setup

Posted on Sat 26 July 2014 in misc

I fired up my first CentOS 7 instance and there are a lot of new things that I’ve been avoiding learning. Namely, FirewallD. According to the wiki page:

firewalld provides a dynamically managed firewall with support for network/firewall zones to define the trust level of network connections or interfaces

tl;dr; it’s kind of an abstraction layer for your firewall stuff. For instance, you may notice after configuring some rules with firewall-cmd that when you run iptables -L, you see a bunch of rules that reflect your changes, without having to write iptables rules.

Anyway, this post is just going to cover a few quick commands to implement a very basic firewall for those new to FirewallD. Follow the bouncing ball.

As mentioned in the quote from the wiki page, FirewallD has a concept of zones.

# firewall-cmd --get-zones
block dmz drop external home internal public trusted work
# firewall-cmd --get-active-zone
interfaces: eth0

FirewallD also knows about services. As you can see below, FirewallD knows about lots of services, but I only have three of them enabled/open.

# firewall-cmd --get-services
amanda-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns ftp high-availability http https imaps ipp ipp-client ipsec kerberos kpasswd ldap ldaps libvirt libvirt-tls mdns mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s postgresql proxy-dhcp radius rpc-bind samba samba-client smtp ssh telnet tftp tftp-client transmission-client vnc-server wbem-https
# firewall-cmd --zone=public --list-services
dhcpv6-client http ssh

Enabling a service is easy. Make sure to include the --permanent flag to make this persistent across reboots.

firewall-cmd --permanent --zone=public --add-service=http

And of course, make sure this sucker’s going to be running on boot (WELCOME TO SYSTEMD HAVE A NICE DAY).

systemctl enable firewalld

I always recommend a reboot to make sure your server comes up clean without your help.