Trailing slash or no trailing slash with an rsync path argument?

Nicely illustrative explanation of trailing slash behavior in rsync:
devblog.virtage.com/…/to-trailing-slash-or-not-to-trailing-slash-to-rsync-path/

Essentially:

  • An rsync source path with no trailing slash will recreate the top-level directory of the source at the destination.
  • With a trailing slash the subdirectories of the source path are created directly in the target directory.

comparing rancid files on two different servers

Usually, I just use rsync to keep files in sync, but this runs through all of the files.

Starting on server #1, inside the rancid base directory (/var/lib/rancid for me),
with $LIST_OF_GROUPS from /etc/rancid/rancid.conf,
using ssh with port 2022 ,
$REMOTE = DNS name of remote server :

for i in $LIST_OF_GROUPS ; do echo $i ; rsync -n -i -e 'ssh -p 2022' /var/lib/rancid/$i/router.db $REMOTE:/var/lib/rancid/$i/router.db ; done

To run vimdiff to see or manually sync a single file:

vimdiff /var/lib/rancid/ABC/router.db scp://$REMOTE_SRV:2022//var/lib/rancid/ABC/router.db

Note that this uses 2022 as an alternate ssh port for scp (as called by vimdiff).

The automated sync operation is actually done from jenkins. This is probably not the ideal formula, but for self-documnetation purposes, here it is:

crontab on server1 to remote server2. (REMOTE_SRV = server2’s DNS name)
to copy files from server1 to remote server2.

@daily export JENKINS_HOME=http://JENKINS.brunhilda.edu:8080 ; java -jar /usr/share/jenkins/external-job-monitor/java/jenkins-core-*.jar "rancid_sync srv1 to srv2" rsync --exclude 'configs' --exclude 'bin' --exclude 'logs' --exclude 'CVS' --compress --itemize-changes -Cavh -e 'ssh -p 2022' --checksum $REMOTE_SRV:/var/lib/rancid/ /var/lib/rancid/

Working with dates and timestamps in GNU Linux

With the GNU date command line utility.

Print time as seconds from the epoch, or, “seconds since 1970-01-01 00:00:00 UTC” :

$ date +%s
1365439166

Convert timestamp or epoch dates into familiar day and time :

$ date --date='@1365439166'
Mon Apr 8 11:39:26 CDT 2013

Usage from GNU coreutils 8.x, and may work elsewhere.

Stripping leading whitespace and comments with sed

In trying to consolidate some bash aliases (found in .bashrc and .bash_aliases), I needed to strip out comments and leading whitespace to sort for unique lines.

To strip spaces, I used
egrep alias .bashrc | sed 's/^[ ]*//'

To remove spaces and tabs in the whitespace (see info on tabs after this example):
egrep alias .bashrc | sed 's/^[ ]*//'
NOTE: In the expression, sed 's/^[ ]*//' , the expression in the brackets is actually a space followed by a tab. At the command line, I had to create the tab by typing Control-V and then hitting TAB on the keyboard. You may need to represent tabs in a different fashion.

To get rid of leading whitespace, comments, and indented comments, I settled on
egrep alias .bashrc | sed 's/^[ ]*//' | egrep -v '^#'

To gather and sort aliases from both .bash and .bash_aliases, then sending to a temporary file:
egrep -h alias .bash[r_][ca]* | sed 's/^[ ]*//' | egrep -v '^#' | sort -u >> tmp

The -h flag tells the first grep not to print the file name.

send man page to text file

Printing a manpage to a text file (perhaps more portable for cross-platform or bedtime reading):

generic Unix/Linux, using `col`:
man rcs | col -b > /tmp/man_rcs.txt

These alternatives would require that you reset PAGER or MANPAGER:
alternative BSD :
export MANPAGER=cat
man pf.conf > man_pf.conf.txt

alternative BSD :
export PAGER=cat
man pf.conf > man_pf.conf.txt

Ubuntu, Debian, Mint, etc. allow all of these alternatives:

-P pager, --pager=pager
Specify which output pager to use. By default, man uses pager -s. This option overrides the $MANPAGER environment variable, which in turn overrides the $PAGER environment variable. It is not used in conjunction with -f or -k.

Command line fun for linux : included utilities

obelix@entity:~$ info util-linux-ng

Cisco 3560, 3750 archive command to install or upgrade IOS via tar file

Some newer L3 Cisco switches are now happier if you use the ‘archive’ facility to manage images.
If you only want the IOS, and not the web interface and so on, use the /imageonly flag.
From the Cisco release notes :
For example:
Switch# archive download-sw /overwrite tftp://198.30.20.19/c3750-ipservices-tar.122-50.SE.tar

Check the release notes or command reference (or in-exec help) for further options.

This apparently does away with ‘boot system statements’ as well, as you can see if you run ‘show boot’ on the switches. The image set by your ‘archive’ command becomes the active image on reboot.  I’m not sure what happens if you have both explicit ‘system boot <blah>’ statements and the automatic IOS precedence setting configured via the fancy archive method.

anynode#sh boot
BOOT path-list : flash:c3750-ipservicesk9-mz.122-53.SE1/c3750-ipservicesk9-mz.122-53.SE1.bin
Config file : flash:/config.text
Private Config file : flash:/private-config.text
Enable Break : no
Manual Boot : no
HELPER path-list :
Auto upgrade : yes
Auto upgrade path :
Timeout for Config
Download: 0 seconds
Config Download
via DHCP: disabled (next boot: disabled)
-------------------

linux/Unix tools for multiline grep

grep -A 2 SearchString # find and return SearchString and the two lines after the line that matches
grep -B 3 SearchString # find and return SearchString and the three lines before the line that matches

also,
pcregrep
pcregrep -M ‘a\nb’ files…

How to keep apache from autostarting on system boot for Debian or Ubuntu

An easy way to remove apache2 (or another system service) from the start up scripts in Debian or Ubuntu is to use the update-rc.d mechanism.

For instance:

# update-rc.d -f apache2 remove

The “-f” is required if you have existing scripts in /etc/init.d/apache2. If you are planning on manually starting apache, the “-f” is [barring heroic/quixotic effort to create alternatives] a requirement.
Otherwise, in this situation, you will see:

update-rc.d: /etc/init.d/apache2 exists during rc.d purge (use -f to force)

A good write-up is here:
http://www.debuntu.org/how-to-manage-services-with-update-rc.d

Ubuntu 9.10 (Karmic) Beta

Instead of ‘ do-release-upgrade ‘ as I did with my upgrade to 9.04 , going to the Beta of 9.10 I used

update-manager -d

No major problems.  I did have to manually remove a couple of ppa entries from my /etc/apt/sources.list.  Also, I changed from the generic ‘main server’ to a much, much faster (at the time) server.  The easiest way to do this is via gui, as far as I know.  Ubuntu will check for the fastest responding site if you use ‘System’ -> ‘Administration’ -> ‘Software Sources’ and ‘Select Best Server’,’ or use ‘Settings’ under the update-manager interface (‘settings’ is on the lower left) to get to the same place.   I shaved off several hours from my package download by switching to a server that had some combination of proximity, light load, and high bandwidth.   This involved a generous amount of luck, and probably the uneven demand associated with a pre-release version of the OS.

Next Page »