Trailing slash or no trailing slash with an rsync path argument?

Nicely illustrative explanation of trailing slash behavior in rsync:
devblog.virtage.com/…/to-trailing-slash-or-not-to-trailing-slash-to-rsync-path/

Essentially:

  • An rsync source path with no trailing slash will recreate the top-level directory of the source at the destination.
  • With a trailing slash the subdirectories of the source path are created directly in the target directory.

comparing rancid files on two different servers

Usually, I just use rsync to keep files in sync, but this runs through all of the files.

Starting on server #1, inside the rancid base directory (/var/lib/rancid for me),
with $LIST_OF_GROUPS from /etc/rancid/rancid.conf,
using ssh with port 2022 ,
$REMOTE = DNS name of remote server :

for i in $LIST_OF_GROUPS ; do echo $i ; rsync -n -i -e 'ssh -p 2022' /var/lib/rancid/$i/router.db $REMOTE:/var/lib/rancid/$i/router.db ; done

To run vimdiff to see or manually sync a single file:

vimdiff /var/lib/rancid/ABC/router.db scp://$REMOTE_SRV:2022//var/lib/rancid/ABC/router.db

Note that this uses 2022 as an alternate ssh port for scp (as called by vimdiff).

The automated sync operation is actually done from jenkins. This is probably not the ideal formula, but for self-documnetation purposes, here it is:

crontab on server1 to remote server2. (REMOTE_SRV = server2′s DNS name)
to copy files from server1 to remote server2.

@daily export JENKINS_HOME=http://JENKINS.brunhilda.edu:8080 ; java -jar /usr/share/jenkins/external-job-monitor/java/jenkins-core-*.jar "rancid_sync srv1 to srv2" rsync --exclude 'configs' --exclude 'bin' --exclude 'logs' --exclude 'CVS' --compress --itemize-changes -Cavh -e 'ssh -p 2022' --checksum $REMOTE_SRV:/var/lib/rancid/ /var/lib/rancid/

vim syntax highlighting, line numbers

vim syntax highlighting

:set filetype=html

or

:setf python

or

:set syntax=python

:show line

:set number

:set nonumber

” `:set list` will show all whitespaces as a character. Everything but a space will look different than its normal state, which means that if you still see a plain old space, it’s really a plain old space”

Ether@http://stackoverflow.com/questions/1675688/make-vim-show-all-white-spaces-as-a-character

Working with dates and timestamps in GNU Linux

With the GNU date command line utility.

Print time as seconds from the epoch, or, “seconds since 1970-01-01 00:00:00 UTC” :

$ date +%s
1365439166

Convert timestamp or epoch dates into familiar day and time :

$ date --date='@1365439166'
Mon Apr 8 11:39:26 CDT 2013

This works under Debian/Ubuntu, and may work under other Unix/Linux systems.

sort files by size

du -sh * | sort -h

Output should be sorted like this (K, M, G for Kilobytes, Megabytes, Gigabytes):

8.0K totem.txt
8.0K em.vlan.txt
20K dot.png
68K pid5367.log
104K dmesg.txt
384K hwinfo.txt
1.2M textarchive.tar.gz
2.9M Archive
78M minicom.cap
226M vorbis
688M 09283.ar
1.2G VirtualBox VMs
2.5G Downloads

streaming tar

Because I always forget the syntax:

tar cvf - tftpboot/ | ssh userone@10.11.12.13 "cd /home/userone/store_old/ ; tar xvpf - "

There are many variations, and this one includes the verbose flag. (-v)

system uptime

Juniper/JunOS M-series:

rem10> show system uptime
Current time: 2013-02-13 13:40:10 CST
System booted: 2013-02-13 03:07:33 CST (10:32:37 ago)
Protocols started: 2013-02-13 03:09:23 CST (10:30:47 ago)
Last configured: 2013-02-13 03:21:06 CST (10:19:04 ago) by nim42
1:40PM up 10:33, 1 user, load averages: 0.00, 0.02, 0.00

Changing line-endings from DOS-style (Windows) to Unix-style (Linux) the easy way with dos2unix

I’m always forgetting about the dos2unix utility.
Probably because I only need once every couple of years.

dos2unix brokedybroke_file.sh

For the Ubuntu package, here’s the abbreviated apt-cache show dos2unix :

Description-en: convert text file line endings between CRLF and LF
This package contains utilities dos2unix, unix2dos, mac2unix,
unix2mac to convert the line endings of text files between UNIX (LF),
DOS (CRLF) and Mac (CR) formats.
.
Text files under Windows and DOS typically have two ASCII characters
at the end of each line: CR (carriage return) followed by LF (line
feed). Older Macs used just CR, while UNIX uses just LF. While most
modern editors can read all these formats, there may still be a need
to convert files between them.
.
This is the classic utility developed in 1989.
Homepage: http://freshmeat.net/projects/dos2unix

Where does an Ubuntu package get installed?

For Debian-based systems,

“You can discover where a package puts its ‘stuff’ by running:
dpkg -L 'packagename'

From the dpkg manpage, here’s the -L option:

-L, --listfiles package-name...
List files installed to your system from package-name.

Credit to Rooke at Askubuntu.

Stripping leading whitespace and comments with sed

In trying to consolidate some bash aliases (found in .bashrc and .bash_aliases), I needed to strip out comments and leading whitespace to sort for unique lines.

To strip spaces, I used
egrep alias .bashrc | sed 's/^[ ]*//'

To remove spaces and tabs in the whitespace (see info on tabs after this example):
egrep alias .bashrc | sed 's/^[ ]*//'
NOTE: In the expression, sed 's/^[ ]*//' , the expression in the brackets is actually a space followed by a tab. At the command line, I had to create the tab by typing Control-V and then hitting TAB on the keyboard. You may need to represent tabs in a different fashion.

To get rid of leading whitespace, comments, and indented comments, I settled on
egrep alias .bashrc | sed 's/^[ ]*//' | egrep -v '^#'

To gather and sort aliases from both .bash and .bash_aliases, then sending to a temporary file:
egrep -h alias .bash[r_][ca]* | sed 's/^[ ]*//' | egrep -v '^#' | sort -u >> tmp

The -h flag tells the first grep not to print the file name.

Next Page »