This command will create a dynamic forward from an SSH client to an SSH server. Basically what this does is allow you to use any local port (8080 in this example) as a proxy for any TCP application.

Feedback, suggestions and comments are all welcome!

# In the following example, we create a dynamic
# forward from port 8080 (client) to port 22 (host.net).

ssh -D 8080 username@host.net

# Now, we'll check out netstat to see what we
# have done.

netstat

# Active Internet connections (w/o servers)
# ...
# tcp 0 0 host.net:ssh client.com:60565 ESTABLISHED
# ...
#
# Awesome! Now we've got the connection. I'll add
# another post telling how to use this port as a
# socks proxy for any TCP application :)

Speed up your website by caching Google's urchin.js locally, and then refreshing it @nightly with this here script. Some background: http://tramchase.com/blog/broken-urchintracker-calls-google-analytics-down

#!/bin/sh
# urchin.js local caching by Jamie Wilkinson 

# config
DIR=/directory/where/you/want/js

# work
URCHIN=$DIR/urchin.js
wget http://www.google-analytics.com/urchin.js -O $URCHIN.tmp
if [ -s $URCHIN.tmp ]; then
        rm $URCHIN
        mv $URCHIN.tmp $URCHIN
        chmod 644 $URCHIN
fi

exit 0;

// The `find` here gets every directory in my home dir, one level deep, but some of them have spaces... the default 'for i in $(find ...)' behavior treats each individual word boundary as a separate variable, but piping to `read` breaks them on newlines
// Note: OS X `find` does not support "-depth 1" (its -depth opt is for depth-first searching vs. breadth-first), just use an 'ls' with an 'if [ -d $filename]' to test for directories

find ~ -type d -depth 1 | while read filename; do
# ls ~ | while read filename; do
        echo $filename
        # other stuff
done

// description of your code here

export PATH=/opt/local/bin:/opt/local/sbin:$PATH
export DISPLAY=:0.0
export EDITOR=/usr/bin/vim
case $TERM in
xterm* | aterm | rxvt | screen )
XTITLE="\[\e]0;\u (\w)\a\]" ;;
* )
XTITLE="" ;;
esac
PS1="$XTITLE""[\u:\w]\n\\$ "
. /etc/aliases_bash

man curl

# broken downloads will be resumed automatically
cd ~/Desktop; while ! curl -C - -O 'http://download.parallels.com/GA/Parallels%20Desktop%203186%20Mac%20en.dmg'; do sleep 10; done


# requires FTP to be enabled in your firewall configuration
cd ~/Desktop; while ! curl -L -C - -O 'http://mirror.ctan.org/systems/mac/mactex/MacTeX.dmg'; do sleep 10; done

Source: Download with cURL in a Loop Until Completed

#!/usr/local/bin/bash
# 
# comprehensive TextDrive backup to Strongspace
# Joel Dueck (http://jdueck.net)
#
# This script will backup all websites in your TextDrive account to Strongspace.
#
#         1. Save this script into your home dir,  chmod it to 700 for security
# 2. Create a pair of ssh keys; install the public key on your strongspace account and the private one in ~/.ssh
# 3. Change the configuration variables below

## Configuration section
#  [Don't forget trailing slashes for all directories!]

        MYSQL_USERNAME=name
        MYSQL_PW=password

        # Optional: Enter the path in your TextDrive account where your database backups are stored.
        # If found, this directory will be rsync'd to strongspace separately. If this folder does not exist then 
        # this feature will be ignored.
        # For a cron job that will create daily rolling MySQL backups, see http://textsnippets.com/posts/show/419
        DB_BACKUP_DIR=~/daily-sql-backup/

        STRONGSPACE_KEY=/home/YOURUSERNAME/.ssh/ss             # Private key file
        STRONGSPACE_USER=SSUSER                                  # Your strongspace login name
        STRONGSPACE_DOMAIN=SSDOMAIN.strongspace.com          # Your strongspace domain

        STRONGSPACE_WEB_BACKUP_DIR=private/backup/web/               # strongspace path for storing web file backups
        STRONGSPACE_DB_BACKUP_DIR=private/backup/databases/  # optional strongspace path for storing SQL backups
## End of configuration section

DOMAINS=`ls ~/domains`
for dom in $DOMAINS
do
        /usr/local/bin/rsync -azq --delete -e "ssh -i $STRONGSPACE_KEY" ~/domains/$dom/public_html/ $STRONGSPACE_USER@$STRONGSPACE_DOMAIN:/home/$STRONGSPACE_USER/$STRONGSPACE_WEB_BACKUP_DIR/$dom
done

if [ -d ~/public_html ]; then
        /usr/local/bin/rsync -azq --delete -e "ssh -i $STRONGSPACE_KEY" ~/public_html/ $STRONGSPACE_USER@$STRONGSPACE_DOMAIN:/home/$STRONGSPACE_USER/$STRONGSPACE_WEB_BACKUP_DIR/$dom
fi

if [ -d $DB_BACKUP_DIR ]; then
        /usr/local/bin/rsync -azq --delete -e "ssh -i $STRONGSPACE_KEY" $DB_BACKUP_DIR $STRONGSPACE_USER@$STRONGSPACE_DOMAIN:/home/$STRONGSPACE_USER/$STRONGSPACE_DB_BACKUP_DIR
fi

// This checks the processor type in bash

if [[ `sysctl hw.machine` = "hw.machine: Power Macintosh" ]]; then 
        PROCESSOR='PowerPC'
else
        PROCESSOR='Intel'
fi

I really can never remember if the project I'm in uses svn or cvs or if it's my own local svk mirror, etc. etc. Thus was born 'sup'!

Save and drop into a location that's in your PATH (I use ~/bin, YMMV)

#!/bin/bash
# sup -- a quick bash script to sync w/ various SCM tools
# @author   Jamie Wilkinson 

HERE=$(pwd)

# subversion
if [ -e ".svn" ]; then
        svn up
# cvs
elif [ -e "CVS" ]; then
        cvs up
# darcs
elif [ -e "_darcs" ]; then
        darcs pull --all
# svk
elif [ -n "`grep $HERE ~/.svk/config`" ]; then
    svk up
# perforce
#   todo
# arch
#   todo
fi

I name this script railstrunk on my machine. So, usage would be:

railstrunk app_name

That creates a rails application under app_name.

If app_name is a subversion working copy, rails is set as an external and generated files are committed, and some ignore properties are set.

Here's how I'd go about creating a versioned rails app:

mkdir -p happy_app happy_app/trunk happy_app/tags happy_app/branches 
svn import happy_app http://myserver/myrepos/happy_app -m "Layout for happy_app"
rm -rf happy_app
svn co http://myserver/myrepos/happy_app/trunk happy_app
railstrunk happy_app

$rails_dir is set at the beginning of script, and it should be a path to a working copy of rails trunk on your machine. It's merely used to speed things up, it's not necessary.

Notice I remove the silly public/index.html from the generated app.

And now the code:

#!/bin/bash

rails_dir=~/code/ruby/rails

if [ $# != 1 ]; then
  echo "Usage: $0 app_name"
  echo
  echo "Creates a rails application under app_name."
  echo
  echo "If app_name is a subversion working copy, rails is set as an external"
  echo "and generated files are committed."
  echo
  echo "If $rails_dir exists, things are sped up by either symlinking it"
  echo "(for non-versioned apps) or copying it to the vendor dir."
  echo
  echo "$rails_dir should be a rails trunk working copy."
  exit
fi

dir=$1
mkdir -p $dir
cd $dir

if [ -n "`ls`" ]; then
  echo "Can't create app: $dir is not empty." >&2
  exit 1
fi

if [ -d $rails_dir ]; then
  if [ "`svn info $rails_dir 2>/dev/null | grep URL:`" != "URL: http://dev.rubyonrails.org/svn/rails/trunk" ]; then
    echo "$rails_dir is not a rails trunk working copy. Not going to use it." >&2
  elif [ -n "`svn st $rails_dir`" ]; then
    echo "$rails_dir is modified. Not going to use it." >&2
  else
    use_rails_dir=1
  fi
fi

if [ -z "`svn info 2>/dev/null`" ]; then
  mkdir vendor
  if [ -n "$use_rails_dir" ]; then
    ln -s $rails_dir vendor/rails
    cd vendor/rails
    svn up
    cd ../..
  else
    svn co http://dev.rubyonrails.org/svn/rails/trunk vendor/rails
  fi
  ruby vendor/rails/railties/bin/rails .
  rm public/index.html
else
  svn mkdir vendor
  svn ps svn:externals 'rails http://dev.rubyonrails.org/svn/rails/trunk' vendor
  svn ci -m 'set rails external to rails trunk'
  [ -n "$use_rails_dir" ] && cp -r $rails_dir vendor/rails
  svn up
  ruby vendor/rails/railties/bin/rails .
  rm public/index.html
  rm log/*
  mv config/database.yml config/database.yml.sample
  svn add . --force
  svn ps svn:ignore '*' tmp/cache tmp/pids tmp/sessions tmp/sockets
  svn ps svn:ignore 'database.yml' config
  svn ps svn:ignore '*.log' log
  svn ci -m "- created rails app
- moved database.yml to database.yml.sample
- deleted public/index.html
- ignored logs and tmp"
  cp config/database.yml.sample config/database.yml
fi

// remember to check that the path is correct.
// This is running on a dedicated server on TextDrive

#!/bin/sh

# This file will run a backup of your desired MySQL database and
# remove any backups older than 7 days.
#
# If youOd like to preserve backups for longer than a week, like say 
# 2 weeks, then set the '-mtime' value from '+7' to '+14'.
#

TIME_STAMP=`date "+%Y-%m-%d"`
echo "starting "$0" on" `date`
for db in db1 db2 db3
do
  DB_STAMP=${db}_${TIME_STAMP}
  echo ${DB_STAMP}
  /opt/csw/mysql5/bin/mysqldump --opt --skip-add-locks --user=username --password=password ${db} | gzip > /domains/backups/mysql/${DB_STAMP}.gz
  /opt/csw/bin/mutt -s "mysql  ${TIME_STAMP}"  -a /domains/backups/mysql/${DB_STAMP}.gz someuser@somedomain.com dev/null
done

cd /domains/backups/mysql/

/usr/bin/find *.gz -mtime +14 -exec rm {} \;
echo "finished "$0" on" `date`

Bash script to add all new files to Subversion

for i in `svn st | grep ? | awk -F "      " '{print $2}'`; do svn add $i; done

Wanna know what gems you have installed without looking at all the descriptions?

gem list | egrep -v "^( |$)"

Here's a (trimmed) sample output:

$ gem list | egrep -v "^ |^$"
*** LOCAL GEMS ***
actionmailer (1.2.1, 1.2.0, 1.1.5)
actionpack (1.12.1, 1.12.0, 1.11.2)
actionwebservice (1.1.2, 1.1.1, 1.1.0, 1.0.0)
activerecord (1.14.2, 1.14.1, 1.14.0, 1.13.2)
activesupport (1.3.1, 1.3.0, 1.2.5)
BlueCloth (1.0.0)
capistrano (1.1.0)
diff-lcs (1.1.2)

Check out man pmset for further command options!

sudo pmset -a disksleep 0 sleep 0 

This is still a work in progress. The objective is to script the entire initial svn hassle that needs to be done with a rails project. The first import, then the first checkout, and then dealing with all the files that need to be ignored. Based heavily on:

http://wiki.rubyonrails.org/rails/pages/HowtoUseRailsWithSubversion

In the future I may add capistrano setup and such.

This worked the last time I tried it, but I don't expect it to work the next time.

#!/bin/bash
set -v                         # verbose output

## USAGE: configure the following variables, and execute in your rails root, ie the directory with
## config/ and app/
## NOTE: This script assumes that your directory of your rails root has the same name as both your 
## application, and your svn repo.
##
## This script also assumes that it's ok to make a backup in the parent directory of your
## rails root.

username="jonshea"               # CHANGE ME!!!!!!!
svn_url="http://jonshea.com/svn/" # CHANGE ME!!!!!

## This is still a work in progress. The objective is to script the entire initial svn hassle that 
## needs to be done with a rails project. The first import, then the first checkout, and then 
## dealing with all the files that need to be ignored. Based heavily on: 
##
## http://wiki.rubyonrails.org/rails/pages/HowtoUseRailsWithSubversion
##
## It designed only for freshly created applications, that haven't really be changed at all from the
## original generation script

app_dir=`pwd`
app_name=`basename $app_dir`
svn_url_to_your_repository=${svn_url}${app_name} # Assumes your repo has the same name as your app
echo $app_name
echo $svn_url_to_your_repository

## Do the initial import
svn import . ${svn_url_to_your_repository}/trunk -m "First Import" --username $username



cd ..                           # Back out a directory from the root

## We're going to make a backup of your app. If one is already there, then remove it.
test -d ./pre_svn_backup_$app_name || rm -rf pre_svn_backup_$app_name
mkdir ../pre_svn_backup_$app_name
mv -f $app_dir ./pre_svn_backup_${app_name} # Move the rails app to the backup dir.

mkdir $app_dir # recreate the application directory
cd $app_dir
svn checkout ${svn_url_to_your_repository}/trunk . # Check out the subversion repo to the app directory

## This section cleans up the svn repo, so that you're not versioning things that shouldn't be versioned.
svn remove log/*
svn commit -m 'removing all log files from subversion'
svn propset svn:ignore "*.log" log/
svn update log/
svn commit -m 'Ignoring all files in /log/ ending in .log'

svn remove tmp/
svn commit -m 'removing the temp directory from subversion'
svn propset svn:ignore "*" tmp/
svn update tmp/
svn commit -m 'Ignore the whole tmp/ directory, might not work on subdirectories?'

svn move config/database.yml config/database.example
svn commit -m 'Moving database.yml to database.example to provide a template for anyone who checks out the code'
svn propset svn:ignore "database.yml" config/
svn update config/
svn commit -m 'Ignoring database.yml'

svn move public/dispatch.rb public/dispatch.rb.example
cp public/dispatch.rb.example public/dispatch.rb
svn move public/dispatch.cgi public/dispatch.cgi.example
cp public/dispatch.cgi.example public/dispatch.cgi
svn move public/dispatch.fcgi public/dispatch.fcgi.example
cp public/dispatch.fcgi.example public/dispatch.fcgi
svn commit -m 'Moving dispatch.(star) to dispatch.(star).example to provide a template.'

svn propedit svn:ignore public/ dispatch.rb
svn propedit svn:ignore public/ dispatch.cgi
svn propedit svn:ignore public/ dispatch.fcgi
svn update public/
svn commit -m 'Ignoring dispatch.* files'

svn propedit svn:ignore db/ *.sqlite
svn propedit svn:ignore db/ *.sqlite3
svn commit -m 'Ignore database files'




#cap --apply-to $app_dir $app_name
#svn add config/deploy.rb     
#svn add lib/tasks/capistrano.rake

exit 0

This isn't a brilliant script, but it sure can be a time saver. When I add a public key by hand I end up doing a lot more commands.

#!/bin/bash

## USAGE: add_to_server.sh remote_server

## This script will add your ssh dsa public key to remote_server's authorized_keys list, 
## assuming that everything is in it's default location

set -v                                 # verbose output
username="USERNAME"              # CHANGE ME!!!!
remote_server=$1              # assigns the first commandline argument to $remote_server


## Pipe the public key to ssh, then remotely touch the file to make sure it will be there, and concat to the end of it.
## Might work without the touch?
cat ~/.ssh/id_dsa.pub | ssh ${username}@${remote_server} "touch ~/.ssh/authorized_keys && cat - >> ~/.ssh/authorized_keys"

exit 0

I have all of my projects in a directory like ~/PROJECTS. Each day when I come to work, I want to ensure that everything on my workstation is up to date. Rather than cd into each directory and issue "svn update" by hand, I kludged the following shell script together. Comments appreciated.

 for X in `find . -maxdepth 2 -type d -name ".svn" | cut -d / -f 2`; do svn update ./$X; done  

Dump all MySQL databases to a location of your choice. I run this as a backup_script from rsnapshot.

This requires the following .my.cnf in the home directory of the user the script runs as (root, when being run from rsnapshot). As it contains an important password, permissions on this file should be 600 (-rw-------).

[client]
user     = root
password = YOUR_MYSQL_ROOT_PASSWORD

#!/bin/bash -e
###########################################################
# Back up MySQL databases
###########################################################

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
PATH="$PATH:/usr/local/mysql/bin"

# My config
#BACKUP_DIR="."

BACKUP_DIR=""

# Do not backup these databases
IGNORE="test"

# Get list of all databases
DB_LIST=`mysql -Bse 'show databases'`

for db in $DB_LIST; do

        # set skip variable
        skip=0

        if [ "$IGNORE" != "" ]; then
                for i in $IGNORE; do
                        [ "$db" == "$i" ] && skip=1 || :
                done
        fi

        if [ "$skip" == "0" ]; then
                mysqldump $db | gzip -9 > $BACKUP_DIR/$db.sql.gz
        fi

done

exit 0

Dump all Subversion repositories to a location of your choice. I run this as a backup_script from rsnapshot.

#!/bin/sh -e
###########################################################
# Back up Subversion repositories
###########################################################

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# Subversion config
###########################################################

# My config
#SVN_DIR="/var/svn"
#BACKUP_DIR="."

SVN_DIR=""
BACKUP_DIR=""
SVN_REPOS=`ls $SVN_DIR`

# Dump repositories
###########################################################

for repos in $SVN_REPOS; do
    svnadmin dump --incremental --deltas --quiet $SVN_DIR/$repos \
        > $BACKUP_DIR/$repos
done

exit 0

This snippet written in bash with calls to perl from the command line measures the number of E-mail messages sent to a mailbox per calendrical day.

#!/bin/bash
grep -h '^Date:' * |
    perl -pe 's!^Date: !!' |
    perl -pe 's!^\w\w\w, !!' |
    perl -pe 's{\d{2}:\d{2}:\d{2}.*$}{}' |
    perl -pe 's!^\s+!!' |
    perl -pae '$_=sprintf("%.2d-%s-%s\n", @F)' |
    sort | uniq -c | sort -n

I used perl for some places where sed would have been more suitable because I find the sed regexp syntax confusing. :-)

#!/bin/sh

USERNAME=username
LIGHTTPD_CONF=/home/$USERNAME/service/lighttpd.conf
PIDFILE=/home/$USERNAME/service/logs/lighttpd.pid
HTTPD=/home/$USERNAME/lighttpd/sbin/lighttpd

PID=0
if [ -e $PIDFILE ]; then
    PID=`cat $PIDFILE`
    if [ "x" == "x$PID" ]; then
        PID=0
    fi
fi

case "$1" in
    start)
        if [ 0 -ne $PID ]; then
            running=`ps --pid $PID | grep $PID`
            if [ $running ]; then
                echo "lighttpd is already running"
                exit 1
            fi
            rm $PIDFILE
            PID=0
        fi
        $HTTPD -f $LIGHTTPD_CONF
        ;;
    stop)
        if [ 0 -eq $PID ]; then
            echo "lighttpd was not running"
            exit 1
        fi
        kill $PID
        tries=""
        while [ -e $PIDFILE ]; do
            if [ "x$tries" == "x.........." ]; then
                break
            fi
            sleep 2
            tries=".$tries"
        done
        if [ -e $PIDFILE ]; then
            echo "lighttpd did not go gentle into that good night, murdering"
            kill -9 $PID
            rm $PIDFILE
        fi
        ;;
    restart)
        $0 stop
        $0 start
        ;;
    reload)
        if [ 0 -eq $PID ]; then
            echo "lighttpd was not running"
        fi
        kill -HUP $PID
        ;;
    *)
        echo "Usage: "`basename $0`" (start|stop|restart|reload)"
        exit 1
        ;;
esac