Never been to TextSnippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world (or not, you can keep them private!)

« Newer Snippets
Older Snippets »
23 total  XML / RSS feed 

sup: bash script for universal SCM syncing (svn, svk, cvs, darcs, etc.)

I really can never remember if the project I'm in uses svn or cvs or if it's my own local svk mirror, etc. etc. Thus was born 'sup'!

Save and drop into a location that's in your PATH (I use ~/bin, YMMV)

#!/bin/bash
# sup -- a quick bash script to sync w/ various SCM tools
# @author   Jamie Wilkinson 

HERE=$(pwd)

# subversion
if [ -e ".svn" ]; then
        svn up
# cvs
elif [ -e "CVS" ]; then
        cvs up
# darcs
elif [ -e "_darcs" ]; then
        darcs pull --all
# svk
elif [ -n "`grep $HERE ~/.svk/config`" ]; then
    svk up
# perforce
#   todo
# arch
#   todo
fi

Recursively remove .svn directories (ruby script)

This snippet traverses a directory tree and removes .svn directories.

require 'find'
require 'fileutils'
Find.find('./') do |path|
  if File.basename(path) == '.svn'
    FileUtils.remove_dir(path, true)
    Find.prune
  end
end

Rails Subversion First Checkout

// I use this for checking out a new rails project

svn checkout http://studicious.com/svn/trunk %1
svn remove log/*
svn commit -m "Removed log files"
svn propset svn:ignore "*.log" log/
svn update log/
svn commit -m "Ignoring log files"
svn remove tmp/*
svn propset svn:ignore "*" tmp/
svn update tmp/
svn commit -m "Ignoring temp folder"

Backup from textdrive to strongspace

#!/bin/sh
# backup script for textdrive to strongspace based on these articles:
# http://help.textdrive.com/index.php?pg=kb.page&id=113
# http://forum.textdrive.com/viewtopic.php?id=10126

# change this!
USER=txd_username
HOME=/users/home/$USER
USER_SS=strongspace_username
PWD_MYSQL=somepassword # chmod 700 this file

# Backup database 
# copy this line for more databases or 
# use --all-databases if you are the main user
# Don't forget to change the database name
/usr/local/bin/mysqldump --opt --skip-add-locks --user=$USER --password=$PWD_MYSQL database1 | gzip > $HOME/backups/database1_`date "+%Y-%m-%d"`.gz
/usr/local/bin/mysqldump --opt --skip-add-locks --user=$USER --password=$PWD_MYSQL database2 | gzip > $HOME/backups/database2_`date "+%Y-%m-%d"`.gz

# Backup subversion (Only works with FSFS)
cd $HOME
tar -z -c -f $HOME/backups/svn_`date "+%Y-%m-%d"`.tar.gz svn

# Add custom dirs here, if you need it, just like the svn example above
# I just keep everything I need in subversion

# Delete old backups
cd $HOME/backups/
/usr/bin/find *.gz -mtime +8 -delete

# Send it to strongspace
/usr/local/bin/rsync -azq --delete -e "ssh -i $HOME/.ssh/ss" $HOME/backups/*.gz $USER_SS@$USER_SS.strongspace.com:/home/$USER_SS/txd-backup/

create a rails app from rails trunk, optionally committing to a subversion repository

I name this script railstrunk on my machine. So, usage would be:

railstrunk app_name


That creates a rails application under app_name.

If app_name is a subversion working copy, rails is set as an external and generated files are committed, and some ignore properties are set.

Here's how I'd go about creating a versioned rails app:

mkdir -p happy_app happy_app/trunk happy_app/tags happy_app/branches 
svn import happy_app http://myserver/myrepos/happy_app -m "Layout for happy_app"
rm -rf happy_app
svn co http://myserver/myrepos/happy_app/trunk happy_app
railstrunk happy_app


$rails_dir is set at the beginning of script, and it should be a path to a working copy of rails trunk on your machine. It's merely used to speed things up, it's not necessary.

Notice I remove the silly public/index.html from the generated app.

And now the code:

#!/bin/bash

rails_dir=~/code/ruby/rails

if [ $# != 1 ]; then
  echo "Usage: $0 app_name"
  echo
  echo "Creates a rails application under app_name."
  echo
  echo "If app_name is a subversion working copy, rails is set as an external"
  echo "and generated files are committed."
  echo
  echo "If $rails_dir exists, things are sped up by either symlinking it"
  echo "(for non-versioned apps) or copying it to the vendor dir."
  echo
  echo "$rails_dir should be a rails trunk working copy."
  exit
fi

dir=$1
mkdir -p $dir
cd $dir

if [ -n "`ls`" ]; then
  echo "Can't create app: $dir is not empty." >&2
  exit 1
fi

if [ -d $rails_dir ]; then
  if [ "`svn info $rails_dir 2>/dev/null | grep URL:`" != "URL: http://dev.rubyonrails.org/svn/rails/trunk" ]; then
    echo "$rails_dir is not a rails trunk working copy. Not going to use it." >&2
  elif [ -n "`svn st $rails_dir`" ]; then
    echo "$rails_dir is modified. Not going to use it." >&2
  else
    use_rails_dir=1
  fi
fi

if [ -z "`svn info 2>/dev/null`" ]; then
  mkdir vendor
  if [ -n "$use_rails_dir" ]; then
    ln -s $rails_dir vendor/rails
    cd vendor/rails
    svn up
    cd ../..
  else
    svn co http://dev.rubyonrails.org/svn/rails/trunk vendor/rails
  fi
  ruby vendor/rails/railties/bin/rails .
  rm public/index.html
else
  svn mkdir vendor
  svn ps svn:externals 'rails http://dev.rubyonrails.org/svn/rails/trunk' vendor
  svn ci -m 'set rails external to rails trunk'
  [ -n "$use_rails_dir" ] && cp -r $rails_dir vendor/rails
  svn up
  ruby vendor/rails/railties/bin/rails .
  rm public/index.html
  rm log/*
  mv config/database.yml config/database.yml.sample
  svn add . --force
  svn ps svn:ignore '*' tmp/cache tmp/pids tmp/sessions tmp/sockets
  svn ps svn:ignore 'database.yml' config
  svn ps svn:ignore '*.log' log
  svn ci -m "- created rails app
- moved database.yml to database.yml.sample
- deleted public/index.html
- ignored logs and tmp"
  cp config/database.yml.sample config/database.yml
fi

Remove missing subversion files

// automatically remove files missing from subversion

svn status | grep '\!' | awk '{print $2;}' | xargs svn rm 

update a working copy and all externals in parallel

If you have an app in subversion with many externals, it may take a bit too long to update it, as updates happen one after another.

This bit of script updates the app and each external in parallel, making it oh so much faster.


#!/usr/local/bin/ruby

puts(
  ( `svn pl -R`.scan(/\S.*'(.*)':\n((?:  .*\n)+)/)\
    .inject({}) { |h, (d, p)| h[d] = p.strip.split(/\s+/); h }\
    .select { |d, ps| ps.include? 'svn:externals' }\
    .map { |xd, ps| [xd, `svn pg svn:externals #{xd}`] }\
    .map { |xd, exts| exts.strip.split(/\s*\n/).map { |l| xd + '/' + l.split(/\s+/).first } }\
    .inject { |a, b| a + b }\
    .map { |d| "cd #{d} && svn up 2>&1" } \
    << 'svn up . --ignore-externals 2>&1'
  )\
  .map { |cmd| [cmd, Thread.new { `#{cmd}` }] }\
  .map { |cmd, thread| "#{cmd}\n#{thread.value}" }.join("\n")
)


(note: if you add an external or change an external property in another way, you'll need to run the standard svn up once)

Rake task that adds all unversioned files to the repository

task :add_all_to_svn do
  `svn status --show-updates`.each { |l|
    l = l.split
    system("svn add #{l.last}") if l.first == "?" # File is not under version control
  }
end

Make a subversion release

Ruby script to make a release from SVN trunk.

Assumes a repository structure of:

/path/to/Trunk
/path/to/Releases/YYYY-MM-DD--XX

Should be easy enough to modify to suit your needs. Comments welcome :)

Example usage:
vi /usr/local/bin/release (and dump snippet into it)
chmod +x /usr/local/bin/release
cd /path/to/checked-out-trunk
release


#!/usr/bin/ruby

today = Time.now.strftime('%Y-%m-%d')

if `svn info`.to_a[1] !~ /^URL: (.*)\/Trunk$/
        puts "Repository invalid. Must be in Trunk."
        exit
end

repo_url = $1

repo = repo_url.split('/').last

release_url = repo_url + "/Releases"

if `svn ls #{release_url}`.to_a.last =~ /#{today}--(\d+)/
        num = $1.to_i + 1
else
        num = 1
end

release = "#{today}--#{num}"

cmd = "svn copy #{repo_url}/Trunk #{release_url}/#{release} -m 'copy #{repo} trunk to release #{release}'"
puts cmd
puts

print "Execute command? "

if gets.chomp.upcase != 'Y'
        puts "Nothing done."
        exit
end

`#{cmd}`
puts "Copied as #{release}."
puts "Execute on the remote machine:"
puts "svn switch #{release_url}/#{release}"

Add all new files to Subversion

Bash script to add all new files to Subversion

for i in `svn st | grep ? | awk -F "      " '{print $2}'`; do svn add $i; done

Bulk svn actions (usefull with rails)

Just run this code from inside your repository. This will do a bulk action like "svn add" on each file marked with the given filter (like '?') when doing "svn status"

Beware: this code might break with multi word filenames...

usage :
# ./svn_bulk ? add ---> this shows a preview of the actions
# ./svn_bulk ? add 1 ---> action !
#!/usr/bin/ruby
sign, action, doit = ARGV
sign = '\?' if sign == '?'
list = IO.popen("svn st")
list.each {|i|
if (i =~ /^#{sign}\s*(.*)/) 
cmd = "svn #{action} '"+$1+"'"
print cmd + "\n"
system(cmd) if doit
end
}

Svn obliterate (dump filter)

Taken and slightly adapted from www.robgonda.com/blog

svnadmin dump /path/to/repos > proj.dump
cat proj.dump | svndumpfilter exclude somefolder > cleanproj.dump
STOP SVN services
BACKUP /path/to/repos/conf /path/to/repos/hooks (all custom configuration for this repository)
DELETE /path/to/repos
svnadmin create /path/to/repos
RESTORE /path/to/repos/conf /path/to/repos/hooks
svnadmin load /path/to/repos < cleanproj.dump
RESTART SVN services

Bash script to perform initial import of a rails project

This is still a work in progress. The objective is to script the entire initial svn hassle that needs to be done with a rails project. The first import, then the first checkout, and then dealing with all the files that need to be ignored. Based heavily on:

http://wiki.rubyonrails.org/rails/pages/HowtoUseRailsWithSubversion

In the future I may add capistrano setup and such.

This worked the last time I tried it, but I don't expect it to work the next time.

#!/bin/bash
set -v                         # verbose output

## USAGE: configure the following variables, and execute in your rails root, ie the directory with
## config/ and app/
## NOTE: This script assumes that your directory of your rails root has the same name as both your 
## application, and your svn repo.
##
## This script also assumes that it's ok to make a backup in the parent directory of your
## rails root.

username="jonshea"               # CHANGE ME!!!!!!!
svn_url="http://jonshea.com/svn/" # CHANGE ME!!!!!

## This is still a work in progress. The objective is to script the entire initial svn hassle that 
## needs to be done with a rails project. The first import, then the first checkout, and then 
## dealing with all the files that need to be ignored. Based heavily on: 
##
## http://wiki.rubyonrails.org/rails/pages/HowtoUseRailsWithSubversion
##
## It designed only for freshly created applications, that haven't really be changed at all from the
## original generation script

app_dir=`pwd`
app_name=`basename $app_dir`
svn_url_to_your_repository=${svn_url}${app_name} # Assumes your repo has the same name as your app
echo $app_name
echo $svn_url_to_your_repository

## Do the initial import
svn import . ${svn_url_to_your_repository}/trunk -m "First Import" --username $username



cd ..                           # Back out a directory from the root

## We're going to make a backup of your app. If one is already there, then remove it.
test -d ./pre_svn_backup_$app_name || rm -rf pre_svn_backup_$app_name
mkdir ../pre_svn_backup_$app_name
mv -f $app_dir ./pre_svn_backup_${app_name} # Move the rails app to the backup dir.

mkdir $app_dir # recreate the application directory
cd $app_dir
svn checkout ${svn_url_to_your_repository}/trunk . # Check out the subversion repo to the app directory

## This section cleans up the svn repo, so that you're not versioning things that shouldn't be versioned.
svn remove log/*
svn commit -m 'removing all log files from subversion'
svn propset svn:ignore "*.log" log/
svn update log/
svn commit -m 'Ignoring all files in /log/ ending in .log'

svn remove tmp/
svn commit -m 'removing the temp directory from subversion'
svn propset svn:ignore "*" tmp/
svn update tmp/
svn commit -m 'Ignore the whole tmp/ directory, might not work on subdirectories?'

svn move config/database.yml config/database.example
svn commit -m 'Moving database.yml to database.example to provide a template for anyone who checks out the code'
svn propset svn:ignore "database.yml" config/
svn update config/
svn commit -m 'Ignoring database.yml'

svn move public/dispatch.rb public/dispatch.rb.example
cp public/dispatch.rb.example public/dispatch.rb
svn move public/dispatch.cgi public/dispatch.cgi.example
cp public/dispatch.cgi.example public/dispatch.cgi
svn move public/dispatch.fcgi public/dispatch.fcgi.example
cp public/dispatch.fcgi.example public/dispatch.fcgi
svn commit -m 'Moving dispatch.(star) to dispatch.(star).example to provide a template.'

svn propedit svn:ignore public/ dispatch.rb
svn propedit svn:ignore public/ dispatch.cgi
svn propedit svn:ignore public/ dispatch.fcgi
svn update public/
svn commit -m 'Ignoring dispatch.* files'

svn propedit svn:ignore db/ *.sqlite
svn propedit svn:ignore db/ *.sqlite3
svn commit -m 'Ignore database files'




#cap --apply-to $app_dir $app_name
#svn add config/deploy.rb     
#svn add lib/tasks/capistrano.rake

exit 0

Update All of the SVN Version Controlled Projects in a Directory

I have all of my projects in a directory like ~/PROJECTS. Each day when I come to work, I want to ensure that everything on my workstation is up to date. Rather than cd into each directory and issue "svn update" by hand, I kludged the following shell script together. Comments appreciated.


 for X in `find . -maxdepth 2 -type d -name ".svn" | cut -d / -f 2`; do svn update ./$X; done  

Check out all TextMate Bundles with SVN

svn --username anon --password anon co http://macromates.com/svn/Bundles/trunk/Bundles /Users/ned/Library/Application\ Support/TextMate/Bundles

Enable svn+ssh remote logins

Installing Subversion for local use is general an easy install, but allowing remote access to your svn repository over SSH can be problomatic dependent upon your OS and the means taken to install.

For Darwinports and Fink on OS X the install location has to be added to users $PATHs, but there are extra steps outlined here for use of the svn+ssh means of access:

http://subversion.tigris.org/faq.html#ssh-svnserve-location

A much easier alternate is to sym link the svn binaries to a place on the default PATH (used by the SSH login):

#For Darwinports
ln -s /opt/local/bin/sv* /usr/bin/

#For Fink
ln -s /sw/bin/sv* /usr/bin/


This links all the binaries at once.

Finding files to be added

svn add `svn st | grep '\?' | tr '\?' ' ' | xargs`

knowning how many commits it actually took you

If you use the same repository for mostly everything, you'll occasionally find it amusing that your brand new project is at r1800 or that that old project not touched since r60 now shares r2385 with everyone else.

Well, amusing != useful, so here's what I use when I'm curious about how many commits a certain project took me:

svn log --stop-on-copy | grep -e "--------" | wc -l

Recursively remove all .svn directories

find . -name .svn -print0 | xargs -0 rm -rf


Update: Thankyou iburrell

Using a pre-commit script (written in python)

For some odd reason tortoise SVN loves to commit empty commit sets. So I managed to get the following pre-commit script working under FreeBSD, seeing that the pre-commit.tmpl does not work as expected.

#!/usr/local/bin/python
"""
Subversion pre-commit hook which currently checks that the commit contains
a commit message to avoid commiting empty changesets which tortoisesvn seems
to have a habbit of committing.

Based on http://svn.collab.net/repos/svn/branches/1.2.x/contrib/hook-scripts/commit-block-joke.py
and hooks/pre-commit.tmpl

Hacked together by Jacques Marneweck 

$Id$
"""

import sys, os, string

SVNLOOK='/usr/local/bin/svnlook'

def main(repos, txn):
    log_cmd = '%s log -t "%s" "%s"' % (SVNLOOK, txn, repos)
    log_msg = os.popen(log_cmd, 'r').readline().rstrip('\n')

    if len(log_msg) < 10:
        sys.stderr.write ("Please enter a commit message which details what has changed during this commit.\n")
        sys.exit(1)
    else:
        sys.exit(0)

if __name__ == '__main__':
    if len(sys.argv) < 3:
        sys.stderr.write("Usage: %s REPOS TXN\n" % (sys.argv[0]))
    else:
        main(sys.argv[1], sys.argv[2])
« Newer Snippets
Older Snippets »
23 total  XML / RSS feed