About this user

AJ http://ajz.textdriven.com

« Newer Snippets
Older Snippets »
3 total  XML / RSS feed 

File buffer chunked processing

For future reference from: http://forum.textdrive.com/viewtopic.php?pid=105585#p105585
(Johan via Bernard)

Reading file buffer in chunks to prevent reaching process memory limits.

File.open("", "wb") do |f|
  while buff = file_field.read(4096)
    f.write(buff)
  end
end

rsrcmeter for TxD

TextDrive: Simple disk usage and HTTP bandwidth counts
This is an updated version of http://textsnippets.com/posts/show/621

#!/usr/local/bin/ruby
Dir.chdir(ENV['HOME'])
# Disk usage
quotaline = `quota -g | tail -n 1`
usage = `echo -n "#{quotaline}" | awk '{print $2}'`.to_f
quota = `echo -n "#{quotaline}" | awk '{print $3}'`.to_f
percent_used = (usage / quota) * 100
puts "Disk usage: " + sprintf("%.4f", usage/1024) + " MiB (Quota: " + sprintf("%.4f", quota/1024/1024) +" GiB; " + sprintf("%.1f", percent_used) + "% used)"

# HTTP Bandwidth
print "Calculating Bandwidth Usage..."
month = `date +"%B %Y"`.chomp
access_logs="access_log." + `date +%Y%m`.chomp + "??"
system("cat logs/access_log 2>/dev/null > temp-bandwidthcount") # Today's log
system("cat domains/*/logs/access_log 2>/dev/null >> temp-bandwidthcount")
system("cat logs/#{access_logs} 2>/dev/null >> temp-bandwidthcount") # Any logs not (yet) gzipped
system("cat domains/*/logs/#{access_logs} 2>/dev/null >> temp-bandwidthcount")
system("zcat logs/#{access_logs}.gz 2>/dev/null >> temp-bandwidthcount") # Gzipped logs from previous days
system("zcat domains/*/logs/#{access_logs}.gz 2>/dev/null >> temp-bandwidthcount")

usage = `cat temp-bandwidthcount | awk '{sum += $10} END {print sum}'`.chomp.to_f / 1024 / 1024
File.delete("temp-bandwidthcount")

30.times {print "\b"}
puts "Bandwidth used for #{month}: " + sprintf("%.4f", usage) + " MiB"


Installing:
Easy as 1..2..3:
curl -o rsrcmeter http://ajz.textdriven.com/rsrcmeter.txt
chmod u+x rsrcmeter
./rsrcmeter


Sample output:
Disk usage: 2.2910 MiB (Quota: 1.9073 GiB; 0.1% used)
Bandwidth used for August 2006: 0.5773 MiB


Confirmed to work on:
- Cardero,
- Davie (by rsimplicio),
- Pendrell (by iolaire),
- Thurlow (by janovetz),
- Bidwell & Jervis (by springworks),
- Chilco (by atl),
- Burnaby (by igner),
- Broughton (Biz Server, by Rich),
- Howe (scoobyfoo),
- Nicola (lderezinski),
- Jervis (mjboyle), and
- One (robertb)
*Should work on any TxD shared/business server.*

showbandwidth (for TxD)

This has been superseded by http://textsnippets.com/posts/show/632 (includes account disk usage)!

#!/usr/local/bin/bash
cd ~
export ACCESS_LOGS=access_log.`date +%Y%m`??
cat logs/access_log 2>/dev/null > temp-bandwidthcount
cat domains/*/logs/access_log 2>/dev/null >> temp-bandwidthcount
cat logs/$ACCESS_LOGS 2>/dev/null >> temp-bandwidthcount
cat domains/*/logs/$ACCESS_LOGS 2>/dev/null >> temp-bandwidthcount
zcat logs/$ACCESS_LOGS.gz 2>/dev/null >> temp-bandwidthcount
zcat domains/*/logs/$ACCESS_LOGS.gz 2>/dev/null >> temp-bandwidthcount
echo -n "Bandwidth used for `date +"%B %Y"`: "
cat temp-bandwidthcount | awk '{sum += $10} END {print sum/1024/1024,"MiB"}'
rm temp-bandwidthcount


1. Upload as showbandwidth
2. chmod u+x it
3. run it (~# ./showbandwidth) to get your current month's (HTTP) bandwidth usage.
Output Sample: Bandwidth used for August 2006: 0.46237 MB

Original credit to Filip: http://textsnippets.com/posts/show/346.

Old version (only use if you have changed where Apache saves its log files):
#!/usr/local/bin/bash
cd ~
export ACCESS_LOGS=access_log.`date +%Y%m`??
cat `find . -name 'access_log' 2>/dev/null` 2>/dev/null > temp-bandwidthcount # Today's log
cat `find . -name "$ACCESS_LOGS" 2>/dev/null` 2>/dev/null >> temp-bandwidthcount # For any not gzipped
zcat `find . -name "$ACCESS_LOGS.gz" 2>/dev/null` 2>/dev/null >> temp-bandwidthcount
echo -n "Bandwidth used for `date +"%B %Y"`: "
cat temp-bandwidthcount | awk '{sum += $10} END {print sum/1024/1024,"MB"}'
rm temp-bandwidthcount
« Newer Snippets
Older Snippets »
3 total  XML / RSS feed