Posts Tagged ‘grep’

ACRNYM: Title of research paper follows in SIGGRAPH paper titles

Thursday, September 22nd, 2011

Recently I posted a list of over 4000 SIGGRAPH technical papers titles. You can see some interesting patterns browsing this list. Since the very beginning of SIGGRAPH (see “Modelaide: a computer graphics program for the evaluation of mathematical models”) there has been a long tradition of papers with titles beginning with a pet name for their algorithm or software. Like recently “MeshFlow: interactive visualization of mesh construction sequences”. With my list its easy to suck out all the paper titles that
begin with “SomeWord:” and look at the names people are using.

MeshFlow
LR
HOT
OverCoat
Spark
GlobFit
ShadowDraw
CATRA
MovieReshape
envyLight
PantaRay
RepFinder
OptiX
VideoMocap
Popup
NETRA
Dinus
GradientShop
DiagSplit
RenderAnts
Sketch2Photo
Bokode
SubEdit
iWIRES
PatchMatch
SkyFinder
FlexiStickers
GRAMPS
VirtualStudio2Go
IGT
AppProp
BSGP
Larrabee
SOHO
Prakash
AppWand
ShapePalettes
VideoTrace
PCU
Ldr2Hdr
TRACKS
SIMBICON
Plushie
FastLSM
FiberMesh
Videogames
SmoothSketch
Glift
SCAPE
MoXi
GoLD
Lightcuts
Geopostors
Color2Gray
RPU
Blister
Lpics
DART
CrossY
VisionWand
GADGET
VisualIDs
TensorTextures
Textureshop
DISCO
SwingWrapper
iLamps
Cg
TreeJuxtaposer
Twister
Matchmaker
VisBio
CHARMS
Chromium
DyRT
BEAT
Kizamu
WordsEye
WireGL
DAB
FoleyAutomatic
Pomegranate
Surfels
QSplat
Deepwave
Hypercosm
la_fabrique
IceBorg
Alice
SimsalaGrimm
Rivet
Teddy
Skin
Feline
LCIS
ArtDefo
VisAD
NeuroAnimator
mediaBlocks
MAPS
Constellation
Wires
InfiniteReality
SCAAT
Alice
NICE
Osmose
VENUS
SKETCH
Improv
Talisman
uisGL
OBBTree
VRML
TicTacToon
AutoKey
Corrigendum
Cantata
Euphrates
Yaged
TBAG
FBRAM
Corrigenda
Placeholder
Pad
EXACT
Leo
VIEW
Radioptimization
PixelFlow
CONDOR
Editorial
Sculpting
Inkwell
Metamouse
ConMan
FRAMES
GRAPE
Corrigendum
MIKE
Rooms
Squeak
SYNGRAPH
INCENSE
MAPQUERY
PICTUREBALM
BUMPS
QUADRIL
ATLAS
ATOMLLL
GAIN
TIGS
CASS
GPGS
Glide
ECOSITE
SIGCHR
ARTES
PRIMS
BGRAF2
Typer
DDA
ARTSPEAK
SUGAR
Grapheasy
WHATSISFACE
IMAGE
Wave
CARTE
TREE
XPLG
DRAW
APLBAGS
MAPS
KARMA
Modelaide

Set up virtual host on mac os x

Tuesday, September 6th, 2011

We’re using SVN to version control our group’s webpage which is all php scripts. If everything were html files then making edits on a locally checked out copy would be easy since I can just open the files in a browser and check my edits. But, being PHP scripts, to properly see the sites I need a php server running which can host the local copy of the site and I can see the edits I make live. Then finally when I’m sure everything looks and executes correctly I can check in the edited version and update the version on the real server. Here’s how I constructed my local setup.

Check out svn to some directory, from now on referred to as “path/to/htdocs-igl”:


svn co https://private.svn.server.com/trunk/htdocs-igl path/to/htdocs-igl

Be sure that no apache servers are not running. The following in order should to do that:


sudo apachectl stop

Use ps and grep to determine if they really stopped. This should return nothing:


ps -ef | grep httpd | grep -v grep

If it returns some lines, then kill each replacing [PID] with each row’s process ID:


sudo kill [PID]

If the process didn’t really die then try:


sudo kill -9 [PID]

And after you think you’ve done enough killing, issuing this again for good measure:


sudo apachectl stop

Now that we’re sure no apache servers are running. Start one:


sudo apachectl start

To check that it’s working browse to http://localhost/. You should see something fully load other than “Not found, blah blah”.

Now, add the following as a new line at the bottom of the file /etc/hosts:


127.0.0.1       localigl

Now you should be able to go to http://localigl/ and see the same thing as you did with http://localhost/.

To have http://localigl/ point to your checked out version of the website, you need to edit the file /etc/apache2/users/[your mac username].conf:


NameVirtualHost *:80

<Directory "/Users/[your mac username]/Sites/">
    Options Indexes MultiViews Includes
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

<VirtualHost *:80>
    ServerName localhost
    DocumentRoot /Users/[your mac username]/Sites/
</VirtualHost>

<Directory "/absolute/path/to/htdocs-igl/">
    Options Indexes MultiViews Includes
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

<VirtualHost *:80>
    ServerName localigl
    DocumentRoot /absolute/path/to/htdocs-igl
</VirtualHost>

Replacing, in the file name and the file contents, [your mac username] with, uh, your mac username.

You must give execute access to parent directory and the root directory of the checked out website:


chmod 755 path/to/htdocs-igl/..
chmod 755 path/to/htdocs-igl

Finally, restart the server with:


sudo apachectl graceful

If you now go to http://localigl/ you should see a listing of the checked out website’s root directory contents (or if you’re using index.html, you’ll see that rendered file). Since we’re using PHP for everything we must enable php scripts.

Uncomment the following line in /etc/apache2/httpd.conf to look like this:


LoadModule php5_module        libexec/apache2/libphp5.so

And again restart the server:


sudo apachectl graceful

Now when you go to http://localigl/ you should see your php scripts executed and rendered.

Source This document is well written but did not enough stress that you must stop all servers (that you may have accidentally started when fooling around) before starting the steps. And it’s nice to repeat his steps with exact names closer to my current set up.

svn get last commit message

Wednesday, August 31st, 2011

Here’s a quick bash one-liner to get the last svn commit message:


svn log -q -v --xml --with-all-revprops -r committed | grep msg | sed -e "s/<msg>\([^<]*\)<\/msg>/\1/g"


Update:
Not sure what the difference between “committed” and “head” is, but now “head” is giving me the correct message:


svn log -q -v --xml --with-all-revprops -r head | grep msg | sed -e "s/<msg>\([^<]*\)<\/msg>/\1/g"

Subtly open Activity Monitor for suspicious CPU hogs

Wednesday, October 13th, 2010

As a first attempt at a subtle HCI program, I’ve written a bash script for Mac that when installed as a cronjob will monitor the running applications and subtly open up the Activity Monitor.app if a process has been hogging the CPU for an extended amount of time.

Here’s the bash code I save in monitor-top-process.sh:


#!/bin/bash
record_file=".monitor-top-process.dat"
MAX_COUNT="3"

# make sure record file exists
touch $record_file

# get process with over 70% cpu usage
top_process=`top -l 2 -n 1 -o cpu | \
  grep "^ *[0-9]\+" | \
  grep -o "[A-z0-9]\+ \+[7-9][0-9.]\+%" | \
  sed -e "s/ *[0-9\.]*%//g"`

if [ -n "$top_process" ]; then 
  old_top_process=`cat $record_file | \
    grep -o "top_process=.*" | \
    sed -e "s/top_process=//g"`
  if [ "$top_process" = "$old_top_process" ]; then
    count=`cat $record_file | \
      grep -o "count=.*" | \
      sed -e "s/count=//g"`
    count=`expr "$count" + 1`
    if [ "$count" -ge "$MAX_COUNT" ]; then
      activity_monitor_running=`top -l 1 | grep Activity`
      if [ -z "$activity_monitor_running" ]; then
        open "/Applications/Utilities/Activity Monitor.app/"
      fi
    fi
  else
    count="1"
  fi
  echo -e "top_process=$top_process\ncount=$count" > $record_file
else
  # no top process clear record
  echo "" > $record_file
fi

Then I issue:


crontab -e

and add the following to run this script every five minutes:


*/5 * * * * bash [path to]/monitor-top-process.sh

It’s not the prettiest way to do it, but a quick proof of concept.

Cron job to warn you when your quota is almost full

Tuesday, May 11th, 2010

When I go over my disk space quota on the CIMS server I’m locked out of my account. I can only issue the ls and cd commands. That’s right, I can’t even issue rm. Which means every time (I realize it shouldn’t happen often but it alas has) I go over the quota I need to call the help desk and have them enlarge my quota temporarily so I can delete some files.

This happened recently over the weekend that we were working on a paper for the SGP deadline and nobody was at the CIMS help desk to answer my desperate call. So from now on I have a little script that will email me when I’m close to filling my quota. Here’s the script, which I save in a file called email-quota-warning.sh:


#!/bin/bash

# grep out percentage of quota used on "home" mount
PERCENT=`quota | grep home | grep -o "[0-9]\+\%" | grep -o "[0-9]\+"`;

# if the percentage is a above some threshold then send an email
if [ "$PERCENT" \> "95" ]; then
  MESSAGE="Your quota on home is getting close to full: $PERCENT%";
  EMAIL="your-email-goes-here";
  SUBJECT="Quota Warning: $PERCENT% full";
  echo $MESSAGE | /bin/mail -s "$SUBJECT" "$EMAIL"
  echo 'echo $MESSAGE | /bin/mail -s "$SUBJECT" "$EMAIL" '
fi;

Then I set up a cron job on a CIMS machine. Because the access.cims.nyu.edu machine has a bogus version of mail (I couldn’t figure out how to make it do subjects), I set up my job on mauler.

Of course then I had trouble editting my crontab. When I issue:


crontab -e

I get the following error:


E486: Pattern not found: 's$
crontab: no changes made to crontab

I couldn’t figure out how to fix this problem so instead I just created a file called .crons with the following in it:


0 * * * * bash ~/bin/email-quota-warning.sh

Of course change the bash to wherever you’re storing the script. Then just issue:


crontab .crons

Find big directories using du and (e)grep

Thursday, March 25th, 2010

Here’s a simple script I’m using to locate big directories (larger than 1GB):


du -h | grep "^ *[0-9][0-9.]*G"

The output looks like this:


1.1G	./Art/2007
2.5G	./Art/2008/Party Freakz Pasta Party
6.3G	./Art/2008
9.2G	./Art
 24G	./Documents/Parallels/Windows 7.pvm/Windows 7-0.hdd
 24G	./Documents/Parallels/Windows 7.pvm
 24G	./Documents/Parallels
 26G	./Documents
2.5G	./Downloads/enstrophy-big-endian
 33G	./Downloads
...
100G	.

Here’s how I find big directories greater than 100MB:


du -h | egrep "^ *([0-9][0-9][0-9][0-9.]*M)|([0-9][0-9.]*G)"

To list files, too, just add the -a flag to du like this:


du -ah | grep "^ *[0-9][0-9.]*G"

Whereami, find out your physical location via command line

Saturday, December 5th, 2009

Using a wget and sed combo I found on go2linux and a little web scraping, I’ve com up with a little command line bash script to find your public ip address and then determine your physical location. Save this in a file called whereami.sh


#!/bin/bash

# get public ip address from checkip.dyndns.org
public_ip=`wget -q -O - checkip.dyndns.org | \
sed -e 's/.*Current IP Address: //' -e 's/<.*$//'`

echo $public_ip

# get physical address from ip address from melissadata.com
wget -q -O - \
http://www.melissadata.com/Lookups/iplocation.asp?ipaddress=$public_ip | \
grep "\(\(City\)\|\(State or Region\)\|\(Country\)\)<\/td>" | \
sed "s/.*<b>\([^<]*\)<\/b>.*/\1/"

Note: If you know of a more stable ip to physical location site to scape leave it below.

Test ssh connection speed

Thursday, November 19th, 2009

Having a few different ssh hosts these days it’s become convenient to be able to test my upload and download transfer rates. Here’s my SSH speed test bash script. It uses dd to generate a test file then uses scp to test the transfer rates across the network.


#!/bin/bash
# scp-speed-test.sh
# Author: Alec Jacobson alecjacobsonATgmailDOTcom
#
# Test ssh connection speed by uploading and then downloading a 10000kB test
# file (optionally user-specified size)
#
# Usage:
#   ./scp-speed-test.sh user@hostname [test file size in kBs]
#

ssh_server=$1
test_file=".scp-test-file"

# Optional: user specified test file size in kBs
if test -z "$2"
then
  # default size is 10kB ~ 10mB
  test_size="10000"
else
  test_size=$2
fi


# generate a 10000kB file of all zeros
echo "Generating $test_size kB test file..."
`dd if=/dev/zero of=$test_file bs=$(echo "$test_size*1024" | bc) \
  count=1 &> /dev/null`

# upload test
echo "Testing upload to $ssh_server..."
up_speed=`scp -v $test_file $ssh_server:$test_file 2>&1 | \
  grep "Bytes per second" | \
  sed "s/^[^0-9]*\([0-9.]*\)[^0-9]*\([0-9.]*\).*$/\1/g"`
up_speed=`echo "($up_speed*0.0009765625*100.0+0.5)/1*0.01" | bc`

# download test
echo "Testing download to $ssh_server..."
down_speed=`scp -v $ssh_server:$test_file $test_file 2>&1 | \
  grep "Bytes per second" | \
  sed "s/^[^0-9]*\([0-9.]*\)[^0-9]*\([0-9.]*\).*$/\2/g"`
down_speed=`echo "($down_speed*0.0009765625*100.0+0.5)/1*0.01" | bc`

# clean up
echo "Removing test file on $ssh_server..."
`ssh $ssh_server "rm $test_file"`
echo "Removing test file locally..."
`rm $test_file`

# print result
echo ""
echo "Upload speed:   $up_speed kB/s"
echo "Download speed: $down_speed kB/s"

Note: I’m still looking for a good way to test general internet upload and download speed via the command line…

Save it in a file called scp-speed-test.sh. Then run it with the command:


bash scp-speed-test.sh user@hostname

Pad images to fit 4 by 6 photo paper bash script using imagemagick

Sunday, August 16th, 2009

I was a little dismayed that I couldn’t use my previously mentioned applescript to pad images with colors other than black. A shrewd observer will also notice that the Image Events suite resamples when it pads resulting in some annoying re-anti-aliasing.

I wrote the following bash script to the same task as the applescript from the command line:


#!/bin/bash

USAGE="Usage: $0 [--replace] [--padcolor=color] ratio imagefile(s)
  ratio              is a float equal to long dimension / short dimension to pad 
                     to fit a 4\"x6\" card, divide 6 by 4 giving ratio = 1.5
  --replace or -f    option to overwrite original images with padded versions,
                     default is to preppend 'pad-[ratio]-' to each naee
  --padcolor= or -c= option to change pad color from default black. Takes a
                     color can be any Imagemagick color or hex color:
                     (See http://www.imagemagick.org/script/color.php)
"
if [ "$#" == "0" ]; then
  echo "$USAGE"
  exit 1
fi

replace=false
if [ "$1" == "--replace" -o "$1" == "-f" ]
then
  echo "Overwriting existing image files..."
  replace=true
  shift
fi

padcolor="black"
if [ `echo "$1" | grep "^\(--padcolor\)\|\(-c\)="` ]
then
  padcolor=`echo "$1" | sed "s/^.*=//"`
  shift
  echo "Using $padcolor as pad color..."
fi

ratio=`echo "$1" | grep "^[0-9]*\.\?[0-9]\+$\|^[0-9]\+\.$"`
if [ "$ratio" == "" ]
then
  echo "$USAGE"
  exit 2
fi
shift
while (( "$#" )); do
  image="$1"
  dir=`dirname "$image"`
  base=`basename "$image"`
  if ! $replace 
  then
    base="pad-$ratio-$base"
  fi
  if wh=`identify -format "%w %h" "$image" 2>/dev/null`
  then
    width=${wh%% *}
    height=${wh#* }
    if [ $height -gt $width ]
    then
      # vertical
      long=`echo "$height/$width > $ratio" | bc -l`
      if [ $long -eq "1" ] 
      then
        # taller than ratio
        newwidth=`echo "$height/$ratio" | bc -l`
        newwidth=`echo "($newwidth+0.5)/1" | bc`
        newheight="$height"
      else
        # shorter than ratio
        newwidth="$width"
        newheight=`echo "$width*$ratio" | bc -l`
        newheight=`echo "($newheight+0.5)/1" | bc`
      fi
    else
      # horizontal
      long=`echo "$width/$height > $ratio" | bc -l`
      if [ $long -eq "1" ] 
      then
        # wider than ratio
        newwidth="$width"
        newheight=`echo "$width/$ratio" | bc -l`
        newheight=`echo "($newheight+0.5)/1" | bc`
      else
        # skinnier than ratio
        newwidth=`echo "$height*$ratio" | bc -l`
        newwidth=`echo "($newwidth+0.5)/1" | bc`
        newheight="$height"
      fi
    fi
    printf "Padding into $base..."
    convert -size "$newwidth"x"$newheight" xc:"$padcolor" .padcanvas.miff
    composite -gravity center "$image" .padcanvas.miff "$dir"/"$base"
    rm .padcanvas.miff
    echo "DONE"
  else
    echo "$image is not a proper image...skipping."
  fi
  shift
done

I have not completely bullet proofed the above but it should get the job done.

Here’s the same before and after shots to show what this script does:

Original Padded with black Padded with transparent pixels
Long horizontal before Long horizontal after Long horizontal after
Short horizontal before Short horizontal after Short horizontal after
Long vertical after Long vertical after Long vertical after
Short vertical before Short vertical after Short vertical after

Look closely at the last column, the images are padded with transparent pixels! Could be useful for web posting.

Remove longest common prefix from all file names in directory

Tuesday, August 11th, 2009

Often when I’m organizing music or especially audiobook folders I run into the tedious problem where I have a folder with a bunch of files or subfolders all starting with the same word or prefix as the parent folder. Here’s an example:


audiobooks/
  ernest hemingway/
    ernest hemingway - old man and the sea/
    ernest hemingway - for whom the bell tolls/
    ernest hemingway - sun also rises/
    ernest hemingway - a farewell to arms/
[and so forth]

Stripping or cropping off "ernest hemingway - " from each name is easily scriptable, and you could probably quickly write an even more generalized script to accept the prefix as an argument. Like strip "ernest hemingway - " ernest\ hemingway/, taking a directory path and a string.

But I wanted to generalize even farther, since for me this usually occurs when there is nothing but prefixed files in the directory, I automated the task of determining the longest common prefix. Then I removed it, with a polite prompt, from each name. Here’s the code, save it in a file called smartcrop.sh:


#!/bin/bash
path=$1
if [ "$path" != "" ]
then
  # be sure path has a single trailing slash
  path="`echo "$path" | sed s/\\\/$//`/"
  if [ ! -d "$path" ]; then
    echo "Usage: ./smartcrop.sh [directory]"
    exit 1
  fi
fi
LS=`ls "$path"`
PREFIX=""
# enclose to keep IFS change from affecting other code
(
  IFS=$'\n'
  for x in $LS
  do
    if [[ $MINLEN -gt ${#x} ]] || [[ $MINLEN == "" ]]
    then
      MINLEN=${#x}
      PREFIX="$x"
    fi
  done
 
  
  # decrement MINLEN in advance because can't `mv "prefix" ""`
  let "MINLEN -= 1"
  while (( 0 < $MINLEN ))
  do
    # throw results away into variable to avoid printing them
    res=`echo "$LS" | grep -v "^$PREFIX"`
    if [ 0 != $? ] 
    then
      break
    fi
    let "MINLEN -= 1"
    PREFIX=${PREFIX:0:MINLEN}
  done
  if (( 0 == $MINLEN))
  then
    echo "No common prefix found."
    exit
  fi
  echo "Longest common prefix: \"$PREFIX\""
  if [ ${#PREFIX} != $MINLEN ]
  then
    echo "  File $path$PREFIX exists."
    echo "  Next longest common prefix: \"${PREFIX:0:MINLEN}\""
  fi
  r=""
  for x in $LS
  do
    if [ "$r" != "A" ]  
    then
      r=""
      while [ "$r" != "y" -a "$r" != "N" -a "$r" != "n" -a "$r" != "A" ]
      do
        read -n 1 -p "rename $path$x to $path${x:MINLEN}? \
[y]es, [n]o, [A]ll, [N]one: " r
        echo
      done
    fi
    if [ "$r" == "A" -o "$r" == "y" -o "$r" == "Y" ]
    then
      mv "$path$x" "$path${x:MINLEN}"
      echo "  moving: $path$x --> $path${x:MINLEN}"
    elif [ "$r" == "N" ]
    then
      exit
    fi
  done
)

To run, execute on the command line in a terminal: ./smartcrop.sh [dir]. So in the above example I would use the command ./smartcrop.sh ernest\ hemingway/

Note: All the mumbojumbo at the beginning of this script is to handle the directory path given as an argument on the command line in the same manner as ls.
Note: the majority of this code can be split into the longest common prefix task and the batch renaming task. Splitting these up into two programs might make more sense for you.