Posts Tagged ‘bash’

Download web file onto a server from client

Tuesday, March 19th, 2013

If I want to download a file from a URL I can use wget. Then I can use scp to copy to my webserver. Or I could ssh to the server and call wget from there. Here’s a one-liner to do just that:


ssh SERVER 'wget URL -O LOCAL_SERVER_OUTPUT_FILE'

Move vim’s undo files

Thursday, November 15th, 2012

vim creates little files to keep a persistent undo. The names are like:


.filename.txt.un~

The dot prefix and the tilde suffix make these files a pain to deal with in bash. Here’s how to move these files to another directory:


mv .*un\~ other_directory

Making mercurial feel like svn

Sunday, November 11th, 2012

Stop asking for my password all the time

The easy way to fix this is to add your password in plain text to your .hgrc file.

The better way is to use SSH. We were currently using RhodeCode to serve up our mercurial repos but its easy to use SSH instead.

For exiting projects just change the [project]/.hg/hgrc file from:


[paths]
default = https://[hg username]@[server]/hg/[project]

to


[paths]
default = ssh://[ssh username]@[server]/hg/[project]

Notice that your [hg username] might be different from your [ssh username].

Update: I’m still figuring this one out…

One-liner commit,push and pull,update

To automatically push after every commit just add the following to your ~/.hgrc:


[hooks]
pre-commit = echo "\x1B[00;32mAutomatically pushing\x1B[00m"
commit.autopush = hg push

The echo serves as a reminder that each commit is really pushing.

To automatically update after each pull, just issue:


hg pull -u

Round images to width and heights divisible by 2, by cropping

Saturday, November 10th, 2012

To make an h264 movie from a bunch of images using ffmpeg I need all the images to have size dimensions divisible by two. Here’s a little one-liner to crop a bunch of images to the nearest size divisible by 2:


for file in *.png; do convert -crop `identify -format "(%[fx:w]/2)*2" $file | bc`x`identify -format "(%[fx:h]/2)*2" $file | bc`+0+0 $file cropped_$file; done

I suppose using mogrify would potentially be faster. But I’m not sure how to introduce the rounding.

One liner to add public key to authorized ssh keys

Monday, November 5th, 2012

cat ~/.ssh/id_rsa.pub | ssh $SERVER 'cat >> ~/.ssh/authorized_keys'

PHP cURL command line app not waiting for response

Monday, September 17th, 2012

I had written a PHP command line app that took a URL as a command line argument:


php foo.php http://www.nytimes.com/2012/09/16/arts/shock-me-if-you-can.html?ref=arts&pagewanted=all 

The foo.php was simply calling curl to grab the page in question, do a little parsing and then spit it back out. When I issued the command above though, the program seemed to finish immediately and then later spit out the results. I was left thinking that the curl commands were asynchronous.
It turned out to be much simpler. I had an ampersand (&) in the url argument. This was forking the process in my bash terminal. All I needed to do was put quotes around the argument:


php foo.php "http://www.nytimes.com/2012/09/16/arts/shock-me-if-you-can.html?ref=arts&pagewanted=all"

Download high definition images from christies

Saturday, July 14th, 2012

Place this in christies.sh:


#!/bin/bash

#id=D5391252
id=`echo "$1" | sed -e "s/.*ObjectID=\([0-9]*\)&.*/d\1/g"`
final=`echo "$1" | sed -e "s/.*lotfinder\\/[^\\/]*\\/\(.*\)-[0-9]*-details.*/\1/g"`
echo $id
echo $final


URL=http://www.christies.com/lotfinderimages/${id:0:6}/$id/TileGroup0/
echo $URL

# Explicitly declare integers
typeset -i i j k max_i max_j max_k tile_x tile_y

if [ $2 ]; then
  let max_i=$2
else
  #Try to find biggest zoom size
  let i=0
  while true; do
    if wget $URL/$i-0-0.jpg --spider -v -O/dev/null 2>&1 | grep "Remote file exists\." 1> /dev/null
    then
      echo "$i-0-0.jpg exists"
      max_i=$i
    else
      break
    fi
    let i++
  done
fi

#let max_i=4

#Try to find biggest first index
let j=0 max_j=0
while true; do
  if wget $URL/$max_i-$j-0.jpg -v 2>&1 | grep "image/jpeg" 1> /dev/null
  then
    echo "$max_i-$j-0.jpg exists"
    max_j=$j
  else
    # clean up
    rm -f $max_i-$j-0.jpg
    break
  fi
  let j++
done

#let max_j=12

#Try to find biggest second index
let k=1 max_k=0
while true; do
  if wget $URL/$max_i-0-$k.jpg -v 2>&1 | grep "image/jpeg" 1> /dev/null
  then
    echo "$max_i-0-$k.jpg exists"
    max_k=$k
  else
    # clean up
    rm -f $max_i-0-$k.jpg
    break
  fi
  let k++
done

#let max_k=8

let j=1
while ((j<=max_j)); do
  let k=1
  while ((k<=max_k)); do
    if wget $URL/$max_i-$j-$k.jpg -v 2>&1 | grep "image/jpeg" 1> /dev/null
    then
      echo "$max_i-$j-$k.jpg exists"
    else
      echo "$max_i-$j-$k.jpg not found!"
    fi
    let k++
  done
  let j++
done

# get list of images in an order that montage will understand
list=""
let k=0
while ((k<=max_k)); do
  let j=0
  while ((j<=max_j)); do
    list="$list $max_i-$j-$k.jpg"
    let j++
  done
  let k++
done

let tile_y=max_j+1
let tile_x=max_k+1

montage $list -tile ${tile_y}x${tile_x} -geometry +0+0 -quality 100 $final.jpg
# clean up
rm -f $list

Then you can issue:


./christies "http://www.christies.com/lotfinder/sculptures-statues-figures/jeff-koons-winter-bears-5408907-details.aspx?from=searchresults&intObjectID=5408907&sid=275b24b4-22e5-4277-8e72-36c4708279d8"

Just be sure to put quotes around the url.

Patch for colormake to properly handle redirecting std err

Friday, February 10th, 2012

I started using colormake which is a great little perl script that wraps the make command, colorizing its output. I liked it so much that I quickly decided to make an alias in my bash .profile so that whenever I called make I would actually run colormake:


alias make=colormake

Note: The original colormake script is called cmake, but I renamed it to colormake to avoid conflict with the cmake.

This was going fine until I want to actually play with my typical make output. One thing I like to be able to do is redirect all of my errors to a file:


make 2>make.err

My alias to colormake broke this because the first thing the colormake wrapper does is redirect all of make’s std err output to std out.

Here’s my patch to the colormake (cmake) script that more properly deals with stderr. I also provide a fix to the errors from using stty incorrectly (though I actually don’t like the extra feature of colormake which trims commands to fit the window):


#!/bin/bash
#
# Wrapper around make to colorize its output.
#
# This patch fixes problems with calling stty in a script and redirecting std
# err
#

## use terminal size but avoid "stty: stdin isn't a terminal" warning from stty 
#STTY_SIZE=`stty size 2>/dev/null`
# Don't pass a terminal size (soft wrap entire commands)
STTY_SIZE=""

if [ -t 1 ];
then
  # Std out is tty
  if [ -t 2 ];
  then
    # Std err is tty
    make $* 2>&1 | colormake.pl $STTY_SIZE                                                              
    ret_val=${PIPESTATUS[0]}
  else
    # Std err is not tty
    # only pipe std out to colormake
    make $* | colormake.pl $STTY_SIZE                                                              
    ret_val=${PIPESTATUS[0]}
  fi
else
  # Std out is not tty
  make $*
  ret_val=$? 
fi                                                                                        
exit $ret_val

Update: Using PIPESTATUS I made sure that this script returns the return code of the make call.

Screen capture a remote desktop into the remote clipboard

Friday, January 6th, 2012

I’m working a bit with Apple Remote Desktop now and I’m having trouble taking screen captures. It seems easy to take a screen capture that ends up as a file on or in the clipboard of the client side. But I’d rather be completely immersed in the remote computer, so ideally when I hit CMD+SHIFT+3 I’d get a screen shot the size and resolution of my remote desktop saved to the remote’s clipboard.

Here’s what I’m doing for now. I prepare my app that I want to screen capture then I switch to terminal and issue


echo "tell application \"skinning\" to activate" | osascript - && screencapture -c

This switches to my app and takes a screen capture into the clipboard.

Self-compiling .cpp file

Thursday, November 17th, 2011

I often put compilation notes in the comments at the top of small .cpp programs. These are just reminders to me of the flags I need to set in g++ to compile the program. For bigger programs I use Xcode or Makefiles, but often if I’m just testing a small idea I don’t want that overhead.

I had a funny idea today that my .cpp file could act as both a bash script and as C++ code. Then I could execute the .cpp file (as a bash script) which would set up the compiler and then compile itself. Here’s a proof of concept, save this in main.cpp


#!/bin/bash
//usr/bin/tail -n +2 $0 | g++ -o main -x c++ - && exit
#include <cstdio>
int main(int argc,char * argv[])
{
  printf("Hello, world\n");
  return 0;
}

Make it executable with


chmod +x main.cpp

To compile just execute the cpp file:


./main.cpp

This creates the real binary of the program which you can execute with:


./main

Update:
If you don’t like the hassle of making the binary you can just run the binary right after you compile it, then delete it immediately. Do this by changing the second line to:


#!/bin/bash
//usr/bin/tail -n +2 $0 | g++ -o main -x c++ - && ./main && rm main && exit
#include <cstdio>
int main(int argc,char * argv[])
{
  printf("Hello, world\n");
  return 0;
}

Update:
With an additional hack you can have a full fledged bash script at the top of your .cpp file:


#!/bin/bash
/*/../bin/ls > /dev/null
# BEGIN BASH SCRIPT
printf "//" | cat - $0 | g++ -o .main -x c++ - && ./.main $@
rm -f .main
# END BASH SCRIPT
exit
*/
#include <cstdio>
int main(int argc,char * argv[])
{
  printf("Hello, world\n");
  return 0;
}

Update: Here’s another one I found especially useful. This allows me to pipe the output to colormake to get colorized terminal compiler output.


#!/bin/bash
/*/../bin/ls > /dev/null
# BEGIN BASH SCRIPT 
export PS4=""
set -o xtrace       
printf "//" | cat - $0 | g++ -o .main -x c++ - && ./.main $@
rm -f .main         
# END BASH SCRIPT
exit
*/
#include <cstdio>
int main(int argc,char * argv[])
{
  printf("Hello, world\n");
  return 0;
}

Then I pipe the output to colormake using:


./main.cpp 2>&1 | colormake.pl