Clean up pen-on-paper line drawings

Alec Jacobson

November 05, 2010

weblog/

The most mechanical and mundane behavior I find myself doing over and over again involves cleaning up my pen and paper line drawings before importing them into Illustrator and turning them into vector graphics. My usual procedure is:

Draw the lines with a real pen on real paper
Scan or photograph the drawing This produces a very raw image of the line drawing.
Use Curves and the Magic Wand Tool in Photoshop to reduce the image to black lines on a white background
Import into Illustrator, use Live Trace with the One Color Logo preset to turn the lines into vector paths. Then color with the Live Paint Bucket.

And in the end I can make something like this:

Vectorizing and coloring with Illustrator

This process is not very hard, but it's rather boring. The fun parts are step #1 and #4, the drawing and coloring steps. Cleaning up the raw images in photoshop is tedious. Sometimes it can be very quick: 5 mins or so if you include opening and closing photoshop. Other times, if there are too many shadows in the image or the lines aren't dark enough or there are too many spurious smudges and marks, I have to spend much more time in photoshop: 10-15 mins. Or worse, I have to retake the photo/scan and start over. There are two ways I thought of solving this problem:

Make better photographs/scans, so cleaning up in Photoshop is easier
Automate the Photoshop tasks

For #1, it seems that just buying a nice scanner would suffice. But getting the scanner hooked up is slow and I can't carry the scanner around with me everywhere. I've also noticed that scanners tend to collect hairs and dust that appear as little marks on the image. Instead I'm working on a way to take better pictures of paper with my crappy cell-phone camera. But that's another post... This post will focus on option #2. The main problem with the Photoshop steps is not even that it takes so long, it's that it needs my attention the entire time. It's in no way automatic. Scripting Photoshop is possible, but actually scripting the curves steps that I do would be too hard and not general enough.

Automatic line extraction

I've come up with the following idea. I will bank on the assumption that the input image is an image of a line drawing on a piece of white paper. The whole problem stems from the fact that shadows and imperfections on the paper cause the paper to appear unevenly gray. If only I had another image of just the paper without the line drawing I could subtract the paper image from the (line + paper) image, leaving just the lines.

lines plus paper minus paper equals line

I wrote up a proof of concept in MATLAB that performs the following steps:

Find edges in the image (note that this is different than finding lines, edges in our case occur on either side of a pen-on-paper line).
Blur these edges using a blur kernel whose width is parameterized based on the expected thickness of the lines (smaller for ball-point pens, larger for thick markers).
Threshold all values over some small amount in the blurred edges mask, also get rid of tiny regions.
Fill use this mask as "holes" to be filled in via Laplacian hole-filling. Notice we now have an approximation of the blank sheet of paper:
Subtract the paper from the original image and intensify the lines so the image is black on white (but not necessarily a binary image):

Results

Here are some results. I compare the original photograph or scan of the line drawing with my best clean up job using (only) Curves in Photoshop and my automatic script. For my script I am not adjusting the parameters for each input. I have found a set of parameters that work well for pen on paper inputs and I used this preset for all the images shown in the table.

Original	Photoshop Curves	MATLAB script

Note: I have implemented my MATLAB script to support all 3 RGB color channels so that color pencil/pen drawings still come out in color. Download: Download the MATLAB source for line drawing clean up algorithm.