Posts Tagged ‘file’

Matlab is stalling after every command

Tuesday, April 1st, 2014

I ran into a strange issue that MATLAB was stalling for a long while each time I issued a command and took a long time to execute scripts. Seems the issue was that the current directory I was working in had over 30000 files. My guess is that this was causing the parser to freak out when searching for available functions. Anyway, moving these files (some generated data) to a subfolder not in the working path fixed the issue.

OBJ reader, mesh viewer in browser with WebGL

Wednesday, April 10th, 2013

mesh in browser

I’m getting around to release source code and demos for my PhD projects and I thought it would be cool to try out writing those demos using WebGL. After toying around (for a few years now), I have a little proof of concept.

This little mesh viewer can load an .obj mesh file (just bare basics of the format) from (1) the remote server, (2) a selected file, or (3) a drag-and-drop file. Just drag an .obj file from your folder onto the viewer! For now there’s not much interface, just a simple zoom mechanism. But I’m going to build on this and keep updated the page.

The source code is heavily borrowed from around the web. See the .js sources.

Convert a bunch of images into a video, batch job

Friday, October 26th, 2012

I was sure I posted this some time before, but just in case, here it is again. My programs can easily dump jpgs, pngs, or tga image files at every frame. Sometimes it suffices to convert these to an animated GIF using imagemagick. But if there are many images and they’re all hi-res then really you need a proper video codec to get a reasonably small file. I use this ffmpeg command:

ffmpeg -f image2 -r 30 -i myimages-%04d.tga -r 30 -vcodec libx264 -pix_fmt yuv420p -q:vscale 0 output.mp4

or using GLOB for a lexicographical, alphabetical file list:

ffmpeg -f image2 -r 30 -pattern_type glob -i 'my-images-*.png' -r 30 -vcodec libx264 -pix_fmt yuv420p -q:vscale 0 output.mp4

Update : The -pix_fmt yuv420p` is important for getting the output video to play in quicktime.

Close all open files in matlab after error

Friday, June 29th, 2012

I got a funny error running a matlab batch script:


/Applications/MATLAB_R2011b.app/toolbox/matlab/lang/message.m:
Too many files open.

Error in dlmread (line 62)

The solution was easy:


fclose('all')

Rename files with number using bash

Tuesday, July 12th, 2011

I had a bunch of jpegs:


 my_file_name.jpg
 my_other_file_name.jpg
 here's_one.more.jpeg
 ...
 my_fiftieth_file.jpg

and I want to quickly rename them to


01.jpg
02.jpg
03.jpg
...
50.jpg

I do this with the following small script in bash. Save this in rename_numbered.sh:

#!/bin/bash
# USAGE: rename_numbered.sh files
x=1;
pad=`echo "(l($#+1)/l(10)) + 0.5" | bc -l`
for var in "$@"
do
  dn=`dirname $var`;
  ext=${var##*.}
  echo $ext
  echo "mv \"$var\" \"$dn/`printf '%0'$pad'd' $x`.$ext\""
#uncomment next line to enable
#  mv "$var" "$dn/`printf '%0'$pad'd' $x`.$ext"
  x=$(($x+1));
done

Then you can rename a bunch of files, like all those in a directory with:


./rename_numbered.sh path/to/dir/*

Vi(m) tip #11: remove swap file of current file (e.g. after a recover operation)

Wednesday, June 22nd, 2011

You can remove the swap file (.*.swp) of the current file in vim with a simple command in command mode:


:!rm .%.swp

Read local file and display contents using javascript

Friday, February 18th, 2011

The upcoming File API for web applications means that HTML 5 pages will be able to ask user for a local file (a file on the user’s computer) and use the data in that file directly. It’s amazing that this hasn’t really existed before. Web applications either used http requests to send the file to the server then have the server send it back, or use super hacky solutions that typically involved using Java or Flash applets to piggy-back into the local file.

Unfortunately, this is not fully supported by browsers. It seems Firefox and Chrom(e|ium) support this File API already, but Safari and Webkit don’t yet. IE may never.

Check it out:

Progress:

File contents:

Here's the source as a full HTML page:


<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
	<head>
		<meta http-equiv='Content-type' content='text/html;charset=UTF-8' >
		<script>
function startRead()
{
  // obtain input element through DOM 
  
  var file = document.getElementById('file').files[0];
  if(file)
	{
    getAsText(file);
  }
}

function getAsText(readFile)
{
	var reader;
	try
	{
    reader = new FileReader();
	}catch(e)
	{
		document.getElementById('output').innerHTML = 
			"Error: seems File API is not supported on your browser";
	  return;
  }
  
  // Read file into memory as UTF-8      
  reader.readAsText(readFile, "UTF-8");
  
  // Handle progress, success, and errors
  reader.onprogress = updateProgress;
  reader.onload = loaded;
  reader.onerror = errorHandler;
}

function updateProgress(evt)
{
  if (evt.lengthComputable)
	{
    // evt.loaded and evt.total are ProgressEvent properties
    var loaded = (evt.loaded / evt.total);
    if (loaded < 1)
		{
      // Increase the prog bar length
      // style.width = (loaded * 200) + "px";
			document.getElementById("bar").style.width = (loaded*100) + "%";
    }
  }
}

function loaded(evt)
{
  // Obtain the read file data    
  var fileString = evt.target.result;
  document.getElementById('output').innerHTML = fileString;
		document.getElementById("bar").style.width = 100 + "%";
}

function errorHandler(evt)
{
  if(evt.target.error.code == evt.target.error.NOT_READABLE_ERR)
	{
    // The file could not be read
		document.getElementById('output').innerHTML = "Error reading file..."
  }
}
		</script>
	</head>

	<body>
		<input id="file" type="file" multiple onchange="startRead()">
		<h3>Progress:</h3>
		<div style="width:100%;height:20px;border:1px solid black;">
		<div id="bar" style="background-color:#45F;width:0px;height:20px;"></div>
		</div>
		<h3>File contents:</h3>
		<pre>
			<code id="output">
			</code>
		</pre>
	</body>
</html>

Note: This WILL NOT WORK if you load your html page as a local file. Meaning the address starts with file:/// instead of http://, for some reason the page must be served to the browser from a server.

Source

Universal File Dialog, a first step

Tuesday, August 31st, 2010

I’m switching from Qt to straight Glut with AntTweakBar for my next project. I’m happy about getting rid of Qt. For my small prototyping projects it was too heavy (hard to get a target working on computers that don’t have Qt installed) and too much work to add simple ui for tweaking parameters and flipping flags. AntTweakBar doesn’t yet solve the deployable problem (I’m trying to figure that out too) but is miles better for prototyping. The only thing I miss from Qt is the native File Save/File Open Dialogs. Since Glut is fairly universal I have taken a stab at writing a small header that could be the start of a universal file dialog “library”. I’d only want this library to depend on standard includes and it only needs to implement get_file_save_path and get_file_open_path.

Here’s what I put in FileDialog.h:


#include <stdio.h>
#define FILE_DIALOG_MAX_BUFFER 1024

// Sets buffer to a path to an existing file 
// buffer[0]=0 on cancel
//
// Usage:
//   char buffer[FILE_DIALOG_MAX_BUFFER];
//   get_open_file_path(buffer);
void get_open_file_path(char buffer[]){
#ifdef __APPLE__
  // For apple use applescript hack
  FILE * output = popen(
    "osascript -e \""
    "   tell application \\\"System Events\\\"\n"
    "           activate\n"
    "           set existing_file to choose file\n"
    "   end tell\n"
    "   set existing_file_path to (POSIX path of (existing_file))\n"
    "\" 2>/dev/null | tr -d '\n' ","r");
  while ( fgets(buffer, FILE_DIALOG_MAX_BUFFER, output) != NULL ){
  }
#else
  // For every other machine type 
  printf("Please enter a file path: ");
  gets(buffer);
#endif
}

// Sets buffer to a path to a new/existing file 
// buffer[0]=0 on cancel
//
// Usage:
//   char buffer[FILE_DIALOG_MAX_BUFFER];
//   get_save_file_path(buffer);
void get_save_file_path(char buffer[]){
#ifdef __APPLE__
  // For apple use applescript hack
  // There is currently a bug in Applescript that strips extensions off
  // of chosen existing files in the "choose file name" dialog
  // I'm assuming that will be fixed soon :-)
  FILE * output = popen(
    "osascript -e \""
    "   tell application \\\"System Events\\\"\n"
    "           activate\n"
    "           set existing_file to choose file name\n"
    "   end tell\n"
    "   set existing_file_path to (POSIX path of (existing_file))\n"
    "\" 2>/dev/null | tr -d '\n' ","r");
  while ( fgets(buffer, FILE_DIALOG_MAX_BUFFER, output) != NULL ){
  }
#else
  // For every other machine type 
  printf("Please enter a file path: ");
  gets(buffer);
#endif
}

And here’s a sample program that calls both functions. Save it in test.c:


#include "FileDialog.h"
#include <stdio.h>

int main(void){
  char buffer[FILE_DIALOG_MAX_BUFFER];

  get_open_file_path(buffer);
  if(buffer[0] == 0)
    printf("Cancelled\n");
  else
    printf("Open file path: %s\n",buffer);

  get_save_file_path(buffer);
  if(buffer[0] == 0)
    printf("Cancelled\n");
  else
    printf("Save file path: %s\n",buffer);

  return 0;
};

Compile and run on a Mac with:


gcc -o test test.c;
./test

Update: I've been looking into a way to do the above for Mac using Carbon or Cocoa but it seems impossible to do correctly without starting up an entire new app...

Update: I've noticed many annoying quirks about the above applescript. There's no way to specify only showing certain file extensions (just UTI types but that's not always what you want). It's quite slow to pop up. But most annoying was that focus was not returned to my GLUT app. Here's the hack I place in my app to use applescript to return the focus to my app. Seems to work great:


// you call to FileDialog.h
get_open_file_path(...)
#ifdef __APPLE__
  // Hack to put focus back on app
  FILE * output = popen(                            
    "osascript -e \""
    "   tell application \\\"NameOfYourApp\\\"\n"           
    "           activate\n"
    "   end tell\"\n"                               
    ,"r");                                          
#endif

Of course, replace NameOfYourApp with the name of your app. If you're unsure what this name is, try using the name that appears above your app's icon on the dock when you hover over it.

Split (large) files in half using bash script

Monday, June 21st, 2010

Here’s a simple bash script that splits a given file into two equal (up to a byte) halves. It uses split which has different syntax between the GNU and BSD implementations, so see the comments for switching between the two.
I tested this on a 6.4 GB dmg file and I was able to reverse the split with cat just fine. I diffed the re-concatenated file and the original and they were byte-wise equal.

Save this in a file called half.sh:


#!/bin/bash
#
# usage: half foo 
#
# This will create foo.aa and foo.ab, where foo.aa is the first half(+1) of foo
# and foo.ab is the second half. Split by bytes not lines.
#
#
# To reverse just issue:
# cat foo.aa foo.ab > foo
#

# get file size
# GNU:
#size=`stat -c "%s" "$1"`
# BSD:
size=`stat -f "%z" "$1"`

# divide by 2 and round up
half_size=`echo "($size+1)/2" | bc`
split -b $half_size "$1" "$1."

Find big directories using du and (e)grep

Thursday, March 25th, 2010

Here’s a simple script I’m using to locate big directories (larger than 1GB):


du -h | grep "^ *[0-9][0-9.]*G"

The output looks like this:


1.1G	./Art/2007
2.5G	./Art/2008/Party Freakz Pasta Party
6.3G	./Art/2008
9.2G	./Art
 24G	./Documents/Parallels/Windows 7.pvm/Windows 7-0.hdd
 24G	./Documents/Parallels/Windows 7.pvm
 24G	./Documents/Parallels
 26G	./Documents
2.5G	./Downloads/enstrophy-big-endian
 33G	./Downloads
...
100G	.

Here’s how I find big directories greater than 100MB:


du -h | egrep "^ *([0-9][0-9][0-9][0-9.]*M)|([0-9][0-9.]*G)"

To list files, too, just add the -a flag to du like this:


du -ah | grep "^ *[0-9][0-9.]*G"