Automate Everything

Gnome and Autospec Notifications

admin — Tue, 04 Aug 2009 15:38:25 +0000

I’m very keen on DBB with RSpec now and I wanted to share how I set up autospec notifications on Ubuntu (Jaunty 9.04) as long as the post I followed to do it is no longer available.

I’m using XCFE but should work perfectly on Gnome as well. This is an screenshot of what you’ll get in the end:

autospec notification XFCE/Gnome

First you need the ZenTest gem, you probably already do:

$ sudo gem install ZenTest
$ sudo gem install redgreen

Then install the libnotify-bin package:

$ sudo apt-get install libnotify-bin

Here is the trick. Create a file called ~/.autotest with this:

#!/bin/ruby
require 'redgreen'
require 'autotest/timestamp'
 
module Autotest::GnomeNotify
  def self.notify title, msg, img
    system "notify-send '#{title}' '#{msg}' -i #{img} -t 3000"
  end
 
  Autotest.add_hook :ran_command do |at|
    image_root = "~/.autotest_images" 
    results = [at.results].flatten.join("\n")
    results.gsub!(/\\e\[\d+m/,'')
    output = results.slice(/(\d+)\sexamples?,\s(\d+)\sfailures?/)
    puts output.inspect
    if output
      if $~[2].to_i > 0
        notify "FAIL", "#{output}", "#{image_root}/fail.png"
      else
        notify "Pass", "#{output}", "#{image_root}/pass.png" 
      end
    end
  end
 
end

As you can see I use the fail.png and pass.png images to show those cheesy smileys : ). You can download them here and copy them to ~/.autotest_images/.

autotest_images.zip

Make your tests go green and have a beer! : )

Update: Added redgreen gem and changed “require redgreen” as Rodrigo Flores and rakk suggested. Thanks!

How to embed .idx subtitles into an mp4 or avi file with mencoder

admin — Wed, 06 May 2009 19:07:30 +0000

I usually watch movies with subtitles on my cellphone but before that I have to transcode the avi to an mp4 with the subtitles “burnt” on the movie, not as a separate file. Recently I came across a movie with subtitles in two files, one .idx and one .sub instead of the common .srt file. And converting the movie to mp4 with that vobsub subtitles embedded wasn’t easy.

First of all I tried converting the idx/sub pair to a single srt. Although possible it’s a pain as long as you have identify the chars with an OCR adding the new ones by hand. Then I tried to get mencoder to read the .idx subs directly as I do with srt. Didn’t work. Finally I came up with a workaround: Merging the avi and the .idx into one Matroska (mkv) file and transcoding that Matroska to mp4 then.

Installing the necessary tools:

sudo apt-get install mencoder mkvtoolnix

Merging the files:

mkvmerge -o video.mkv video.avi video.idx

This is the output:

mkvmerge v2.4.1 ('Use Me') built on Dec 13 2008 21:03:46
'video.avi': Using the AVI demultiplexer. Opening file. This may take some time depending on the file's size.
'video.idx': Using the VobSub subtitle reader (SUB file 'video.sub').
'video.avi' track 0: Using the MPEG-4 part 2 video output module.
'video.avi' track 1: Using the MPEG audio output module.
'video.idx' track 0: Using the VobSub subtitle output module (language: eng).
'video.idx' track 1: Using the VobSub subtitle output module (language: fre).
'video.idx' track 2: Using the VobSub subtitle output module (language: spa).
The file 'video.mkv' has been opened for writing.
Progress: 100%
The cue entries (the index) are being written...
Muxing took 23 seconds.

As you can see the idx file contains subs for three languages, as long as we want to overlay the English one we will pass mencoder the option -slang eng:

mencoder -slang eng -of lavf -lavfopts format=mp4 -oac lavc -ovc lavc -lavcopts aglobal=1:vglobal=1:acodec=libfaac:vcodec=mpeg4:abitrate=128:vbitrate=1200:keyint=250:mbd=1:vqmax=10:lmax=10  -vf harddup 'video.mkv' -o 'video.mp4'

And the resultang mp4 should have the idx subtitles rendered inside, not in a very beautiful way but works : ).

How to fix audio synchronization of an AVI on Linux

admin — Mon, 04 May 2009 19:15:54 +0000

I just downloaded an AVI file with audio out of sync. Everything sounded one second before it should during the whole movie so it was really annoying. Fixing this with mencoder is easy and fast.

mencoder video.avi  -oac copy -ovc copy  -delay -1 -o video-fixed.avi

That will delay sound for one second. Notice that “-1″ represents a negative value, so you could write “2.5″ to advance (not delay) audio for 2 seconds and a half.

Here comes the tedious part. How do I know how much do I have to delay audio? What I did is trial and error. But don’t scream yet it’s not as horrible as it sounds. You can make tries less painful encoding just the first 150 seconds of video adding “-endpos 150″:

mencoder video.avi  -oac copy -ovc copy  -delay -1 -endpos 150 -o video-fixed.avi

Prevent apt-get from upgrading a package on Ubuntu

admin — Sat, 25 Apr 2009 15:16:18 +0000

Yesterday I updated my Ubuntu to Jaunty Jackalope and it overwrote my custom version of rxvt-unicode that supports 256 colors. I got it following this entry Get rxvt-unicode with 256 color support on Ubuntu.

To recover it I reinstalled the custom deb package. It worked, but every time I try to update my packages apt-get tries to “upgrade” rxvt-unicode too discarding my version.

I found the solution here: Keep a package at specified version. It’s just a matter of pinning a package. Just go:

sudo vim /etc/apt/preferences

And add the following:

Package: rxvt-unicode
Pin: version 9.05*
Pin-Priority: 1001

And no more whinning : )

Converting Avi files to Mp4 for Nokia 5800

admin — Sun, 29 Mar 2009 16:30:28 +0000

Recently I acquired a Nokia 5800 XpressMusic mobile phone. Great one! It comes with a native video player capable of playing video up to a resolution of 640×480 at 30fps. The video player, in combination with its big screen and the TV-out capability, makes this phone a good device for watching a movie with an acceptable quality. It can play videos in the following formats:

3GPP formats (H.263)
Flash Video
H.264/AVC
MPEG-4
RealVideo 7,8,9/10
WMV 9

But as you can see, there is no AVI playback. Right now there is no AVI player that supports Symbian s60 5th. So we’ll have to transcode our .avi files to .mp4 before playing them on the Nokia 5800. Playing with mencoder I’ve come to following commands for converting an AVI to MP4, Nokia 5800 ready. Subtitles included and 2 pass (which means high quality results):

mencoder -of lavf -lavfopts format=mp4 -oac lavc -ovc lavc -lavcopts aglobal=1:vglobal=1:acodec=libfaac:vcodec=mpeg4:abitrate=96:vbitrate=800:keyint=250:mbd=1:vqmax=10:lmax=10:vpass=1:turbo -ofps 25 -af lavcresample=44100 -vf harddup,scale=640:-3 -sub 'input.srt' 'input.avi' -o 'output.mp4'
mencoder -of lavf -lavfopts format=mp4 -oac lavc -ovc lavc -lavcopts aglobal=1:vglobal=1:acodec=libfaac:vcodec=mpeg4:abitrate=96:vbitrate=800:keyint=250:mbd=1:vqmax=10:lmax=10:vpass=2 -ofps 25 -af lavcresample=44100 -vf harddup,scale=640:-3 -sub 'input.srt' 'input.avi' -o 'output.mp4'

Messy huh? What I did is a Ruby script that takes AVI files as an argument an transcode them taking care of everything. I mean:

Automatically names the output as “file.mp4″ from “file.avi”
Checks if there are subtitles (as an SRT file) and add them.
Performs the 2 passes.
Deletes the useless “divx2pass.log” once finished.
Resize and resample video and audio to fit the phone specs.

Here is the script:

#!/usr/bin/ruby
 
ARGV.each do |input_f|
  everything_ok = true
  log_file = "divx2pass.log"
 
  # Escapes single quotes
  input_f = input_f.split("'").join("\'\\'\'")
 
  # Extracts the file name without extension
  file_name = input_f.split(".")[0..-2].join(".")  
 
  # Output file
  output_f = file_name + ".mp4"
  puts "\nConverting \'#{input_f}\' into \'#{output_f}\':"
 
  # Check if there are subtitles to add to the file
  subs_args = File.exists?(file_name + ".srt") ? " -sub \'#{file_name}.srt\'" : ""
 
  # Encodes with two pass
  [":vpass=1:turbo",":vpass=2"].each do |vpass|
    command = "mencoder"
    command << " -of lavf -lavfopts format=mp4" # Set container format to mp4
    command << " -oac lavc" # Audio will be encoded with a libavcodec codec
    command << " -ovc lavc" # Video will be encoded with a libavcodec codec
 
    command << " -lavcopts" # Begin of options for libavcodec
    command << " aglobal=1:vglobal=1" # Needed for mp4
    command << ":acodec=libfaac" # Encode audio as AAC
    command << ":vcodec=mpeg4" # Encode video as mp4 too
    command << ":abitrate=96" # Audio bitrate in Kbps
    command << ":vbitrate=800" # Video bitrate in Kbps
    command << ":keyint=250" # Number of frames between keyframes, a bit lower than the default to save some Megabytes
    command << ":mbd=1" # Macroblock decision algorithm set to minimize size of file
    command << ":vqmax=10:lmax=10" # Don't know whay but raises the video quality
    command << vpass # First pass or second pass
 
    command << " -ofps 25" # Convert the video to 25 fps to avoid incompatible fps values
    command << " -af lavcresample=44100" # Resample the audio
    command << " -vf harddup,scale=640:-3" # Scale the video to 640 px width mantaining aspect ratio
    command << subs_args # Add subs
 
    # command << " -endpos 10" # Encode just the first 10 seconds of video, useful for testing
 
    command << " \'#{input_f}\' -o \'#{output_f}\'" # file names
 
    puts command
    everything_ok = system command
 
    if !everything_ok
      puts "There were errors encoding #{input_f}, aborting."
      break
    end
  end
 
  File.delete log_file if File.exists? log_file
  puts "File succesfully encoded as \'#{output_f}\'" if everything_ok
end

Use it like this:

avi2mp4 myfile.avi

And it will generate “myfile.mp4″. It also can receive several files as argument and will encode each of them separately. Hope you find this useful!

Adding all new files recursively to Subversion

admin — Thu, 26 Mar 2009 18:25:45 +0000

Sometimes you want to add a lot of new files to a subversion repository. An smart solution can be found on this post:

svn status | grep "^?" | awk '{print $2}' | xargs svn add

That will add the new files to the repository in just one command, not looping around for each file. Cool!

Piston and git

admin — Thu, 26 Mar 2009 18:15:51 +0000

As many Rails developers I have some projects that I haven’t migrated to git yet. Those are still under Subversion and I manage its plugins with Piston. But what happen when you want to import a plugin hosted on a git repository? That the gem “piston” doesn’t work.

The trick is that the last version of piston that supports git is not the gem called “piston” but “francois-piston”. So let’s go:

gem sources -a http://gems.github.com 
sudo gem install francois-piston

Now we can do:

piston import git://github.com/activescaffold/active_scaffold.git vendor/plugins/active_scaffold

Easy :).

How to migrate a Subversion repository from one server to another

admin — Tue, 10 Feb 2009 10:50:51 +0000

Now I got a new server I had to migrate my old Subversion repository from the old server to the new one. Thanks to Subversion it was a cake easy task:

First, we have to dump our old repository.

ssh old-server.com
svnadmin dump svn/my-project > my-project.dump
exit

Then, we transfer the dump from one server to the other:

cd /tmp
rsync -Praz old-server.com:my-project.dump .
rsync -Praz my-project.dump new-server.com:~

We are now ready to load the dump into the new repository:

ssh new-server.com
mkdir svn
svnadmin create svn/my-project
svnadmin load svn/my-project < my-project.dump
exit

Finally, we update our local copy of the code:

mv my-project my-project.old
svn co svn+ssh://new-server.com/home/your-user-name/svn/my-project/trunk my-project

And you are ready to code!

How I got Mephisto working on Dreamhost

admin — Tue, 10 Feb 2009 10:49:24 +0000

As long as Dreamhost supports Phusion Passenger now, installing the Ruby on Rails blog engine Mephisto should be an easy task. However, I found this trickier than I expected. I followed the basic instructions and I was getting the following error on my log/production.log:

ActionView::TemplateError (can't convert nil into String) on line #6 of app/views/admin/design/_sidebar.rhtml:
3: Modify a template by selecting it from the list below.  Add a new layout by creating a Liquid template with an
4:   *_layout suffix (e.g custom_layout).
5: 
6:   <% @theme.templates.template_types(@theme.extension).each do |template| -%>
7:     <%= link_to h(template), url_for_theme(:controller => 'templates', :action => 'edit', :filename => template) %>
8:   <% end -%>
9:   <% @theme.templates.custom(@theme.extension).each_with_index do |template, i| -%>
 
    /home/manuelmorales/mephisto/app/models/templates.rb:6:in `+'
    /home/manuelmorales/mephisto/app/models/templates.rb:6:in `template_types'
    /home/manuelmorales/mephisto/app/models/templates.rb:6:in `collect'
    /home/manuelmorales/mephisto/app/models/templates.rb:6:in `template_types'
    ....

The problem is that, in spite of the right tzinfo gem already comes with Mephisto, Phusion Passenger is ignoring local gems, taking the old version of tzinfo that Dreamhost provides. The solution is simple: Force Rails to load exactly that gem adding the following to your config/environment.rb.

...
require '/home//mephisto/vendor/gems/tzinfo-0.3.12/lib/tzinfo'
...

And restarted the server:

touch tmp/restart.txt

Visited a working page of the blog twice to force the restart to make effect and tzinfo error gone.

In case that you are trying to install another application that doesn’t already come with the gem you have to override, you will have to install it locally before forcing Rails to load it. Directions about how to install gems locally in the Dreamhost wiki.

But that wasn’t enough. Then I was getting the following error:

ActionView::TemplateError (attempted to output tainted string: email@domain.com) on line #14 of app/views/admin/sites/show.rhtml:
11: <% form_for :site, :url => { :action => "update", :id => @site } do |f| -%>
12: <%= f.error_messages %>
13: 
14:   <%=h @site.title %> (<%= mail_to @site.email %>)
15:   
16:             Host
17:             <%= f.text_field :host %>
 
    /home/manuelmorales/mephisto/vendor/gems/emk-safe_erb-0.1.1/lib/safe_erb/common.rb:48:in `concat_unless_tainted'
    /home/manuelmorales/mephisto/app/views/admin/sites/show.rhtml:14
    /usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/helpers/form_helper.rb:313:in `fields_for'
    ...

As said here on the Mephisto blog:

One debugging tip: If you see an error like ActionView::TemplateError (attempted to output tainted string), you’ve run afoul of SafeERB and you probably need to insert an h(…) somewhere. Don’t hesitate to ask for help on #mephisto.

That’s it. I had to go to file app/views/admin/sites/show.rhtml, line 14, and modify this:

<%=h @site.title %> (<%=mail_to @site.email %>)

to surround the @site.email with h():

<%=h @site.title %> (<%=mail_to h(@site.email) %>)

And here is it. Fully working now! : )