sysadmin

Getting Netflix to Run on Mint 15 (Olivia)

This turned out to be a bit tricky and required some digging. Here’s what worked for me:

Install Pipelight

Follow the instructions here to install Pipelight (which uses Wine to run the Silverlight plugin that Netflix needs):

Pipelight Installation Instructions

For me, this meant running the following commands:

sudo add-apt-repository ppa:pipelight/stable
sudo apt-get update
sudo apt-get install --install-recommends pipelight-multi
sudo pipelight-plugin --update

User Agent Switching

Before you can test anything, you need to install a user agent switcher in your browser. I normally run Firefox, but switched to Chromium for Netflix because I kept getting DRM errors (Turns out that Firefox was not the issue, but a clean install of Chromium eliminated some variables anyhow).

While testing in Firefox, I ran across these instructions to delete the contents of “/~/.wine-pipelight/drive_c/users/Public/Application\ Data/Microsoft/PlayReady/” to eliminate the DRM error. This didn’t seem to help for me, so I switched to Chromium, just in case. In fact, the real issue was that I needed to set my root filesystem to mount with extended attributes (see below). Both Chromium and Firefox worked in the end.

In Chromium, I installed this extension:

User-Agent Switcher for Chrome

In Firefox, this one:

User-Agent Overrider

I set the user-agent to “Firefox/Windows Firefox 15” in Chromium and “Windows/Firefox 29″ in Firefox (both worked – tips on which user-agent to choose are here).

Mount Filesystem with Extended Attributes

This turned out to be the key step. You need to mount the root filesystem with user_xattr enabled (as described here.

For me this meant editing /etc/fstab as root and adding “user_xattr,” (note the comma) before “errors=remount=ro” for my root filesystem (more info here). I recommend that you be very careful about doing this, because a typo could prevent you from mounting the operating system on reboot.

After rebooting my machine, everything worked like a charm in both Chromium and Firefox.

Installing Tiny Tiny RSS on Raspbian

Just finished installing Tiny-Tiny RSS on my pi. I was able to use the automatic configuration wizard, but had to do a bit of fiddling to get it to work.

Here are the steps I followed:

1) download the latest tt-rss package: http://tt-rss.org/redmine/projects/tt-rss/wiki/InstallationNotes

2) Move, decompress, and rename it in my web server directory

sudo mv 1.1.tar.gz /var/www
sudo tar xvzf /var/www/1.1.tar.gz
sudo mv Tiny-Tiny-RSS-1.11/ tt-rss

3) In my case, I had to forward port 3306 on my cable modem to my pi server for mysql connections.

4) Install mysql and choose a root user password when prompted

sudo apt-get update; sudo apt-get install mysql-server

5) Create a database and user, then assign privileges

mysql -u root -p [enter root password at prompt]

In mysql:

create database ttrss;
create user 'ttrss'@'localhost' identified by 'whatever_password';
grant all on ttrss.* to ttrssuser;
exit

6) Connect to url: http://999.999.999.999/tt-rss [where 999.999.999.999 is my server ip address]. This will take you to the install page. Enter your settings:

Database Type: MySQL
User Name: ttrssuser
Password: whatever_password
Database: ttrss
Host: [leave blank]
Port: [leave blank]
Tiny Tiny RSS URL: [leave as default]

Click “Test Configuration”

7) As instructed by the installer, I needed to copy the configuration from the installer webpage and use an editor to paste the contentts into a new /var/www/tt-rss/config.php. At this point, the TT-rss page wouldn’t load, but it helpfully told me to change the following permissions:

sudo chmod -R 777 cache/images
sudo chmod -R 777 cache/upload
sudo chmod -R 777 cache/export
sudo chmod -R 777 cache/js
sudo chmod -R 777 feed-icons
sudo chmod -R 777 lock

8) Last but not least, I had to set up regular updates by adding the following to my crontab

*/30 * * * * /usr/bin/php /var/www/tt-rss/update.php --feeds --quiet

And that’s it!

Using Smugmug as a Photo Backup Location

I’ve finally gotten around to uploading my photo collection. I’ve decided to pay for a subscription to Smugmug.com and so far I’m happy with it. I chose not to use Flickr or other options because I wanted a site which I could use as a private one-to-one backup of my personal store of photos. I can make albums or photos public using other services or through Smugmug. I also wanted to be able to synchronize tags and metadata between the on-line store and my digikam-managed local version. If I ever lose my local copy, I should be able to download a full copy of everything in its original form. Smugmug appears to be able to do all of that (and has other good features, too).

To do the synchronization, I’m using the php uploader found here. It seems to be working, although I had some trouble at first pointing it at the right directory on my system. Running the upload program is simple:

php /path/to/multi-album-upload.php /path/to/pictures

NOTE: I initially had a little trouble because I was pointing to the wrong level in the hierarchy of my picture directory tree. The program assumes an arrangement something like this:

home
user
photo_library
category
album
image.jpg

My reading of the instructions implied that you could point to “/home/user/photo_library/category” but that doesn’t work. You need to point to “/home/user/photo_library.”

Scripts for Monkeying with Cinelerra XML

I’ve moved my Cinelerra projects from one directory path to another and scrambled a few things in the process. I’ve written a few bash scripts which helped me sort things out.

In the process, I’ve learned a bit about the xml format which Cinelerra uses to store references to files and edits. When you add a resource, it appears twice between opening and closing “ASSET” tags with the format <ASSET SRC=”/path/to/filename”>[asset details]</ASSET>. Then every time you include that file in a track of your project, the filename appears again in an “EDIT” tag complex in the format <FILE SRC=”/path/to/filename”>[edit details]</FILE>

First, I needed to check the paths for all the resources in the file to make sure they are still valid. I called this program test_xml_paths.sh. I corrected the incorrect paths with a text editor, but found that Cinelerra was still crashing with a memory error when I loaded the project.

Second, I decided to find and eliminate all of the unused assets in the file. I originally had a forty minute video in one file, but some time ago I split the project into four ten minute chunks. When I created the original project, I had added a couple of thousands of clips for possible inclusion, so now that I’m working on the final cut, I don’t need all that bloat.

I created a function to figure out which assets were being used in the edit list and which were not: list_unused_assets.sh.

Finally, I wrote a script which would recreate a new xml project file with all the redundant assets removed: remove_unused_assets.sh.

I can’t offer any warranty on these scripts, of course, but I hope they may help somebody else. If nothing else, there’s a great example of multi-line sed editing in the last one.

Remove Unused Assets from a Cinelerra XML Project File

This is a bash script to remove all the unwanted assets from a Cinelerra project XML file. I needed it to cut out bloat from a project I was working on which had hundreds of unused resources after I split the project up into ten minute chunks. Doing so stopped Cinelerra from crashing when I loaded the project: YMMV.

Save this script as “remove_unused_assets.sh”. It needs the function in “list_unused_assets.sh” which I posted here.

#! /bin/bash
#kk remove_unused_assets.sh
#kk
#kk For cinelerra xml files
#kk this script removes the contents of asset tags which are not also used for edits

if [ -z "$1" -o "$1" = "-h" ]
then
echo “Usage: remove_unused_assets [xml_filename]“
echo “For cinelerra xml files, this script removes the contents of asset tags which are not being used for edits”
exit
fi

TMPDIR=”/tmp”
SCRIPTDIR=”$HOME/Video/video_new/common/scripts”

. $SCRIPTDIR/list_unused_assets.sh
. $SCRIPTDIR/include.inc

#assign the parameter to a variable
XMLFILE=$1

#location of the temporary files
ASSETS_FILE=”$TMPDIR/ASSETS_FILE”
EDITS_FILE=”$TMPDIR/EDITS_FILE”
XMLFILE_NEW=”$1″_new
XMLFILE_TMP=”$TMPDIR/XMLFILE_TMP”

#check if the xml file exists and exit if it does not
if [ ! -e $XMLFILE ]; then error “XML file does not exist”; fi

list_unused_assets $XMLFILE
if [ $? != 0 ] ; then error “list_unused_assets failed”; fi

cp $XMLFILE $XMLFILE_TMP

echo
echo Removing Unused Assets…
echo
echo “Working…”
echo

# loop through the list of unused assets and remove the ASSET tag lines for each
# sed command from here: http://ilfilosofo.com/blog/2008/04/26/sed-multi-line-search-and-replace/

for f in $( comm -23 $ASSETS_FILE $EDITS_FILE ); do

#to make the filename string work in sed, I have to escape all of the periods and slashes
FORMATTED_f=`echo “$f” | sed -e ‘s:\/:\\\/:g’ -e ‘s:\.:\\\.:g’`

#this complicated sed script came from the sed FAQ: http://sed.sourceforge.net/sedfaq4.html#s4.21
#in order to make the variable substitution work, I had to close the sed script with a single quote
#then enclose the variable in double quotes, then re-open the sed script with a single quote
#the -i option is for editing a file “in place”
sed -i -e ‘
# sed script to delete a block if /regex/ matches inside it
:Top
/\/ { # For each line between these block markers..
/\/ASSET\>/!{ # If we are not at the /end/ marker
$!{ # nor the last line of the file,
N; # add the Next line to the pattern space
b Top
} # and branch (loop back) to the :Top label.
} # This line matches the /end/ marker.
/’”$FORMATTED_f”‘/d; # If /regex/ matches, delete the block.
} # Otherwise, the block will be printed.
#—end of script—
‘ $XMLFILE_TMP

#diff $XMLFILE $XMLFILE_TMP

echo -n ‘ \r’
echo -n $f’\r’

done

cp $XMLFILE_TMP $XMLFILE_NEW

echo
list_unused_assets $XMLFILE_NEW

echo
echo Finished.
echo
echo The new version of the xml file has been saved as $XMLFILE_NEW.
echo

This program depends on include.inc which contains the following:

#!/bin/bash

#use colour in the output just for fun
GREEN=$(printf “\033[32m”)
RED=$(printf “\033[31m”)
NC=$(printf “\033[0m”)

error()
{
echo ERROR: $1
exit 1
}

I tried to make these scripts as tidy and portable as possible — you will, however, need to change the path for $SCRIPTDIR to the location you are using (and $TMPDIR if you don’t want to put your temporary files in /tmp).

List Unused Assets in a Cinelerra XML Project File

This is a bash script which contains a function used to figure out which assets in a Cinelerra project are not required for the edit list. I created it because I had pulled in a couple of thousand media resources for a project and wanted to eliminate bloat. I moved the project and found it kept crashing on me until I eliminated the crud. I wrote another program to call this function and then actually clean out the xml. You can view “remove_unused_assets.sh” here.

Running this script doesn’t do anything until you call the enclosed function. This function can be called from the command line with:

. list_unused_assets.sh;list_unused_assets
#! /bin/bash
#kk list_unused_assets.sh
#kk
#kk check a cinelerra xml file to see which assets have been loaded, but not used in the project
#kk provide filename to check as parameter
#kk from the command line, type “. list_unused_assets.sh;list_unused_assets

list_unused_assets()
{

SCRIPTDIR=”$HOME/Video/video_new/common/scripts”
TMPDIR=”/tmp”

. $SCRIPTDIR/include.inc

#location of the temporary files
ASSETS_FILE=”$TMPDIR/ASSETS_FILE”
EDITS_FILE=”$TMPDIR/EDITS_FILE”

#assign the parameter to a variable
XMLFILE=$1

#check if the xml file exists and exit if it does not
if [ ! -e $XMLFILE ]; then error “XML file does not exist”; fi

#if the temp file exists, delete it
if [ -e $ASSETS_FILE ]; then
rm $ASSETS_FILE
fi

if [ -e $EDITS_FILE ]; then
rm $EDITS_FILE
fi

echo
echo Checking $XMLFILE for assets unused by the project…
echo

#find all of the filenames in the xml file and strip the extra junk
grep -E “ASSET SRC=” $XMLFILE | sed -e ‘s/^.*SRC=”//g’ -e ‘s/”>.*$//g’ | sort | uniq > $ASSETS_FILE
grep -E “FILE SRC=” $XMLFILE | sed -e ‘s/^.*SRC=”//g’ -e ‘s/”>.*$//g’ | sort | uniq > $EDITS_FILE

#output the counts
echo `grep -E “ASSET SRC=|FILE SRC=” $XMLFILE | sed -e ‘s/^.*SRC=”//g’ -e ‘s/”>.*$//g’ | sort | uniq | wc -l` Total Assets
echo $RED`comm -23 $ASSETS_FILE $EDITS_FILE | wc -l` Unused Assets$NC
echo $GREEN`comm -12 $ASSETS_FILE $EDITS_FILE | wc -l` Used Assets$NC

}

This program depends on include.inc which contains the following:

#!/bin/bash

#use colour in the output just for fun
GREEN=$(printf “\033[32m”)
RED=$(printf “\033[31m”)
NC=$(printf “\033[0m”)

error()
{
echo ERROR: $1
exit 1
}

I tried to make these scripts as tidy and portable as possible — you will, however, need to change the path for $SCRIPTDIR to the location you are using (and $TMPDIR if you don’t want to put your temporary files in /tmp).

Test File Paths in a Cinelerra XML File

This is a bash script to test all the filenames and paths in a Cinelerra project to make sure they are still valid.

#!/bin/bash
#kk test_xml_paths.sh
#kk
#kk check a cinelerra xml file to make sure the file references are all valid
#kk provide filename to check as parameter

if [ -z "$1" -o "$1" = "-h" ]
then
echo “Usage: test_xml_paths [xml_filename]“
echo “For cinelerra xml files, this script checks that file paths are valid”
exit
fi

SCRIPTDIR=”$HOME/Video/video_new/common/scripts”
TMPDIR=”/tmp”

. $SCRIPTDIR/include.inc

#assign the parameter to a variable
XMLFILE=$1

#check if the xml file exists and exit if it does not
if [ ! -e $XMLFILE ]; then error “XML file does not exist”; fi

#location of the temporary file
FILENAMES=”$TMPDIR/TEST_XML_PATHS”

#set counters to zero
GOODCOUNT=0
BADCOUNT=0

#if the temp file exists, delete it
if [ -e $FILENAMES ]; then
rm $FILENAMES
fi

echo
echo Checking $XMLFILE for valid file paths…
echo

#find all of the filenames in the xml file and strip the extra junk
#send the output to a file
grep -E “SRC=|PATH=” $XMLFILE | sed -e ‘s/^.*SRC=”//g’ -e ‘s/^.*PATH=”//g’ -e ‘s/”>.*$//g’ > $FILENAMES
if [ $? != 0 ] ; then error “grep failed”; fi

# loop through the output and test if the filename is either a link or a file
for f in $( cat $FILENAMES ); do
if [ ! -h $f -a ! -e $f ]; then
BADCOUNT=`expr $BADCOUNT + 1`
echo ERROR: Missing $f
else
GOODCOUNT=`expr $GOODCOUNT + 1`
fi
done

#output the counts
echo
echo $RED$BADCOUNT Total Bad Pathss $NC
echo $GREEN$GOODCOUNT Total Good Paths$NC
echo

This program depends on include.inc which contains the following:

#!/bin/bash

#use colour in the output just for fun
GREEN=$(printf “\033[32m”)
RED=$(printf “\033[31m”)
NC=$(printf “\033[0m”)

error()
{
echo ERROR: $1
exit 1
}

I tried to make these scripts as tidy and portable as possible — you will, however, need to change the path for $SCRIPTDIR to the location you are using (and $TMPDIR if you don’t want to put your temporary files in /tmp).

Reinstalling A Corrupted Ubuntu Karmic Upgrade

I have previously posted about my own Karmic upgrade problems. In my case, the system crashed in the middle of the upgrade (due to my own fault, nothing to do with the upgrade itself as far as I know), causing my half-upgraded system to be unable to mount the root filesystem and dropping me into a maintenance shell.

I’ve always tried to promote Linux use among friends and family which means I also provide support as required. Yesterday, a friend of mine brought me her laptop with a similar problem to the one I had: in her case, she thinks that she lost her internet connection in the middle of the Jaunty to Karmic upgrade. In any case, her half-upgraded system would neither boot nor allow a root prompt. The error message on the console was:

mountall: symbol lookup error: mountall: undefined symbol: udev_monitor_filter_add_match_subsystem_devtype
init: mountall main process (310) terminated with status 127

I tried a couple of things: first, I used an old Jaunty disk to boot the machine, mounted the old root filesystem and attempted to use chroot so that I could complete the upgrade.

mkdir /media/disk
mount /dev/sda5 /media/disk
chroot /media/disk

Unfortunately, this didn’t work because I had no network devices available in the chroot jail, and I couldn’t find any obvious way to sort that out. Similarly, I downloaded a new iso of Ubuntu Karmic hoping to use it as the apt source for the upgrade, but I couldn’t access the cd drive either, even using apt-cdrom to set it up. As before, a bit more effort might have sorted this out, but I decided to solve the problem by doing a complete re-install of Karmic over top of the previous one.

I backed up the /etc and /home directories to an external hard drive first to be safe. To my delight, the reinstall worked fine and didn’t clobber any of the existing user data on the drive. During the partioning step, I selected the manual option. I changed /dev/sda5 (which in this case was the old Ubuntu root filesystem) to mount as “/” but didn’t check the box for it to be formatted. In the “import settings” step, I also told the installer to import user settings from the old Ubuntu install. When I rebooted, everything was up to date, and the user settings and documents were all in place as expected. Good to know!

As a final note, I should add that when I set up this system for my friend, I configured Dropbox and told her how to use it. That meant that I could’ve easily restored everything even if I’d had to do a complete format and reinstall. On my own machine, I’ve placed a symbolic link to /etc in my Dropbox folder (it doesn’t take up much space) along with linked versions of all of my important dot files — that means that, with a little effort, I can reimage and restore at any time. I will set up all my friends the same way from now on.

Logging out with the Cairo dock

I’ve decided to take advantage of my new powerful laptop and enable a little eye candy. I’ve tweaked Compiz to my liking and enabled the Cairo dock (I used the Avant Window Navigator for a couple of days and liked it as well, but Cairo is even better). I’ve been able to set up my Cairo dock to give me everything I appreciated in the old gnome panels with one exception: the user switcher applet. Cairo gives you a shut-down menu, but no way to log out of your session or switch to another user.

I have discovered the command “gdmXnestchooser” which at least lets me switch to the face browser and log in as another user, but I’d like to be able to log out of my current session sometimes in order to refresh things. I have also enabled restarting the X server (similar results to logging out) as described here. Essentially, you must go to the preferences/keyboard/layout and enable the ctrl-alt-backspace key combination. Another tool, “gdmXnestchooser” allows you to create a nested X session, but you need to install gdm-2.20 to make it work and I don’t want to risk buggering up my current setup.

One thing I love about the combination of Compiz and Cairo is the ability to dump all of my widgets (weather, clock, conky, system tray etc.) onto the compiz widget layer. I have it configured to pop up when I move the mouse to the top edge of the screen so I can instantly see the status of everything without having to use precious screen real estate or flip back to the desktop. I have the dock on auto-hide, so I can use the full screen without sacrificing usability. It is an elegant solution to the problem of trying to use a single display for many simultaneous purposes.

UPDATE: Turns out it’s easy to log out from the Cairo dock…see comment below.

Ubuntu Karmic Koala Upgrade

I have upgraded from Jaunty to Karmic on my Dell XPS 1610 and everything seems to work great.

My upgrade was interrupted (my own fault for running another application at the same time and having it crash) and caused some serious problems for a few hours while I tried to fix it. The symptom was that the system wouldn’t boot and told me that it couldn’t read from the root filesystem. In the end I solved the problem by dropping to a shell, re-mounting the root disk so I could write to it (“mount -o remount,rw /”), then running apt-get and sorting out dependency problems with python2.6 and mono-common (I finally had to delete /var/lib/binfmts/python2.5 in order for python to install properly). This process took hours and finally I was able to perform a full “apt-get dist-upgrade” from the shell, then reboot. I still had a disk error which prevented booting, but I discovered a great tip someplace to force a disk check on the root filesystem on reboot with “tune2fs -C 30 /dev/sda5″ (or whatever device).

The experience reminded me that Linux is great for people like me, but maybe not so great for others. It isn’t that Windows doesn’t have similar problems (like I said, it was my fault for crashing the system in the middle of a major upgrade), but when it does, a less-savvy user can take the machine to the corner computer shop to get it fixed.

In spite of that, Karmic has lots of great features that make it perfect for any user. I’m still noticing small changes, but so far, so good. I like the new face browser login window and I’m thrilled with the new audio control panel, which is simpler and clearer than before with new features — for example, you can now see which active applications are using the sound server and turn them on and off with a single click.