You can see that most of the signal in this sample is contained within a few peaks. Sometimes those peaks saturate (max-out, overblow) the detector, which is bad if I am interested in comparing the heights of the peaks (a controversial subject, I should note I am only doing bulk, not individual, comparisons). Of course, I could just add less DNA and run it again, except that then I would be liable to lose some of the smaller peaks (also, it’s not practical for me to re-run these specific samples). So I’ve written a script in the open-source statistical package R to estimate the heights of the saturated peaks by fitting a Gaussian function of the form

where ‘y_0′ is the y-minimum, ‘x_0′ is the center of the peak, ‘b’ is a scaling factor, and ‘d’ is related to the standard deviation of the distribution.

You can download the script here: gaussfit.r

The figures below show (A) a fitted regular-sized peak, and (B) a fitted saturated peak. In my case, the fitted function has a maximum that is 1.6 ± 2.5% of the observed maximum for regular-sized peaks.

Briefly, the way it works is that you set up a central repository for your files, preferably on some computer that is connected to the internet all the time, even better if it has a static domain name. Into this repository you place the files you want to be version controlled. This works best for plain text files (like LaTeX source files) because it is simple to compare versions if necessary, but it will also work for binary files like OpenOffice documents (which have built-in version control, if you didn’t know) and images. If you are working on the computer with the central repository, great, just make sure to update the files each time before you work on them and commit them each time you are done. You could probably write a cron job to do this for you if you are liable to forget. If you are working on a different computer, you just have to ask the central repository for the files and it will give you the latest versions. After you edit them you have to commit the changes to a local repository, which can then be merged with the central repository.

To get started:

• Install bzr
• Make a new directory and copy the files you’ll be starting with into it. If you’re using LaTeX, you’ll probably only want the files that are not compiled, e.g. *.tex, *.bib, *.bst and NOT *.aux, *.log, etc.
• Tell bzr who you are:
  bzr whoami "me <me@me.com>"
• Initialize tracking in this directory:
  bzr init
• Add all the files in the directory, recursively:
  bzr add
• Commit the files to the first revision. Make sure to add a message (-m), or you might find yourself in ‘vi’ (shudder). Do this after every work session.
  bzr commit -m "very important message"
• Next time you work from this computer with the central repository, make sure you are using the most recent version:
  bzr update
• The first time you work on your project from a different computer, pull a new copy via ssh from that computer:
  bzr branch sftp://user@host.domain:port/location/to/remote/files /new/local/location
• In future sessions, pull new edits from the central repository before working on them:
  bzr merge sftp://user@host.domain:port/location/to/remote/files
  bzr commit -m "merge from parent"
• To submit edits to the central repository, first commit them locally, then push edits via ssh to that computer:
  bzr commit -m "local changes"
  bzr push sftp://user@host.domain:port/location/to/remote/files

Some further usage, much of which is intuitive because it works just like regular unix commands:

• Clean up a directory by deleting non-versioned files and reloading the last commit:
  bzr commit
   rm *
  bzr revert
• Make a new versioned subdirectory:
  bzr mkdir ./newdirectory
• Add a new file to the repository:
  bzr add ./newfile.txt
• Remove a file from the repository:
  bzr rm ./oldfile.txt
• Rename or move a file or directory in the repository:
  bzr mv oldfile.txt newfile.txt

and installed the packages arb (base installation), transfig (something to do with LaTeX), xfig (for exporting trees), and libmotif3 (for library libXm.so.3)

More info here: http://techno.mikro.biologie.tu-muenchen.de/

1. get sequences into ARB, via e.g. greengenes or SILVA or RDP. If importing an alignment in FASTA format you may need to use the following IFT (save with rest of arb .ift files, e.g. usr/arb/lib/import/fasta_wgap.ift):
   AUTODETECT      ">*"
           #Global settings:
   KEYWIDTH        1
   BEGIN   ">??*"
   MATCH   ">*"
           SRT "* *=*1:*\t*=*1"
           WRITE "name"
   MATCH   ">*"
           SRT "*|*=*1"
           WRITE "full_name"
   SEQUENCEAFTER   "*"
   SEQUENCESRT     ""
   SEQUENCECOLUMN  0
   SEQUENCEEND     ">*"
   # DONT_GEN_NAMES
   CREATE_ACC_FROM_SEQUENCE
   END     "//"
2. in ARB: prune tree to closest isolates, closest clones/from environment, and enough internal branches for context. make a copy for each tree you work on.
3. in ARB: mark all in tree, look at alignment, unselect those not full length/long enough, note start and end of full length section by position.
4. in ARB: export sequence to fasta (File–>Export–>Export to foreign format). You may want to use a simple export file (EFT) like the following (save it with the others e.g. usr/arb/lib/import/fasta_simple.eft or wherever) :
   SUFFIX          fasta
   BEGIN
   >*(name)
   *(|export_sequence)

   During the export use a hypervariable SAI (made by parsimony) to filter by quality (Lane 1991 mask) “-=.0123456″ and region to export, using the starting and ending positions you determined previously

5. in SEAVIEW, open exported FASTA file and save as NEXUS file (the export filter from ARB to PAUP doesn’t seem to work very well)
6. PAUP may get grouchy if you have digit-only sequence names, so you can run the following script to temporarily change the names.
   s/'\([0-9]\{4,\}\)'/'\1tmpname'/g

   Save the script as ‘fromdigits.sed’ (it’s easier as a file because the apostrophes in the regular expression complicate things on the command line) and run

   sed -f fromdigits.sed [filename.nexus] > [filename.converted.nexus]

   After running PAUP you can change the names back in the PHYLIP tree files by using the following sed script:

   sed -e 's/\([0-9]\{4,\}\)tmp[name]*/\1/g' [treefile.phy] > [treefile.converted.phy]

   Or, from NEXUS tree files:

   sed -e 's/\([0-9]\{4,\}\)tmp[name]*/\1/g' [treefile.nexus] > [treefile.converted.nexus]
7. in a text editor, open NEXUS file and append PAUP block to end, save as new file. for bootstrapping, the PAUP block may come from MODELTEST (NOTE bug in PAUP 4.10b means you have to add an extra command to MODELTEST input block: “default lscores longfmt=yes”). for ratcheting the PAUP block may come from PRAP2
8. run in PAUP to get parsimony/likelihood trees, save trees in phylip format. if you only have a NEXUS tree, open it in a texteditor and delete the introductory NEXUS block. keep only what is between the outermost parentheses.
9. in ARB: input consensus tree with bootstraps back into arb using Tree–>TreeAdmin–>Import. make sure to remove the period (.) from the tree name or it won’t import.
10. in ARB: save distance matrix to ‘infile’ by doing Tree–>Build Tree–>Distance Methods–>Phylip Distance Matrix, then use the same filter as above for trees. Hit ‘y’ and try to save the file that opens up as ‘/tmp/infile’. If no file pops up, make sure you’ve installed ‘xedit’ or make a soft link from e.g. /usr/bin/xedit to some other text editor e.g. gedit, kedit. If you can’t save the file anywhere, copy it from the ~/.arb_tmp directory in your home folder.
11. in ARB: save tree to /tmp/arbtree (Tree–>Tree Admin–>Export).
12. here is a script that does the following 3 steps:
    #
    # Script file for automating the process of adding distances to likelihood tree
    # REC
    
    # turns tree from arb ('arbtree') and distance matrix from dnadist ('distmatrix') into arb tree plus distances ('arbtree.fitch.outtree')
    
    filedate=`date +%s`
    
    mkdir tmp$filedate
    cd tmp$filedate
    
    # clean up tree file
    # the sed '1d' removes the notes line -- if you have zero or more than one you'll need to change this or do it manually
    # the tr -d [:space:] gets rid of all the whitespace
    sed '1d' < ./../arbtree | tr -d [:space:] > intree
    
    # unroot tree file
    (echo y; echo w; echo u; echo q)|(retree)
    mv intree arbtree.retree.intree
    mv outtree intree
    
    # run fitch to add the distances to the tree we've supplied
    cp ./../distmatrix infile
    (echo d; echo u; echo -; echo y)|(fitch)
    mv intree arbtree.fitch.intree
    mv infile arbtree.fitch.infile
    mv outtree arbtree.fitch.outtree
    mv outfile arbtree.fitch.outfile
    cd ..

to do it manually: in a text editor: delete first line and despace tree for phylip, or do the following on the command line:

sed '1d' < arbtree | tr -d [:space:] > intree

with PHYLIP: run retree –> write unrooted tree, then mv outtree intree

with PHYLIP: run fitch, choose options D –> minimum evolution, U –> input file, – –> no negative lengths

13. in ARB: import outtree with Tree–>Tree Admin–>Import
14. get NDS info organized and displayed correctly, e.g.
    • –to rename accession numbers to accession numbers from greengenes
      copy full_name to acc
      :*=*(full_name)

      then get rid of extraneous info:

      /[0-9][0-9]* [A-Z][A-Z]*[0-9]*\.*[0-9]* //
      /\..*//
    • –to rename full_name by sequence info
      copy full_name to tmp and to backup (just in case)
      :*=*(full_name)

      then get rid of extraneous info:

      /[0-9][0-9]* [A-Z][A-Z]*[0-9]*\.*[0-9]* //
    • then copy tmp to full_name
      :*=*(tmp)
15. in ARB: export to XFIG (Tree–>Export to xfig) with no handles, no colors, full tree
16. in XFIG: immediately export to SVG
17. in INKSCAPE: open svg tree file and edit as necessary
18. in INKSCAPE: resize page (File –> document properties –> fit page to selection), resize selection to maximize page, export bitmap 300dpi png
]]> http://staff.washington.edu/rec3141/wordpress/archives/410/feed 0 http://staff.washington.edu/rec3141/wordpress/archives/386 http://staff.washington.edu/rec3141/wordpress/archives/386#comments Tue, 06 Jan 2009 23:35:05 +0000 eric http://staff.washington.edu/rec3141/wordpress/archives/386 Trying to get a Parsimony (and likelihood) Ratchet protocol going I spent a couple hours trying to get PAUPRat working, including compiling an old compiler and finding old versions of libraries and editing Makefiles… then I found a newer Java program that does the same thing: PRAP2. The program includes the Likelihood ratchet protocol described above but not the Parsimony. Here are the settings for the Parsimony ratchet protocol, as described by Sikes et al., which are then replicated by PRAP2 into PAUP* blocks.
[!* startcmd: * * * * * * * * * * * * *  *]  
[!* ----- Parsimony Ratchet  -----*]
[!*       as in PAUPrat               *]                                                                                  
[!*           January, 2009          *]                                                                                  
Time;
Log File=paupratchet.log;                                                                                                    
Set Increase=auto;
hs status=no nrep=1 swap=tbr start=stepwise addseq=random nchuck=1 chuckscore=1;
savetrees file=mydata.tre replace;
savetrees file=mydata.tmp replace;
[!* paupcmd: * * * * * * * * * * * * *  *]
pset mstaxa=uncertain;
hsearch status=no start=1 swap=tbr multrees=no;
[!* rewtdcmd:* * * * * * * * * * * * *  *]  
[!* normcmd:* * * * * * * * * * * * *  *]
savetrees file=mydata.tmp replace;
gettrees file=mydata.tre mode=7;
savetrees file=mydata.tre replace;
gettrees file=mydata.tmp mode=3 warntree=no;
Time;
[!* stopcmd: * * * * * * * * * * * * *  *]
gettrees file=mydata.tre mode=3;
pscores all;
Time;                                
Log Stop;                            
[!* * * * * * * * * * * * * * * * *]
[!* -- THIS SEARCH IS COMPLETE -- *]
[!*  A LOG FILE HAS BEEN WRITTEN  *]
[!* AND ALL TREES HAVE BEEN SAVED *]
[!*    IT IS OKAY TO QUIT PAUP    *]
[!* * * * * * * * * * * * * * * * *]
Quit;

Update: PRAP2 has a built-in parsimony ratchet that is more cleanly written than this one, except I had to remove the command ‘dropmode=execute’ because PAUP* 4b10 wasn’t recognizing it. Using the pre-configured ratchet you can choose the number of runs to code automatically, but using a custom script you can’t. So here’s a bash shell script to automate the script above, after generating some number (here N=20) script files named ’sequences.[1-N].nexus’.

#!/bin/bash
for i in `jot 20 1 20`;
do
 `paup sequences.$i.nexus`
 `mv -f mydata.tmp mydata.$i.tmp`
 `mv -f mydata.tre mydata.$i.tre`
 `mv -f paupratchet.log paupratchet.$i.log`
done

Since the parsimony ratchets run quickly it is fine to do them all on one processor, but I’m starting the likelihood runs separately on each processor. This script will run them one at a time, but if you add ’screen -S like.$i’ to the line before ‘/bin/paup…’ it should start up to 10 simultaneously in separate “screens“, assuming your nexus files are all in the same directory and named ratchet.like.[1-10].nexus

#!/bin/bash
for i in `jot 10 1 10`;
do
 `mkdir like$i`
 `cp ratchet.like.$i.nexus ./like$i/`
  cd like$i
 `/bin/paup4b10-ppc-macosx -n ./ratchet.like.$i.nexus`
 cd ..
done
]]> http://staff.washington.edu/rec3141/wordpress/archives/386/feed 0 http://staff.washington.edu/rec3141/wordpress/archives/290 http://staff.washington.edu/rec3141/wordpress/archives/290#comments Mon, 03 Nov 2008 02:48:28 +0000 eric http://staff.washington.edu/rec3141/?p=290 I just bought an aluminum macbook5,1 and successfully installed kubuntu 8.10 intrepid ibex onto it.

Mostly, I used this wiki page: https://help.ubuntu.com/community/MacBook Aluminum
and this forum thread: http://ubuntuforums.org/showthread.php?t=947947

UPDATE 2009-01-08: I’ve got the macbook back, now with it’s 3rd logic board and 2nd magsafe board. The tech said he thinks there was a ‘bad batch’ of logic boards in the late summer and fall that were ‘thinner’ than usual. One guy got 4 boards replaced before he got one that worked. How about some Quality Control, Apple?!

UPDATE 2009-01-07: The honeymoon is over. My MacBook has been in the Apple Hospital for the better part of 3 weeks. First, the lcd screen dissolved to white for no apparent reason and would not display anything but white although I could hear things booting up in the background. After a couple of attempts at fixing it by resetting the NVRAM and SMC it wouldn’t boot up at all. I took it in and they replaced the Logic Board. Then they tried to blame it on rEFIt, something about how it was a software package that overwrote the EFI and therefore invalidated the warranty and it would cost $1000 to replace the board blah blah. I think the manager knew he was full of it and ended up replacing it for free (rEFIt apparently does not invalidate the warranty but you might want to remove it anyway — I’d rather not have to hire a lawyer to prove it). Then, less than a week later, the battery slowly stopped charging. I thought it was the power adapter because the display model adapter at the Apple store worked (green light came on) so I got a replacement but it didn’t work. I had to take it back in and they still have it. Now they’re replacing the magsafe board.


First in OSX I partitioned the harddrive using bootcamp into a 50Gb OSX partition and the rest for ‘windows’.

Next, I installed refit v0.12 (http://refit.sourceforge.net/) in OSX. [I later edited refit.conf to boot linux first by uncomment the legacyboot line.]

Then I burned an ISO of the kubuntu 8.10 livecd and restarted the computer, holding ‘c’ at the white screen

Then I installed kubuntu. At the partition step, I went to the ‘manual’ screen and deleted the FAT32 partition created by bootcamp and instead made a 103Gb ext3 partition (mounted to /) and a ~4Gb swap partition. At the confirmation screen, I clicked the ‘Advanced’ button and unchecked the box for where to install grub because I wasn’t sure what to put there. [Probably the answer was (hd0,2)].

After installation, I went to konsole and ran grub manually. This step could also be done by booting into the livecd, and may in fact require first syncing the partition tables using the refit Partitioning Tool.

sudo grub
find /boot/grub/stage1

I got (hd0,2), then typed

root (hd0,2)
setup (hd0,2)
exit

After this, I shut down the computer and turned it back on. [rebooting from kubuntu doesn't seem to go well, it either goes to a black screen or starts making this hideous beeping sound].

At the refit menu, I made sure the partition tables were synced using the Partitioning Tool.

After selecting the linux option from the refit menu it went to a white screen with Tux and sat there… and sat there. So I hard-rebooted and tried it again with the same result. Then I went investigating and found that no menu.lst had been installed in /boot/grub/ so I made my own. I probably should not have done this. I probably should have kept cycling the power until it worked, that seems to be what other people did. But anyway it worked eventually and here is my /boot/grub/menu.lst:

# Boot automatically after 3 secs.
timeout 3
# By default, boot the first entry.
default 0
# For booting GNU/Linux
title  GNU/Linux
root (hd0,2)
kernel /vmlinuz root=/dev/sda3
initrd /initrd.img

After that, once I was able to boot reliably into Kubuntu, I started messing with the accessories. In particular, the trackpad is terrible to start off and manageable after a few tweaks. After performing the following actions I can scroll with two fingers and right-click with a two-finger click, but the movement can still be a bit jerky. The main problem is that there is no dedicated button, it has been integrated into the trackpad, so I cannot simultaneously click something with the thumb and move it with a finger. Instead, I have to click and drag with the same finger, which is pretty annoying for dragging files or resizing windows or selecting text. It also means I can’t rest my thumb on the trackpad at all, which goes all the way to the bottom of the laptop, so it just has to hover there. If it touches the trackpad then the pointer won’t move. I tried setting up the palm detection options in the file below but that turned into a disaster so I gave up. It also means I can’t have my pointer finger on the trackpad when I click, so there’s a constant rocking back and forth between finger and thumb which is annoying.

So first I added the PPA repository to /etc/apt/sources.list

deb http://ppa.launchpad.net/mactel-support/ubuntu intrepid main

then installed some apple-specific modules:

sudo aptitude install hal-applesmc applesmc-dkms bcm5974-dkms

Only the first two are listed in the wiki page but I think I needed the third as well, which also provides access to the hardware function keys, few of which seem to work so far (F7-9 are back/play/forward; F10 mutes; the rest don’t work). This might also be a good place to mention that if you install bcm5974-dkms the function keys stop working as normal and to access them you have to hit fn+F-key, e.g. to go to application quick-launcher you would have to hit alt+fn+F2 instead of just alt+F1, although I ended up changing this key-combination to command+escape in KDE system settings anyway. I also figured out that you can do a real delete by using fn+delete.

Anyway, once those are installed I edited /etc/modules to include ‘applesmc’ on a new line.

Then I started working with the synaptics touchpad options.

sudo cp /usr/share/hal/fdi/policy/20thirdparty/11-x11-synaptics.fdi /etc/hal/fdi/policy/
sudo pico /etc/hal/fdi/policy/20thirdparty/11-x11-synaptics.fdi

I added the following line which seems to be important though I’m not well enough informed to know why.

&lt;merge key="input.x11_options.SHMConfig" type="string"&gt;True&lt;/merge&gt;

As I mentioned before, other edits I made changed the behaviour of the touchpad unpredictably. Some reminders: to scroll horizontally or vertically just drag two fingers either direction. To right-click, click with two fingers on the touchpad. To paste from the clipboard (middle-click) click with three fingers on the touchpad.

For the video driver, the program that popped up on booting Kubuntu refused to install the nvidia driver so I did it manually:

sudo apt-get install nvidia-glx-177

Sound: I added the following line to my /etc/modprobe.d/options file but I still have no sound through the speakers (only headphone jack).

options snd_hda_intel model=mbp3

Wireless works.

update: to change the brightness, and have it go back to that brightness on re-suspend:

sudo aptitude install mbp-nvidia-bl-dkms
sudo modprobe mbp-nvidia-bl-dkms

then the brightness can be changed by:

echo 1 | sudo tee /sys/class/backlight/mbp_backlight/brightness
]]> http://staff.washington.edu/rec3141/wordpress/archives/290/feed 0 http://staff.washington.edu/rec3141/wordpress/archives/206 http://staff.washington.edu/rec3141/wordpress/archives/206#comments Wed, 10 Sep 2008 19:48:21 +0000 eric http://staff.washington.edu/rec3141/?p=206 5.2.5 (thanks to steve at sixthform for [...]]]> Using WordPress 2.6 and wp-latexrender and an installer. Success after jiggling with the paths and editing class.latexrender.php lines 197–224 to replace each instance of ‘\f’ with ‘\\f’.

For example, line 203 had to be changed from

$string .= "\\newsavebox{\formulabox}\n";

to

$string .= "\\newsavebox{\\formulabox}\n";

This change is apparently only necessary in PHP versions >5.2.5 (thanks to steve at sixthform for figuring that out).



ta da!

]]> http://staff.washington.edu/rec3141/wordpress/archives/206/feed 0 http://staff.washington.edu/rec3141/wordpress/archives/167 http://staff.washington.edu/rec3141/wordpress/archives/167#comments Fri, 15 Aug 2008 23:58:42 +0000 eric http://staff.washington.edu/rec3141/wordpress/archives/167
watch -n 1 ‘dmesg | tail’

watches a file (or other input) and outputs it every ‘1′ second

]]> http://staff.washington.edu/rec3141/wordpress/archives/167/feed 0