© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program uses the compatibility method for unrooted two-state characters to obtain the largest cliques of characters and the trees which they suggest. This approach originated in the work of Le Quesne (1969), though the algorithms were not precisely specified until the later work of Estabrook, Johnson, and McMorris (1976a, 1976b). These authors proved the theorem that a group of two-state characters which were pairwise compatible would be jointly compatible. This program uses an algorithm inspired by the Kent Fiala - George Estabrook program CLINCH, though closer in detail to the algorithm of Bron and Kerbosch (1973). I am indebted to Kent Fiala for pointing out that paper to me, and to David Penny for decribing to me his branch-and-bound approach to finding the largest cliques, from which I have also borrowed. I am particularly grateful to Kent Fiala for catching a bug in versions 2.0 and 2.1 which resulted in those versions failing to find all of the cliques which they should. The program computes a compatibility matrix for the characters, then uses a recursive procedure to examine all possible cliques of characters.
After one pass through all possible cliques, the program knows the size of the largest clique, and during a second pass it prints out the cliques of the right size. It also, along with each clique, prints out the tree suggested by that clique.
Input to the algorithm is standard, but the "?", "P", and "B" states are not allowed. This is a serious limitation of this program. If you want to find large cliques in data that has "?" states, I recommend that you use MIX instead with the T (Threshold) option and the value of the threshold set to 2.0. The theory underlying this is given in my paper on character weighting (Felsenstein, 1981b).
The options are chosen from a menu, which looks like this:
Largest clique program, version 3.69 Settings for this run: A Use ancestral states in input file? No F Use factors information? No W Sites weighted? No C Specify minimum clique size? No O Outgroup root? No, use as outgroup species 1 M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out compatibility matrix No 4 Print out tree Yes 5 Write out trees onto tree file? Yes Y to accept these or type the letter for one to change |
The A (Ancestors), F (Factors), O (Outgroup) ,M (Multiple Data Sets), and W (Weights) options are the usual ones, described in the main documentation file.
If you use option A (Ancestors) you should also choose it in the menu. The compatibility matrix calculation in effect assumes if the Ancestors option is invoked that there is in the data another species that has all the ancestral states. This changes the compatibility patterns in the proper way. The Ancestors option also requires information on the ancestral states of each character to be in the input file.
The O (Outgroup) option will take effect only if the tree is not rooted by the Ancestral States option.
The C (Clique Size) option indicates that you wish to specify a minimum clique size and print out all cliques (and their associated trees) greater than or equal to that size. The program prompts you for the minimum clique size.
Note that this allows you to list all cliques (each with its tree) by simply setting the minimum clique size to 1. If you do one run and find that the largest clique has 23 characters, you can do another run with the minimum clique size set at 18, thus listing all cliques within 5 characters of the largest one.
Output involves a compatibility matrix (using the symbols "." and "1") and the cliques and trees.
If you have used the F option there will be two lists of characters for each clique, one the original multistate characters and the other the binary characters. It is the latter that are shown on the tree. When the F option is not used the output and the cliques reflect only the binary characters.
The trees produced have it indicated on each branch the points at which derived character states arise in the characters that define the clique. There is a legend above the tree showing which binary character is involved. Of course if the tree is unrooted you can read the changes as going in either direction.
The program runs very quickly but if the maximum number of characters is large it will need a good deal of storage, since the compatibility matrix requires ActualChars x ActualChars boolean variables, where ActualChars is the number of characters (in the case of the factors option, the total number of true multistate characters).
Basically the following assumptions are made:
The assumptions of compatibility methods have been treated in several of my papers (1978b, 1979, 1981b, 1988b), especially the 1981 paper. For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
A constant available for alteration at the beginning of the program is the form width, "FormWide", which you may want to change to make it as large as possible consistent with the page width available on your output device, so as to avoid the output of cliques and of trees getting wrapped around unnecessarily.
5 6
Alpha 110110
Beta 110000
Gamma 100110
Delta 001001
Epsilon 001110
|
Largest clique program, version 3.69
5 species, 6 characters
Species Character states
------- --------- ------
Alpha 11011 0
Beta 11000 0
Gamma 10011 0
Delta 00100 1
Epsilon 00111 0
Character Compatibility Matrix (1 if compatible)
--------- ------------- ------ -- -- -----------
111..1
111..1
111..1
...111
...111
111111
Largest Cliques
------- -------
Characters: ( 1 2 3 6)
Tree and characters:
2 1 3 6
0 0 1 1
+1-Delta
+0--1-+
+--0-+ +--Epsilon
! !
! +--------Gamma
!
+-------------Alpha
!
+-------------Beta
remember: this is an unrooted tree!
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
Consense reads a file of computer-readable trees and prints out (and may also write out onto a file) a consensus tree. At the moment it carries out a family of consensus tree methods called the Ml methods (Margush and McMorris, 1981). These include strict consensus and majority rule consensus. Basically the consensus tree consists of monophyletic groups that occur as often as possible in the data. If a group occurs in more than a fraction l of all the input trees it will definitely appear in the consensus tree.
The tree printed out has at each fork a number indicating how many times the group which consists of the species to the right of (descended from) the fork occurred. Thus if we read in 15 trees and find that a fork has the number 15, that group occurred in all of the trees. The strict consensus tree consists of all groups that occurred 100% of the time, the rest of the resolution being ignored. The tree printed out here includes groups down to 50%, and below it until the tree is fully resolved.
The majority rule consensus tree consists of all groups that occur more than 50% of the time. Any other percentage level between 50% and 100% can also be used, and that is why the program in effect carries out a family of methods. You have to decide on the percentage level, figure out for yourself what number of occurrences that would be (e.g. 15 in the above case for 100%), and resolutely ignore any group below that number. Do not use numbers at or below 50%, because some groups occurring (say) 35% of the time will not be shown on the tree. The collection of all groups that occur 35% or more of the time may include two groups that are mutually self contradictory and cannot appear in the same tree. In this program, as the default method I have included groups that occur less than 50% of the time, working downwards in their frequency of occurrence, as long as they continue to resolve the tree and do not contradict more frequent groups. In this respect the method is similar to the Nelson consensus method (Nelson, 1979) as explicated by Page (1989) although it is not identical to it.
The program can also carry out Strict consensus, Majority Rule consensus without the extension which adds groups until the tree is fully resolved, and other members of the Ml family, where the user supplied the fraction of times the group must appear in the input trees to be included in the consensus tree. For the moment the program cannot carry out any other consensus tree method, such as Adams consensus (Adams, 1972, 1986) or methods based on quadruples of species (Estabrook, McMorris, and Meacham, 1985).
Input is a tree file (called intree) which contains a series of trees in the Newick standard form -- the form used when many of the programs in this package write out tree files. Each tree starts on a new line. Each tree can have a weight, which is a real number and is located in comment brackets "[" and "]" just before the final ";" which ends the description of the tree. When the input trees have weights (like [0.01000]) then the total number of trees will be the total of those weights, which is often a number like 1.00. When the a tree doesn't have a weight it will be assigned a weight of 1. This means that when we have tied trees (as from a parsimony program) three alternative tied trees will be counted as if each was 1/3 of a tree.
Note that this program can correctly read trees whether or not they are bifurcating: in fact they can be multifurcating at any level in the tree.
The options are selected from a menu, which looks like this:
Consensus tree program, version 3.69 Settings for this run: C Consensus type (MRe, strict, MR, Ml): Majority rule (extended) O Outgroup root: No, use as outgroup species 1 R Trees to be treated as Rooted: No T Terminal type (IBM PC, ANSI, none): ANSI 1 Print out the sets of species: Yes 2 Print indications of progress of run: Yes 3 Print out tree: Yes 4 Write out trees onto tree file: Yes Are these settings correct? (type Y or the letter for one to change) |
Option C (Consensus method) selects which of four methods the program uses. The program defaults to using the extended Majority Rule method. Each time the C option is chosen the program moves on to another method, the others being in order Strict, Majority Rule, and Ml. Here are descriptions of the methods. In each case the fraction of times a set appears among the input trees is counted by weighting by the weights of the trees (the numbers like [0.6000] that appear at the ends of trees in some cases).
Option R (Rooted) toggles between the default assumption that the input trees are unrooted trees and the selection that specifies that the tree is to be treated as a rooted tree and not re-rooted. Otherwise the tree will be treated as outgroup-rooted and will be re-rooted automatically at the first species encountered on the first tree (or at a species designated by the Outgroup option).
Option O is the usual Outgroup rooting option. It is in effect only if the Rooted option selection is not in effect. The trees will be re-rooted with a species of your choosing. You will be asked for the number of the species that is to be the outgroup. If we want to outgroup-root the tree on the line leading to a species which appears as the third species (counting left-to-right) in the first computer-readable tree in the input file, we would invoke select menu option O and specify species 3.
Output is a list of the species (in the order in which they appear in the first tree, which is the numerical order used in the program), a list of the subsets that appear in the consensus tree, a list of those that appeared in one or another of the individual trees but did not occur frequently enough to get into the consensus tree, followed by a diagram showing the consensus tree. The lists of subsets consists of a row of symbols, each either "." or "*". The species that are in the set are marked by "*". Every ten species there is a blank, to help you keep track of the alignment of columns. The order of symbols corresponds to the order of species in the species list. Thus a set that consisted of the second, seventh, and eighth out of 13 species would be represented by:
.*....**.. ...
Note that if the trees are unrooted the final tree will have one group, consisting of every species except the Outgroup (which by default is the first species encountered on the first tree), which always appears. It will not be listed in either of the lists of sets, but it will be shown in the final tree as occurring all of the time. This is hardly surprising: in telling the program that this species is the outgroup we have specified that the set consisting of all of the others is always a monophyletic set. So this is not to be taken as interesting information, despite its dramatic appearance.
Option 2 in the menu gives you the option of turning off the writing of these sets into the output file. This may be useful if you are primarily interested in getting the tree file.
Option 3 is the usual tree file option. If this is on (it is by default) then the final tree will be written onto an output tree file (whose default name is "outtree").
Note that the lengths on the tree on the output tree file are not branch lengths but the number of times that each group appeared in the input trees. This number is the sum of the weights of the trees in which it appeared, so that if there are 11 trees, ten of them having weight 0.1 and one weight 1.0, a group that appeared in the last tree and in 6 others would be shown as appearing 1.6 times and its branch length will be 1.6. This means that if you take the consensus tree from the output tree file and try to draw it, the branch lengths will be strange. I am often asked how to put the correct branch lengths on these (this is one of our Frequently Asked Questions).
There is no simple answer to this. It depends on what "correct" means. For example, if you have a group of species that shows up in 80% of the trees, and the branch leading to that group has average length 0.1 among that 80%, is the "correct" length 0.1? Or is it (0.80 x 0.1)? There is no simple answer.
However, if you want to take the consensus tree as an estimate of the true tree (rather than as an indicator of the conflicts among trees) you may be able to use the User Tree (option U) mode of the phylogeny program that you used, and use it to put branch lengths on that tree. Thus, if you used Dnaml, you can take the consensus tree, make sure it is an unrooted tree, and feed that to Dnaml using the original data set (before bootstrapping) and Dnaml's option U. As Dnaml wants an unrooted tree, you may have to use Retree to make the tree unrooted (using the W option of Retree and choosing the unrooted option within it). Of course you will also want to change the tree file name from "outree" to "intree".
If you used a phylogeny program that does not infer branch lengths, you might want to use a different one (such as Fitch or Dnaml) to infer the branch lengths, again making sure the tree is unrooted, if the program needs that.
The program uses the consensus tree algorithm originally designed for the bootstrap programs. It is quite fast, and execution time is unlikely to be limiting for you (assembling the input file will be much more of a limiting step). In the future, if possible, more consensus tree methods will be incorporated (although the current methods are the ones needed for the component analysis of bootstrap estimates of phylogenies, and in other respects I also think that the present ones are among the best).
TEST SET OF INPUT TREES
(A,(B,(H,(D,(J,(((G,E),(F,I)),C)))))); (A,(B,(D,((J,H),(((G,E),(F,I)),C))))); (A,(B,(D,(H,(J,(((G,E),(F,I)),C)))))); (A,(B,(E,(G,((F,I),((J,(H,D)),C)))))); (A,(B,(E,(G,((F,I),(((J,H),D),C)))))); (A,(B,(E,((F,I),(G,((J,(H,D)),C)))))); (A,(B,(E,((F,I),(G,(((J,H),D),C)))))); (A,(B,(E,((G,(F,I)),((J,(H,D)),C))))); (A,(B,(E,((G,(F,I)),(((J,H),D),C))))); |
Consensus tree program, version 3.69
Species in order:
1. A
2. B
3. H
4. D
5. J
6. G
7. E
8. F
9. I
10. C
Sets included in the consensus tree
Set (species in order) How many times out of 9.00
.......**. 9.00
..******** 9.00
..****.*** 6.00
..***..... 6.00
..***....* 6.00
..*.*..... 4.00
..***..*** 2.00
Sets NOT included in consensus tree:
Set (species in order) How many times out of 9.00
.....**... 3.00
.....***** 3.00
..**...... 3.00
.....****. 3.00
..****...* 2.00
.....*.**. 2.00
..*.****** 2.00
....****** 2.00
...******* 1.00
Extended majority rule consensus tree
CONSENSUS TREE:
the numbers on the branches indicate the number
of times the partition of the species into the two sets
which are separated by that branch occurred
among the trees, out of 9.00 trees
+-----------------------C
|
+--6.00-| +-------H
| | +--4.00-|
| +--6.00-| +-------J
+--2.00-| |
| | +---------------D
| |
+--6.00-| | +-------F
| | +------------------9.00-|
| | +-------I
+--9.00-| |
| | +---------------------------------------G
+-------| |
| | +-----------------------------------------------E
| |
| +-------------------------------------------------------B
|
+---------------------------------------------------------------A
remember: this is an unrooted tree!
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
The programs in this group use gene frequencies and quantitative character values. One (Contml) constructs maximum likelihood estimates of the phylogeny, another (Gendist) computes genetic distances for use in the distance matrix programs, and the third (Contrast) examines correlation of traits as they evolve along a given phylogeny.
When the gene frequencies data are used in Contml or Gendist, this involves the following assumptions:
How these assumptions affect the methods will be seen in my papers on inference of phylogenies from gene frequency and continuous character data (Felsenstein, 1973b, 1981c, 1985c).
The input formats are fairly similar to the discrete-character programs, but with one difference. When Contml is used in the gene-frequency mode (its usual, default mode), or when Gendist is used, the first line contains the number of species (or populations) and the number of loci and the options information. There then follows a line which gives the numbers of alleles at each locus, in order. This must be the full number of alleles, not the number of alleles which will be input: i. e. for a two-allele locus the number should be 2, not 1. There then follow the species (population) data, each species beginning on a new line. The first 10 characters are taken as the name, and thereafter the values of the individual characters are read free-format, preceded and separated by blanks. They can go to a new line if desired, though of course not in the middle of a number. Missing data is not allowed - an important limitation. In the default configuration, for each locus, the numbers should be the frequencies of all but one allele. The menu option A (All) signals that the frequencies of all alleles are provided in the input data -- the program will then automatically ignore the last of them. So without the A option, for a three-allele locus there should be two numbers, the frequencies of two of the alleles (and of course it must always be the same two!). Here is a typical data set without the A option:
5 3
2 3 2
Alpha 0.90 0.80 0.10 0.56
Beta 0.72 0.54 0.30 0.20
Gamma 0.38 0.10 0.05 0.98
Delta 0.42 0.40 0.43 0.97
Epsilon 0.10 0.30 0.70 0.62
|
whereas here is what it would have to look like if the A option were invoked:
5 3
2 3 2
Alpha 0.90 0.10 0.80 0.10 0.10 0.56 0.44
Beta 0.72 0.28 0.54 0.30 0.16 0.20 0.80
Gamma 0.38 0.62 0.10 0.05 0.85 0.98 0.02
Delta 0.42 0.58 0.40 0.43 0.17 0.97 0.03
Epsilon 0.10 0.90 0.30 0.70 0.00 0.62 0.38
|
The first line has the number of species (or populations) and the number of loci. The second line has the number of alleles for each of the 3 loci. The species lines have names (filled out to 10 characters with blanks) followed by the gene frequencies of the 2 alleles for the first locus, the 3 alleles for the second locus, and the 2 alleles for the third locus. You can start a new line after any of these allele frequencies, and continue to give the frequencies on that line (without repeating the species name).
If all alleles of a locus are given, it is important to have them add up to 1. Roundoff of the frequencies may cause the program to conclude that the numbers do not sum to 1, and stop with an error message.
While many compilers may be more tolerant, it is probably wise to make sure that each number, including the first, is preceded by a blank, and that there are digits both preceding and following any decimal points.
Contml and Contrast also treat quantitative characters (the continuous-characters mode in Contml, which is option C). It is assumed that each character is evolving according to a Brownian motion model, at the same rate, and independently. In reality it is almost always impossible to guarantee this. The issue is discussed at length in my review article in Annual Review of Ecology and Systematics (Felsenstein, 1988a), where I point out the difficulty of transforming the characters so that they are not only genetically independent but have independent selection acting on them. If you are going to use Contml to model evolution of continuous characters, then you should at least make some attempt to remove genetic correlations between the characters (usually all one can do is remove phenotypic correlations by transforming the characters so that there is no within-population covariance and so that the within-population variances of the characters are equal -- this is equivalent to using Canonical Variates). However, this will only guarantee that one has removed phenotypic covariances between characters. Genetic covariances could only be removed by knowing the coheritabilities of the characters, which would require genetic experiments, and selective covariances (covariances due to covariation of selection pressures) would require knowledge of the sources and extent of selection pressure in all variables.
Contrast is a program designed to infer, for a given phylogeny that is provided to the program, the covariation between characters in a data set. Thus we have a program in this set that allows us to take information about the covariation and rates of evolution of characters and make an estimate of the phylogeny (Contml), and a program that takes an estimate of the phylogeny and infers the variances and covariances of the character changes. But we have no program that infers both the phylogenies and the character covariation from the same data set.
In the quantitative characters mode, a typical small data set would be:
5 6
Alpha 0.345 0.467 1.213 2.2 -1.2 1.0
Beta 0.457 0.444 1.1 1.987 -0.2 2.678
Gamma 0.6 0.12 0.97 2.3 -0.11 1.54
Delta 0.68 0.203 0.888 2.0 1.67
Epsilon 0.297 0.22 0.90 1.9 1.74
|
Note that in the latter case, there is no line giving the numbers of alleles at each locus. In this latter case no square-root transformation of the coordinates is done: each is assumed to give directly the position on the Brownian motion scale.
For further discussion of options and modifiable constants in Contml, Gendist, and Contrast see the documentation files for those programs. phylip-3.697/doc/contml.html 0000644 0047320 0047320 00000037101 12406201172 015471 0 ustar joe felsenst_g
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program estimates phylogenies by the restricted maximum likelihood method based on the Brownian motion model. It is based on the model of Edwards and Cavalli-Sforza (1964; Cavalli-Sforza and Edwards, 1967). Gomberg (1966), Felsenstein (1973b, 1981c) and Thompson (1975) have done extensive further work leading to efficient algorithms. Contml uses restricted maximum likelihood estimation (REML), which is the criterion used by Felsenstein (1973b). The actual algorithm is an iterative EM Algorithm (Dempster, Laird, and Rubin, 1977) which is guaranteed to always give increasing likelihoods. The algorithm is described in detail in a paper of mine (Felsenstein, 1981c), which you should definitely consult if you are going to use this program. Some simulation tests of it are given by Rohlf and Wooten (1988) and Kim and Burgman (1988).
The default (gene frequency) mode treats the input as gene frequencies at a series of loci, and square-root-transforms the allele frequencies (constructing the frequency of the missing allele at each locus first). This enables us to use the Brownian motion model on the resulting coordinates, in an approximation equivalent to using Cavalli-Sforza and Edwards's (1967) chord measure of genetic distance and taking that to give distance between particles undergoing pure Brownian motion. It assumes that each locus evolves independently by pure genetic drift.
The alternative continuous characters mode (menu option C) treats the input as a series of coordinates of each species in N dimensions. It assumes that we have transformed the characters to remove correlations and to standardize their variances.
Many current users of Contml use it to analyze microsatellite data. There are three ways to do this:
http://evolution.gs.washington.edu/phylip/software.html.
Those distance measures allow for mutation during the divergence of the populations. But even they are not perfect -- they do not allow us to use all the information contained in the gene frequency differences within a copy number allele. There is a need for a more complete statistical treatment of inference of phylogenies from microsatellite models, ones that take both mutation and genetic drift fully into account.
0.10 X 18 + 0.24 X 19 + 0.60 X 20 + 0.06 X 21 = 19.62
copies. These values can, I believe, be calculated by a spreadsheet program. Each microsatellite is represented by one character, and the continuous character mode of Contml is used (not the gene frequencies mode). This coding allows for mutation that changes copy number. It does not make complete use of all data, but neither does the treatment of microsatellite gene frequencies as changing only by genetic drift.
The input file is as described in the continuous characters documentation file above. Options are selected using a menu:
Continuous character Maximum Likelihood method version 3.69 Settings for this run: U Search for best tree? Yes C Gene frequencies or continuous characters? Gene frequencies A Input file has all alleles at each locus? No, one allele missing at each O Outgroup root? No, use as outgroup species 1 G Global rearrangements? No J Randomize input order of species? No. Use input order M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Write out trees onto tree file? Yes Y to accept these or type the letter for one to change |
Option U is the usual User Tree option. Options C (Continuous Characters) and A (All alleles present) have been described in the Gene Frequencies and Continuous Characters Programs documentation file. The options G, J, O and M are the usual Global Rearrangements, Jumble order of species, Outgroup root, and Multiple Data Sets options.
The M (Multiple data sets) option does not allow multiple sets of weights instead of multiple data sets, as there are no weights in this program.
The G and J options have no effect if the User Tree option is selected. User trees are given with a trifurcation (three-way split) at the base. They can start from any interior node. Thus the tree:
A ! *--B ! *-----C ! *--D ! E
can be represented by any of the following:
(A,B,(C,(D,E))); ((A,B),C,(D,E)); (((A,B),C),D,E);
(there are of course 69 other representations as well obtained from these by swapping the order of branches at an interior node).
The output has a standard appearance. The topology of the tree is given by an unrooted tree diagram. The lengths (in time or in expected amounts of variance) are given in a table below the topology, and a rough confidence interval given for each length. Negative lower bounds on length indicate that rearrangements may be acceptable.
The units of length are amounts of expected accumulated variance (not time). The log likelihood (natural log) of each tree is also given, and it is indicated how many topologies have been tried. The tree does not necessarily have all tips contemporary, and the log likelihood may be either positive or negative (this simply corresponds to whether the density function does or does not exceed 1) and a negative log likelihood does not indicate any error. The log likelihood allows various formal likelihood ratio hypothesis tests. The description of the tree includes approximate standard errors on the lengths of segments of the tree. These are calculated by considering only the curvature of the likelihood surface as the length of the segment is varied, holding all other lengths constant. As such they are most probably underestimates of the variance, and hence may give too much confidence in the given tree.
One should use caution in interpreting the likelihoods that are printed out. If the model is wrong, it will not be possible to use the likelihoods to make formal statistical statements. Thus, if gene frequencies are being analyzed, but the gene frequencies change not only by genetic drift, but also by mutation, the model is not correct. It would be as well-justified in this case to use Gendist to compute the Nei (1972) genetic distance and then use Fitch, Kitsch or Neighbor to make a tree. If continuous characters are being analyzed, but if the characters have not been transformed to new coordinates that evolve independently and at equal rates, then the model is also violated and no statistical analysis is possible. Doing such a transformation is not easy, and usually not even possible.
If the U (User Tree) option is used and more than one tree is supplied, the program also performs a statistical test of each of these trees against the one with highest likelihood. If there are two user trees, the test done is one which is due to Kishino and Hasegawa (1989), a version of a test originally introduced by Templeton (1983). In this implementation it uses the mean and variance of log-likelihood differences between trees, taken across loci. If the two trees' means are more than 1.96 standard deviations different then the trees are declared significantly different. This use of the empirical variance of log-likelihood differences is more robust and nonparametric than the classical likelihood ratio test, and may to some extent compensate for any lack of realism in the model underlying this program.
If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. The version used here is a multivariate normal approximation to their test; it is due to Shimodaira (1998). The variances and covariances of the sum of log likelihoods across loci are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected log-likelihood, log-likelihoods for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the highest log-likelihood exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.
In either the KHT or the SH test the program prints out a table of the log-likelihoods of each tree, the differences of each from the highest one, the variance of that quantity as determined by the log-likelihood differences at individual sites, and a conclusion as to whether that tree is or is not significantly worse than the best one.
One problem which sometimes arises is that the program is fed two species (or populations) with identical transformed gene frequencies: this can happen if sample sizes are small and/or many loci are monomorphic. In this case the program "gets its knickers in a twist" and can divide by zero, usually causing a crash. If you suspect that this has happened, check for two species with identical coordinates. If you find them, eliminate one from the problem: the two must always show up as being at the same point on the tree anyway.
The constants available for modification at the beginning of the program include "epsilon1", a small quantity used in the iterations of branch lengths, "epsilon2", another not quite so small quantity used to check whether gene frequencies that were fed in for all alleles do not add up to 1, "smoothings", the number of passes through a given tree in the iterative likelihood maximization for a given topology, "maxtrees", the maximum number of user trees that will be used for the Kishino-Hasegawa-Templeton test, and "namelength", the length of species names. There is no provision in this program for saving multiple trees that are tied for having the highest likelihood, mostly because an exact tie is unlikely anyway.
The algorithm does not run as quickly as the discrete character methods but is not enormously slower. Like them, its execution time should rise as the cube of the number of species.
This data set was compiled by me from the compilation of human gene frequencies by Mourant (1976). It appeared in a paper of mine (Felsenstein, 1981c) on maximum likelihood phylogenies from gene frequencies. The names of the loci and alleles are given in that paper.
5 10
2 2 2 2 2 2 2 2 2 2
European 0.2868 0.5684 0.4422 0.4286 0.3828 0.7285 0.6386 0.0205
0.8055 0.5043
African 0.1356 0.4840 0.0602 0.0397 0.5977 0.9675 0.9511 0.0600
0.7582 0.6207
Chinese 0.1628 0.5958 0.7298 1.0000 0.3811 0.7986 0.7782 0.0726
0.7482 0.7334
American 0.0144 0.6990 0.3280 0.7421 0.6606 0.8603 0.7924 0.0000
0.8086 0.8636
Australian 0.1211 0.2274 0.5821 1.0000 0.2018 0.9000 0.9837 0.0396
0.9097 0.2976
|
Continuous character Maximum Likelihood method version 3.69
5 Populations, 10 Loci
Numbers of alleles at the loci:
------- -- ------- -- --- -----
2 2 2 2 2 2 2 2 2 2
Name Gene Frequencies
---- ---- -----------
locus: 1 2 3 4 5 6
7 8 9 10
European 0.28680 0.56840 0.44220 0.42860 0.38280 0.72850
0.63860 0.02050 0.80550 0.50430
African 0.13560 0.48400 0.06020 0.03970 0.59770 0.96750
0.95110 0.06000 0.75820 0.62070
Chinese 0.16280 0.59580 0.72980 1.00000 0.38110 0.79860
0.77820 0.07260 0.74820 0.73340
American 0.01440 0.69900 0.32800 0.74210 0.66060 0.86030
0.79240 0.00000 0.80860 0.86360
Australian 0.12110 0.22740 0.58210 1.00000 0.20180 0.90000
0.98370 0.03960 0.90970 0.29760
+-----------------------------------------------------------African
!
! +-------------------------------Australian
1-------------3
! ! +-----------------------American
! +-----2
! +Chinese
!
+European
remember: this is an unrooted tree!
Ln Likelihood = 38.71914
Between And Length Approx. Confidence Limits
------- --- ------ ------- ---------- ------
1 African 0.09693444 ( 0.03123910, 0.19853605)
1 3 0.02252816 ( 0.00089799, 0.05598045)
3 Australian 0.05247406 ( 0.01177094, 0.11542376)
3 2 0.00945315 ( -0.00897717, 0.03795670)
2 American 0.03806240 ( 0.01095938, 0.07997877)
2 Chinese 0.00208822 ( -0.00960622, 0.02017434)
1 European 0.00000000 ( -0.01627246, 0.02516630)
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program implements the contrasts calculation described in my 1985 paper on the comparative method (Felsenstein, 1985d). It reads in a data set of the standard quantitative characters sort, and also a tree from the treefile. It then forms the contrasts between species that, according to that tree, are statistically independent. This is done for each character. The contrasts are all standardized by branch lengths (actually, square roots of branch lengths).
The method is explained in the 1985 paper. It assumes a Brownian motion model. This model was introduced by Edwards and Cavalli-Sforza (1964; Cavalli-Sforza and Edwards, 1967) as an approximation to the evolution of gene frequencies. I have discussed (Felsenstein, 1973b, 1981c, 1985d, 1988b) the difficulties inherent in using it as a model for the evolution of quantitative characters. Chief among these is that the characters do not necessarily evolve independently or at equal rates. This program allows one to evaluate this, if there is independent information on the phylogeny. You can compute the variance of the contrasts for each character, as a measure of the variance accumulating per unit branch length. You can also test covariances of characters.
The input file is as described in the continuous characters documentation file above, for the case of continuous quantitative characters (not gene frequencies). Options are selected using a menu:
Continuous character comparative analysis, version 3.69 Settings for this run: W Within-population variation in data? No, species values are means R Print out correlations and regressions? Yes C Print out contrasts? No M Analyze multiple trees? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes Y to accept these or type the letter for one to change |
Option W makes the program expect not means of the phenotypes in each species, but phenotypes of individual specimens. The details of the input file format in that case are given below. In that case the program estimates the covariances of the phenotypic change, as well as covariances of within-species phenotypic variation. The model used is similar to (but not identical to) that of Lynch (1990). The algorithms used differ from the ones he gives in that paper. They are described in a recent paper (Felsenstein, 2008). In the case that has within-species samples contrasts are used by the program, but it does not make sense to write them out to an output file for direct analysis. They are of two kinds, contrasts within species and contrasts between species. The former are affected only by the within-species phenotypic covariation, but the latter are affected by both within- and between-species covariation. Contrast infers these two kinds of covariances and writes the estimates out.
M is similar to the usual multiple data sets input option, but is used here to allow multiple trees to be read from the treefile, not multiple data sets to be read from the input file. In this way you can use bootstrapping on the data that estimated these trees, get multiple bootstrap estimates of the tree, and then use the M option to make multiple analyses of the contrasts and the covariances, correlations, and regressions. In this way (Felsenstein, 1988b) you can assess the effect of the inaccuracy of the trees on your estimates of these statistics.
R allows you to turn off or on the printing out of the statistics. If it is off only the contrasts will be printed out (unless option 1 is selected). With only the contrasts printed out, they are in a simple array that is in a form that many statistics packages should be able to read. The contrasts are rows, and each row has one contrast for each character. Any multivariate statistics package should be able to analyze these (but keep in mind that the contrasts have, by virtue of the way they are generated, expectation zero, so all regressions must pass through the origin). If the W option has been set to analyze within-species as well as between-species variation, the R option does not appear in the menu as the regression and correlation statistics should always be computed in that case.
As usual, the tree file has the default name intree. It should contain the desired tree or trees. These can be either in bifurcating form, or may have the bottommost fork be a trifurcation (it should not matter which of these ways you present the tree). Note that the tree may not contain any multifurcations aside from a trifurcation at the root! If there are any, the program may not work, or may give misleading results.
The tree must, of course, have branch lengths. These cannot be negative. Trees from some distance methods, particularly Neighbor-Joining, are sometimes inferred to have negative branch lengths, so be sure to choose options in those programs that prevent negative branch lengths.
If you have a molecular data set (for example) and also, on the same species, quantitative measurements, here is how you can allow for the uncertainty of your estimate of the tree. Use Seqboot to generate multiple data sets from your molecular data. Then, whichever method you use to analyze it (the relevant ones are those that produce estimates of the branch lengths: Dnaml, Dnamlk, Fitch, Kitsch, and Neighbor -- the latter three require you to use Dnadist to turn the bootstrap data sets into multiple distance matrices), you should use the Multiple Data Sets option of that program. This will result in a tree file with many trees on it. Then use this tree file with the input file containing your continuous quantitative characters, choosing the Multiple Trees (M) option. You will get one set of contrasts and statistics for each tree in the tree file. At the moment there is no overall summary: you will have to tabulate these by hand. A similar process can be followed if you have restriction sites data (using Restml) or gene frequencies data.
The statistics that are printed out include the covariances between all pairs of characters, the regressions of each character on each other (column j is regressed on row i), and the correlations between all pairs of characters. In assessing degress of freedom it is important to realize that each contrast was taken to have expectation zero, which is known because each contrast could as easily have been computed xi-xj instead of xj-xi. Thus there is no loss of a degree of freedom for estimation of a mean. The degrees of freedom are thus the same as the number of contrasts, namely one less than the number of species (tips). If you feed these contrasts into a multivariate statistics program make sure that it knows that each variable has expectation exactly zero.
10 5 Alpha 2 2.01 5.3 1.5 -3.41 0.3 1.98 4.3 2.1 -2.98 0.45 Gammarus 3 6.57 3.1 2.0 -1.89 0.6 7.62 3.4 1.9 -2.01 0.7 6.02 3.0 1.9 -2.03 0.6 ... |
number of species, number of characters name of 1st species, # of individuals data for individual #1 data for individual #2 name of 2nd species, # of individuals data for individual #1 data for individual #2 data for individual #3 (and so on) |
The covariances, correlations, and regressions for the "additive" (between-species evolutionary variation) and "environmental" (within-species phenotypic variation) are printed out (the maximum likelihood estimates of each). The program also estimates the within-species phenotypic variation in the case where the between-species evolutionary covariances are forced to be zero. The log-likelihoods of these two cases are compared and a likelihood ratio test (LRT) is carried out. The program prints the result of this test as a chi-square variate, and gives the number of degrees of freedom of the LRT. You have to look up the chi-square variable on a table of the chi-square distribution. The A option is available (if the W option is invoked) to allow you to turn off the doing of this test if you want to.
The program prints out the log-likelihood of the data under the models with and without between-species variation. It shows the degrees of freedom and chi-square value for a likelihood ratio test of the absence of between-species variation. For the moment the program cannot handle the case where within-species variation is to be taken into account but where only species means are available. (It can handle cases where some species have only one member in their sample).
We hope to fix this soon. We are also on our way to incorporating full-sib, half-sib, or clonal groups within species, so as to do one analysis for within-species genetic and between-species phylogenetic variation.
The data set used as an example below is the example from a paper by Michael Lynch (1990), his characters having been log-transformed. In the case where there is only one specimen per species, Lynch's model is identical to our model of within-species variation (for multiple individuals per species it is not a subcase of his model).
5 2
Homo 4.09434 4.74493
Pongo 3.61092 3.33220
Macaca 2.37024 3.36730
Ateles 2.02815 2.89037
Galago -1.46968 2.30259
|
((((Homo:0.21,Pongo:0.21):0.28,Macaca:0.49):0.13,Ateles:0.62):0.38,Galago:1.00); |
Continuous character contrasts analysis, version 3.69
5 Populations, 2 Characters
Name Phenotypes
---- ----------
Homo 4.09434 4.74493
Pongo 3.61092 3.33220
Macaca 2.37024 3.36730
Ateles 2.02815 2.89037
Galago -1.46968 2.30259
Contrasts (columns are different characters)
--------- -------- --- --------- -----------
0.74593 2.17989
1.58474 0.71761
1.19293 0.86790
3.35832 0.89706
Covariance matrix
---------- ------
3.9423 1.7028
1.7028 1.7062
Regressions (columns on rows)
----------- -------- -- -----
1.0000 0.4319
0.9980 1.0000
Correlations
------------
1.0000 0.6566
0.6566 1.0000
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
These programs are intended for the use of morphological systematists who are dealing with discrete characters, or by molecular evolutionists dealing with presence-absence data on restriction sites. One of the programs (Pars) allows multistate characters, with up to 8 states, plus the unknown state symbol "?". For the others, the characters are assumed to be coded into a series of (0,1) two-state characters. For most of the programs there are two other states possible, "P", which stands for the state of Polymorphism for both states (0 and 1), and "?", which stands for the state of ignorance: it is the state "unknown", or "does not apply". The state "P" can also be denoted by "B", for "both".
There is a method invented by Sokal and Sneath (1963) for linear sequences of character states, and fully developed for branching sequences of character states by Kluge and Farris (1969) for recoding a multistate character into a series of two-state (0,1) characters. Suppose we had a character with four states whose character-state tree had the rooted form:
1 ---> 0 ---> 2
|
|
V
3
so that 1 is the ancestral state and 0, 2 and 3 derived states. We can represent this as three two-state characters:
Old State New States
--- ----- --- ------
0 001
1 000
2 011
3 101
The three new states correspond to the three arrows in the above character state tree. Possession of one of the new states corresponds to whether or not the old state had that arrow in its ancestry. Thus the first new state corresponds to the bottommost arrow, which only state 3 has in its ancestry, the second state to the rightmost of the top arrows, and the third state to the leftmost top arrow. This coding will guarantee that the number of times that states arise on the tree (in programs Mix, Move, Penny and Boot) or the number of polymorphic states in a tree segment (in the Polymorphism option of Dollop, Dolmove, Dolpenny and Dolboot) will correctly correspond to what would have been the case had our programs been able to take multistate characters into account. Although I have shown the above character state tree as rooted, the recoding method works equally well on unrooted multistate characters as long as the connections between the states are known and contain no loops.
However, in the default option of programs Dollop, Dolmove, Dolpenny and Dolboot the multistate recoding does not necessarily work properly, as it may lead the program to reconstruct nonexistent state combinations such as 010. An example of this problem is given in my paper on alternative phylogenetic methods (1979).
If you have multistate character data where the states are connected in a branching "character state tree" you may want to do the binary recoding yourself. Thanks to Christopher Meacham, the package contains a program, Factor, which will do the recoding itself. For details see the documentation file for Factor.
We now also have the program Pars, which can do parsimony for unordered character states.
The methods used in these programs make different assumptions about evolutionary rates, probabilities of different kinds of events, and our knowledge about the characters or about the character state trees. Basic references on these assumptions are my 1979, 1981b and 1983b papers, particularly the latter. The assumptions of each method are briefly described in the documentation file for the corresponding program. In most cases my assertions about what are the assumptions of these methods are challenged by others, whose papers I also cite at that point. Personally, I believe that they are wrong and I am right. I must emphasize the importance of understanding the assumptions underlying the methods you are using. No matter how fancy the algorithms, how maximum the likelihood or how minimum the number of steps, your results can only be as good as the correspondence between biological reality and your assumptions!
The input format is as described in the general documentation file. The input starts with a line containing the number of species and the number of characters.
In Pars, each character can have up to 8 states plus a "?" state. In any character, the first 8 symbols encountered will be taken to represent these states. Any of the digits 0-9, letters A-Z and a-z, and even symbols such as + and -, can be used (and in fact which 8 symbols are used can be different in different characters).
In the other discrete characters programs the allowable states are, 0, 1, P, B, and ?. Blanks may be included between the states (i. e. you can have a species whose data is DISCOGLOSS0 1 1 0 1 1 1). It is possible for extraneous information to follow the end of the character state data on the same line. For example, if there were 7 characters in the data set, a line of species data could read "DISCOGLOSS0110111 Hello there").
The discrete character data can continue to a new line whenever needed. The characters are not in the "aligned" or "interleaved" format used by the molecular sequence programs: they have the name and entire set of characters for one species, then the name and entire set of characters for the next one, and so on. This is known as the sequential format. Be particularly careful when you use restriction sites data, which can be in either the aligned or the sequential format for use in Restml but must be in the sequential format for these discrete character programs.
For Pars the discrete character data can be in either Sequential or Interleaved format; the latter is the default.
Errors in the input data will often be detected by the programs, and this will cause them to issue an error message such as 'BAD OUTGROUP NUMBER: ' together with information as to which species, character, or in this case outgroup number is the incorrect one. The program will then terminate; you will have to look at the data and figure out what went wrong and fix it. Often an error in the data causes a lack of synchronization between what is in the data file and what the program thinks is to be there. Thus a missing character may cause the program to read part of the next species name as a character and complain about its value. In this type of case you should look for the error earlier in the data file than the point about which the program is complaining.
Specific information on options will be given in the documentation file associated with each program. However, some options occur in many programs. Options are selected from the menu in each program.
An example is:
001??11
The ancestor information can be continued to a new line and can have blanks between any of the characters in the same way that species character data can. In the program Clique the ancestor is instead to be included as a regular species and no A option is available.
For example, if there were 20 binary characters that had been generated by nine multistate characters having respectively 4, 3, 3, 2, 1, 2, 2, 2, and 1 binary factors you would make the factors file be:
11112223334456677889
although it could equivalently be:
aaaabbbaaabbabbaabba
All that is important is that the symbol for each binary character change only when adjacent binary characters correspond to different mutlistate characters. The factors file contents can continue to a new line at any time except during the initial characters filling out the length of a species name.
WWWCC WWCWC
Note that blanks in the seqence of characters (after the first ones that are as long as the species names) will be ignored, and the information can go on to a new line at any point. So this could equally well have been specified by
WW CCCWWCWC
On the line in that table corresponding to each branch of the tree will also be printed "yes", "no" or "maybe" as an answer to the question of whether this branch is of nonzero length. If there is no evidence that any character has changed in that branch, then "no" will be printed. If there is definite evidence that one has changed, then "yes" will be printed. If the matter is ambiguous, then "maybe" will be printed. You should keep in mind that all of these conclusions assume that we are only interested in the assignment of states that require the least amount of change. In reality, the confidence limit on tree topology usually includes many different topologies, and presumably also then the confidence limits on amounts of change in branches are also very broad.
In addition to the table showing numbers of events, a table may be printed out showing which ancestral state causes the fewest events for each character. This will not always be done, but only when the tree is rooted and some ancestral states are unknown. This can be used to infer states of occurred and making it easy for the user to reconstruct all the alternative patterns of the characters states in the hypothetical ancestral nodes. In Pars you can, using the menu, turn off this dot-differencing convention and see all states at all hypothetical ancestral nodes of the tree.
If you select the proper menu option, a table of the number of events required in each character can also be printed, to help in reconstructing the placement of changes on the tree.
This table may not be obvious at first. A typical example looks like this:
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 2 2 2 2 1 1 2 2 1
10! 1 2 3 1 1 1 1 1 1 2
20! 1 2 2 1 2 2 1 1 1 2
30! 1 2 1 1 1 2 1 3 1 1
40! 1
The numbers across the top and down the side indicate which character is being
referred to. Thus character 23 is column "3" of row "20" and has 2 steps in
this case.
I cannot emphasize too strongly that just because the tree diagram which the program prints out contains a particular branch MAY NOT MEAN THAT WE HAVE EVIDENCE THAT THE BRANCH IS OF NONZERO LENGTH. In some of the older programs, the procedure which prints out the tree cannot cope with a trifurcation, nor can the internal data structures used in some of my programs. Therefore, even when we have no resolution and a multifurcation, successive bifurcations may be printed out, although some of the branches shown will in fact actually be of zero length. To find out which, you will have to work out character by character where the placements of the changes on the tree are, under all possible ways that the changes can be placed on that tree.
In Pars, Mix, Penny, Dollop, and Dolpenny the trees will be (if the user selects the option to see them) accompanied by tables showing the reconstructed states of the characters in the hypothetical ancestral nodes in the interior of the tree. This will enable you to reconstruct where the changes were in each of the characters. In some cases the state shown in an interior node will be "?", which means that either 0 or 1 would be possible at that point. In such cases you have to work out the ambiguity by hand. A unique assignment of locations of changes is often not possible in the case of the Wagner parsimony method. There may be multiple ways of assigning changes to segments of the tree with that method. Printing only one would be misleading, as it might imply that certain segments of the tree had no change, when another equally valid assignment would put changes there. It must be emphasized that all these multiple assignments have exactly equal numbers of total changes, so that none is preferred over any other.
I have followed the convention of having a "." printed out in the table of character states of the hypothetical ancestral nodes whenever a state is 0 or 1 and its immediate ancestor is the same. This has the effect of highlighting the places where changes might have occurred and making it easy for the user to reconstruct all the alternative patterns of the characters states in the hypothetical ancestral nodes. In Pars you can, using the menu, turn off this dot-differencing convention and see all states at all hypothetical ancestral nodes of the tree.
On the line in that table corresponding to each branch of the tree will also be printed "yes", "no" or "maybe" as an answer to the question of whether this branch is of nonzero length. If there is no evidence that any character has changed in that branch, then "no" will be printed. If there is definite evidence that one has changed, then "yes" will be printed. If the matter is ambiguous, then "maybe" will be printed. You should keep in mind that all of these conclusions assume that we are only interested in the assignment of states that requires the least amount of change. In reality, the confidence limit on tree topology usually includes many different topologies, and presumably also then the confidence limits on amounts of change in branches are also very broad.
In addition to the table showing numbers of events, a table may be printed out showing which ancestral state causes the fewest events for each character. This will not always be done, but only when the tree is rooted and some ancestral states are unknown. This can be used to infer states of ancestors. For example, if you use the O (Outgroup) and A (Ancestral states) options together, with at least some of the ancestral states being given as "?", then inferences will be made for those characters, as the outgroup makes the tree rooted if it was not already.
In programs Mix and Penny, if you are using the Camin-Sokal parsimony option with ancestral state "?" and it turns out that the program cannot decide between ancestral states 0 and 1, it will fail to even attempt reconstruction of states of the hypothetical ancestors, printing them all out as "." for those characters. This is done for internal bookkeeping reasons -- to reconstruct their changes would require a fair amount of additional code and additional data structures. It is not too hard to reconstruct the internal states by hand, trying the two possible ancestral states one after the other. A similar comment applies to the use of ancestral state "?" in the Dollo or Polymorphism parsimony methods (programs Dollop and Dolpenny) which also can result in a similar hesitancy to print the estimate of the states of the hypothetical ancestors. In all of these cases the program will print "?" rather than "no" when it describes whether there are any changes in a branch, since there might or might not be changes in those characters which are not reconstructed.
For further information see the documentation files for the individual programs. phylip-3.697/doc/distance.html 0000644 0047320 0047320 00000040014 12406201172 015764 0 ustar joe felsenst_g
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
The programs Fitch, Kitsch, and Neighbor are for dealing with data which comes in the form of a matrix of pairwise distances between all pairs of taxa, such as distances based on molecular sequence data, gene frequency genetic distances, amounts of DNA hybridization, or immunological distances. In analyzing these data, distance matrix programs implicitly assume that:
These assumptions can be traced in the least squares methods of programs Fitch and Kitsch but it is not quite so easy to see them in operation in the Neighbor-Joining method of Neighbor, where the independence assumption is less obvious.
THESE TWO ASSUMPTIONS ARE DUBIOUS IN MOST CASES: independence will not be expected to be true in most kinds of data, such as genetic distances from gene frequency data. For genetic distance data in which pure genetic drift without mutation can be assumed to be the mechanism of change Contml may be more appropriate. However, Fitch, Kitsch, and Neighbor will not give positively misleading results (they will not make a statistically inconsistent estimate) provided that additivity holds, which it will if the distance is computed from the original data by a method which corrects for reversals and parallelisms in evolution. If additivity is not expected to hold, problems are more severe. A short discussion of these matters will be found in a review article of mine (1984a). For detailed, if sometimes irrelevant, controversy see the papers by Farris (1981, 1985, 1986) and myself (1986, 1988b).
For genetic distances from gene frequencies, Fitch, Kitsch, and Neighbor may be appropriate if a neutral mutation model can be assumed and Nei's genetic distance is used, or if pure drift can be assumed and either Cavalli-Sforza's chord measure or Reynolds, Weir, and Cockerham's (1983) genetic distance is used. However, in the latter case (pure drift) Contml should be better.
Restriction site and restriction fragment data can be treated by distance matrix methods if a distance such as that of Nei and Li (1979) is used. Distances of this sort can be computed in PHYLIP by the program Restdist.
For nucleic acid sequences, the distances computed in Dnadist allow correction for multiple hits (in different ways) and should allow one to analyse the data under the presumption of additivity. In all of these cases independence will not be expected to hold. DNA hybridization and immunological distances may be additive and independent if transformed properly and if (and only if) the standards against which each value is measured are independent. (This is rarely exactly true).
Fitch and the Neighbor-Joining option of Neighbor fit a tree which has the branch lengths unconstrained. Kitsch and the UPGMA option of Neighbor, by contrast, assume that an "evolutionary clock" is valid, according to which the true branch lengths from the root of the tree to each tip are the same: the expected amount of evolution in any lineage is proportional to elapsed time.
The input format for distance data is straightforward. The first line of the input file contains the number of species. There follows species data, starting, as with all other programs, with a species name. The species name is ten characters long, and must be padded out with blanks if shorter. For each species there then follows a set of distances to all the other species (options selected in the programs' menus allow the distance matrix to be upper or lower triangular or square). The distances can continue to a new line after any of them. If the matrix is lower-triangular, the diagonal entries (the distances from a species to itself) will not be read by the programs. If they are included anyway, they will be ignored by the programs, except for the case where one of them starts a new line, in which case the program will mistake it for a species name and get very confused.
For example, here is a sample input matrix, with a square matrix:
5
Alpha 0.000 1.000 2.000 3.000 3.000
Beta 1.000 0.000 2.000 3.000 3.000
Gamma 2.000 2.000 0.000 3.000 3.000
Delta 3.000 3.000 3.000 0.000 1.000
Epsilon 3.000 3.000 3.000 1.000 0.000
|
and here is a sample lower-triangular input matrix with distances continuing to new lines as needed:
14 Mouse Bovine 1.7043 Lemur 2.0235 1.1901 Tarsier 2.1378 1.3287 1.2905 Squir Monk 1.5232 1.2423 1.3199 1.7878 Jpn Macaq 1.8261 1.2508 1.3887 1.3137 1.0642 Rhesus Mac 1.9182 1.2536 1.4658 1.3788 1.1124 0.1022 Crab-E.Mac 2.0039 1.3066 1.4826 1.3826 0.9832 0.2061 0.2681 BarbMacaq 1.9431 1.2827 1.4502 1.4543 1.0629 0.3895 0.3930 0.3665 Gibbon 1.9663 1.3296 1.8708 1.6683 0.9228 0.8035 0.7109 0.8132 0.7858 Orang 2.0593 1.2005 1.5356 1.6606 1.0681 0.7239 0.7290 0.7894 0.7140 0.7095 Gorilla 1.6664 1.3460 1.4577 1.5935 0.9127 0.7278 0.7412 0.8763 0.7966 0.5959 0.4604 Chimp 1.7320 1.3757 1.7803 1.7119 1.0635 0.7899 0.8742 0.8868 0.8288 0.6213 0.5065 0.3502 Human 1.7101 1.3956 1.6661 1.7599 1.0557 0.6933 0.7118 0.7589 0.8542 0.5612 0.4700 0.3097 0.2712 |
Note that the name "Mouse" in this matrix must be padded out by blanks to the full length of 10 characters.
In general the distances are assumed to all be present: at the moment there is only one way we can have missing entries in the distance matrix. If the S option (which allows the user to specify the degree of replication of each distance) is invoked, with some of the entries having degree of replication zero, if the U (User Tree) option is in effect, and if the tree being examined is such that every branch length can be estimated from the data, it will be possible to solve for the branch lengths and sum of squares when there is some missing data. You may not get away with this if the U option is not in effect, as a tree may be tried on which the program will calculate a branch length by dividing zero by zero, and get upset.
The present version of Neighbor does allow the Subreplication option to be used and the number of replicates to be in the input file, but it actally does nothing with this information except read it in. It makes use of the average distances in the cells of the input data matrix. This means that you cannot use the S option to treat zero cells. We hope to modify Neighbor in the future to allow Subreplication. Of course the U (User tree) option is not available in Neighbor in any case.
The present versions of Fitch and Kitsch will do much better on missing values than did previous versions, but you will still have to be careful about them. Nevertheless you might (just) be able to explore relevant alternative tree topologies one at a time using the U option when there is missing data.
Alternatively, if the missing values in one cell always correspond to a cell with non-missing values on the opposite side of the main diagonal (i.e., if D(i,j) missing implies that D(j,i) is not missing), then use of the S option will always be sufficient to cope with missing values. When it is used, the missing distances should be entered as if present (any number can be used) and the degree of replication for them should be given as 0.
Note that the algorithm for searching among topologies in Fitch and Kitsch is the same one used in other programs, so that it is necessary to try different orders of species in the input data. The J (Jumble) menu option may be sufficient for most purposes.
The programs Fitch and Kitsch carry out the method of Fitch and Margoliash (1967) for fitting trees to distance matrices. They also are able to carry out the least squares method of Cavalli-Sforza and Edwards (1967), plus a variety of other methods of the same family (see the discussion of the P option below). They can also be set to use the Minimum Evolution method (Nei and Rzhetsky, 1993; Kidd and Sgaramella-Zonta, 1971).
The objective of these methods is to find that tree which minimizes
__ __
\ \ nij ( Dij - dij)2
Sum of squares = /_ /_ ------------------
i j Dijp
(the symbol made up of \, / and _ characters is of course a summation sign) where D is the observed distance between species i and j and d is the expected distance, computed as the sum of the lengths (amounts of evolution) of the segments of the tree from species i to species j. The quantity n is the number of times each distance has been replicated. In simple cases this is taken to be one, but the user can, as an option, specify the degree of replication for each distance. The distance is then assumed to be a mean of those replicates. The power P is what distinguished the various methods. For the Fitch- Margoliash method, which is the default method with this program, P is 2.0. For the Cavalli-Sforza and Edwards least squares method it should be set to 0 (so that the denominator is always 1). An intermediate method is also available in which P is 1.0, and any other value of P, such as 4.0 or -2.3, can also be used. This generates a whole family of methods.
The P (Power) option is not available in the Neighbor-Joining program Neighbor. Implicitly, in this program P is 0.0 (though it is hard to prove this). The UPGMA option of Neighbor will assign the same branch lengths to the particular tree topology that it finds as will Kitsch when given the same tree and Power = 0.0.
All these methods make the assumptions of additivity and independent errors. The difference between the methods is how they weight departures of observed from expected. In effect, these methods differ in how they assume that the variance of measurement of a distance will rise as a function of the expected value of the distance.
These methods assume that the variance of the measurement error is proportional to the P-th power of the expectation (hence the standard deviation will be proportional to the P/2-th power of the expectation). If you have reason to think that the measurement error of a distance is the same for small distances as it is for large, then you should set P=0 and use the least squares method, but if you have reason to think that the relative (percentage) error is more nearly constant than the absolute error, you should use P=2, the Fitch-Margoliash method. In between, P=1 would be appropriate if the sizes of the errors were proportional to the square roots of the expected distance.
One question which arises frequently is what the units of branch length are in the resulting trees. In general, they are not time but units of distance. Thus if two species have a distance 0.3 between them, they will tend to be separated by branches whose total length is about 0.3. In the case of DNA distances, for example, the unit of branch length will be substitutions per base. (In the case of protein distances, it will be amino acid substitutions per amino acid position.)
Here are the options available in all three programs. They are selected using the menu of options.
((A,B),C,(D,E));
while in Kitsch they are to be regarded as rooted and have a bifurcation at the base:
((A,B),(C,(D,E)));
Be careful not to move User trees from Fitch to Kitsch without changing their form appropriately (you can use Retree to do this). User trees are not available in Neighbor. In Fitch if you specify the branch lengths on one or more branches, you can select the L (use branch Lengths) option to avoid having those branches iterated, so that the tree is evaluated with their lengths fixed.
Delta 3.00 5 3.21 3 1.84 9
the 5, 3, and 9 being the number of times the measurement was replicated. When the number of replicates is zero, a distance value must still be provided, although its value will not afect the result. This option is not available in Neighbor.
The numerical options are the usual ones and should be clear from the menu.
Note that when the options L or R are used one of the species, the first or last one, will have its name on an otherwise empty line. Even so, the name should be padded out to full length with blanks. Here is a sample lower- triangular data set.
5
Alpha
Beta 1.00
Gamma 3.00 3.00
Delta 3.00 3.00 2.00
Epsilon 3.00 3.00 2.00 1.00
| <--- note: five blanks should follow the name "Alpha" |
Be careful if you are using lower- or upper-triangular trees to make the corresponding selection from the menu (L or R), as the program may get horribly confused otherwise, but it still gives a result even though the result is then meaningless. With the menu option selected all should be well. phylip-3.697/doc/dnacomp.html 0000644 0047320 0047320 00000027526 12406201172 015630 0 ustar joe felsenst_g
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program implements the compatibility method for DNA sequence data. For a four-state character without a character-state tree, as in DNA sequences, the usual clique theorems cannot be applied. The approach taken in this program is to directly evaluate each tree topology by counting how many substitutions are needed in each site, comparing this to the minimum number that might be needed (one less than the number of bases observed at that site), and then evaluating the number of sites which achieve the minimum number. This is the evaluation of the tree (the number of compatible sites), and the topology is chosen so as to maximize that number.
Compatibility methods originated with Le Quesne's (1969) suggestion that one ought to look for trees supported by the largest number of perfectly fitting (compatible) characters. Fitch (1975) showed by counterexample that one could not use the pairwise compatibility methods used in Clique to discover the largest clique of jointly compatible characters.
The assumptions of this method are similar to those of Clique. In a paper in the Biological Journal of the Linnean Society (1981b) I discuss this matter extensively. In effect, the assumptions are that:
That these are the assumptions of compatibility methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that arguments such as mine are invalid and that parsimony (and perhaps compatibility) methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b, 1988), but also read the exchange between Felsenstein and Sober (1986).
There is, however, some reason to believe that the present criterion is not the proper way to correct for the presence of some sites with high rates of change in nucleotide sequence data. It can be argued that sites showing more than two nucleotide states, even if those are compatible with the other sites, are also candidates for sites with high rates of change. It might then be more proper to use Dnapars with the Threshold option with a threshold value of 2.
Change from an occupied site to a gap is counted as one change. Reversion from a gap to an occupied site is allowed and is also counted as one change. Note that this in effect assumes that a gap N bases long is N separate events. This may be an overcorrection. When we have nonoverlapping gaps, we could instead code a gap as a single event by changing all but the first "-" in the gap into "?" characters. In this way only the first base of the gap causes the program to infer a change.
The input data is standard. The first line of the input file contains the number of species and the number of sites.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
DNA compatibility algorithm, version 3.69 Settings for this run: U Search for best tree? Yes J Randomize input order of sequences? No. Use input order O Outgroup root? No, use as outgroup species 1 W Sites weighted? No M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Print steps & compatibility at sites No 5 Print sequences at all nodes of tree No 6 Write out trees onto tree file? Yes Are these settings correct? (type Y or the letter for one to change) |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The options U, J, O, W, M, and 0 are the usual ones. They are described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
The O (outgroup) option has no effect if the U (user-defined tree) option is in effect. The user-defined trees (option U) fed in must be strictly bifurcating, with a two-way split at their base.
The interpretation of weights (option W) in the case of a compatibility method is that they count how many times the character (in this case the site) is counted in the analysis. Thus a character can be dropped from the analysis by assigning it zero weight. On the other hand, giving it a weight of 5 means that in any clique it is in, it is counted as 5 characters when the size of the clique is evaluated. Generally, weights other than 0 or 1 do not have much meaning when dealing with DNA sequences.
Output is standard: if option 1 is toggled on, the data is printed out, with the convention that "." means "the same as in the first species". Then comes a list of equally parsimonious trees, and (if option 2 is toggled on) a table of the number of changes of state required in each character. If option 5 is toggled on, a table is printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. If the inferred state is a "?" or one of the IUB ambiguity symbols, there will be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand. A "?" in the reconstructed states means that in addition to one or more bases, a gap may or may not be present. If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.
If the U (User Tree) option is used and more than one tree is supplied, the program also performs a statistical test of each of these trees against the one with highest likelihood. If there are two user trees, the test done is one which is due to Kishino and Hasegawa (1989), a version of a test originally introduced by Templeton (1983). In this implementation it uses the mean and variance of weighted compatibility differences between trees, taken across sites. If the two trees' compatibilities are more than 1.96 standard deviations different then the trees are declared significantly different.
If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. In the version used here the variances and covariances of the sum of weighted compatibilities of sites are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected compatibility, compatibilities for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the highest compatibility exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.
In either the KHT or the SH test the program prints out a table of the compatibility of each tree, the differences of each from the highest one, the variance of that quantity as determined by the compatibility differences at individual sites, and a conclusion as to whether that tree is or is not significantly worse than the best one.
The algorithm is a straightforward modification of Dnapars, but with some extra machinery added to calculate, as each species is added, how many base changes are the minimum which could be required at that site. The program runs fairly quickly.
The constants which can be changed at the beginning of the program are: the name length "nmlngth", "maxtrees", the maximum number of trees which the program will store for output, and "maxuser", the maximum number of user trees that can be used in the paired sites test.
5 13
Alpha AACGUGGCCAAAU
Beta AAGGUCGCCAAAC
Gamma CAUUUCGUCACAA
Delta GGUAUUUCGGCCU
Epsilon GGGAUCUCGGCCC
|
DNA compatibility algorithm, version 3.69
5 species, 13 sites
Name Sequences
---- ---------
Alpha AACGUGGCCA AAU
Beta ..G..C.... ..C
Gamma C.UU.C.U.. C.A
Delta GGUA.UU.GG CC.
Epsilon GGGA.CU.GG CCC
One most parsimonious tree found:
+--Epsilon
+--4
+--3 +--Delta
! !
+--2 +-----Gamma
! !
1 +--------Beta
!
+-----------Alpha
remember: this is an unrooted tree!
total number of compatible sites is 11.0
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 2 1 3 2 0 2 1 1 1
10| 1 1 1 3
compatibility (Y or N) of each site with this tree:
0123456789
*----------
0 ! YYNYYYYYY
10 !YYYN
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AABGTSGCCA AAY
1 2 maybe .....C.... ...
2 3 yes V.KD...... C..
3 4 yes GG.A..T.GG .C.
4 Epsilon maybe ..G....... ..C
4 Delta yes ..T..T.... ..T
3 Gamma yes C.TT...T.. ..A
2 Beta maybe ..G....... ..C
1 Alpha maybe ..C..G.... ..T
|
© Copyright 1986-2008 by the University of Washington. Written by Joseph Felsenstein. Permission is granted to copy this document provided that no fee is charged for it and that this copyright notice is not removed.
This program uses nucleotide sequences to compute a distance matrix, under four different models of nucleotide substitution. It can also compute a table of similarity between the nucleotide sequences. The distance for each pair of species estimates the total branch length between the two species, and can be used in the distance matrix programs Fitch, Kitsch or Neighbor. This is an alternative to using the sequence data itself in the maximum likelihood program Dnaml or the parsimony program Dnapars.
The program reads in nucleotide sequences and writes an output file containing the distance matrix, or else a table of similarity between sequences. The four models of nucleotide substitution are those of Jukes and Cantor (1969), Kimura (1980), the F84 model (Kishino and Hasegawa, 1989; Felsenstein and Churchill, 1996), and the model underlying the LogDet distance (Barry and Hartigan, 1987; Lake, 1994; Steel, 1994; Lockhart et. al., 1994). All except the LogDet distance can be made to allow for for unequal rates of substitution at different sites, as Jin and Nei (1990) did for the Jukes-Cantor model. The program correctly takes into account a variety of sequence ambiguities, although in cases where they exist it can be slow.
Jukes and Cantor's (1969) model assumes that there is independent change at all sites, with equal probability. Whether a base changes is independent of its identity, and when it changes there is an equal probability of ending up with each of the other three bases. Thus the transition probability matrix (this is a technical term from probability theory and has nothing to do with transitions as opposed to transversions) for a short period of time dt is:
To: A G C T
---------------------------------
A | 1-3a a a a
From: G | a 1-3a a a
C | a a 1-3a a
T | a a a 1-3a
where a is u dt, the product of the rate of substitution per unit time (u) and the length dt of the time interval. For longer periods of time this implies that the probability that two sequences will differ at a given site is:
p = 3/4 ( 1 - e- 4/3 u t)
and hence that if we observe p, we can compute an estimate of the branch length ut by inverting this to get
ut = - 3/4 loge ( 1 - 4/3 p )
The Kimura "2-parameter" model is almost as symmetric as this, but allows for a difference between transition and transversion rates. Its transition probability matrix for a short interval of time is:
To: A G C T
---------------------------------
A | 1-a-2b a b b
From: G | a 1-a-2b b b
C | b b 1-a-2b a
T | b b a 1-a-2b
where a is u dt, the product of the rate of transitions per unit time and dt is the length dt of the time interval, and b is v dt, the product of half the rate of transversions (i.e., the rate of a specific transversion) and the length dt of the time interval.
The F84 model incorporates different rates of transition and transversion, but also allows for different frequencies of the four nucleotides. It is the model which is used in Dnaml, the maximum likelihood nucelotide sequence phylogenies program in this package. You will find the model described in the document for that program. The transition probabilities for this model are given by Kishino and Hasegawa (1989), and further explained in a paper by me and Gary Churchill (1996).
The LogDet distance allows a fairly general model of substitution. It computes the distance from the determinant of the empirically observed matrix of joint probabilities of nucleotides in the two species. An explanation of it is available in the chapter by Swofford et al. (1996).
The first three models are closely related. The Dnaml model reduces to Kimura's two-parameter model if we assume that the equilibrium frequencies of the four bases are equal. The Jukes-Cantor model in turn is a special case of the Kimura 2-parameter model where a = b. Thus each model is a special case of the ones that follow it, Jukes-Cantor being a special case of both of the others.
The Jin and Nei (1990) correction for variation in rate of evolution from site to site can be adapted to all of the first three models. It assumes that the rate of substitution varies from site to site according to a gamma distribution, with a coefficient of variation that is specified by the user. The user is asked for it when choosing this option in the menu.
Each distance that is calculated is an estimate, from that particular pair of species, of the divergence time between those two species. For the Jukes- Cantor model, the estimate is computed using the formula for ut given above, as long as the nucleotide symbols in the two sequences are all either A, C, G, T, U, N, X, ?, or - (the latter four indicate a deletion or an unknown nucleotide). This estimate is a maximum likelihood estimate for that model. For the Kimura 2-parameter model, with only these nucleotide symbols, formulas special to that estimate are also computed. These are also, in effect, computing the maximum likelihood estimate for that model. In the Kimura case it depends on the observed sequences only through the sequence length and the observed number of transition and transversion differences between those two sequences. The calculation in that case is a maximum likelihood estimate and will differ somewhat from the estimate obtained from the formulas in Kimura's original paper. That formula was also a maximum likelihood estimate, but with the transition/transversion ratio estimated empirically, separately for each pair of sequences. In the present case, one overall preset transition/transversion ratio is used which makes the computations harder but achieves greater consistency between different comparisons.
For the F84 model, or for any of the models where one or both sequences contain at least one of the other ambiguity codons such as Y, R, etc., a maximum likelihood calculation is also done using code which was originally written for Dnaml. Its disadvantage is that it is slow. The resulting distance is in effect a maximum likelihood estimate of the divergence time (total branch length between) the two sequences. However the present program will be much faster than versions earlier than 3.5, because I have speeded up the iterations.
The LogDet model computes the distance from the determinant of the matrix of co-occurrence of nucleotides in the two species, according to the formula
D = - 1/4(loge(|F|) - 1/2loge(fA1fC1fG1fT1fA2fC2fG2fT2))Where F is a matrix whose (i,j) element is the fraction of sites at which base i occurs in one species and base j occurs in the other. fji is the fraction of sites at which species i has base j. The LogDet distance cannot cope with ambiguity codes. It must have completely defined sequences. One limitation of the LogDet distance is that it may be infinite sometimes, if there are too many changes between certain pairs of nucleotides. This can be particularly noticeable with distances computed from bootstrapped sequences.
Note that there is an assumption that we are looking at all sites, including those that have not changed at all. It is important not to restrict attention to some sites based on whether or not they have changed; doing that would bias the distances by making them too large, and that in turn would cause the distances to misinterpret the meaning of those sites that had changed.
For all of these distance methods, the program allows us to specify that "third position" bases have a different rate of substitution than first and second positions, that introns have a different rate than exons, and so on. The Categories option which does this allows us to make up to 9 categories of sites and specify different rates of change for them.
In addition to the four distance calculations, the program can also compute a table of similarities between nucleotide sequences. These values are the fractions of sites identical between the sequences. The diagonal values are 1.0000. No attempt is made to count similarity of nonidentical nucleotides, so that no credit is given for having (for example) different purines at corresponding sites in the two sequences. This option has been requested by many users, who need it for descriptive purposes. It is not intended that the table be used for inferring the tree.
Input is fairly standard, with one addition. As usual the first line of the file gives the number of species and the number of sites.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion -- neither is dot (".").
The options are selected using an interactive menu. The menu looks like this:
Nucleic acid sequence Distance Matrix program, version 3.69 Settings for this run: D Distance (F84, Kimura, Jukes-Cantor, LogDet)? F84 G Gamma distributed rates across sites? No T Transition/transversion ratio? 2.0 C One category of substitution rates? Yes W Use weights for sites? No F Use empirical base frequencies? Yes L Form of distance matrix? Square M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes Y to accept these or type the letter for one to change |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The D option selects one of the four distance methods, or the similarity table. It toggles among the five methods. The default method, if none is specified, is the F84 model.
If the G (Gamma distribution) option is selected, the user will be asked to supply the coefficient of variation of the rate of substitution among sites. This is different from the parameters used by Nei and Jin but related to them: their parameter a is also known as "alpha", the shape parameter of the Gamma distribution. It is related to the coefficient of variation by
CV = 1 / a1/2
or
a = 1 / (CV)2
(their parameter b is absorbed here by the requirement that time is scaled so that the mean rate of evolution is 1 per unit time, which means that a = b). As we consider cases in which the rates are less variable we should set a larger and larger, as CV gets smaller and smaller.
The F (Frequencies) option appears when the Maximum Likelihood distance is selected. This distance requires that the program be provided with the equilibrium frequencies of the four bases A, C, G, and T (or U). Its default setting is one which may save users much time. If you want to use the empirical frequencies of the bases, observed in the input sequences, as the base frequencies, you simply use the default setting of the F option. These empirical frequencies are not really the maximum likelihood estimates of the base frequencies, but they will often be close to those values (what they are is maximum likelihood estimates under a "star" or "explosion" phylogeny). If you change the setting of the F option you will be prompted for the frequencies of the four bases. These must add to 1 and are to be typed on one line separated by blanks, not commas.
The T option in this program does not stand for Threshold, but instead is the Transition/transversion option. The user is prompted for a real number greater than 0.0, as the expected ratio of transitions to transversions. Note that this is not the ratio of the first to the second kinds of events, but the resulting expected ratio of transitions to transversions. The exact relationship between these two quantities depends on the frequencies in the base pools. The default value of the T parameter if you do not use the T option is 2.0.
The C option allows user-defined rate categories. The user is prompted for the number of user-defined rates, and for the rates themselves, which cannot be negative but can be zero. These numbers, which must be nonnegative (some could be 0), are defined relative to each other, so that if rates for three categories are set to 1 : 3 : 2.5 this would have the same meaning as setting them to 2 : 6 : 5. The assignment of rates to sites is then made by reading a file whose default name is "categories". It should contain a string of digits 1 through 9. A new line or a blank can occur after any character in this string. Thus the categories file might look like this:
122231111122411155 1155333333444
If both user-assigned rate categories and Gamma-distributed rates are allowed, the program assumes that the actual rate at a site is the product of the user-assigned category rate and the Gamma-distributed rate. This allows you to specify that certain sites have higher or lower rates of change while also allowing the program to allow variation of rates in addition to that. (This may not always make perfect biological sense: it would be more natural to assume some upper bound to the rate, as we have discussed in the Felsenstein and Churchill paper). Nevertheless you may want to use both types of rate variation.
The L option specifies that the output file is to have the distance matrix in lower triangular form.
The W (Weights) option is invoked in the usual way, with only weights 0 and 1 allowed. It selects a set of sites to be analyzed, ignoring the others. The sites selected are those with weight 1. If the W option is not invoked, all sites are analyzed. The Weights (W) option takes the weights from a file whose default name is "weights". The weights follow the format described in the main documentation file.
The M (multiple data sets) option will ask you whether you want to use multiple sets of weights (from the weights file) or multiple data sets from the input file. The ability to use a single data set with multiple weights means that much less disk space will be used for this input data. The bootstrapping and jackknifing tool Seqboot has the ability to create a weights file with multiple weights. Note also that when we use multiple weights for bootstrapping we can also then maintain different rate categories for different sites in a meaningful way. If you use the multiple data sets option rather than multiple weights, you should not at the same time use the user-defined rate categories option (option C), because the user-defined rate categories could then be associated with the wrong sites. This is not a concern when the M option is used by using multiple weights.
The option 0 is the usual one. It is described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
As the distances are computed, the program prints on your screen or terminal the names of the species in turn, followed by one dot (".") for each other species for which the distance to that species has been computed. Thus if there are ten species, the first species name is printed out, followed by nine dots, then on the next line the next species name is printed out followed by eight dots, then the next followed by seven dots, and so on. The pattern of dots should form a triangle. When the distance matrix has been written out to the output file, the user is notified of that.
The output file contains on its first line the number of species. The distance matrix is then printed in standard form, with each species starting on a new line with the species name, followed by the distances to the species in order. These continue onto a new line after every nine distances. If the L option is used, the matrix of distances is in lower triangular form, so that only the distances to the other species that precede each species are printed. Otherwise the distance matrix is square with zero distances on the diagonal. In general the format of the distance matrix is such that it can serve as input to any of the distance matrix programs.
If the option to print out the data is selected, the output file will precede the data by more complete information on the input and the menu selections. The output file begins by giving the number of species and the number of characters, and the identity of the distance measure that is being used.
If the C (Categories) option is used a table of the relative rates of expected substitution at each category of sites is printed, and a listing of the categories each site is in.
There will then follow the equilibrium frequencies of the four bases. If the Jukes-Cantor or Kimura distances are used, these will necessarily be 0.25 : 0.25 : 0.25 : 0.25. The output then shows the transition/transversion ratio that was specified or used by default. In the case of the Jukes-Cantor distance this will always be 0.5. The transition-transversion parameter (as opposed to the ratio) is also printed out: this is used within the program and can be ignored. There then follow the data sequences, with the base sequences printed in groups of ten bases along the lines of the Genbank and EMBL formats.
The distances printed out are scaled in terms of expected numbers of substitutions, counting both transitions and transversions but not replacements of a base by itself, and scaled so that the average rate of change, averaged over all sites analyzed, is set to 1.0 if there are multiple categories of sites. This means that whether or not there are multiple categories of sites, the expected fraction of change for very small branches is equal to the branch length. Of course, when a branch is twice as long this does not mean that there will be twice as much net change expected along it, since some of the changes may occur in the same site and overlie or even reverse each other. The branch length estimates here are in terms of the expected underlying numbers of changes. That means that a branch of length 0.26 is 26 times as long as one which would show a 1% difference between the nucleotide sequences at the beginning and end of the branch. But we would not expect the sequences at the beginning and end of the branch to be 26% different, as there would be some overlaying of changes.
One problem that can arise is that two or more of the species can be so dissimilar that the distance between them would have to be infinite, as the likelihood rises indefinitely as the estimated divergence time increases. For example, with the Jukes-Cantor model, if the two sequences differ in 75% or more of their positions then the estimate of divergence time would be infinite. Since there is no way to represent an infinite distance in the output file, the program regards this as an error, issues an error message indicating which pair of species are causing the problem, and stops. It might be that, had it continued running, it would have also run into the same problem with other pairs of species. If the Kimura distance is being used there may be no error message; the program may simply give a large distance value (it is iterating towards infinity and the value is just where the iteration stopped). Likewise some maximum likelihood estimates may also become large for the same reason (the sequences showing more divergence than is expected even with infinite branch length). I hope in the future to add more warning messages that would alert the user the this.
If the similarity table is selected, the table that is produced is not in a format that can be used as input to the distance matrix programs. It has a heading, and the species names are also put at the tops of the columns of the table (or rather, the first 8 characters of each species name is there, the other two characters omitted to save space). There is not an option to put the table into a format that can be read by the distance matrix programs, nor is there one to make it into a table of fractions of difference by subtracting the similarity values from 1. This is done deliberately to make it more difficult for to use these values to construct trees. The similarity values are not corrected for multiple changes, and their use to construct trees (even after converting them to fractions of difference) would be wrong, as it would lead to severe conflict between the distant pairs of sequences and the close pairs of sequences.
The constants that are available to be changed by the user at the beginning of the program include "maxcategories", the maximum number of site categories, "iterations", which controls the number of times the program iterates the EM algorithm that is used to do the maximum likelihood distance, "namelength", the length of species names in characters, and "epsilon", a parameter which controls the accuracy of the results of the iterations which estimate the distances. Making "epsilon" smaller will increase run times but result in more decimal places of accuracy. This should not be necessary.
The program spends most of its time doing real arithmetic. The algorithm, with separate and independent computations occurring for each pattern, lends itself readily to parallel processing.
5 13 Alpha AACGTGGCCACAT Beta AAGGTCGCCACAC Gamma CAGTTCGCCACAA Delta GAGATTTCCGCCT Epsilon GAGATCTCCGCCC |
(Note that when the options for displaying the input data are turned off, the output is in a form suitable for use as an input file in the distance matrix programs).
Nucleic acid sequence Distance Matrix program, version 3.69
5 species, 13 sites
F84 Distance
Transition/transversion ratio = 2.000000
Name Sequences
---- ---------
Alpha AACGTGGCCA CAT
Beta ..G..C.... ..C
Gamma C.GT.C.... ..A
Delta G.GA.TT..G .C.
Epsilon G.GA.CT..G .CC
Empirical Base Frequencies:
A 0.24615
C 0.36923
G 0.21538
T(U) 0.16923
5
Alpha 0.000000 0.303900 0.857544 1.158927 1.542899
Beta 0.303900 0.000000 0.339727 0.913522 0.619671
Gamma 0.857544 0.339727 0.000000 1.631729 1.293713
Delta 1.158927 0.913522 1.631729 0.000000 0.165882
Epsilon 1.542899 0.619671 1.293713 0.165882 0.000000
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program reads in nucleotide sequences for four species and computes the phylogenetic invariants discovered by James Cavender (Cavender and Felsenstein, 1987) and James Lake (1987). Lake's method is also called by him "evolutionary parsimony". I prefer Cavender's more mathematically precise term "invariants", as the method bears somewhat more relationship to likelihood methods than to parsimony. The invariants are mathematical formulas (in the present case linear or quadratic) in the EXPECTED frequencies of site patterns which are zero for all trees of a given tree topology, irrespective of branch lengths. We can consider at a given site that if there are no ambiguities, we could have for four species the nucleotide patterns (considering the same site across all four species) AAAA, AAAC, AAAG, ... through TTTT, 256 patterns in all.
The invariants are formulas in the expected pattern frequencies, not the observed pattern frequencies. When they are computed using the observed pattern frequencies, we will usually find that they are not precisely zero even when the model is correct and we have the correct tree topology. Only as the number of nucleotides scored becomes infinite will the observed pattern frequencies approach their expectations; otherwise, we must do a statistical test of the invariants.
Some explanation of invariants will be found in the above papers, and also in my review article on statistical aspects of inferring phylogenies (Felsenstein, 1988b). Although invariants have some important advantages, their validity also depends on symmetry assumptions that may not be satisfied. In the discussion below suppose that the possible unrooted phylogenies are I: ((A,B),(C,D)), II: ((A,C),(B,D)), and III: ((A,D),(B,C)).
Lake's invariants are fairly simple to describe: the patterns involved are only those in which there are two purines and two pyrimidines at a site. Thus a site with AACT would affect the invariants, but a site with AAGG would not. Let us use (as Lake does) the symbols 1, 2, 3, and 4, with the proviso that 1 and 2 are either both of the purines or both of the pyrimidines; 3 and 4 are the other two nucleotides. Thus 1 and 2 always differ by a transition; so do 3 and 4. Lake's invariants, expressed in terms of expected frequencies, are the three quantities:
(1) P(1133) + P(1234) - P(1134) - P(1233),
(2) P(1313) + P(1324) - P(1314) - P(1323),
(3) P(1331) + P(1342) - P(1341) - P(1332),
He showed that invariants (2) and (3) are zero under Topology I, (1) and (3) are zero under topology II, and (1) and (2) are zero under Topology III. If, for example, we see a site with pattern ACGC, we can start by setting 1=A. Then 2 must be G. We can then set 3=C (so that 4 is T). Thus its pattern type, making those substitutions, is 1323. P(1323) is the expected probability of the type of pattern which includes ACGC, TGAG, GTAT, etc.
Lake's invariants are easily tested with observed frequencies. For example, the first of them is a test of whether there are as many sites of types 1133 and 1234 as there are of types 1134 and 1233; this is easily tested with a chi-square test or, as in this program, with an exact binomial test. Note that with several invariants to test, we risk overestimating the significance of results if we simply accept the nominal 95% levels of significance (Li and Guoy, 1990).
Lake's invariants assume that each site is evolving independently, and that starting from any base a transversion is equally likely to end up at each of the two possible bases (thus, an A undergoing a transversion is equally likely to end up as a C or a T, and similarly for the other four bases from which one could start. Interestingly, Lake's results do not assume that rates of evolution are the same at all sites. The result that the total of 1133 and 1234 is expected to be the same as the total of 1134 and 1233 is unaffected by the fact that we may have aggregated the counts over classes of sites evolving at different rates.
Cavender's invariants (Cavender and Felsenstein, 1987) are for the case of a character with two states. In the nucleic acid case we can classify nucleotides into two states, R and Y (Purine and Pyrimidine) and then use the two-state results. Cavender starts, as before, with the pattern frequencies. Coding purines as R and pyrimidines as Y, the patterns types are RRRR, RRRY, and so on until YYYY, a total of 16 types. Cavender found quadratic functions of the expected frequencies of these 16 types that were expected to be zero under a given phylogeny, irrespective of branch lengths. Two invariants (called K and L) were found for each tree topology. The L invariants are particularly easy to understand. If we have the tree topology ((A,B),(C,D)), then in the case of two symmetric states, the event that A and B have the same state should be independent of whether C and D have the same state, as the events determining these happen in different parts of the tree. We can set up a contingency table:
C = D C =/= D
------------------------------
|
A = B | YYYY, YYRR, YYYR, YYRY,
| RRRR, RRYY RRYR, RRRY
|
A =/= B | YRYY, YRRR, YRYR, YRRY,
| RYYY, RYRR RYYR, RYRY
where "=/=" means "is not equal to." We expect that the events C = D and A = B will be independent. Cavender's L invariant for this tree topology is simply the negative of the crossproduct difference,
P(A=/=B and C=D) P(A=B and C=/=D) - P(A=B and C=D) P(A=/=B and C=/=D).
One of these L invariants is defined for each of the three tree topologies. They can obviously be tested simply by doing a chi-square test on the contingency table. The one corresponding to the correct topology should be statistically indistinguishable from zero. Again, there is a possible multiple tests problem if all three are tested at a nominal value of 95%.
The K invariants are differences between the L invariants. When one of the tables is expected to have crossproduct difference zero, the other two are expected to be nonzero, and also to be equal. So the difference of their crossproduct differences can be taken; this is the K invariant. It is not so easily tested.
The assumptions of Cavender's invariants are different from those of Lake's. One obviously need not assume anything about the frequencies of, or transitions among, the two different purines or the two different pyrimidines. However one does need to assume independent events at each site, and one needs to assume that the Y and R states are symmetric, that the probability per unit time that a Y changes into an R is the same as the probability that an R changes into a Y, so that we expect equal frequencies of the two states. There is also an assumption that all sites are changing between these two states at the same expected rate. This assumption is not needed for Lake's invariants, since expectations of sums are equal to sums of expectations, but for Cavender's it is, since products of expectations are not equal to expectations of products.
It is helpful to have both sorts of invariants available; with further work we may appreciate what other invaraints there are for various models of nucleic acid change.
The input data for Dnainvar is standard. The first line of the input file contains the number of species (which must always be 4 for this version of Dnainvar) and the number of sites.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
Nucleic acid sequence Invariants method, version 3.69 Settings for this run: W Sites weighted? No M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out the counts of patterns Yes 4 Print out the invariants Yes Y to accept these or type the letter for one to change |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The options W, M and 0 are the usual ones. They are described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
The output consists first (if option 1 is selected) of a reprinting of the input data, then (if option 2 is on) tables of observed pattern frequencies and pattern type frequencies. A table will be printed out, in alphabetic order AAAA through TTTT of all the patterns that appear among the sites and the number of times each appears. This table will be invaluable for computation of any other invariants. There follows another table, of pattern types, using the 1234 notation, in numerical order 1111 through 1234, of the number of times each type of pattern appears. In this computation all sites at which there are any ambiguities or deletions are omitted. Cavender's invariants could actually be computed from sites that have only Y or R ambiguities; this will be done in the next release of this program.
If option 3 is on the invariants are then printed out, together with their statistical tests. For Lake's invariants the two sums which are expected to be equal are printed out, and then the result of an one-tailed exact binomial test which tests whether the difference is expected to be this positive or more. The P level is given (but remember the multiple-tests problem!).
For Cavender's L invariants the contingency tables are given. Each is tested with a one-tailed chi-square test. It is possible that the expected numbers in some categories could be too small for valid use of this test; the program does not check for this. It is also possible that the chi-square could be significant but in the wrong direction; this is not tested in the current version of the program. To check it beware of a chi-square greater than 3.841 but with a positive invariant. The invariants themselves are computed, as the difference of cross-products. Their absolute magnitudes are not important, but which one is closest to zero may be indicative. Significantly nonzero invariants should be negative if the model is valid. The K invariants, which are simply differences among the L invariants, are also printed out without any test on them being conducted. Note that it is possible to use the bootstrap utility Seqboot to create multiple data sets, and from the output from summing all of these get the empirical variability of these quadratic invariants.
The constants that are defined at the beginning of the program include "maxsp", which must always be 4 and should not be changed.
The program is very fast, as it has rather little work to do; these methods are just a little bit beyond the reach of hand tabulation. Execution speed should never be a limiting factor.
In a future version I hope to allow for Y and R codes in the calculation of the Cavender invariants, and to check for significantly negative cross-product differences in them, which would indicate violation of the model. By then there should be more known about invariants for larger number of species, and any such advances will also be incorporated.
4 13 Alpha AACGTGGCCAAAT Beta AAGGTCGCCAAAC Gamma CATTTCGTCACAA Delta GGTATTTCGGCCT |
Nucleic acid sequence Invariants method, version 3.69
4 species, 13 sites
Name Sequences
---- ---------
Alpha AACGTGGCCA AAT
Beta ..G..C.... ..C
Gamma C.TT.C.T.. C.A
Delta GGTA.TT.GG CC.
Pattern Number of times
AAAC 1
AAAG 2
AACC 1
AACG 1
CCCG 1
CCTC 1
CGTT 1
GCCT 1
GGGT 1
GGTA 1
TCAT 1
TTTT 1
Symmetrized patterns (1, 2 = the two purines and 3, 4 = the two pyrimidines
or 1, 2 = the two pyrimidines and 3, 4 = the two purines)
1111 1
1112 2
1113 3
1121 1
1132 2
1133 1
1231 1
1322 1
1334 1
Tree topologies (unrooted):
I: ((Alpha,Beta),(Gamma,Delta))
II: ((Alpha,Gamma),(Beta,Delta))
III: ((Alpha,Delta),(Beta,Gamma))
Lake's linear invariants
(these are expected to be zero for the two incorrect tree topologies.
This is tested by testing the equality of the two parts
of each expression using a one-sided exact binomial test.
The null hypothesis is that the first part is no larger than the second.)
Tree Exact test P value Significant?
I 1 - 0 = 1 0.5000 no
II 0 - 0 = 0 1.0000 no
III 0 - 0 = 0 1.0000 no
Cavender's quadratic invariants (type L) using purines vs. pyrimidines
(these are expected to be zero, and thus have a nonsignificant
chi-square, for the correct tree topology)
They will be misled if there are substantially
different evolutionary rate between sites, or
different purine:pyrimidine ratios from 1:1.
Tree I:
Contingency Table
2 8
1 2
Quadratic invariant = 4.0
Chi-square = 0.23111 (not significant)
Tree II:
Contingency Table
1 5
1 6
Quadratic invariant = -1.0
Chi-square = 0.01407 (not significant)
Tree III:
Contingency Table
1 2
6 4
Quadratic invariant = 8.0
Chi-square = 0.66032 (not significant)
Cavender's quadratic invariants (type K) using purines vs. pyrimidines
(these are expected to be zero for the correct tree topology)
They will be misled if there are substantially
different evolutionary rate between sites, or
different purine:pyrimidine ratios from 1:1.
No statistical test is done on them here.
Tree I: -9.0
Tree II: 4.0
Tree III: 5.0
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program implements the maximum likelihood method for DNA sequences. The present version is faster than earlier versions of Dnaml. Details of the algorithm are published in the paper by Felsenstein and Churchill (1996). The model of base substitution allows the expected frequencies of the four bases to be unequal, allows the expected frequencies of transitions and transversions to be unequal, and has several ways of allowing different rates of evolution at different sites.
The assumptions of the present model are:
The ratio of the two purines in the purine replacement pool is the same as their ratio in the overall pool, and similarly for the pyrimidines.
The ratios of transitions to transversions can be set by the user. The substitution process can be diagrammed as follows: Suppose that you specified A, C, G, and T base frequencies of 0.24, 0.28, 0.27, and 0.21.
Purine pool: Pyrimidine pool:
| | | |
| 0.4706 A | | 0.5714 C |
| 0.5294 G | | 0.4286 T |
| (ratio is | | (ratio is |
| 0.24 : 0.27) | | 0.28 : 0.21) |
|_______________| |_______________|
Draw from the overall pool:
| |
| 0.24 A |
| 0.28 C |
| 0.27 G |
| 0.21 T |
|__________________|
Note that if the existing base is, say, an A, the first kind of event has a 0.4706 probability of "replacing" it by another A. The second kind of event has a 0.24 chance of replacing it by another A. This rather disconcerting model is used because it has nice mathematical properties that make likelihood calculations far easier. A closely similar, but not precisely identical model having different rates of transitions and transversions has been used by Hasegawa et. al. (1985b). The transition probability formulas for the current model were given (with my permission) by Kishino and Hasegawa (1989). Another explanation is available in the paper by Felsenstein and Churchill (1996).
Note the assumption that we are looking at all sites, including those that have not changed at all. It is important not to restrict attention to some sites based on whether or not they have changed; doing that would bias branch lengths by making them too long, and that in turn would cause the method to misinterpret the meaning of those sites that had changed.
This program uses a Hidden Markov Model (HMM) method of inferring different rates of evolution at different sites. This was described in a paper by me and Gary Churchill (1996). It allows us to specify to the program that there will be a number of different possible evolutionary rates, what the prior probabilities of occurrence of each is, and what the average length of a patch of sites all having the same rate is. The rates can also be chosen by the program to approximate a Gamma distribution of rates, or a Gamma distribution plus a class of invariant sites. The program computes the the likelihood by summing it over all possible assignments of rates to sites, weighting each by its prior probability of occurrence.
For example, if we have used the C and A options (described below) to specify that there are three possible rates of evolution, 1.0, 2.4, and 0.0, that the prior probabilities of a site having these rates are 0.4, 0.3, and 0.3, and that the average patch length (number of consecutive sites with the same rate) is 2.0, the program will sum the likelihood over all possibilities, but give less weight to those that (say) assign all sites to rate 2.4, or that fail to have consecutive sites that have the same rate.
The Hidden Markov Model framework for rate variation among sites was independently developed by Yang (1993, 1994, 1995). We have implemented a general scheme for a Hidden Markov Model of rates; we allow the rates and their prior probabilities to be specified arbitrarily by the user, or by a discrete approximation to a Gamma distribution of rates (Yang, 1995), or by a mixture of a Gamma distribution and a class of invariant sites.
This feature effectively removes the artificial assumption that all sites have the same rate, and also means that we need not know in advance the identities of the sites that have a particular rate of evolution.
Another layer of rate variation also is available. The user can assign categories of rates to each site (for example, we might want first, second, and third codon positions in a protein coding sequence to be three different categories). This is done with the categories input file and the C option. We then specify (using the menu) the relative rates of evolution of sites in the different categories. For example, we might specify that first, second, and third positions evolve at relative rates of 1.0, 0.8, and 2.7.
If both user-assigned rate categories and Hidden Markov Model rates are allowed, the program assumes that the actual rate at a site is the product of the user-assigned category rate and the Hidden Markov Model regional rate. (This may not always make perfect biological sense: it would be more natural to assume some upper bound to the rate, as we have discussed in the Felsenstein and Churchill paper). Nevertheless you may want to use both types of rate variation.
Subject to these assumptions, the program is a correct maximum likelihood method. The input is fairly standard, with one addition. As usual the first line of the file gives the number of species and the number of sites.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
Nucleic acid sequence Maximum Likelihood method, version 3.69 Settings for this run: U Search for best tree? Yes T Transition/transversion ratio: 2.0000 F Use empirical base frequencies? Yes C One category of sites? Yes R Rate variation among sites? constant rate W Sites weighted? No S Speedier but rougher analysis? Yes G Global rearrangements? No J Randomize input order of sequences? No. Use input order O Outgroup root? No, use as outgroup species 1 M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Write out trees onto tree file? Yes 5 Reconstruct hypothetical sequences? No Y to accept these or type the letter for one to change |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The options U, W, J, O, M, and 0 are the usual ones. They are described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
The T option in this program does not stand for Threshold, but instead is the Transition/transversion option. The user is prompted for a real number greater than 0.0, as the expected ratio of transitions to transversions. Note that this is not the ratio of the first to the second kinds of events, but the resulting expected ratio of transitions to transversions. The exact relationship between these two quantities depends on the frequencies in the base pools. The default value of the T parameter if you do not use the T option is 2.0.
The F (Frequencies) option is one which may save users much time. If you want to use the empirical frequencies of the bases, observed in the input sequences, as the base frequencies, you simply use the default setting of the F option. These empirical frequencies are not really the maximum likelihood estimates of the base frequencies, but they will often be close to those values (they are the maximum likelihood estimates under a "star" or "explosion" phylogeny). If you change the setting of the F option you will be prompted for the frequencies of the four bases. These must add to 1 and are to be typed on one line separated by blanks, not commas.
The R (Hidden Markov Model rates) option allows the user to approximate a Gamma distribution of rates among sites, or a Gamma distribution plus a class of invariant sites, or to specify how many categories of substitution rates there will be in a Hidden Markov Model of rate variation, and what are the rates and probabilities for each. By repeatedly selecting the R option one toggles among no rate variation, the Gamma, Gamma+I, and general HMM possibilities.
If you choose Gamma or Gamma+I the program will ask how many rate categories you want. If you have chosen Gamma+I, keep in mind that one rate category will be set aside for the invariant class and only the remaining ones used to approximate the Gamma distribution. For the approximation we do not use the quantile method of Yang (1995) but instead use a quadrature method using generalized Laguerre polynomials. This should give a good approximation to the Gamma distribution with as few as 5 or 6 categories.
In the Gamma and Gamma+I cases, the user will be asked to supply the coefficient of variation of the rate of substitution among sites. This is different from the parameters used by Nei and Jin (1990) but related to them: their parameter a is also known as "alpha", the shape parameter of the Gamma distribution. It is related to the coefficient of variation by
CV = 1 / a1/2
or
a = 1 / (CV)2
(their parameter b is absorbed here by the requirement that time is scaled so that the mean rate of evolution is 1 per unit time, which means that a = b). As we consider cases in which the rates are less variable we should set a larger and larger, as CV gets smaller and smaller.
If the user instead chooses the general Hidden Markov Model option, they are first asked how many HMM rate categories there will be (for the moment there is an upper limit of 9, which should not be restrictive). Then the program asks for the rates for each category. These rates are only meaningful relative to each other, so that rates 1.0, 2.0, and 2.4 have the exact same effect as rates 2.0, 4.0, and 4.8. Note that an HMM rate category can have rate of change 0, so that this allows us to take into account that there may be a category of sites that are invariant. Note that the run time of the program will be proportional to the number of HMM rate categories: twice as many categories means twice as long a run. Finally the program will ask for the probabilities of a random site falling into each of these regional rate categories. These probabilities must be nonnegative and sum to 1. The default for the program is one category, with rate 1.0 and probability 1.0 (actually the rate does not matter in that case).
If more than one HMM rate category is specified, then another option, A, becomes visible in the menu. This allows us to specify that we want to assume that sites that have the same HMM rate category are expected to be clustered so that there is autocorrelation of rates. The program asks for the value of the average patch length. This is an expected length of patches that have the same rate. If it is 1, the rates of successive sites will be independent. If it is, say, 10.25, then the chance of change to a new rate will be 1/10.25 after every site. However the "new rate" is randomly drawn from the mix of rates, and hence could even be the same. So the actual observed length of patches with the same rate will be a bit larger than 10.25. Note below that if you choose multiple patches, there will be an estimate in the output file as to which combination of rate categories contributed most to the likelihood.
Note that the autocorrelation scheme we use is somewhat different from Yang's (1995) autocorrelated Gamma distribution. I am unsure whether this difference is of any importance -- our scheme is chosen for the ease with which it can be implemented.
The C option allows user-defined rate categories. The user is prompted for the number of user-defined rates, and for the rates themselves, which cannot be negative but can be zero. These numbers, which must be nonnegative (some could be 0), are defined relative to each other, so that if rates for three categories are set to 1 : 3 : 2.5 this would have the same meaning as setting them to 2 : 6 : 5. The assignment of rates to sites is then made by reading a file whose default name is "categories". It should contain a string of digits 1 through 9. A new line or a blank can occur after any character in this string. Thus the categories file might look like this:
122231111122411155 1155333333444
With the current options R, A, and C the program has gained greatly in its ability to infer different rates at different sites and estimate phylogenies under a more realistic model. Note that Likelihood Ratio Tests can be used to test whether one combination of rates is significantly better than another, provided one rate scheme represents a restriction of another with fewer parameters. The number of parameters needed for rate variation is the number of regional rate categories, plus the number of user-defined rate categories less 2, plus one if the regional rate categories have a nonzero autocorrelation.
The G (global search) option causes, after the last species is added to the tree, each possible group to be removed and re-added. This improves the result, since the position of every species is reconsidered. It approximately triples the run-time of the program.
The User tree (option U) is read from a file whose default name is intree. The trees can be multifurcating.
If the U (user tree) option is chosen another option appears in the menu, the L option. If it is selected, it signals the program that it should take any branch lengths that are in the user tree and simply evaluate the likelihood of that tree, without further altering those branch lengths. This means that if some branches have lengths and others do not, the program will estimate the lengths of those that do not have lengths given in the user tree. Note that the program Retree can be used to add and remove lengths from a tree.
The U option can read a multifurcating tree. This allows us to test the hypothesis that a certain branch has zero length (we can also do this by using Retree to set the length of that branch to 0.0 when it is present in the tree). By doing a series of runs with different specified lengths for a branch we can plot a likelihood curve for its branch length while allowing all other branches to adjust their lengths to it. If all branches have lengths specified, none of them will be iterated. This is useful to allow a tree produced by another method to have its likelihood evaluated. The L option has no effect and does not appear in the menu if the U option is not used.
The W (Weights) option is invoked in the usual way, with only weights 0 and 1 allowed. It selects a set of sites to be analyzed, ignoring the others. The sites selected are those with weight 1. If the W option is not invoked, all sites are analyzed. The Weights (W) option takes the weights from a file whose default name is "weights". The weights follow the format described in the main documentation file.
The M (multiple data sets) option will ask you whether you want to use multiple sets of weights (from the weights file) or multiple data sets from the input file. The ability to use a single data set with multiple weights means that much less disk space will be used for this input data. The bootstrapping and jackknifing tool Seqboot has the ability to create a weights file with multiple weights. Note also that when we use multiple weights for bootstrapping we can also then maintain different rate categories for different sites in a meaningful way. If you use the multiple data sets option rather than multiple weights, you should not at the same time use the user-defined rate categories option (option C), because the user-defined rate categories could then be associated with the wrong sites. This is not a concern when the M option is used by using multiple weights.
The algorithm used for searching among trees is faster than it was in version 3.5, thanks to using a technique invented by David Swofford and J. S. Rogers. This involves not iterating most branch lengths on most trees when searching among tree topologies, This is of necessity a "quick-and-dirty" search but it saves much time. There is a menu option (option S) which can turn off this search and revert to the earlier search method which iterated branch lengths in all topologies. This will be substantially slower but will also be a bit more likely to find the tree topology of highest likelihood.
The output starts by giving the number of species, the number of sites, and the base frequencies for A, C, G, and T that have been specified. It then prints out the transition/transversion ratio that was specified or used by default. It also uses the base frequencies to compute the actual transition/transversion ratio implied by the parameter.
If the R (HMM rates) option is used a table of the relative rates of expected substitution at each category of sites is printed, as well as the probabilities of each of those rates.
There then follow the data sequences, if the user has selected the menu option to print them out, with the base sequences printed in groups of ten bases along the lines of the Genbank and EMBL formats. The trees found are printed as an unrooted tree topology (possibly rooted by outgroup if so requested). The internal nodes are numbered arbitrarily for the sake of identification. The number of trees evaluated so far and the log likelihood of the tree are also given. Note that the trees printed out have a trifurcation at the base. The branch lengths in the diagram are roughly proportional to the estimated branch lengths, except that very short branches are printed out at least three characters in length so that the connections can be seen.
A table is printed showing the length of each tree segment (in units of expected nucleotide substitutions per site), as well as (very) rough confidence limits on their lengths. If a confidence limit is negative, this indicates that rearrangement of the tree in that region is not excluded, while if both limits are positive, rearrangement is still not necessarily excluded because the variance calculation on which the confidence limits are based results in an underestimate, which makes the confidence limits too narrow.
In addition to the confidence limits, the program performs a crude Likelihood Ratio Test (LRT) for each branch of the tree. The program computes the ratio of likelihoods with and without this branch length forced to zero length. This is done by comparing the likelihoods changing only that branch length. A truly correct LRT would force that branch length to zero and also allow the other branch lengths to adjust to that. The result would be a likelihood ratio closer to 1. Therefore the present LRT will err on the side of being too significant. YOU ARE WARNED AGAINST TAKING IT TOO SERIOUSLY. If you want to get a better likelihood curve for a branch length you can do multiple runs with different prespecified lengths for that branch, as discussed above in the discussion of the L option.
One should also realize that if you are looking not at a previously-chosen branch but at all branches, that you are seeing the results of multiple tests. With 20 tests, one is expected to reach significance at the P = .05 level purely by chance. You should therefore use a much more conservative significance level, such as .05 divided by the number of tests. The significance of these tests is shown by printing asterisks next to the confidence interval on each branch length. It is important to keep in mind that both the confidence limits and the tests are very rough and approximate, and probably indicate more significance than they should. Nevertheless, maximum likelihood is one of the few methods that can give you any indication of its own error; most other methods simply fail to warn the user that there is any error! (In fact, whole philosophical schools of taxonomists exist whose main point seems to be that there isn't any error, that the "most parsimonious" tree is the best tree by definition and that's that).
The log likelihood printed out with the final tree can be used to perform various likelihood ratio tests. One can, for example, compare runs with different values of the expected transition/transversion ratio to determine which value is the maximum likelihood estimate, and what is the allowable range of values (using a likelihood ratio test, which you will find described in mathematical statistics books). One could also estimate the base frequencies in the same way. Both of these, particularly the latter, require multiple runs of the program to evaluate different possible values, and this might get expensive.
If the U (User Tree) option is used and more than one tree is supplied, and the program is not told to assume autocorrelation between the rates at different sites, the program also performs a statistical test of each of these trees against the one with highest likelihood. If there are two user trees, the test done is one which is due to Kishino and Hasegawa (1989), a version of a test originally introduced by Templeton (1983). In this implementation it uses the mean and variance of log-likelihood differences between trees, taken across sites. If the two trees' means are more than 1.96 standard deviations different then the trees are declared significantly different. This use of the empirical variance of log-likelihood differences is more robust and nonparametric than the classical likelihood ratio test, and may to some extent compensate for the any lack of realism in the model underlying this program.
If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. In the version used here the variances and covariances of the sum of log likelihoods across sites are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected log-likelihood, log-likelihoods for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the highest log-likelihood exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.
In either the KHT or the SH test the program prints out a table of the log-likelihoods of each tree, the differences of each from the highest one, the variance of that quantity as determined by the log-likelihood differences at individual sites, and a conclusion as to whether that tree is or is not significantly worse than the best one. However the test is not available if we assume that there is autocorrelation of rates at neighboring sites (option A) and is not done in those cases.
The branch lengths printed out are scaled in terms of expected numbers of substitutions, counting both transitions and transversions but not replacements of a base by itself, and scaled so that the average rate of change, averaged over all sites analyzed, is set to 1.0 if there are multiple categories of sites. This means that whether or not there are multiple categories of sites, the expected fraction of change for very small branches is equal to the branch length. Of course, when a branch is twice as long this does not mean that there will be twice as much net change expected along it, since some of the changes occur in the same site and overlie or even reverse each other. The branch length estimates here are in terms of the expected underlying numbers of changes. That means that a branch of length 0.26 is 26 times as long as one which would show a 1% difference between the nucleotide sequences at the beginning and end of the branch. But we would not expect the sequences at the beginning and end of the branch to be 26% different, as there would be some overlaying of changes.
Confidence limits on the branch lengths are also given. Of course a negative value of the branch length is meaningless, and a confidence limit overlapping zero simply means that the branch length is not necessarily significantly different from zero. Because of limitations of the numerical algorithm, branch length estimates of zero will often print out as small numbers such as 0.00001. If you see a branch length that small, it is really estimated to be of zero length. Note that versions 2.7 and earlier of this program printed out the branch lengths in terms of expected probability of change, so that they were scaled differently.
Another possible source of confusion is the existence of negative values for the log likelihood. This is not really a problem; the log likelihood is not a probability but the logarithm of a probability. When it is negative it simply means that the corresponding probability is less than one (since we are seeing its logarithm). The log likelihood is maximized by being made more positive: -30.23 is worse than -29.14.
At the end of the output, if the R option is in effect with multiple HMM rates, the program will print a list of what site categories contributed the most to the final likelihood. This combination of HMM rate categories need not have contributed a majority of the likelihood, just a plurality. Still, it will be helpful as a view of where the program infers that the higher and lower rates are. Note that the use in this calculation of the prior probabilities of different rates, and the average patch length, gives this inference a "smoothed" appearance: some other combination of rates might make a greater contribution to the likelihood, but be discounted because it conflicts with this prior information. See the example output below to see what this printout of rate categories looks like. A second list will also be printed out, showing for each site which rate accounted for the highest fraction of the likelihood. If the fraction of the likelihood accounted for is less than 95%, a dot is printed instead.
Option 3 in the menu controls whether the tree is printed out into the output file. This is on by default, and usually you will want to leave it this way. However for runs with multiple data sets such as bootstrapping runs, you will primarily be interested in the trees which are written onto the output tree file, rather than the trees printed on the output file. To keep the output file from becoming too large, it may be wisest to use option 3 to prevent trees being printed onto the output file.
Option 4 in the menu controls whether the tree estimated by the program is written onto a tree file. The default name of this output tree file is "outtree". If the U option is in effect, all the user-defined trees are written to the output tree file.
Option 5 in the menu controls whether ancestral states are estimated at each node in the tree. If it is in effect, a table of ancestral sequences is printed out (including the sequences in the tip species which are the input sequences). In that table, if a site has a base which accounts for more than 95% of the likelihood, it is printed in capital letters (A rather than a). If the best nucleotide accounts for less than 50% of the likelihood, the program prints out an ambiguity code (such as M for "A or C") for the set of nucleotides which, taken together, account for more than half of the likelihood. The ambiguity codes are listed in the sequence programs documentation file. One limitation of the current version of the program is that when there are multiple HMM rates (option R) the reconstructed nucleotides are based on only the single assignment of rates to sites which accounts for the largest amount of the likelihood. Thus the assessment of 95% of the likelihood, in tabulating the ancestral states, refers to 95% of the likelihood that is accounted for by that particular combination of rates.
The constants defined at the beginning of the program include "maxtrees", the maximum number of user trees that can be processed. It is small (100) at present to save some further memory but the cost of increasing it is not very great. Other constants include "maxcategories", the maximum number of site categories, "namelength", the length of species names in characters, and three others, "smoothings", "iterations", and "epsilon", that help "tune" the algorithm and define the compromise between execution speed and the quality of the branch lengths found by iteratively maximizing the likelihood. Reducing iterations and smoothings, and increasing epsilon, will result in faster execution but a worse result. These values will not usually have to be changed.
The program spends most of its time doing real arithmetic. The algorithm, with separate and independent computations occurring for each pattern, lends itself readily to parallel processing.
This program, which in version 2.6 replaced the old version of Dnaml, is not derived directly from it but instead was developed by modifying Contml, with which it shares many of its data structures and much of its strategy. It was speeded up by two major developments, the use of aliasing of nucleotide sites (version 3.1) and pretabulation of some exponentials (added by Akiko Fuseki in version 3.4). In version 3.5 the Hidden Markov Model code was added and the method of iterating branch lengths was changed from an EM algorithm to direct search. The Hidden Markov Model code slows things down, especially if there is autocorrelation between sites, so this version is slower than version 3.4. Nevertheless we hope that the sacrifice is worth it.
One change that is needed in the future is to put in some way of allowing for base composition of nucleotide sequences in different parts of the phylogeny.
5 13 Alpha AACGTGGCCAAAT Beta AAGGTCGCCAAAC Gamma CATTTCGTCACAA Delta GGTATTTCGGCCT Epsilon GGGATCTCGGCCC |
(It was run with HMM rates having gamma-distributed rates approximated by 5 rate categories, with coefficient of variation of rates 1.0, and with patch length parameter = 1.5. Two user-defined rate categories were used, one for the first 6 sites, the other for the last 7, with rates 1.0 : 2.0. Weights were used, with sites 1 and 13 given weight 0, and all others weight 1.)
Nucleic acid sequence Maximum Likelihood method, version 3.69
5 species, 13 sites
Site categories are:
1111112222 222
Sites are weighted as follows:
01111 11111 110
Name Sequences
---- ---------
Alpha AACGTGGCCA AAT
Beta ..G..C.... ..C
Gamma C.TT.C.T.. C.A
Delta GGTA.TT.GG CC.
Epsilon GGGA.CT.GG CCC
Empirical Base Frequencies:
A 0.23636
C 0.29091
G 0.25455
T(U) 0.21818
Transition/transversion ratio = 2.000000
Discrete approximation to gamma distributed rates
Coefficient of variation of rates = 1.000000 (alpha = 1.000000)
State in HMM Rate of change Probability
1 0.264 0.522
2 1.413 0.399
3 3.596 0.076
4 7.086 0.0036
5 12.641 0.000023
Expected length of a patch of sites having the same rate = 1.500
Site category Rate of change
1 1.000
2 2.000
+Beta
|
| +Epsilon
| +----------------------------------------------3
1---2 +-Delta
| |
| +--Gamma
|
+-Alpha
remember: this is an unrooted tree!
Ln Likelihood = -58.41388
Between And Length Approx. Confidence Limits
------- --- ------ ------- ---------- ------
1 Alpha 0.32320 ( zero, 0.93246) **
1 Beta 0.02699 ( zero, 0.49959)
1 2 0.65789 ( zero, 2.29501)
2 3 7.11637 ( zero, 20.73855) **
3 Epsilon 0.00006 ( zero, 0.52703)
3 Delta 0.30602 ( zero, 0.83268) **
2 Gamma 0.43465 ( zero, 2.10073)
* = significantly positive, P < 0.05
** = significantly positive, P < 0.01
Combination of categories that contributes the most to the likelihood:
1122121111 111
Most probable category at each site if > 0.95 probability ("." otherwise)
.......... ...
Probable sequences at interior nodes:
node Reconstructed sequence (caps if > 0.95)
1 .AgGTCGCCA AA.
Beta AAGGTCGCCA AAC
2 .AkkTcGtCA cA.
3 .GGATCTCGG CC.
Epsilon GGGATCTCGG CCC
Delta GGTATTTCGG CCT
Gamma CATTTCGTCA CAA
Alpha AACGTGGCCA AAT
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program implements the maximum likelihood method for DNA sequences under the constraint that the trees estimated must be consistent with a molecular clock. The molecular clock is the assumption that the tips of the tree are all equidistant, in branch length, from its root. This program is indirectly related to Dnaml. Details of the algorithm are not yet published, but many aspects of it are similar to Dnaml, and these are published in the paper by Felsenstein and Churchill (1996). The model of base substitution allows the expected frequencies of the four bases to be unequal, allows the expected frequencies of transitions and transversions to be unequal, and has several ways of allowing different rates of evolution at different sites.
The assumptions of the model are:
The ratio of the two purines in the purine replacement pool is the same as their ratio in the overall pool, and similarly for the pyrimidines.
The ratios of transitions to transversions can be set by the user. The substitution process can be diagrammed as follows: Suppose that you specified A, C, G, and T base frequencies of 0.24, 0.28, 0.27, and 0.21.
Purine pool: Pyrimidine pool:
| | | |
| 0.4706 A | | 0.5714 C |
| 0.5294 G | | 0.4286 T |
| (ratio is | | (ratio is |
| 0.24 : 0.27) | | 0.28 : 0.21) |
|_______________| |_______________|
Draw from the overall pool:
| |
| 0.24 A |
| 0.28 C |
| 0.27 G |
| 0.21 T |
|__________________|
Note that if the existing base is, say, an A, the first kind of event has a 0.4706 probability of "replacing" it by another A. The second kind of event has a 0.24 chance of replacing it by another A. This rather disconcerting model is used because it has nice mathematical properties that make likelihood calculations far easier. A closely similar, but not precisely identical model having different rates of transitions and transversions has been used by Hasegawa et. al. (1985b). The transition probability formulas for the current model were given (with my permission) by Kishino and Hasegawa (1989). Another explanation is available in the paper by Felsenstein and Churchill (1996).
Note the assumption that we are looking at all sites, including those that have not changed at all. It is important not to restrict attention to some sites based on whether or not they have changed; doing that would bias branch lengths by making them too long, and that in turn would cause the method to misinterpret the meaning of those sites that had changed.
This program uses a Hidden Markov Model (HMM) method of inferring different rates of evolution at different sites. This was described in a paper by me and Gary Churchill (1996). It allows us to specify to the program that there will be a number of different possible evolutionary rates, what the prior probabilities of occurrence of each is, and what the average length of a patch of sites all having the same rate is. The rates can also be chosen by the program to approximate a Gamma distribution of rates, or a Gamma distribution plus a class of invariant sites. The program computes the the likelihood by summing it over all possible assignments of rates to sites, weighting each by its prior probability of occurrence.
For example, if we have used the C and A options (described below) to specify that there are three possible rates of evolution, 1.0, 2.4, and 0.0, that the prior probabilities of a site having these rates are 0.4, 0.3, and 0.3, and that the average patch length (number of consecutive sites with the same rate) is 2.0, the program will sum the likelihood over all possibilities, but give less weight to those that (say) assign all sites to rate 2.4, or that fail to have consecutive sites that have the same rate.
The Hidden Markov Model framework for rate variation among sites was independently developed by Yang (1993, 1994, 1995). We have implemented a general scheme for a Hidden Markov Model of rates; we allow the rates and their prior probabilities to be specified arbitrarily by the user, or by a discrete approximation to a Gamma distribution of rates (Yang, 1995), or by a mixture of a Gamma distribution and a class of invariant sites.
This feature effectively removes the artificial assumption that all sites have the same rate, and also means that we need not know in advance the identities of the sites that have a particular rate of evolution.
Another layer of rate variation also is available. The user can assign categories of rates to each site (for example, we might want first, second, and third codon positions in a protein coding sequence to be three different categories). This is done with the categories input file and the C option. We then specify (using the menu) the relative rates of evolution of sites in the different categories. For example, we might specify that first, second, and third positions evolve at relative rates of 1.0, 0.8, and 2.7.
If both user-assigned rate categories and Hidden Markov Model rates are allowed, the program assumes that the actual rate at a site is the product of the user-assigned category rate and the Hidden Markov Model regional rate. (This may not always make perfect biological sense: it would be more natural to assume some upper bound to the rate, as we have discussed in the Felsenstein and Churchill paper). Nevertheless you may want to use both types of rate variation.
Subject to these assumptions, the program is a correct maximum likelihood method. The input is fairly standard, with one addition. As usual the first line of the file gives the number of species and the number of sites.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
Nucleic acid sequence Maximum Likelihood method with molecular clock, version 3.69 Settings for this run: U Search for best tree? Yes T Transition/transversion ratio: 2.0 F Use empirical base frequencies? Yes C One category of substitution rates? Yes R Rate variation among sites? constant rate G Global rearrangements? No W Sites weighted? No J Randomize input order of sequences? No. Use input order M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Write out trees onto tree file? Yes 5 Reconstruct hypothetical sequences? No Are these settings correct? (type Y or the letter for one to change) |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The options U, W, J, O, M, and 0 are the usual ones. They are described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
The T option in this program does not stand for Threshold, but instead is the Transition/transversion option. The user is prompted for a real number greater than 0.0, as the expected ratio of transitions to transversions. Note that this is not the ratio of the first to the second kinds of events, but the resulting expected ratio of transitions to transversions. The exact relationship between these two quantities depends on the frequencies in the base pools. The default value of the T parameter if you do not use the T option is 2.0.
The F (Frequencies) option is one which may save users much time. If you want to use the empirical frequencies of the bases, observed in the input sequences, as the base frequencies, you simply use the default setting of the F option. These empirical frequencies are not really the maximum likelihood estimates of the base frequencies, but they will often be close to those values (they are maximum likelihood estimates under a "star" or "explosion" phylogeny). If you change the setting of the F option you will be prompted for the frequencies of the four bases. These must add to 1 and are to be typed on one line separated by blanks, not commas.
The R (Hidden Markov Model rates) option allows the user to approximate a Gamma distribution of rates among sites, or a Gamma distribution plus a class of invariant sites, or to specify how many categories of substitution rates there will be in a Hidden Markov Model of rate variation, and what are the rates and probabilities for each. By repeatedly selecting the R option one toggles among no rate variation, the Gamma, Gamma+I, and general HMM possibilities.
If you choose Gamma or Gamma+I the program will ask how many rate categories you want. If you have chosen Gamma+I, keep in mind that one rate category will be set aside for the invariant class and only the remaining ones used to approximate the Gamma distribution. For the approximation we do not use the quantile method of Yang (1995) but instead use a quadrature method using generalized Laguerre polynomials. This should give a good approximation to the Gamma distribution with as few as 5 or 6 categories.
In the Gamma and Gamma+I cases, the user will be asked to supply the coefficient of variation of the rate of substitution among sites. This is different from the parameters used by Nei and Jin (1990) but related to them: their parameter a is also known as "alpha", the shape parameter of the Gamma distribution. It is related to the coefficient of variation by
CV = 1 / a1/2
or
a = 1 / (CV)2
(their parameter b is absorbed here by the requirement that time is scaled so that the mean rate of evolution is 1 per unit time, which means that a = b). As we consider cases in which the rates are less variable we should set a larger and larger, as CV gets smaller and smaller.
If the user instead chooses the general Hidden Markov Model option, they are first asked how many HMM rate categories there will be (for the moment there is an upper limit of 9, which should not be restrictive). Then the program asks for the rates for each category. These rates are only meaningful relative to each other, so that rates 1.0, 2.0, and 2.4 have the exact same effect as rates 2.0, 4.0, and 4.8. Note that an HMM rate category can have rate of change 0, so that this allows us to take into account that there may be a category of sites that are invariant. Note that the run time of the program will be proportional to the number of HMM rate categories: twice as many categories means twice as long a run. Finally the program will ask for the probabilities of a random site falling into each of these regional rate categories. These probabilities must be nonnegative and sum to 1. The default for the program is one category, with rate 1.0 and probability 1.0 (actually the rate does not matter in that case).
If more than one category is specified, then another option, A, becomes visible in the menu. This allows us to specify that we want to assume that sites that have the same HMM rate category are expected to be clustered so that there is autocorrelation of rates. The program asks for the value of the average patch length. This is an expected length of patches that have the same rate. If it is 1, the rates of successive sites will be independent. If it is, say, 10.25, then the chance of change to a new rate will be 1/10.25 after every site. However the "new rate" is randomly drawn from the mix of rates, and hence could even be the same. So the actual observed length of patches with the same rate will be a bit larger than 10.25. Note below that if you choose multiple patches, there will be an estimate in the output file as to which combination of rate categories contributed most to the likelihood.
Note that the autocorrelation scheme we use is somewhat different from Yang's (1995) autocorrelated Gamma distribution. I am unsure whether this difference is of any importance -- our scheme is chosen for the ease with which it can be implemented.
The C option allows user-defined rate categories. The user is prompted for the number of user-defined rates, and for the rates themselves, which cannot be negative but can be zero. These numbers, which must be nonnegative (some could be 0), are defined relative to each other, so that if rates for three categories are set to 1 : 3 : 2.5 this would have the same meaning as setting them to 2 : 6 : 5. The assignment of rates to sites is then made by reading a file whose default name is "categories". It should contain a string of digits 1 through 9. A new line or a blank can occur after any character in this string. Thus the categories file might look like this:
122231111122411155 1155333333444
With the current options R, A, and C the program has gained greatly in its ability to infer different rates at different sites and estimate phylogenies under a more realistic model. Note that Likelihood Ratio Tests can be used to test whether one combination of rates is significantly better than another, provided one rate scheme represents a restriction of another with fewer parameters. The number of parameters needed for rate variation is the number of regional rate categories, plus the number of user-defined rate categories less 2, plus one if the regional rate categories have a nonzero autocorrelation.
The G (global search) option causes, after the last species is added to the tree, each possible group to be removed and re-added. This improves the result, since the position of every species is reconsidered. It approximately triples the run-time of the program.
The User tree (option U) is read from a file whose default name is intree. The trees can be multifurcating. This allows us to test the hypothesis that a given branch has zero length.
If the U (user tree) option is chosen another option appears in the menu, the L option. If it is selected, it signals the program that it should take any branch lengths that are in the user tree and simply evaluate the likelihood of that tree, without further altering those branch lengths. In the case of a clock, if some branches have lengths and others do not, the program does not estimate the lengths of those that do not have lengths given in the user tree. If any of the branches do not have lengths, the program re-estimates the lengths of all of them. This is done because estimating some and not others is hard in the case of a clock.
The W (Weights) option is invoked in the usual way, with only weights 0 and 1 allowed. It selects a set of sites to be analyzed, ignoring the others. The sites selected are those with weight 1. If the W option is not invoked, all sites are analyzed. The Weights (W) option takes the weights from a file whose default name is "weights". The weights follow the format described in the main documentation file.
The M (multiple data sets) option will ask you whether you want to use multiple sets of weights (from the weights file) or multiple data sets from the input file. The ability to use a single data set with multiple weights means that much less disk space will be used for this input data. The bootstrapping and jackknifing tool Seqboot has the ability to create a weights file with multiple weights. Note also that when we use multiple weights for bootstrapping we can also then maintain different rate categories for different sites in a meaningful way. If you use the multiple data sets option rather than multiple weights, you should not at the same time use the user-defined rate categories option (option C), because the user-defined rate categories could then be associated with the wrong sites. This is not a concern when the M option is used by using multiple weights.
The algorithm used for searching among trees is faster than it was in version 3.5, thanks to using a technique invented by David Swofford and J. S. Rogers. This involves not iterating most branch lengths on most trees when searching among tree topologies, This is of necessity a "quick-and-dirty" search but it saves much time.
The output starts by giving the number of species, the number of sites, and the base frequencies for A, C, G, and T that have been specified. It then prints out the transition/transversion ratio that was specified or used by default. It also uses the base frequencies to compute the actual transition/transversion ratio implied by the parameter.
If the R (HMM rates) option is used a table of the relative rates of expected substitution at each category of sites is printed, as well as the probabilities of each of those rates.
There then follow the data sequences, if the user has selected the menu option to print them out, with the base sequences printed in groups of ten bases along the lines of the Genbank and EMBL formats. The trees found are printed as a rooted tree topology. The internal nodes are numbered arbitrarily for the sake of identification. The number of trees evaluated so far and the log likelihood of the tree are also given. The branch lengths in the diagram are roughly proportional to the estimated branch lengths, except that very short branches are printed out at least three characters in length so that the connections can be seen.
A table is printed showing the length of each tree segment, and the time (in units of expected nucleotide substitutions per site) of each fork in the tree, measured from the root of the tree. I have not attempted to include code for approximate confidence limits on branch points, as I have done for branch lengths in Dnaml, both because of the extreme crudeness of that test, and because the variation of times for different forks would be highly correlated.
The log likelihood printed out with the final tree can be used to perform various likelihood ratio tests. One can, for example, compare runs with different values of the expected transition/transversion ratio to determine which value is the maximum likelihood estimate, and what is the allowable range of values (using a likelihood ratio test, which you will find described in mathematical statistics books). One could also estimate the base frequencies in the same way. Both of these, particularly the latter, require multiple runs of the program to evaluate different possible values, and this might get expensive.
This program makes possible a (reasonably) legitimate statistical test of the molecular clock. To do such a test, run Dnaml and Dnamlk on the same data. If the trees obtained are of the same topology (when considered as unrooted), it is legitimate to compare their likelihoods by the likelihood ratio test. In Dnaml the likelihood has been computed by estimating 2n-3 branch lengths, if there are n tips on the tree. In Dnamlk it has been computed by estimating n-1 branching times (in effect, n-1 branch lengths). The difference in the number of parameters is (2n-3)-(n-1) = n-2. To perform the test take the difference in log likelihoods between the two runs (Dnaml should be the higher of the two, barring numerical iteration difficulties) and double it. Look this up on a chi-square distribution with n-2 degrees of freedom. If the result is significant, the log likelihood has been significantly increased by allowing all 2n-3 branch lengths to be estimated instead of just n-1, and molecular clock may be rejected.
If the U (User Tree) option is used and more than one tree is supplied, and the program is not told to assume autocorrelation between the rates at different sites, the program also performs a statistical test of each of these trees against the one with highest likelihood. If there are two user trees, the test done is one which is due to Kishino and Hasegawa (1989), a version of a test originally introduced by Templeton (1983). In this implementation it uses the mean and variance of log-likelihood differences between trees, taken across sites. If the two trees' means are more than 1.96 standard deviations different then the trees are declared significantly different. This use of the empirical variance of log-likelihood differences is more robust and nonparametric than the classical likelihood ratio test, and may to some extent compensate for the any lack of realism in the model underlying this program.
If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. In the version used here the variances and covariances of the sum of log likelihoods across sites are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected log-likelihood, log-likelihoods for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the highest log-likelihood exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.
In either the KHT or the SH test the program prints out a table of the log-likelihoods of each tree, the differences of each from the highest one, the variance of that quantity as determined by the log-likelihood differences at individual sites, and a conclusion as to whether that tree is or is not significantly worse than the best one. However the test is not available if we assume that there is autocorrelation of rates at neighboring sites (option A) and is not done in those cases.
The branch lengths printed out are scaled in terms of expected numbers of substitutions, counting both transitions and transversions but not replacements of a base by itself, and scaled so that the average rate of change, averaged over all sites analyzed, is set to 1.0 if there are multiple categories of sites. This means that whether or not there are multiple categories of sites, the expected fraction of change for very small branches is equal to the branch length. Of course, when a branch is twice as long this does not mean that there will be twice as much net change expected along it, since some of the changes occur in the same site and overlie or even reverse each other. The branch length estimates here are in terms of the expected underlying numbers of changes. That means that a branch of length 0.26 is 26 times as long as one which would show a 1% difference between the nucleotide sequences at the beginning and end of the branch. But we would not expect the sequences at the beginning and end of the branch to be 26% different, as there would be some overlaying of changes.
Because of limitations of the numerical algorithm, branch length estimates of zero will often print out as small numbers such as 0.00001. If you see a branch length that small, it is really estimated to be of zero length.
Another possible source of confusion is the existence of negative values for the log likelihood. This is not really a problem; the log likelihood is not a probability but the logarithm of a probability. When it is negative it simply means that the corresponding probability is less than one (since we are seeing its logarithm). The log likelihood is maximized by being made more positive: -30.23 is worse than -29.14.
At the end of the output, if the R option is in effect with multiple HMM rates, the program will print a list of what site categories contributed the most to the final likelihood. This combination of HMM rate categories need not have contributed a majority of the likelihood, just a plurality. Still, it will be helpful as a view of where the program infers that the higher and lower rates are. Note that the use in this calculation of the prior probabilities of different rates, and the average patch length, gives this inference a "smoothed" appearance: some other combination of rates might make a greater contribution to the likelihood, but be discounted because it conflicts with this prior information. See the example output below to see what this printout of rate categories looks like.
A second list will also be printed out, showing for each site which rate accounted for the highest fraction of the likelihood. If the fraction of the likelihood accounted for is less than 95%, a dot is printed instead.
Option 3 in the menu controls whether the tree is printed out into the output file. This is on by default, and usually you will want to leave it this way. However for runs with multiple data sets such as bootstrapping runs, you will primarily be interested in the trees which are written onto the output tree file, rather than the trees printed on the output file. To keep the output file from becoming too large, it may be wisest to use option 3 to prevent trees being printed onto the output file.
Option 4 in the menu controls whether the tree estimated by the program is written onto a tree file. The default name of this output tree file is "outtree". If the U option is in effect, all the user-defined trees are written to the output tree file.
Option 5 in the menu controls whether ancestral states are estimated at each node in the tree. If it is in effect, a table of ancestral sequences is printed out (including the sequences in the tip species which are the input sequences). In that table, if a site has a base which accounts for more than 95% of the likelihood, it is printed in capital letters (A rather than a). If the best nucleotide accounts for less than 50% of the likelihood, the program prints out an ambiguity code (such as M for "A or C") for the set of nucleotides which, taken together, account for more than half of the likelihood. The ambiguity codes are listed in the sequence programs documentation file. One limitation of the current version of the program is that when there are multiple HMM rates (option R) the reconstructed nucleotides are based on only the single assignment of rates to sites which accounts for the largest amount of the likelihood. Thus the assessment of 95% of the likelihood, in tabulating the ancestral states, refers to 95% of the likelihood that is accounted for by that particular combination of rates.
The constants defined at the beginning of the program include "maxtrees", the maximum number of user trees that can be processed. It is small (100) at present to save some further memory but the cost of increasing it is not very great. Other constants include "maxcategories", the maximum number of site categories, "namelength", the length of species names in characters, and three others, "smoothings", "iterations", and "epsilon", that help "tune" the algorithm and define the compromise between execution speed and the quality of the branch lengths found by iteratively maximizing the likelihood. Reducing iterations and smoothings, and increasing epsilon, will result in faster execution but a worse result. These values will not usually have to be changed.
The program spends most of its time doing real arithmetic. The algorithm, with separate and independent computations occurring for each pattern, lends itself readily to parallel processing.
This program was developed in 1989 by combining code from Dnapars and from Dnaml. It was speeded up by two major developments, the use of aliasing of nucleotide sites (version 3.1) and pretabulation of some exponentials (added by Akiko Fuseki in version 3.4). In version 3.5 the Hidden Markov Model code was added and the method of iterating branch lengths was changed from an EM algorithm to direct search. The Hidden Markov Model code slows things down, especially if there is autocorrelation between sites, so this version is slower than version 3.4. Nevertheless we hope that the sacrifice is worth it.
One change that is needed in the future is to put in some way of allowing for base composition of nucleotide sequences in different parts of the phylogeny.
5 13 Alpha AACGTGGCCAAAT Beta AAGGTCGCCAAAC Gamma CATTTCGTCACAA Delta GGTATTTCGGCCT Epsilon GGGATCTCGGCCC |
(It was run with HMM rates having gamma-distributed rates approximated by 5 rate categories, with coefficient of variation of rates 1.0, and with patch length parameter = 1.5. Two user-defined rate categories were used, one for the first 6 sites, the other for the last 7, with rates 1.0 : 2.0. Weights were used, with sites 1 and 13 given weight 0, and all others weight 1.)
Nucleic acid sequence
Maximum Likelihood method with molecular clock, version 3.69
5 species, 13 sites
Site categories are:
1111112222 222
Sites are weighted as follows:
01111 11111 110
Name Sequences
---- ---------
Alpha AACGTGGCCA AAT
Beta ..G..C.... ..C
Gamma C.TT.C.T.. C.A
Delta GGTA.TT.GG CC.
Epsilon GGGA.CT.GG CCC
Empirical Base Frequencies:
A 0.23636
C 0.29091
G 0.25455
T(U) 0.21818
Transition/transversion ratio = 2.000000
Discrete approximation to gamma distributed rates
Coefficient of variation of rates = 1.000000 (alpha = 1.000000)
State in HMM Rate of change Probability
1 0.264 0.522
2 1.413 0.399
3 3.596 0.076
4 7.086 0.0036
5 12.641 0.000023
Expected length of a patch of sites having the same rate = 1.500
Site category Rate of change
1 1.000
2 2.000
+-Epsilon
+---------------------------------------------------4
! +-Delta
--3
! +-------Gamma
+---------------------------------------------2
! +-Beta
+-----1
+-Alpha
Ln Likelihood = -58.51728
Ancestor Node Node Height Length
-------- ---- ---- ------ ------
root 3
3 4 4.14820 4.14820
4 Epsilon 4.29769 0.14949
4 Delta 4.29769 0.14949
3 2 3.67522 3.67522
2 Gamma 4.29769 0.62247
2 1 4.12429 0.44907
1 Beta 4.29769 0.17340
1 Alpha 4.29769 0.17340
Combination of categories that contributes the most to the likelihood:
1122121111 111
Most probable category at each site if > 0.95 probability ("." otherwise)
.......... ...
Probable sequences at interior nodes:
node Reconstructed sequence (caps if > 0.95)
3 .ayrtykcsr cm.
4 .GkaTctCgg Cc.
Epsilon GGGATCTCGG CCC
Delta GGTATTTCGG CCT
2 .AykTcgtcA ca.
Gamma CATTTCGTCA CAA
1 .AcgTcGCCA AA.
Beta AAGGTCGCCA AAC
Alpha AACGTGGCCA AAT
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
Dnamove is an interactive DNA parsimony program, inspired by Wayne Maddison and David and Wayne Maddison's marvellous program MacClade, which is written for Macintosh computers. Dnamove reads in a data set which is prepared in almost the same format as one for the DNA parsimony program Dnapars. It allows the user to choose an initial tree, and displays this tree on the screen. The user can look at different sites and the way the nucleotide states are distributed on that tree, given the most parsimonious reconstruction of state changes for that particular tree. The user then can specify how the tree is to be rearraranged, rerooted or written out to a file. By looking at different rearrangements of the tree the user can manually search for the most parsimonious tree, and can get a feel for how different sites are affected by changes in the tree topology.
This program uses graphic characters that show the tree to best advantage on some computer systems. Its graphic characters will work best on MSDOS systems or MSDOS windows in Windows, and to any system whose screen or terminals emulate ANSI standard terminals such as old Digital VT100 terminals, Telnet programs, or VT100-compatible windows in the X windowing system. For any other screen types, (such as Macintosh windows) there is a generic option which does not make use of screen graphics characters. The program will work well in those cases, but the tree it displays will look a bit uglier.
The input data file is set up almost identically to the data files for Dnapars. The code for nucleotide sequences is the standard one, as described in the molecular sequence programs document. The user trees are contained in the input tree file which is used for input of the starting tree (if desired). The output tree file is used for the final tree.
The user interaction starts with the program presenting a menu. The menu looks like this:
Interactive DNA parsimony, version 3.69 Settings for this run: O Outgroup root? No, use as outgroup species 1 W Sites weighted? No T Use Threshold parsimony? No, use ordinary parsimony I Input sequences interleaved? Yes U Initial tree (arbitrary, user, specify)? Arbitrary 0 Graphics type (IBM PC, ANSI, none)? ANSI S Width of terminal screen? 80 L Number of lines on screen? 24 Are these settings correct? (type Y or the letter for one to change) |
The O (Outgroup), W (Weights), T (Threshold), and 0 (Graphics type) options are the usual ones and are described in the main documentation file. The I (Interleaved) option is the usual one and is described in the main documentation file and the molecular sequences programs documentation file. The U (initial tree) option allows the user to choose whether the initial tree is to be arbitrary, interactively specified by the user, or read from a tree file. Typing U causes the program to change among the three possibilities in turn. I would recommend that for a first run, you allow the tree to be set up arbitrarily (the default), as the "specify" choice is difficult to use and the "user tree" choice requires that you have available a tree file with the tree topology of the initial tree, which must be a rooted tree. Its default name is intree. The program will ask you for its name if it looks for the input tree file and does not find one of this name. If you wish to set up some particular tree you can also do that by the rearrangement commands specified below.
The W (Weights) option allows only weights of 0 or 1.
The T (threshold) option allows a continuum of methods between parsimony and compatibility. Thresholds less than or equal to 1.0 do not have any meaning and should not be used: they will result in a tree dependent only on the input order of species and not at all on the data!
The S (Screen width) option allows the width in characters of the display to be adjusted when more than 80 characters can be displayed on the user's screen.
The L (screen Lines) option allows the user to change the height of the screen (in lines of characters) that is assumed to be available on the display. This may be particularly helpful when displaying large trees on terminals that have more than 24 lines per screen, or on workstation or X-terminal screens that can emulate the ANSI terminals with more than 24 lines.
After the initial menu is displayed and the choices are made, the program then sets up an initial tree and displays it. Below it will be a one-line menu of possible commands, which looks like this:
NEXT? (Options: R # + - S . T U W O F C H ? X Q) (H or ? for Help)
If you type H or ? you will get a single screen showing a description of each of these commands in a few words. Here are slightly more detailed descriptions:
As we have seen, the initial menu of the program allows you to choose among three screen types (PCDOS, Ansi, and none). If you want to avoid having to make this choice every time, you can change some of the constants in the file phylip.h to have the terminal type initialize itself in the proper way, and recompile. We have tried to have the default values be correct for PC, Macintosh, and Unix screens. If the setting is "none" (which is necessary on Macintosh MacOS 9 screens), the special graphics characters will not be used to indicate nucleotide states, but only letters will be used for the four nucleotides. This is less easy to look at.
The constants that need attention are ANSICRT and IBMCRT. Currently these are both set to "false" on Macintosh MacOS 9 systems, to "true" on MacOS X and on Unix/Linux systems, and IBMCRT is set to "true" on Windows systems. If your system has an ANSI compatible terminal, you might want to find the definition of ANSICRT in phylip.h and set it to "true", and IBMCRT to "false".
This program carries out unrooted parsimony (analogous to Wagner trees) (Eck and Dayhoff, 1966; Kluge and Farris, 1969) on DNA sequences. The method of Fitch (1971) is used to count the number of changes of base needed on a given tree. The assumptions of this method are exactly analogous to those of MIX:
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
Change from an occupied site to a deletion is counted as one change. Reversion from a deletion to an occupied site is allowed and is also counted as one change.
Below is a test data set, but we cannot show the output it generates because of the interactive nature of the program.
5 13 Alpha AACGUGGCCA AAU Beta AAGGUCGCCA AAC Gamma CAUUUCGUCA CAA Delta GGUAUUUCGG CCU Epsilon GGGAUCUCGG CCC |
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program carries out unrooted parsimony (analogous to Wagner trees) (Eck and Dayhoff, 1966; Kluge and Farris, 1969) on DNA sequences. The method of Fitch (1971) is used to count the number of changes of base needed on a given tree. The assumptions of this method are analogous to those of MIX:
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b, 1988), but also read the exchange between Felsenstein and Sober (1986).
Change from an occupied site to a deletion is counted as one change. Reversion from a deletion to an occupied site is allowed and is also counted as one change. Note that this in effect assumes that a deletion N bases long is N separate events.
Dnapars can handle both bifurcating and multifurcating trees. In doing its search for most parsimonious trees, it adds species not only by creating new forks in the middle of existing branches, but it also tries putting them at the end of new branches which are added to existing forks. Thus it searches among both bifurcating and multifurcating trees. If a branch in a tree does not have any characters which might change in that branch in the most parsimonious tree, it does not save that tree. Thus in any tree that results, a branch exists only if some character has a most parsimonious reconstruction that would involve change in that branch.
It also saves a number of trees tied for best (you can alter the number it saves using the V option in the menu). When rearranging trees, it tries rearrangements of all of the saved trees. This makes the algorithm slower than earlier versions of Dnapars.
The input data is standard. The first line of the input file contains the number of species and the number of sites.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
DNA parsimony algorithm, version 3.69 Setting for this run: U Search for best tree? Yes S Search option? More thorough search V Number of trees to save? 10000 J Randomize input order of sequences? No. Use input order O Outgroup root? No, use as outgroup species 1 T Use Threshold parsimony? No, use ordinary parsimony N Use Transversion parsimony? No, count all steps W Sites weighted? No M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Print out steps in each site No 5 Print sequences at all nodes of tree No 6 Write out trees onto tree file? Yes Y to accept these or type the letter for one to change |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The S (search) option controls how, and how much, rearrangement is done on the tied trees that are saved by the program. If the "More thorough search" option (the default) is chosen, the program will save multiple tied trees, without collapsing internal branches that have no evidence of change on them. It will subsequently rearrange on all parts of each of those trees. If the "Less thorough search" option is chosen, before saving, the program will collapse all branches that have no evidence that there is any change on that branch. This leads to less attempted rearrangement. If the "Rearrange on one best tree" option is chosen, only the first of the tied trees is used for rearrangement. This is faster but less thorough. If your trees are likely to have large multifurcations, do not use the default "More thorough search" option as it could result in too large a number of trees being saved.
The N option allows you to choose transversion parsimony, which counts only transversions (changes between one of the purines A or G and one of the pyrimidines C or T). This setting is turned off by default.
The Weights (W) option takes the weights from a file whose default name is "weights". The weights follow the format described in the main documentation file, with integer weights from 0 to 35 allowed by using the characters 0, 1, 2, ..., 9 and A, B, ... Z.
The User tree (option U) is read from a file whose default name is intree. The trees can be multifurcating. They must be preceded in the file by a line giving the number of trees in the file.
The options J, O, T, M, and 0 are the usual ones. They are described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
The M (multiple data sets option) will ask you whether you want to use multiple sets of weights (from the weights file) or multiple data sets. The ability to use a single data set with multiple weights means that much less disk space will be used for this input data. The bootstrapping and jackknifing tool Seqboot has the ability to create a weights file with multiple weights.
The O (outgroup) option will have no effect if the U (user-defined tree) option is in effect. The T (threshold) option allows a continuum of methods between parsimony and compatibility. Thresholds less than or equal to 1.0 do not have any meaning and should not be used: they will result in a tree dependent only on the input order of species and not at all on the data!
Output is standard: if option 1 is toggled on, the data is printed out, with the convention that "." means "the same as in the first species". Then comes a list of equally parsimonious trees. Each tree has branch lengths. These are computed using an algorithm published by Hochbaum and Pathria (1997) which I first heard of from Wayne Maddison who invented it independently of them. This algorithm averages the number of reconstructed changes of state over all sites over all possible most parsimonious placements of the changes of state among branches. Note that it does not correct in any way for multiple changes that overlay each other.
If option 2 is toggled on a table of the number of changes of state required in each character is also printed. If option 5 is toggled on, a table is printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. This is a reconstruction of the ancestral sequences in the tree. If you choose option 5, a menu item "." appears which gives you the opportunity to turn off dot-differencing so that complete ancestral sequences are shown. If the inferred state is a "?" or one of the IUB ambiguity symbols, there will be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand. A "?" in the reconstructed states means that in addition to one or more bases, a deletion may or may not be present. If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs.
If the U (User Tree) option is used and more than one tree is supplied, and the program is not told to assume autocorrelation between the rates at different sites, the program also performs a statistical test of each of these trees against the one with highest likelihood. If there are two user trees, this is a version of the test proposed by Alan Templeton (1983) and evaluated in a test case by me (1985a). It is closely parallel to a test using log likelihood differences due to Kishino and Hasegawa (1989) It uses the mean and variance of the differences in the number of steps between trees, taken across sites. If the two trees' means are more than 1.96 standard deviations different, then the trees are declared significantly different.
If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. In the version used here the variances and covariances of the sums of steps across sites are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected number of steps, numbers of steps for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the lowest number of steps exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.
In either the KHT or the SH test the program prints out a table of the number of steps for each tree, the differences of each from the lowest one, the variance of that quantity as determined by the differences of the numbers of steps at individual sites, and a conclusion as to whether that tree is or is not significantly worse than the best one.
Option 6 in the menu controls whether the tree estimated by the program is written onto a tree file. The default name of this output tree file is "outtree". If the U option is in effect, all the user-defined trees are written to the output tree file. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.
The program is a straightforward relative of MIX and runs reasonably quickly, especially with many sites and few species.
5 13 Alpha AACGUGGCCAAAU Beta AAGGUCGCCAAAC Gamma CAUUUCGUCACAA Delta GGUAUUUCGGCCU Epsilon GGGAUCUCGGCCC |
DNA parsimony algorithm, version 3.69
5 species, 13 sites
Name Sequences
---- ---------
Alpha AACGUGGCCA AAU
Beta ..G..C.... ..C
Gamma C.UU.C.U.. C.A
Delta GGUA.UU.GG CC.
Epsilon GGGA.CU.GG CCC
One most parsimonious tree found:
+-----Epsilon
+----------------------------3
+------------2 +-------Delta
| |
| +----------------Gamma
|
1----Beta
|
+---------Alpha
requires a total of 19.000
between and length
------- --- ------
1 2 0.217949
2 3 0.487179
3 Epsilon 0.096154
3 Delta 0.134615
2 Gamma 0.275641
1 Beta 0.076923
1 Alpha 0.173077
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 2 1 3 2 0 2 1 1 1
10| 1 1 1 3
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AABGTCGCCA AAY
1 2 yes V.KD...... C..
2 3 yes GG.A..T.GG .C.
3 Epsilon maybe ..G....... ..C
3 Delta yes ..T..T.... ..T
2 Gamma yes C.TT...T.. ..A
1 Beta maybe ..G....... ..C
1 Alpha yes ..C..G.... ..T
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
Dnapenny is a program that will find all of the most parsimonious trees implied by your data when the nucleic acid sequence parsimony criterion is employed. It does so not by examining all possible trees, but by using the more sophisticated "branch and bound" algorithm, a standard computer science search strategy first applied to phylogenetic inference by Hendy and Penny (1982). (J. S. Farris [personal communication, 1975] had also suggested that this strategy, which is well-known in computer science, might be applied to phylogenies, but he did not publish this suggestion).
There is, however, a price to be paid for the certainty that one has found all members of the set of most parsimonious trees. The problem of finding these has been shown (Graham and Foulds, 1982; Day, 1983) to be NP-complete, which is equivalent to saying that there is no fast algorithm that is guaranteed to solve the problem in all cases (for a discussion of NP-completeness, see the Scientific American article by Lewis and Papadimitriou, 1978). The result is that this program, despite its algorithmic sophistication, is VERY SLOW.
The program should be slower than the other tree-building programs in the package, but usable up to about ten species. Above this it will bog down rapidly, but exactly when depends on the data and on how much computer time you have. IT IS VERY IMPORTANT FOR YOU TO GET A FEEL FOR HOW LONG THE PROGRAM WILL TAKE ON YOUR DATA. This can be done by running it on subsets of the species, increasing the number of species in the run until you either are able to treat the full data set or know that the program will take unacceptably long on it. (Making a plot of the logarithm of run time against species number may help to project run times).
The search strategy used by Dnapenny starts by making a tree consisting of the first two species (the first three if the tree is to be unrooted). Then it tries to add the next species in all possible places (there are three of these). For each of the resulting trees it evaluates the number of base substitutions. It adds the next species to each of these, again in all possible spaces. If this process would continue it would simply generate all possible trees, of which there are a very large number even when the number of species is moderate (34,459,425 with 10 species). Actually it does not do this, because the trees are generated in a particular order and some of them are never generated.
This is because the order in which trees are generated is not quite as implied above, but is a "depth-first search". This means that first one adds the third species in the first possible place, then the fourth species in its first possible place, then the fifth and so on until the first possible tree has been produced. For each tree the number of steps is evaluated. Then one "backtracks" by trying the alternative placements of the last species. When these are exhausted one tries the next placement of the next-to-last species. The order of placement in a depth-first search is like this for a four-species case (parentheses enclose monophyletic groups):
Make tree of first two species: (A,B)
Add C in first place: ((A,B),C)
Add D in first place: (((A,D),B),C)
Add D in second place: ((A,(B,D)),C)
Add D in third place: (((A,B),D),C)
Add D in fourth place: ((A,B),(C,D))
Add D in fifth place: (((A,B),C),D)
Add C in second place: ((A,C),B)
Add D in first place: (((A,D),C),B)
Add D in second place: ((A,(C,D)),B)
Add D in third place: (((A,C),D),B)
Add D in fourth place: ((A,C),(B,D))
Add D in fifth place: (((A,C),B),D)
Add C in third place: (A,(B,C))
Add D in first place: ((A,D),(B,C))
Add D in second place: (A,((B,D),C))
Add D in third place: (A,(B,(C,D)))
Add D in fourth place: (A,((B,C),D))
Add D in fifth place: ((A,(B,C)),D)
Among these fifteen trees you will find all of the four-species rooted trees, each exactly once (the parentheses each enclose a monophyletic group). As displayed above, the backtracking depth-first search algorithm is just another way of producing all possible trees one at a time. The branch and bound algorithm consists of this with one change. As each tree is constructed, including the partial trees such as (A,(B,C)), its number of steps is evaluated. In addition a prediction is made as to how many steps will be added, at a minimum, as further species are added.
This is done by counting how many sites which are invariant in the data up to the most recent species added will ultimately show variation when further species are added. Thus if 20 sites vary among species A, B, and C and their root, and if tree ((A,C),B) requires 24 steps, then if there are 8 more sites which will be seen to vary when species D is added, we can immediately say that no matter how we add D, the resulting tree can have no less than 24 + 8 = 32 steps. The point of all this is that if a previously-found tree such as ((A,B),(C,D)) required only 30 steps, then we know that there is no point in even trying to add D to ((A,C),B). We have computed the bound that enables us to cut off a whole line of inquiry (in this case five trees) and avoid going down that particular branch any farther.
The branch-and-bound algorithm thus allows us to find all most parsimonious trees without generating all possible trees. How much of a saving this is depends strongly on the data. For very clean (nearly "Hennigian") data, it saves much time, but on very messy data it will still take a very long time.
The algorithm in the program differs from the one outlined here in some essential details: it investigates possibilities in the order of their apparent promise. This applies to the order of addition of species, and to the places where they are added to the tree. After the first two-species tree is constructed, the program tries adding each of the remaining species in turn, each in the best possible place it can find. Whichever of those species adds (at a minimum) the most additional steps is taken to be the one to be added next to the tree. When it is added, it is added in turn to places which cause the fewest additional steps to be added. This sounds a bit complex, but it is done with the intention of eliminating regions of the search of all possible trees as soon as possible, and lowering the bound on tree length as quickly as possible. This process of evaluating which species to add in which order goes on the first time the search makes a tree; thereafter it uses that order.
The program keeps a list of all the most parsimonious trees found so far. Whenever it finds one that has fewer losses than these, it clears out the list and restarts it with that tree. In the process the bound tightens and fewer possibilities need be investigated. At the end the list contains all the shortest trees. These are then printed out. It should be mentioned that the program Clique for finding all largest cliques also works by branch-and-bound. Both problems are NP-complete but for some reason Clique runs far faster. Although their worst-case behavior is bad for both programs, those worst cases occur far more frequently in parsimony problems than in compatibility problems.
Among the quantities available to be set from the menu of Dnapenny, two (howoften and howmany) are of particular importance. As Dnapenny goes along it will keep count of how many trees it has examined. Suppose that howoften is 100 and howmany is 1000, the default settings. Every time 100 trees have been examined, Dnapenny will print out a line saying how many multiples of 100 trees have now been examined, how many steps the most parsimonious tree found so far has, how many trees with that number of steps have been found, and a very rough estimate of what fraction of all trees have been looked at so far.
When the number of these multiples printed out reaches the number howmany (say 1000), the whole algorithm aborts and prints out that it has not found all most parsimonious trees, but prints out what is has gotten so far anyway. These trees need not be any of the most parsimonious trees: they are simply the most parsimonious ones found so far. By setting the product (howoften times howmany) large you can make the algorithm less likely to abort, but then you risk getting bogged down in a gigantic computation. You should adjust these constants so that the program cannot go beyond examining the number of trees you are reasonably willing to pay for (or wait for). In their initial setting the program will abort after looking at 100,000 trees. Obviously you may want to adjust howoften in order to get more or fewer lines of intermediate notice of how many trees have been looked at so far. Of course, in small cases you may never even reach the first multiple of howoften, and nothing will be printed out except some headings and then the final trees.
The indication of the approximate percentage of trees searched so far will be helpful in judging how much farther you would have to go to get the full search. Actually, since that fraction is the fraction of the set of all possible trees searched or ruled out so far, and since the search becomes progressively more efficient, the approximate fraction printed out will usually be an underestimate of how far along the program is, sometimes a serious underestimate.
A constant at the beginning of the program that affects the result is "maxtrees", which controls the maximum number of trees that can be stored. Thus if maxtrees is 25, and 32 most parsimonious trees are found, only the first 25 of these are stored and printed out. If maxtrees is increased, the program does not run any slower but requires a little more intermediate storage space. I recommend that maxtrees be kept as large as you can, provided you are willing to look at an output with that many trees on it! Initially, maxtrees is set to 100 in the distribution copy.
The counting of the length of trees is done by an algorithm nearly identical to the corresponding algorithms in Dnapars, and thus the remainder of this document will be nearly identical to the Dnapars document.
This program carries out unrooted parsimony (analogous to Wagner trees) (Eck and Dayhoff, 1966; Kluge and Farris, 1969) on DNA sequences. The method of Fitch (1971) is used to count the number of changes of base needed on a given tree. The assumptions of this method are exactly analogous to those of Dnapars:
Change from an occupied site to a deletion is counted as one change. Reversion from a deletion to an occupied site is allowed and is also counted as one change.
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
Change from an occupied site to a deletion is counted as one change. Reversion from a deletion to an occupied site is allowed and is also counted as one change. Note that this in effect assumes that a deletion N bases long is N separate events.
The input data is standard. The first line of the input file contains the number of species and the number of sites. If the Weights option is being used, there must also be a W in this first line to signal its presence. There are only two options requiring information to be present in the input file, W (Weights) and U (User tree). All options other than W (including U) are invoked using the menu.
Next come the species data. Each sequence starts on a new line, has a ten-character species name that must be blank-filled to be of that length, followed immediately by the species data in the one-letter code. The sequences must either be in the "interleaved" or "sequential" formats described in the Molecular Sequence Programs document. The I option selects between them. The sequences can have internal blanks in the sequence but there must be no extra blanks at the end of the terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
Penny algorithm for DNA, version 3.69 branch-and-bound to find all most parsimonious trees Settings for this run: H How many groups of 100 trees: 1000 F How often to report, in trees: 100 S Branch and bound is simple? Yes O Outgroup root? No, use as outgroup species 1 T Use Threshold parsimony? No, use ordinary parsimony W Sites weighted? No M Analyze multiple data sets? No I Input sequences interleaved? Yes 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Print out steps in each site No 5 Print sequences at all nodes of tree No 6 Write out trees onto tree file? Yes Are these settings correct? (type Y or the letter for one to change) |
The user either types "Y" (followed, of course, by a carriage-return) if the settings shown are to be accepted, or the letter or digit corresponding to an option that is to be changed.
The options O, T, W, M, and 0 are the usual ones. They are described in the main documentation file of this package. Option I is the same as in other molecular sequence programs and is described in the documentation file for the sequence programs.
The T (threshold) option allows a continuum of methods between parsimony and compatibility. Thresholds less than or equal to 1.0 do not have any meaning and should not be used: they will result in a tree dependent only on the input order of species and not at all on the data!
The W (Weights) option allows only weights of 0 or 1.
The options H, F, and S are not found in the other molecular sequence programs. H (How many) allows the user to set the quantity howmany, which we have already seen controls number of times that the program will report on its progress. F allows the user to set the quantity howoften, which sets how often it will report -- after scanning how many trees.
The S (Simple) option alters a step in Dnapenny which reconsiders the order in which species are added to the tree. Normally the decision as to what species to add to the tree next is made as the first tree is being constructed; that ordering of species is not altered subsequently. The S option causes it to be continually reconsidered. This will probably result in a substantial increase in run time, but on some data sets of intermediate messiness it may help. It is included in case it might prove of use on some data sets. The Simple option, in which the ordering is kept the same after being established by trying alternatives during the construction of the first tree, is the default. Continual reconsideration can be selected as an alternative.
Output is standard: if option 1 is toggled on, the data is printed out, with the convention that "." means "the same as in the first species". Then comes a list of equally parsimonious trees, and (if option 2 is toggled on) a table of the number of changes of state required in each character. If option 5 is toggled on, a table is printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. If the inferred state is a "?" or one of the IUB ambiguity symbols, there will be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand. A "?" in the reconstructed states means that in addition to one or more bases, a deletion may or may not be present. If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.
8 6
Alpha1 AAGAAG
Alpha2 AAGAAG
Beta1 AAGGGG
Beta2 AAGGGG
Gamma1 AGGAAG
Gamma2 AGGAAG
Delta GGAGGA
Epsilon GGAAAG
|
Penny algorithm for DNA, version 3.69
branch-and-bound to find all most parsimonious trees
8 species, 6 sites
Name Sequences
---- ---------
Alpha1 AAGAAG
Alpha2 ......
Beta1 ...GG.
Beta2 ...GG.
Gamma1 .G....
Gamma2 .G....
Delta GGAGGA
Epsilon GGA...
requires a total of 8.000
9 trees in all found
+--------------------Alpha1
!
! +-----------Alpha2
! !
1 +-----4 +--Epsilon
! ! ! +-----6
! ! ! ! +--Delta
! ! +--5
+--2 ! +--Gamma2
! +-----7
! +--Gamma1
!
! +--Beta2
+--------------3
+--Beta1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 2 no ......
2 4 no ......
4 Alpha2 no ......
4 5 yes .G....
5 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 7 no ......
7 Gamma2 no ......
7 Gamma1 no ......
2 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
+--------------------Alpha1
!
! +-----------Alpha2
! !
1 +-----4 +--------Gamma2
! ! ! !
! ! +--7 +--Epsilon
! ! ! +--6
+--2 +--5 +--Delta
! !
! +-----Gamma1
!
! +--Beta2
+--------------3
+--Beta1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 2 no ......
2 4 no ......
4 Alpha2 no ......
4 7 yes .G....
7 Gamma2 no ......
7 5 no ......
5 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 Gamma1 no ......
2 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
+--------------------Alpha1
!
! +-----------Alpha2
! !
1 +-----4 +-----Gamma2
! ! ! +--7
! ! ! ! ! +--Epsilon
! ! +--5 +--6
+--2 ! +--Delta
! !
! +--------Gamma1
!
! +--Beta2
+--------------3
+--Beta1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 2 no ......
2 4 no ......
4 Alpha2 no ......
4 5 yes .G....
5 7 no ......
7 Gamma2 no ......
7 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 Gamma1 no ......
2 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
+--------------------Alpha1
!
1 +-----------------Alpha2
! !
! ! +--------Gamma2
+--2 !
! +-----7 +--Epsilon
! ! ! +--6
! ! +--5 +--Delta
+--4 !
! +-----Gamma1
!
! +--Beta2
+-----------3
+--Beta1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 2 no ......
2 Alpha2 no ......
2 4 no ......
4 7 yes .G....
7 Gamma2 no ......
7 5 no ......
5 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 Gamma1 no ......
4 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
+--------------------Alpha1
!
! +-----------------Alpha2
1 !
! ! +--Epsilon
! ! +-----6
+--2 ! +--Delta
! +-----5
! ! ! +--Gamma2
! ! +-----7
+--4 +--Gamma1
!
! +--Beta2
+-----------3
+--Beta1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 2 no ......
2 Alpha2 no ......
2 4 no ......
4 5 yes .G....
5 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 7 no ......
7 Gamma2 no ......
7 Gamma1 no ......
4 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
+--------------------Alpha1
!
! +-----------------Alpha2
1 !
! ! +-----Gamma2
! ! +--7
+--2 ! ! +--Epsilon
! +-----5 +--6
! ! ! +--Delta
! ! !
+--4 +--------Gamma1
!
! +--Beta2
+-----------3
+--Beta1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 2 no ......
2 Alpha2 no ......
2 4 no ......
4 5 yes .G....
5 7 no ......
7 Gamma2 no ......
7 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 Gamma1 no ......
4 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
+--------------------Alpha1
!
! +-----Alpha2
1 +-----------2
! ! ! +--Beta2
! ! +--3
+--4 +--Beta1
!
! +--------Gamma2
! !
+--------7 +--Epsilon
! +--6
+--5 +--Delta
!
+-----Gamma1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 4 no ......
4 2 no ......
2 Alpha2 no ......
2 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
4 7 yes .G....
7 Gamma2 no ......
7 5 no ......
5 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 Gamma1 no ......
+--------------------Alpha1
!
! +-----Alpha2
1 +-----------2
! ! ! +--Beta2
! ! +--3
! ! +--Beta1
+--4
! +-----Gamma2
! +--7
! ! ! +--Epsilon
+--------5 +--6
! +--Delta
!
+--------Gamma1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 4 no ......
4 2 no ......
2 Alpha2 no ......
2 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
4 5 yes .G....
5 7 no ......
7 Gamma2 no ......
7 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 Gamma1 no ......
+--------------------Alpha1
!
! +-----Alpha2
1 +-----------2
! ! ! +--Beta2
! ! +--3
! ! +--Beta1
+--4
! +--Epsilon
! +-----6
! ! +--Delta
+--------5
! +--Gamma2
+-----7
+--Gamma1
remember: this is an unrooted tree!
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 AAGAAG
1 Alpha1 no ......
1 4 no ......
4 2 no ......
2 Alpha2 no ......
2 3 yes ...GG.
3 Beta2 no ......
3 Beta1 no ......
4 5 yes .G....
5 6 yes G.A...
6 Epsilon no ......
6 Delta yes ...GGA
5 7 no ......
7 Gamma2 no ......
7 Gamma1 no ......
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
Penny is a program that will find all of the most parsimonious trees implied by your data. It does so not by examining all possible trees, but by using the more sophisticated "branch and bound" algorithm, a standard computer science search strategy first applied to phylogenetic inference by Hendy and Penny (1982). (J. S. Farris [personal communication, 1975] had also suggested that this strategy, which is well-known in computer science, might be applied to phylogenies, but he did not publish this suggestion).
There is, however, a price to be paid for the certainty that one has found all members of the set of most parsimonious trees. The problem of finding these has been shown (Graham and Foulds, 1982; Day, 1983) to be NP-complete, which is equivalent to saying that there is no fast algorithm that is guaranteed to solve the problem in all cases (for a discussion of NP-completeness, see the Scientific American article by Lewis and Papadimitriou, 1978). The result is that this program, despite its algorithmic sophistication, is VERY SLOW.
The program should be slower than the other tree-building programs in the package, but useable up to about ten species. Above this it will bog down rapidly, but exactly when depends on the data and on how much computer time you have. IT IS VERY IMPORTANT FOR YOU TO GET A FEEL FOR HOW LONG THE PROGRAM WILL TAKE ON YOUR DATA. This can be done by running it on subsets of the species, increasing the number of species in the run until you either are able to treat the full data set or know that the program will take unacceptably long on it. (Making a plot of the logarithm of run time against species number may help to project run times).
The search strategy used by Penny starts by making a tree consisting of the first two species (the first three if the tree is to be unrooted). Then it tries to add the next species in all possible places (there are three of these). For each of the resulting trees it evaluates the number of steps. It adds the next species to each of these, again in all possible spaces. If this process would continue it would simply generate all possible trees, of which there are a very large number even when the number of species is moderate (34,459,425 with 10 species). Actually it does not do this, because the trees are generated in a particular order and some of them are never generated.
Actually the order in which trees are generated is not quite as implied above, but is a "depth-first search". This means that first one adds the third species in the first possible place, then the fourth species in its first possible place, then the fifth and so on until the first possible tree has been produced. Its number of steps is evaluated. Then one "backtracks" by trying the alternative placements of the last species. When these are exhausted one tries the next placement of the next-to-last species. The order of placement in a depth-first search is like this for a four-species case (parentheses enclose monophyletic groups):
Make tree of first two species (A,B)
Add C in first place ((A,B),C)
Add D in first place (((A,D),B),C)
Add D in second place ((A,(B,D)),C)
Add D in third place (((A,B),D),C)
Add D in fourth place ((A,B),(C,D))
Add D in fifth place (((A,B),C),D)
Add C in second place: ((A,C),B)
Add D in first place (((A,D),C),B)
Add D in second place ((A,(C,D)),B)
Add D in third place (((A,C),D),B)
Add D in fourth place ((A,C),(B,D))
Add D in fifth place (((A,C),B),D)
Add C in third place (A,(B,C))
Add D in first place ((A,D),(B,C))
Add D in second place (A,((B,D),C))
Add D in third place (A,(B,(C,D)))
Add D in fourth place (A,((B,C),D))
Add D in fifth place ((A,(B,C)),D)
Among these fifteen trees you will find all of the four-species rooted bifurcating trees, each exactly once (the parentheses each enclose a monophyletic group). As displayed above, the backtracking depth-first search algorithm is just another way of producing all possible trees one at a time. The branch and bound algorithm consists of this with one change. As each tree is constructed, including the partial trees such as (A,(B,C)), its number of steps is evaluated. In addition a prediction is made as to how many steps will be added, at a minimum, as further species are added.
This is done by counting how many binary characters which are invariant in the data up the species most recently added will ultimately show variation when further species are added. Thus if 20 characters vary among species A, B, and C and their root, and if tree ((A,C),B) requires 24 steps, then if there are 8 more characters which will be seen to vary when species D is added, we can immediately say that no matter how we add D, the resulting tree can have no less than 24 + 8 = 32 steps. The point of all this is that if a previously-found tree such as ((A,B),(C,D)) required only 30 steps, then we know that there is no point in even trying to add D to ((A,C),B). We have computed the bound that enables us to cut off a whole line of inquiry (in this case five trees) and avoid going down that particular branch any farther.
The branch-and-bound algorithm thus allows us to find all most parsimonious trees without generating all possible trees. How much of a saving this is depends strongly on the data. For very clean (nearly "Hennigian") data, it saves much time, but on very messy data it will still take a very long time.
The algorithm in the program differs from the one outlined here in some essential details: it investigates possibilities in the order of their apparent promise. This applies to the order of addition of species, and to the places where they are added to the tree. After the first two-species tree is constructed, the program tries adding each of the remaining species in turn, each in the best possible place it can find. Whichever of those species adds (at a minimum) the most additional steps is taken to be the one to be added next to the tree. When it is added, it is added in turn to places which cause the fewest additional steps to be added. This sounds a bit complex, but it is done with the intention of eliminating regions of the search of all possible trees as soon as possible, and lowering the bound on tree length as quickly as possible.
The program keeps a list of all the most parsimonious trees found so far. Whenever it finds one that has fewer steps than these, it clears out the list and restarts the list with that tree. In the process the bound tightens and fewer possibilities need be investigated. At the end the list contains all the shortest trees. These are then printed out. It should be mentioned that the program Clique for finding all largest cliques also works by branch-and-bound. Both problems are NP-complete but for some reason Clique runs far faster. Although their worst-case behavior is bad for both programs, those worst cases occur far more frequently in parsimony problems than in compatibility problems.
Among the quantities available to be set at the beginning of a run of Penny, two (howoften and howmany) are of particular importance. As Penny goes along it will keep count of how many trees it has examined. Suppose that howoften is 100 and howmany is 1000, the default settings. Every time 100 trees have been examined, Penny will print out a line saying how many multiples of 100 trees have now been examined, how many steps the most parsimonious tree found so far has, how many trees with that number of steps have been found, and a very rough estimate of what fraction of all trees have been looked at so far.
When the number of these multiples printed out reaches the number howmany (say 1000), the whole algorithm aborts and prints out that it has not found all most parsimonious trees, but prints out what is has got so far anyway. These trees need not be any of the most parsimonious trees: they are simply the most parsimonious ones found so far. By setting the product (howoften times howmany) large you can make the algorithm less likely to abort, but then you risk getting bogged down in a gigantic computation. You should adjust these constants so that the program cannot go beyond examining the number of trees you are reasonably willing to wait for. In their initial setting the program will abort after looking at 100,000 trees. Obviously you may want to adjust howoften in order to get more or fewer lines of intermediate notice of how many trees have been looked at so far. Of course, in small cases you may never even reach the first multiple of howoften and nothing will be printed out except some headings and then the final trees.
The indication of the approximate percentage of trees searched so far will be helpful in judging how much farther you would have to go to get the full search. Actually, since that fraction is the fraction of the set of all possible trees searched or ruled out so far, and since the search becomes progressively more efficient, the approximate fraction printed out will usually be an underestimate of how far along the program is, sometimes a serious underestimate.
A constant at the beginning of the program that affects the result is "maxtrees", which controls the maximum number of trees that can be stored. Thus if "maxtrees" is 25, and 32 most parsimonious trees are found, only the first 25 of these are stored and printed out. If "maxtrees" is increased, the program does not run any slower but requires a little more intermediate storage space. I recommend that "maxtrees" be kept as large as you can, provided you are willing to look at an output with that many trees on it! Initially, "maxtrees" is set to 100 in the distribution copy.
The counting of the length of trees is done by an algorithm nearly identical to the corresponding algorithms in Mix, and thus the remainder of this document will be nearly identical to the Mix document. Mix is a general parsimony program which carries out the Wagner and Camin-Sokal parsimony methods in mixture, where each character can have its method specified. The program defaults to carrying out Wagner parsimony.
The Camin-Sokal parsimony method explains the data by assuming that changes 0 --> 1 are allowed but not changes 1 --> 0. Wagner parsimony allows both kinds of changes. (This under the assumption that 0 is the ancestral state, though the program allows reassignment of the ancestral state, in which case we must reverse the state numbers 0 and 1 throughout this discussion). The criterion is to find the tree which requires the minimum number of changes. The Camin-Sokal method is due to Camin and Sokal (1965) and the Wagner method to Eck and Dayhoff (1966) and to Kluge and Farris (1969).
Here are the assumptions of these two methods:
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
The input for Penny is the standard input for discrete characters programs, described above in the documentation file for the discrete-characters programs. States "?", "P", and "B" are allowed.
The options are selected using a menu:
Penny algorithm, version 3.696 branch-and-bound to find all most parsimonious trees Settings for this run: X Use Mixed method? No P Parsimony method? Wagner F How often to report, in trees: 100 H How many groups of 100 trees: 1000 O Outgroup root? No, use as outgroup species 1 S Branch and bound is simple? Yes T Use Threshold parsimony? No, use ordinary parsimony A Use ancestral states in input file? No W Sites weighted? No M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Print out steps in each character No 5 Print states at all nodes of tree No 6 Write out trees onto tree file? Yes Are these settings correct? (type Y or the letter for one to change) |
The options X, O, T, A, and M are the usual miXed Methods, Outgroup, Threshold, Ancestral States, and Multiple Data Sets options. They are described in the Main documentation file and in the Discrete Characters Programs documentation file. The O option is only acted upon if the final tree is unrooted.
The option P toggles between the Camin-Sokal parsimony criterion and the Wagner parsimony criterion. Options F and H reset the variables howoften (F) and howmany (H). The user is prompted for the new values. By setting these larger the program will report its progress less often (howoften) and will run longer (howmany times howoften). These values default to 100 and 1000 which guarantees a search of 100,000 trees, but these can be changed. Note that option F in this program is not the Factors option available in some of the other programs in this section of the package.
The A (Ancestral states) option works in the usual way, described in the Discrete Characters Programs documentation file. If the A option is not used, then the program will assume 0 as the ancestral state for those characters following the Camin-Sokal method, and will assume that the ancestral state is unknown for those characters following Wagner parsimony. If any characters have unknown ancestral states, and if the resulting tree is rooted (even by outgroup), a table will be printed out showing the best guesses of which are the ancestral states in each character.
The S (Simple) option alters a step in Penny which reconsiders the order in which species are added to the tree. Normally the decision as to what species to add to the tree next is made as the first tree is being constructed; that ordering of species is not altered subsequently. The S option causes it to be continually reconsidered. This will probably result in a substantial increase in run time, but on some data sets of intermediate messiness it may help. It is included in case it might prove of use on some data sets. The Simple option, in which the ordering is kept the same after being established by trying alternatives during the construction of the first tree, is the default. Continual reconsideration can be selected as an alternative.
The F (Factors) option is not available in this program, as it would have no effect on the result even if that information were provided in the input file.
The final output is standard: a set of trees, which will be printed as rooted or unrooted depending on which is appropriate, and if the user elects to see them, tables of the number of changes of state required in each character. If the Wagner option is in force for a character, it may not be possible to unambiguously locate the places on the tree where the changes occur, as there may be multiple possibilities. A table is available to be printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. If the inferred state is a "?" there will be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand.
If the Camin-Sokal parsimony method (option C or S) is invoked and the A option is also used, then the program will infer, for any character whose ancestral state is unknown ("?") whether the ancestral state 0 or 1 will give the fewest state changes. If these are tied, then it may not be possible for the program to infer the state in the internal nodes, and these will all be printed as ".". If this has happened and you want to know more about the states at the internal nodes, you will find helpful to use Move to display the tree and examine its interior states, as the algorithm in Move shows all that can be known in this case about the interior states, including where there is and is not amibiguity. The algorithm in Penny gives up more easily on displaying these states.
If the A option is not used, then the program will assume 0 as the ancestral state for those characters following the Camin-Sokal method, and will assume that the ancestral state is unknown for those characters following Wagner parsimony. If any characters have unknown ancestral states, and if the resulting tree is rooted (even by outgroup), a table will be printed out showing the best guesses of which are the ancestral states in each character. You will find it useful to understand the difference between the Camin-Sokal parsimony criterion with unknown ancestral state and the Wagner parsimony criterion.
If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.
At the beginning of the program are a series of constants, which can be changed to help adapt the program to different computer systems. Two are the initial values of howmany and howoften, constants "often" and "many". Constant "maxtrees" is the maximum number of tied trees that will be stored.
7 6
Alpha1 110110
Alpha2 110110
Beta1 110000
Beta2 110000
Gamma1 100110
Delta 001001
Epsilon 001110
|
Penny algorithm, version 3.69
branch-and-bound to find all most parsimonious trees
7 species, 6 characters
Wagner parsimony method
Name Characters
---- ----------
Alpha1 11011 0
Alpha2 11011 0
Beta1 11000 0
Beta2 11000 0
Gamma1 10011 0
Delta 00100 1
Epsilon 00111 0
requires a total of 8.000
3 trees in all found
+-----------------Alpha1
!
! +--------Alpha2
--1 !
! +-----4 +--Epsilon
! ! ! +--6
! ! +--5 +--Delta
+--2 !
! +-----Gamma1
!
! +--Beta2
+-----------3
+--Beta1
remember: this is an unrooted tree!
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 11011 0
1 Alpha1 no ..... .
1 2 no ..... .
2 4 no ..... .
4 Alpha2 no ..... .
4 5 yes .0... .
5 6 yes 0.1.. .
6 Epsilon no ..... .
6 Delta yes ...00 1
5 Gamma1 no ..... .
2 3 yes ...00 .
3 Beta2 no ..... .
3 Beta1 no ..... .
+-----------------Alpha1
!
--1 +--------------Alpha2
! !
! ! +--Epsilon
+--2 +--6
! +-----5 +--Delta
! ! !
+--4 +-----Gamma1
!
! +--Beta2
+--------3
+--Beta1
remember: this is an unrooted tree!
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 11011 0
1 Alpha1 no ..... .
1 2 no ..... .
2 Alpha2 no ..... .
2 4 no ..... .
4 5 yes .0... .
5 6 yes 0.1.. .
6 Epsilon no ..... .
6 Delta yes ...00 1
5 Gamma1 no ..... .
4 3 yes ...00 .
3 Beta2 no ..... .
3 Beta1 no ..... .
+-----------------Alpha1
!
! +-----Alpha2
--1 +--------2
! ! ! +--Beta2
! ! +--3
+--4 +--Beta1
!
! +--Epsilon
! +--6
+--------5 +--Delta
!
+-----Gamma1
remember: this is an unrooted tree!
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 11011 0
1 Alpha1 no ..... .
1 4 no ..... .
4 2 no ..... .
2 Alpha2 no ..... .
2 3 yes ...00 .
3 Beta2 no ..... .
3 Beta1 no ..... .
4 5 yes .0... .
5 6 yes 0.1.. .
6 Epsilon no ..... .
6 Delta yes ...00 1
5 Gamma1 no ..... .
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program carries out the Dollo and polymorphism parsimony methods. The Dollo parsimony method was first suggested in print by Le Quesne (1974) and was first well-specified by Farris (1977). The method is named after Louis Dollo since he was one of the first to assert that in evolution it is harder to gain a complex feature than to lose it. The algorithm explains the presence of the state 1 by allowing up to one forward change 0-->1 and as many reversions 1-->0 as are necessary to explain the pattern of states seen. The program attempts to minimize the number of 1-->0 reversions necessary.
The assumptions of this method are in effect:
One problem can arise when using additive binary recoding to represent a multistate character as a series of two-state characters. Unlike the Camin-Sokal, Wagner, and Polymorphism methods, the Dollo method can reconstruct ancestral states which do not exist. An example is given in my 1979 paper. It will be necessary to check the output to make sure that this has not occurred.
The polymorphism parsimony method was first used by me, and the results published (without a clear specification of the method) by Inger (1967). The method was independently published by Farris (1978a) and by me (1979). The method assumes that we can explain the pattern of states by no more than one origination (0-->1) of state 1, followed by retention of polymorphism along as many segments of the tree as are necessary, followed by loss of state 0 or of state 1 where necessary. The program tries to minimize the total number of polymorphic characters, where each polymorphism is counted once for each segment of the tree in which it is retained.
The assumptions of the polymorphism parsimony method are in effect:
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
The input format is the standard one, with "?", "P", "B" states allowed. The options are selected using a menu:
Dollo and polymorphism parsimony algorithm, version 3.69 Settings for this run: U Search for best tree? Yes P Parsimony method? Dollo J Randomize input order of species? No. Use input order T Use Threshold parsimony? No, use ordinary parsimony A Use ancestral states in input file? No W Sites weighted? No M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Print out steps in each character No 5 Print states at all nodes of tree No 6 Write out trees onto tree file? Yes Are these settings correct? (type Y or the letter for one to change) |
The options U, J, T, A, and M are the usual User Tree, Jumble, Ancestral States, and Multiple Data Sets options, described either in the main documentation file or in the Discrete Characters Programs documentation file. The A (Ancestral States) option allows implementation of the unordered Dollo parsimony and unordered polymorphism parsimony methods which I have described elsewhere (1984b). When the A option is used the ancestor is not to be counted as one of the species. The O (outgroup) option is not available since the tree produced is already rooted. Since the Dollo and polymorphism methods produce a rooted tree, the user-defined trees required by the U option have two-way forks at each level.
The P (Parsimony Method) option is the one that toggles between polymorphism parsimony and Dollo parsimony. The program defaults to Dollo parsimony.
The T (Threshold) option has already been described in the Discrete Characters programs documentation file. Setting T at or below 1.0 but above 0 causes the criterion to become compatibility rather than polymorphism parsimony, although there is no advantage to using this program instead of MIX to do a compatibility method. Setting the threshold value higher brings about an intermediate between the Dollo or polymorphism parsimony methods and the compatibility method, so that there is some rationale for doing that. Since the Dollo and polymorphism methods produces a rooted tree, the user-defined trees required by the U option have two-way forks at each level.
Using a threshold value of 1.0 or lower, but above 0, one can obtain a rooted (or, if the A option is used with ancestral states of "?", unrooted) compatibility criterion, but there is no particular advantage to using this program for that instead of MIX. Higher threshold values are of course meaningful and provide intermediates between Dollo and compatibility methods.
The X (Mixed parsimony methods) option is not available in this program. The Factors option is also not available in this program, as it would have no effect on the result even if that information were provided in the input file.
Output is standard: a list of equally parsimonious trees, and, if the user selects menu option 4, a table of the numbers of reversions or retentions of polymorphism necessary in each character. If any of the ancestral states has been specified to be unknown, a table of reconstructed ancestral states is also provided. When reconstructing the placement of forward changes and reversions under the Dollo method, keep in mind that each polymorphic state in the input data will require one "last minute" reversion. This is included in the tabulated counts. Thus if we have both states 0 and 1 at a tip of the tree the program will assume that the lineage had state 1 up to the last minute, and then state 0 arose in that population by reversion, without loss of state 1.
If the user selects menu option 5, a table is printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. If the inferred state is a "?" there may be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand.
If the A option is used, then the program will infer, for any character whose ancestral state is unknown ("?") whether the ancestral state 0 or 1 will give the best tree. If these are tied, then it may not be possible for the program to infer the state in the internal nodes, and these will all be printed as ".". If this has happened and you want to know more about the states at the internal nodes, you will find it helpful to use Dolmove to display the tree and examine its interior states, as the algorithm in Dolmove shows all that can be known in this case about the interior states, including where there is and is not ambiguity. The algorithm in Dollop gives up more easily on displaying these states.
If the U (User Tree) option is used and more than one tree is supplied, the program also performs a statistical test of each of these trees against the best tree. This test is a version of the test proposed by Alan Templeton (1983), evaluated in a test case by me (1985a). It is closely parallel to a test using log likelihood differences invented by Kishino and Hasegawa (1989), and uses the mean and variance of step differences between trees, taken across characters. If the mean is more than 1.96 standard deviations different then the trees are declared significantly different. The program prints out a table of the steps for each tree, the differences of each from the highest one, the variance of that quantity as determined by the step differences at individual characters, and a conclusion as to whether that tree is or is not significantly worse than the best one. It is important to understand that the test assumes that all the binary characters are evolving independently, which is unlikely to be true for many suites of morphological characters.
If there are more than two trees, the test done is an extension of the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out that a correction for the number of trees was necessary, and they introduced a resampling method to make this correction. In the version used here the variances and covariances of the sums of steps across characters are computed for all pairs of trees. To test whether the difference between each tree and the best one is larger than could have been expected if they all had the same expected number of steps, numbers of steps for all trees are sampled with these covariances and equal means (Shimodaira and Hasegawa's "least favorable hypothesis"), and a P value is computed from the fraction of times the difference between the tree's value and the lowest number of steps exceeds that actually observed. Note that this sampling needs random numbers, and so the program will prompt the user for a random number seed if one has not already been supplied. With the two-tree KHT test no random numbers are used.
In either the KHT or the SH test the program prints out a table of the number of steps for each tree, the differences of each from the lowest one, the variance of that quantity as determined by the differences of the numbers of steps at individual characters, and a conclusion as to whether that tree is or is not significantly worse than the best one.
If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.
The algorithm is a fairly simple adaptation of the one used in the program Sokal, which was formerly in this package and has been superseded by Mix. It requires two passes through each tree to count the numbers of reversions.
5 6
Alpha 110110
Beta 110000
Gamma 100110
Delta 001001
Epsilon 001110
|
Dollo and polymorphism parsimony algorithm, version 3.69
Dollo parsimony method
5 species, 6 characters
Name Characters
---- ----------
Alpha 11011 0
Beta 11000 0
Gamma 10011 0
Delta 00100 1
Epsilon 00111 0
One most parsimonious tree found:
+-----------Delta
--3
! +--------Epsilon
+--4
! +-----Gamma
+--2
! +--Beta
+--1
+--Alpha
requires a total of 3.000
reversions in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 0 0 1 1 1 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
root 3 yes ..1.. .
3 Delta yes ..... 1
3 4 yes ...11 .
4 Epsilon no ..... .
4 2 yes 1.0.. .
2 Gamma no ..... .
2 1 yes .1... .
1 Beta yes ...00 .
1 Alpha no ..... .
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
Dolmove is an interactive parsimony program which uses the Dollo and Polymorphism parsimony criteria. It was inspired by Wayne Maddison and David Maddison's marvellous program MacClade, which was written for Macintosh computers. Dolmove reads in a data set which is prepared in almost the same format as one for the Dollo and polymorhism parsimony program Dollop. It allows the user to choose an initial tree, and displays this tree on the screen. The user can look at different characters and the way their states are distributed on that tree, given the most parsimonious reconstruction of state changes for that particular tree. The user then can specify how the tree is to be rearraranged, rerooted or written out to a file. By looking at different rearrangements of the tree the user can manually search for the most parsimonious tree, and can get a feel for how different characters are affected by changes in the tree topology.
This program is compatible with fewer computer systems than the other programs in PHYLIP. It can be adapted to PCDOS systems or to any system whose screen or terminals emulate DEC VT100 terminals (such as Telnet programs for logging in to remote computers over a TCP/IP network, VT100-compatible windows in the X windowing system, and any terminal compatible with ANSI standard terminals). For any other screen types, there is a generic option which does not make use of screen graphics characters to display the character states. This will be less effective, as the states will be less easy to see when displayed.
The input data file is set up almost identically to the input file for Dollop.
The user interaction starts with the program presenting a menu. The menu looks like this:
Interactive Dollo or polymorphism parsimony, version 3.69 Settings for this run: P Parsimony method? Dollo A Use ancestral states? No F Use factors information? No W Sites weighted? No T Use Threshold parsimony? No, use ordinary parsimony A Use ancestral states in input file? No U Initial tree (arbitrary, user, specify)? Arbitrary 0 Graphics type (IBM PC, ANSI, none)? ANSI L Number of lines on screen? 24 S Width of terminal screen? 80 Are these settings correct? (type Y or the letter for one to change) |
The P (Parsimony Method) option is the one that toggles between polymorphism parsimony and Dollo parsimony. The program defaults to Dollo parsimony.
The T (Threshold), F (Factors), A (Ancestors), and 0 (Graphics type) options are the usual ones and are described in the main documentation page and in the Discrete Characters Program documentation page.
The F (Factors) option is used to inform the program which groups of characters are to be counted together in computing the number of characters compatible with the tree. Thus if three binary characters are all factors of the same multistate character, the multistate character will be counted as compatible with the tree only if all three factors are compatible with it.
The X (miXed methods) option is not available in Dolmove.
The usual W (Weights) option is available in Dolmove. It allows integer weights up to 36, using the symbols 0-9 and A-Z. Increased weight on a step increases both the number of parsimony steps on the character and the contribution it makes to the number of compatibilities.
The T (threshold) option allows a continuum of methods between parsimony and compatibility. Thresholds less than or equal to 0 do not have any meaning and should not be used: they will result in a tree dependent only on the input order of species and not at all on the data!
The U (initial tree) option allows the user to choose whether the initial tree is to be arbitrary, interactively specified by the user, or read from a tree file. Typing U causes the program to change among the three possibilities in turn. I would recommend that for a first run, you allow the tree to be set up arbitrarily (the default), as the "specify" choice is difficult to use and the "user tree" choice requires that you have available a tree file with the tree topology of the initial tree. Its default name is intree. The program will ask you for its name if it looks for the input tree file and does not find one of this name. If you wish to set up some particular tree you can also do that by the rearrangement commands specified below.
The S (Screen width) option allows the width in characters of the display to be adjusted when more then 80 characters can be displayed on the user's screen.
The L (screen Lines) option allows the user to change the height of the screen (in lines of characters) that is assumed to be available on the display. This may be particularly helpful when displaying large trees on terminals that have more than 24 lines per screen, or on workstation or X-terminal screens that can emulate the ANSI terminals with more than 24 lines.
After the initial menu is displayed and the choices are made, the program then sets up an initial tree and displays it. Below it will be a one-line menu of possible commands, which looks like this:
NEXT? (Options: R # + - S . T U W O F C H ? X Q) (H or ? for Help)
If you type H or ? you will get a single screen showing a description of each of these commands in a few words. Here are slightly more detailed descriptions:
If the A option is used, then the program will infer, for any character whose ancestral state is unknown ("?") whether the ancestral state 0 or 1 will give the fewest changes (according to the criterion in use). If these are tied, then it may not be possible for the program to infer the state in the internal nodes, and many of these will be shown as "?". If the A option is not used, then the program will assume 0 as the ancestral state.
When reconstructing the placement of forward changes and reversions under the Dollo method, keep in mind that each polymorphic state in the input data will require one "last minute" reversion. This is included in the counts. Thus if we have both states 0 and 1 at a tip of the tree the program will assume that the lineage had state 1 up to the last minute, and then state 0 arose in that population by reversion, without loss of state 1.
When Dolmove calculates the number of characters compatible with the tree, it will take the F option into account and count the multistate characters as units, counting a character as compatible with the tree only when all of the binary characters corresponding to it are compatible with the tree.
As we have seen, the initial menu of the program allows you to choose among three screen types (PCDOS, Ansi, and none). If you want to avoid having to make this choice every time, you can change some of the constants in the file phylip.h to have the terminal type initialize itself in the proper way, and recompile. We have tried to have the default values be correct for PC, Macintosh, and Unix screens. If the setting is "none" (which is necessary on Macintosh MacOS 9 screens), the special graphics characters will not be used to indicate nucleotide states, but only letters will be used for the four nucleotides. This is less easy to look at.
The constants that need attention are ANSICRT and IBMCRT. Currently these are both set to "false" on Macintosh MacOS 9 systems, to "true" on MacOS X and on Unix/Linux systems, and IBMCRT is set to "true" on Windows systems. If your system has an ANSI compatible terminal, you might want to find the definition of ANSICRT in phylip.h and set it to "true", and IBMCRT to "false".
Dolmove uses as its numerical criterion the Dollo and polymorphism parsimony methods. The program defaults to carrying out Dollo parsimony.
The Dollo parsimony method was first suggested in print by Le Quesne (1974) and was first well-specified by Farris (1977). The method is named after Louis Dollo since he was one of the first to assert that in evolution it is harder to gain a complex feature than to lose it. The algorithm explains the presence of the state 1 by allowing up to one forward change 0-->1 and as many reversions 1-->0 as are necessary to explain the pattern of states seen. The program attempts to minimize the number of 1-->0 reversions necessary.
The assumptions of this method are in effect:
One problem can arise when using additive binary recoding to represent a multistate character as a series of two-state characters. Unlike the Camin-Sokal, Wagner, and Polymorphism methods, the Dollo method can reconstruct ancestral states which do not exist. An example is given in my 1979 paper. It will be necessary to check the output to make sure that this has not occurred.
The polymorphism parsimony method was first used by me, and the results published (without a clear specification of the method) by Inger (1967). The method was independently published by Farris (1978a) and by me (1979). The method assumes that we can explain the pattern of states by no more than one origination (0-->1) of state 1, followed by retention of polymorphism along as many segments of the tree as are necessary, followed by loss of state 0 or of state 1 where necessary. The program tries to minimize the total number of polymorphic characters, where each polymorphism is counted once for each segment of the tree in which it is retained.
The assumptions of the polymorphism parsimony method are in effect:
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
Below is a test data set, but we cannot show the output it generates because of the interactive nature of the program.
5 6
Alpha 110110
Beta 110000
Gamma 100110
Delta 001001
Epsilon 001110
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
Dolpenny is a program that will find all of the most parsimonious trees implied by your data when the Dollo or polymorphism parsimony criteria are employed. It does so not by examining all possible trees, but by using the more sophisticated "branch and bound" algorithm, a standard computer science search strategy first applied to phylogenetic inference by Hendy and Penny (1982). (J. S. Farris [personal communication, 1975] had also suggested that this strategy, which is well-known in computer science, might be applied to phylogenies, but he did not publish this suggestion).
There is, however, a price to be paid for the certainty that one has found all members of the set of most parsimonious trees. The problem of finding these has been shown (Graham and Foulds, 1982; Day, 1983) to be NP-complete, which is equivalent to saying that there is no fast algorithm that is guaranteed to solve the problem in all cases (for a discussion of NP-completeness, see the Scientific American article by Lewis and Papadimitriou, 1978). The result is that this program, despite its algorithmic sophistication, is VERY SLOW.
The program should be slower than the other tree-building programs in the package, but useable up to about ten species. Above this it will bog down rapidly, but exactly when depends on the data and on how much computer time you have (it may be more effective in the hands of someone who can let a microcomputer grind all night than for someone who has the "benefit" of paying for time on the campus mainframe computer). IT IS VERY IMPORTANT FOR YOU TO GET A FEEL FOR HOW LONG THE PROGRAM WILL TAKE ON YOUR DATA. This can be done by running it on subsets of the species, increasing the number of species in the run until you either are able to treat the full data set or know that the program will take unacceptably long on it. (Making a plot of the logarithm of run time against species number may help to project run times).
The search strategy used by Dolpenny starts by making a tree consisting of the first two species (the first three if the tree is to be unrooted). Then it tries to add the next species in all possible places (there are three of these). For each of the resulting trees it evaluates the number of losses. It adds the next species to each of these, again in all possible spaces. If this process would continue it would simply generate all possible trees, of which there are a very large number even when the number of species is moderate (34,459,425 with 10 species). Actually it does not do this, because the trees are generated in a particular order and some of them are never generated.
Actually the order in which trees are generated is not quite as implied above, but is a "depth-first search". This means that first one adds the third species in the first possible place, then the fourth species in its first possible place, then the fifth and so on until the first possible tree has been produced. Its number of steps is evaluated. Then one "backtracks" by trying the alternative placements of the last species. When these are exhausted one tries the next placement of the next-to-last species. The order of placement in a depth-first search is like this for a four-species case (parentheses enclose monophyletic groups):
Make tree of first two species (A,B)
Add C in first place ((A,B),C)
Add D in first place (((A,D),B),C)
Add D in second place ((A,(B,D)),C)
Add D in third place (((A,B),D),C)
Add D in fourth place ((A,B),(C,D))
Add D in fifth place (((A,B),C),D)
Add C in second place: ((A,C),B)
Add D in first place (((A,D),C),B)
Add D in second place ((A,(C,D)),B)
Add D in third place (((A,C),D),B)
Add D in fourth place ((A,C),(B,D))
Add D in fifth place (((A,C),B),D)
Add C in third place (A,(B,C))
Add D in first place ((A,D),(B,C))
Add D in second place (A,((B,D),C))
Add D in third place (A,(B,(C,D)))
Add D in fourth place (A,((B,C),D))
Add D in fifth place ((A,(B,C)),D)
Among these fifteen trees you will find all of the four-species rooted bifurcating trees, each exactly once (the parentheses each enclose a monophyletic group). As displayed above, the backtracking depth-first search algorithm is just another way of producing all possible trees one at a time. The branch and bound algorithm consists of this with one change. As each tree is constructed, including the partial trees such as (A,(B,C)), its number of losses (or retentions of polymorphism) is evaluated.
The point of this is that if a previously-found tree such as ((A,B),(C,D)) required fewer losses, then we know that there is no point in even trying to add D to ((A,C),B). We have computed the bound that enables us to cut off a whole line of inquiry (in this case five trees) and avoid going down that particular branch any farther.
The branch-and-bound algorithm thus allows us to find all most parsimonious trees without generating all possible trees. How much of a saving this is depends strongly on the data. For very clean (nearly "Hennigian") data, it saves much time, but on very messy data it will still take a very long time.
The algorithm in the program differs from the one outlined here in some essential details: it investigates possibilities in the order of their apparent promise. This applies to the order of addition of species, and to the places where they are added to the tree. After the first two-species tree is constructed, the program tries adding each of the remaining species in turn, each in the best possible place it can find. Whichever of those species adds (at a minimum) the most additional steps is taken to be the one to be added next to the tree. When it is added, it is added in turn to places which cause the fewest additional steps to be added. This sounds a bit complex, but it is done with the intention of eliminating regions of the search of all possible trees as soon as possible, and lowering the bound on tree length as quickly as possible.
The program keeps a list of all the most parsimonious trees found so far. Whenever it finds one that has fewer losses than these, it clears out the list and restarts the list with that tree. In the process the bound tightens and fewer possibilities need be investigated. At the end the list contains all the shortest trees. These are then printed out. It should be mentioned that the program Clique for finding all largest cliques also works by branch-and-bound. Both problems are NP-complete but for some reason Clique runs far faster. Although their worst-case behavior is bad for both programs, those worst cases occur far more frequently in parsimony problems than in compatibility problems.
Among the quantities available to be set at the beginning of a run of Dolpenny, two (howoften and howmany) are of particular importance. As Dolpenny goes along it will keep count of how many trees it has examined. Suppose that howoften is 100 and howmany is 300, the default settings. Every time 100 trees have been examined, Dolpenny will print out a line saying how many multiples of 100 trees have now been examined, how many steps the most parsimonious tree found so far has, how many trees of with that number of steps have been found, and a very rough estimate of what fraction of all trees have been looked at so far.
When the number of these multiples printed out reaches the number howmany (say 1000), the whole algorithm aborts and prints out that it has not found all most parsimonious trees, but prints out what is has got so far anyway. These trees need not be any of the most parsimonious trees: they are simply the most parsimonious ones found so far. By setting the product (howoften X howmany) large you can make the algorithm less likely to abort, but then you risk getting bogged down in a gigantic computation. You should adjust these constants so that the program cannot go beyond examining the number of trees you are reasonably willing to pay for (or wait for). In their initial setting the program will abort after looking at 100,000 trees. Obviously you may want to adjust howoften in order to get more or fewer lines of intermediate notice of how many trees have been looked at so far. Of course, in small cases you may never even reach the first multiple of howoften and nothing will be printed out except some headings and then the final trees.
The indication of the approximate percentage of trees searched so far will be helpful in judging how much farther you would have to go to get the full search. Actually, since that fraction is the fraction of the set of all possible trees searched or ruled out so far, and since the search becomes progressively more efficient, the approximate fraction printed out will usually be an underestimate of how far along the program is, sometimes a serious underestimate.
A constant that affects the result is "maxtrees", which controls the maximum number of trees that can be stored. Thus if "maxtrees" is 25, and 32 most parsimonious trees are found, only the first 25 of these are stored and printed out. If "maxtrees" is increased, the program does not run any slower but requires a little more intermediate storage space. I recommend that "maxtrees" be kept as large as you can, provided you are willing to look at an output with that many trees on it! Initially, "maxtrees" is set to 100 in the distribution copy.
The counting of the length of trees is done by an algorithm nearly identical to the corresponding algorithms in Dollop, and thus the remainder of this document will be nearly identical to the Dollop document. The Dollo parsimony method was first suggested in print by Le Quesne (1974) and was first well-specified by Farris (1977). The method is named after Louis Dollo since he was one of the first to assert that in evolution it is harder to gain a complex feature than to lose it. The algorithm explains the presence of the state 1 by allowing up to one forward change 0-->1 and as many reversions 1-->0 as are necessary to explain the pattern of states seen. The program attempts to minimize the number of 1-->0 reversions necessary.
The assumptions of this method are in effect:
That these are the assumptions is established in several of my papers (1973a, 1978b, 1979, 1981b, 1983). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
One problem can arise when using additive binary recoding to represent a multistate character as a series of two-state characters. Unlike the Camin-Sokal, Wagner, and Polymorphism methods, the Dollo method can reconstruct ancestral states which do not exist. An example is given in my 1979 paper. It will be necessary to check the output to make sure that this has not occurred.
The polymorphism parsimony method was first used by me, and the results published (without a clear specification of the method) by Inger (1967). The method was published by Farris (1978a) and by me (1979). The method assumes that we can explain the pattern of states by no more than one origination (0-->1) of state 1, followed by retention of polymorphism along as many segments of the tree as are necessary, followed by loss of state 0 or of state 1 where necessary. The program tries to minimize the total number of polymorphic characters, where each polymorphism is counted once for each segment of the tree in which it is retained.
The assumptions of the polymorphism parsimony method are in effect:
That these are the assumptions of parsimony methods has been documented in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For an opposing view arguing that the parsimony methods make no substantive assumptions such as these, see the papers by Farris (1983) and Sober (1983a, 1983b), but also read the exchange between Felsenstein and Sober (1986).
The input format is the standard one, with "?", "P", "B" states allowed. Most of the options are selected using a menu:
Penny algorithm for Dollo or polymorphism parsimony, version 3.69 branch-and-bound to find all most parsimonious trees Settings for this run: P Parsimony method? Dollo H How many groups of 100 trees: 1000 F How often to report, in trees: 100 S Branch and bound is simple? Yes T Use Threshold parsimony? No, use ordinary parsimony A Use ancestral states? No W Sites weighted? No M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Print out steps in each character No 5 Print states at all nodes of tree No 6 Write out trees onto tree file? Yes Are these settings correct? (type Y or the letter for one to change) |
The P option toggles between the Polymorphism parsimony method and the default Dollo parsimony method.
The options T, A, and M are the usual Threshold, Ancestral States, and Multiple Data Sets options. They are described in the Main documentation file and in the Discrete Characters Programs documentation file.
Options F and H reset the variables howoften (F) and howmany (H). The user is prompted for the new values. By setting these larger the program will report its progress less often (howoften) and will run longer (howmany times howoften). These values default to 100 and 1000 which guarantees a search of 100,000 trees, but these can be changed. Note that option F in this program is not the Factors option available in some of the other programs in this section of the package.
The use of the A option allows implementation of the unordered Dollo parsimony and unordered polymorphism parsimony methods which I have described elsewhere (1984b). When the A option is used the ancestor is not to be counted as one of the species. The O (outgroup) option is not available since the tree produced is already rooted.
Setting T at or below 1.0 but above 0 causes the criterion to become compatibility rather than polymorphism parsimony, although there is no advantage to using this program instead of Penny to do a compatibility method. Setting the threshold value higher brings about an intermediate between the Dollo or polymorphism parsimony methods and the compatibility method, so that there is some rationale for doing that.
Using a threshold value of 1.0 or lower, but above 0, one can obtain a rooted (or, if the A option is used with ancestral states of "?", unrooted) compatibility criterion, but there is no particular advantage to using this program for that instead of MIX. Higher threshold values are of course meaningful and provide intermediates between Dollo and compatibility methods.
The S (Simple) option alters a step in Dolpenny which reconsiders the order in which species are added to the tree. Normally the decision as to what species to add to the tree next is made as the first tree is being constructucted; that ordering of species is not altered subsequently. The R option causes it to be continually reconsidered. This will probably result in a substantial increase in run time, but on some data sets of intermediate messiness it may help. It is included in case it might prove of use on some data sets. The Simple option, in which the ordering is kept the same after being established by trying alternatives during the construction of the first tree, is the default. Continual reconsideration can be selected as an alternative.
The Factors option is not available in this program, as it would have no effect on the result even if that information were provided in the input file.
The output format is also standard. It includes a rooted tree and, if the user selects option 4, a table of the numbers of reversions or retentions of polymorphism necessary in each character. If any of the ancestral states has been specified to be unknown, a table of reconstructed ancestral states is also provided. When reconstructing the placement of forward changes and reversions under the Dollo method, keep in mind that each polymorphic state in the input data will require one "last minute" reversion. This is included in the tabulated counts. Thus if we have both states 0 and 1 at a tip of the tree the program will assume that the lineage had state 1 up to the last minute, and then state 0 arose in that population by reversion, without loss of state 1.
A table is available to be printed out after each tree, showing for each branch whether there are known to be changes in the branch, and what the states are inferred to have been at the top end of the branch. If the inferred state is a "?" there will be multiple equally-parsimonious assignments of states; the user must work these out for themselves by hand.
If the A option is used, then the program will infer, for any character whose ancestral state is unknown ("?") whether the ancestral state 0 or 1 will give the best tree. If these are tied, then it may not be possible for the program to infer the state in the internal nodes, and these will all be printed as ".". If this has happened and you want to know more about the states at the internal nodes, you will find it helpful to use Dolmove to display the tree and examine its interior states, as the algorithm in Dolmove shows all that can be known in this case about the interior states, including where there is and is not ambiguity. The algorithm in Dolpenny gives up more easily on displaying these states.
If option 6 is left in its default state the trees found will be written to a tree file, so that they are available to be used in other programs. If the program finds multiple trees tied for best, all of these are written out onto the output tree file. Each is followed by a numerical weight in square brackets (such as [0.25000]). This is needed when we use the trees to make a consensus tree of the results of bootstrapping or jackknifing, to avoid overrepresenting replicates that find many tied trees.
At the beginning of the program are a series of constants, which can be changed to help adapt the program to different computer systems. Two are the initial values of howmany and howoften, constants "often" and "many". Constant "maxtrees" is the maximum number of tied trees that will be stored.
7 6
Alpha1 110110
Alpha2 110110
Beta1 110000
Beta2 110000
Gamma1 100110
Delta 001001
Epsilon 001110
|
Penny algorithm for Dollo or polymorphism parsimony, version 3.69
branch-and-bound to find all most parsimonious trees
7 species, 6 characters
Dollo parsimony method
Name Characters
---- ----------
Alpha1 11011 0
Alpha2 11011 0
Beta1 11000 0
Beta2 11000 0
Gamma1 10011 0
Delta 00100 1
Epsilon 00111 0
requires a total of 3.000
3 trees in all found
+-----------------Delta
!
--2 +--------------Epsilon
! !
+--3 +-----------Gamma1
! !
+--6 +--------Alpha2
! !
+--1 +--Beta2
! +--5
+--4 +--Beta1
!
+-----Alpha1
reversions in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 0 0 1 1 1 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
root 2 yes ..1.. .
2 Delta yes ..... 1
2 3 yes ...11 .
3 Epsilon no ..... .
3 6 yes 1.0.. .
6 Gamma1 no ..... .
6 1 yes .1... .
1 Alpha2 no ..... .
1 4 no ..... .
4 5 yes ...00 .
5 Beta2 no ..... .
5 Beta1 no ..... .
4 Alpha1 no ..... .
+-----------------Delta
!
--2 +--------------Epsilon
! !
+--3 +-----------Gamma1
! !
+--6 +--Beta2
! +-----5
! ! +--Beta1
+--4
! +--Alpha2
+-----1
+--Alpha1
reversions in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 0 0 1 1 1 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
root 2 yes ..1.. .
2 Delta yes ..... 1
2 3 yes ...11 .
3 Epsilon no ..... .
3 6 yes 1.0.. .
6 Gamma1 no ..... .
6 4 yes .1... .
4 5 yes ...00 .
5 Beta2 no ..... .
5 Beta1 no ..... .
4 1 no ..... .
1 Alpha2 no ..... .
1 Alpha1 no ..... .
+-----------------Delta
!
--2 +--------------Epsilon
! !
+--3 +-----------Gamma1
! !
! ! +--Beta2
+--6 +--5
! +--4 +--Beta1
! ! !
+--1 +-----Alpha2
!
+--------Alpha1
reversions in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 0 0 1 1 1 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
root 2 yes ..1.. .
2 Delta yes ..... 1
2 3 yes ...11 .
3 Epsilon no ..... .
3 6 yes 1.0.. .
6 Gamma1 no ..... .
6 1 yes .1... .
1 4 no ..... .
4 5 yes ...00 .
5 Beta2 no ..... .
5 Beta1 no ..... .
4 Alpha2 no ..... .
1 Alpha1 no ..... .
|
Written by Joseph Felsenstein and James McGill.
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Drawtree and Drawgram are interactive tree-plotting programs that take a tree description in a file and read it, and then let you interactively make various settings and then make a plot of the tree in a file in some graphical file format, or plot the tree on a laser printer, plotter, or dot matrix printer. In most cases you can preview the resulting tree. This allows you to modify the tree until you like the result, then plot the result. Drawtree plots unrooted trees and Drawgram plots rooted cladograms and phenograms. With a plot to a file whose format is one acceptable to publishers both programs can produce fully publishable results.
These programs are descended from PLOTGRAM and PLOTREE, written by Christopher Meacham in 1984 and contributed to PHYLIP. I have incorporated his code for fonts and his plotter drivers, and in Drawtree have used some of his code for drawing unrooted trees. In both programs I have also included some plotter driver code by David Swofford, Julian Humphries and George D.F. "Buz" Wilson, to all of whom I am very grateful. Mostly, however, they consist of my own code and that of my programmers. The font files are printable-character recodings of the public-domain Hershey fonts, recoded by Christopher Meacham. The Java interface for the programs was created by Jim McGill.
This document will describe the features common to both programs. The documents for Drawtree and Drawgram describe the particular choices you can make in each of those programs. The Appendix to this documentation file contains some pieces of C code that can be inserted to make the program handle another plotting device -- the plotters by Calcomp.
To use Drawtree and Drawgram, you must have
Once you have all these pieces, the programs should be fairly self explanatory, particular if you can preview your plots so that you can discover the meaning of the different options by trying them out.
Once you have an executable version of the appropriate program (say Drawgram), and a file called (say) intree with the tree in it, and if necessary a font file (say font2 which you have copied as a file called fontfile), all you do is run the Drawgram program. It should automatically read the tree file and any font file needed, and will allow you to change the graphics device. Then it will let you see the options it has chosen, and ask you if you want to change these. Once you have modified those that you want to, you can tell it to accept those. The version of the program that has a Java interface will then allow you to preview the tree on the computer screen.
After you are done previewing the tree, the program will want to know whether you are ready to plot the tree. In the Java GUI version of the programs, you press on the Create Plot File button when you want to produce the final plot.
In the character-mode menu-driven versions of the programs, options can be changed but previewing does not occur. Plotting will occur after you close the menu by making the Y (yes) choice when you are asked whether you want to accept the plot as is. If you say no, it will once again allow you to change options, as many times as you want. If you say yes, then it will write a file called (say) plotfile. If you then copy this file to your printer or plotter, it should result in a beautifully plotted tree. You may need to change the filename to have the file format recognized by your operating system (for example, you may want to change plotfile to plotfile.ps if the file is in Postscript format).
If you don't want to print the file immediately, but want to edit the figure first, you should have chosen an output format that is readable by a draw program. Postscript format is readable by drawing programs such as Adobe Illustrator, Canvas, Freehand, and Coreldraw, and can be displayed by the Unix utilities Ghostscript and Ghostview. It can also be imported into word processors such as Microsoft Word as a figure. The PICT format was created for earlier Macintosh drawing programs such as MacDraw, and can be read by some other drawing programs and word processors. A widely-available bitmap drawing editor is GIMP (the Gnu Image Manipulation Program). On Windows systems bitmap drawing editors such as Paint can read Windows Bitmap files. We have provided output formats here for Xfig and Idraw drawing programs available on Linux or Unix systems.
Drawing programs can be used to add branch length numbers (something too hard for us to do automatically in these programs) and to make scale bars. Another use is as a way of printing out the trees, as most drawing programs are set up to print out their figures.
Having read the above, you may be ready to run the program. Below you will find more information about representation of trees in the tree file, on the different kinds of graphics devices supported by this program, and on how to recompile these programs.
The Newick Standard for representing trees in computer-readable form makes use of the correspondence between trees and nested parentheses, noticed in 1857 by the famous English mathematician Arthur Cayley. If we have this rooted tree:
A D
\ E /
\ C / /
\ ! / /
\ ! / /
B \!/ /
\ o /
\ ! /
\ ! /
\ ! /
\ ! /
\!/
o
!
!
then in the tree file it is represented by the following sequence of printable characters, starting at the beginning of the file:
(B,(A,C,E),D);
The tree ends with a semicolon. Everything after the semicolon in the input file is ignored, including any other trees. The bottommost node in the tree is an interior node, not a tip. Interior nodes are represented by a pair of matched parentheses. Between them are representations of the nodes that are immediately descended from that node, separated by commas. In the above tree, the immediate descendants are B, another interior node, and D. The other interior node is represented by a pair of parentheses, enclosing representations of its immediate descendants, A, C, and E.
Tips are represented by their names. A name can be any string of printable characters except blanks, colons, semcolons, parentheses, and square brackets. In the programs a maximum of 20 characters are allowed for names: this limit can easily be increased by recompiling the program and changing the constant declaration for "MAXNCH" in phylip.h.
Because you may want to include a blank in a name, it is assumed that an underscore character ("_") stands for a blank; any of these in a name will be converted to a blank when it is read in. Any name may also be empty: a tree like
(,(,,),);
is allowed. Trees can be multifurcating at any level (while in many of the programs multifurcations of user-defined trees are not allowed or restricted to a trifurcation at the bottommost level, these programs do make any such restriction).
Branch lengths can be incorporated into a tree by putting a real number, with or without decimal point, after a node and preceded by a colon. This represents the length of the branch immediately below that node. Thus the above tree might have lengths represented as:
(B:6.0,(A:5.0,C:3.0,E:4.0):5.0,D:11.0);
These programs will be able to make use of this information only if lengths exist for every branch, except the one at the bottom of the tree.
The tree starts on the first line of the file, and can continue to subsequent lines. It is best to proceed to a new line, if at all, immediately after a comma. Blanks can be inserted at any point except in the middle of a species name or a branch length.
The above description is of a subset of the Newick Standard. For example, interior nodes can have names in that standard, but if any are included the present programs will omit them.
To help you understand this tree representation, here are some trees in the above form:
((raccoon:19.19959,bear:6.80041):0.84600,((sea_lion:11.99700, seal:12.00300):7.52973,((monkey:100.85930,cat:47.14069):20.59201, weasel:18.87953):2.09460):3.87382,dog:25.46154);(Bovine:0.69395,(Gibbon:0.36079,(Orang:0.33636,(Gorilla:0.17147,(Chimp:0.19268, Human:0.11927):0.08386):0.06124):0.15057):0.54939,Mouse:1.21460);
(Bovine:0.69395,(Hylobates:0.36079,(Pongo:0.33636,(G._Gorilla:0.17147, (P._paniscus:0.19268,H._sapiens:0.11927):0.08386):0.06124):0.15057):0.54939, Rodent:1.21460);
();
((A,B),(C,D));
(Alpha,Beta,Gamma,Delta,,Epsilon,,,);
The Newick standard is based on a standard invented by Christopher Meacham for his programs PLOTREE and PLOTGRAM. The Newick Standard was adopted June 26, 1986 by an informal committee that met during the Society for the Study of Evolution meetings in Durham, New Hampshire and consisted of James Archie, William H.E. Day, Wayne Maddison, Christopher Meacham, F. James Rohlf, David Swofford, and myself. A web page describing it will be found at http://evolution.gs.washington.edu/phylip/newicktree.html.
When the programs run they have a menu which allows you to set (on its option P) the final plotting device, and another menu which allows you to set the type of preview screen. The choices for previewing are a subset of those available for plotting, and they can be different (the most useful combination will usually be a previewing graphics screen with a hard-copy plotter or a drawing program graphics file format).
In the Java interface the "Final plot file type" menu gives you the choices
Which plotter or printer will the tree be drawn on?
(many other brands or models are compatible with these)
type: to choose one compatible with:
L Postscript printer file format
M PICT format (for drawing programs)
J HP Laserjet PCL file format
W MS-Windows Bitmap
F FIG 2.0 drawing program format
A Idraw drawing program format
Z VRML Virtual Reality Markup Language file
P PCX file format (for drawing programs)
K TeKtronix 4010 graphics terminal
X X Bitmap format
V POVRAY 3D rendering program file
R Rayshade 3D rendering program file
H Hewlett-Packard pen plotter (HPGL file format)
D DEC ReGIS graphics (VT240 terminal)
E Epson MX-80 dot-matrix printer
C Prowriter/Imagewriter dot-matrix printer
O Okidata dot-matrix printer
B Houston Instruments plotter
U other: one you have inserted code for
Choose one:
|
Here are the choices, with some comments on each:
Postscript printer file format. This means that the program will generate a file containing Postscript commands as its plot file. This can be printed on any Postscript-compatible laser printer, and can be incorporated into Microsoft Word documents or into PDF documents. The page size is assumed to be 8.5 by 11 inches, but as plotting is within this limit A4 metric paper should work well too. This is the best quality output option. For this printer the menu options in Drawgram and Drawtree that allow you to select one of the built-in fonts will work. The programs default to Times-Roman when this plotting option is in effect. I have been able to use fonts Courier, Times-Roman, and Helvetica. The others have eluded me for some reason known only to those who really understand Postscript. The font name is written into the file, so any name that works there is possible.
PICT format (for drawing programs). This file format is read by many drawing programs (an early example was MacDraw). It has support for some fonts, though if fonts are used the species names can only be drawn horizontally or vertically, not at other angles in between. The control over line widths is a bit rough also, so that some lines at different angles may turn out to be different widths when you do not want them to be. If you are working on a Mac OS X system and have not been able to persuade it to print a Postscript file, even after adding a .ps extension to the file name, this option may be the best solution, as you could then read the file into a drawing program and then order it to print the resulting screen. The PICT file format has font support, and the default font for this plotting option is set to Times. You can also choose font attributes for the labels such as Bold, Italic, Outline, and Shadowed. PICT files can be read and supported by various drawing programs, but support for it in Adobe Photoshop has recently been dropped. It has been replaced by PDF format as the default graphics file format in Mac OS X.
HP Laserjet PCL file format. Hewlett-Packard's extremely popular line of laser printers has been emulated by many other brands of laser printer, so that for many years this format was compatible with many printers. It was also the default format for many inkjet printers. More recently almost all of these printers have support for the Postscript format, and their support for the PCL format may ultimately disappear. One limitation of the early versions of the PCL command language for these printers was that they did not have primitive operations for drawing arbitrary diagonal lines. This means that they must be treated by these programs as if they were dot matrix printers with a great many dots. This makes output files large, and output can be slow. The user will be asked to choose the dot resolution (75, 150, or 300 dots per inch). The 300 dot per inch setting should not be used if the laser printer's memory is less than 512k bytes. The quality of output is also not as good as it might be so that the Postscript file format will usually produce better results even at the same resolution. I am grateful to Kevin Nixon for inadvertently assisting me by pointing out that on Laserjets one does not have to dump the complete bitmap of a page to plot a tree.
MS-Windows Bitmap. This file format is used by most Windows drawing and paint programs, including Windows Paint which comes with the Windows operating system. It asks you to choose the height and width of the graphic image in pixels. For the moment, the image is set to be a monochrome image which can only be black or white. We hope to change that soon, but note that by pasting the image into a copy of Paint that is set to have a color image of the appropriate size, one can get a version whose color can be changed. Note also that large enough Windows Bitmap files can be used as "wallpaper" images for the background of a desktop.
FIG 2.0 drawing program format. This is the file format of the free drawing program Xfig, available for X-windows systems on Unix or Linux systems. Xfig can be downloaded from these places:
Xfig but may draw them with lines. This often makes the names look rather bumpy. We hope to change this soon.
Idraw drawing program format. Idraw is a free drawing program for X windows systems (such as Unix and Linux systems). Its interface is loosely based on MacDraw, and I find it much more useable than Xfig (almost no one else seems to agree with me). Though it was unsupported for a number of years, it has more recently been actively supported by Scott Johnston, of Vectaport, Inc. (http://www.vectaport.com). He has produced, in his ivtools package, a number of specialized versions of Idraw, and he also distributes the original Idraw as part of it. ivtools is available as a package on the Debian family of Linux distributions, as packages ivtools-bin, libiv-unidraw1, libiv1 and (for development) ivtools-dev. Thus on a Debia-family Linux system such as Ubuntu Linux, Linux Mint, SUSE, etc. you may simply need to type:
sudo apt-get install ivtools-bin sudo apt-get install libiv-unidraw1 sudo apt-get install libiv1in order to install Ivtools.
The Idraw file format that our programs produce can be read into Idraw, and also can be imported into the other Ivtools programs such as Drawtool. The file format saved from Idraw (or which can be exported from the other Ivtools programs) is Postscript, and if one does not print directly from Idraw one can simply send the file to the printer. But the format that we produce is missing some of the header information and will not work directly as a Postscript file. However if you read it into Idraw and then save it (or import it into one of the other Ivtools drawing programs such as Drawtool, and then export it) you will get a Postscript version that is fully useable.
Drawgram and Drawtree have font support in their Idraw file format options. The default font is Times-Bold but you can also enter the name of any other font that is supported by your Postscript printer. Idraw labels can be rotated to any angle. Some of these fonts are directly supported by the Idraw program. There is also a way to install new Postscript Type 1 fonts in the Ivtools programs.
Note that the Idraw drawing program from Ivtools is not related to the drawing program iDraw, which is produced by Indeeo, Inc.
VRML Virtual Reality Markup Language file. This is by far the most interesting plotting file format. VRML files describe objects in 3-dimensional space with lighting on them. A number of freely available "virtual reality browsers" or browser plugins can read VRML files. A list of available virtual reality browsers and browser plugins can be found at http://cic.nist.gov/vrml/vbdetect.html, a site that also automatically detects which VRML plugins are appropriate for your web browser. VRML plugins for your web browser or standalone browsers allow you to wander around looking at the tree from various angles, including from behind! I found VRMLView particularly easy to download -- it is distributed as an executable. It is not particulary fast and somewhat mysterious to use (try your mouse buttons). At the moment our VRML output is unsophisticated. The branches are made of tubes, with spheres at their joints. The tree is made of three-dimensional tubes but is basically flat. Names are made of connected tubes (to get this make sure you use a simple default font such as the Hershey font in file font1). This has the interesting effect that if you (virtually) move around and look at the tree from behind, the names will be backwards. VRML itself has VRML itself has been superseded by a standard called X3D (see http://www.web3d.org/), and we will be moving toward X3D support. Fortunately X3D is backwards compatible with VRML. What's next? Trees whose branches stick out in three dimensions? Animated trees whose forks rotate slowly? A video game involving combat among schools of systematists?
PCX file format (for drawing programs). A bitmap format that was formerly much used on the PC platform, this has been largely superseded by the Windows Bitmap (BMP) format, but it is still useful. This file format is simple and is read by many other programs as well. The user must choose one of three resolutions for the file, 640x480, 800x600, or 1024x768. The file is a monochrome paint file. Our PCX format is correct but is not read correctly by versions of Microsoft Paint (PBrush) that are running on systems that have loaded Word97. The version of the Paint utility provided with Windows 7 also does not support the PCX format. The free image manipulation program GIMP (Gnu Image Manipulation Program) is able to read the PCX format.
The plot devices from here on are only available in the non-Java-interface version of the programs:
Tektronix 4010 graphics terminal. The plot file will contain commands for driving the Tektronix series of graphics terminals. Other graphics terminals were compatible with the Tektronix 4010 and its immediate descendants. There are terminal emulation programs for Macintoshes that emulate Tektronix graphics. On workstations with X windows you can use one option of the "xterm" utility to create a Tektronix-compatible window. On Sun workstations there used to be a Tektronix emulator you can run called "tektool" which can be used to view the trees.
X Bitmap format. This produces an X-bitmap for the X Windows system on Unix or Linux systems, which can be displayed on X screens. You will be asked for the size of the bitmap (e.g., 16x16, or 256x256, etc.). This format cannot be printed out without further format conversion but is usable for backgrounds of windows ("wallpaper"). This can be a very bulky format if you choose a large bitmap. The bitmap is a structure that can actually be compiled into a C program (and thus built in to it), if you should have some reason for doing that.
POVRAY 3D rendering program file. This produces a file for the free ray-tracing program POVRay (Persistence of Vision Raytracer), which is available at http://www.povray.org/. It shows a tree floating above a flat landscape. The tree is flat but made out of tubes (as are the letters of the species names). It casts a realistic shadow across the landscape. lit from over the left shoulder of the viewer. You will be asked to confirm the colors of the tree branches, the species names, the background, and the bottom plane. These default to Blue, Yellow, White, and White respectively.
Rayshade 3D rendering program file. The input format for the free ray-tracing program "rayshade" which is available at http://www-graphics.stanford.edu/~cek/rayshade/rayshade.html for many kinds of systems. Rayshade takes files of this format and turns them into color scenes in "raw" raster format (also called "MTV" format after a raytracing program of that name). If you get the Netpbm package (available from http://netpbm.sourceforge.net/projects/netpbm/). and compile it on your system you can use the "mtvtoppm" and "ppmtogif" programs to convert this into the widely-used GIF raster format. (the Netpbm package will also allow you to convert into tiff, pcx and many other formats) The resultant image will show a tree floating above a landscape, rendered in a real-looking 3-dimensional scene with shadows and illumination. It is possible to use Rayshade to make two scenes that together are a stereo pair. When producing output for Rayshade you will be asked by the Drawgram or Drawtree whether you want to reset the values for the colors you want for the tree, the species names, the background, and the desired resolution.
Hewlett-Packard pen plotter (HPGL file format). This means that the program will generate a file as its plot file which uses the HPGL graphics language. Hewlett-Packard 7470, 7475, and many other plotters are compatible with this. The paper size is again assumed to be 8.5 by 11 inches (again, A4 should work well too). It is assumed that there are two pens, a finer one for drawing names, and the HPGL commands will call for switching between these. Few people have HP plotters these days, the PCL printer control language found in but recent Hewlett-Packard printers can emulate an HP plotter, as this feature is included in its PCL5 command language (but not in the PCL4 command languages of earlier Hewlett-Packard models).
DEC ReGIS graphics (VT240 terminal). The DEC ReGIS standard is used by the VT240 and VT340 series terminals by DEC (Digital Equipment Corporation). There used to be many graphics terminals that emulate the VT240 or VT340 as well. The DECTerm windows in many versions of Digital's (now Compaq's) DECWindows windowing system also did so. These days DEC ReGIS graphics is rarely seen: it is most likely to be encountered as an option in X11 Xterm windows.
Epson MX-80 dot-matrix printer. This file format is for the dot-matrix printers by Epson (starting with the MX80 and continuing on to many other models), as well as the IBM Graphics printers. The code here plots in double-density graphics mode. Many of the later models are capable of higher-density graphics but not with every dot printed. This density was chosen for reasonably wide compatibility. Many other dot-matrix printers on the market have graphics modes compatible with the Epson printers. I cannot guarantee that the plot files generated by these programs would be compatible with all of these, but they do work on Epsons. They have also worked, in our hands, on IBM Graphics Printers. There used to be many printers that claimed compatibility with these too, but I do not know whether it will work on all of them. If you have trouble with any of these you might consider trying in the epson option of procedure initplotter to put in a fprintf statement that writes to plotfile an escape sequence that changes line spacing. As dot matrix printers are rare these days, used mostly to print multipart receipts in business, I suspect this option will not get much testing.
Prowriter/Imagewriter dot-matrix printer. The trading firm C. Itoh distributed this line of dot-matrix printers, which was made by Tokyo Electric (TEC), now a subsidiary of Toshiba. It was also sold by NEC under the product number PC8023. These were 9-pin dot matrix printers. In a slightly modified form they were also the Imagewriter printer sold by Apple for their Macintosh line. The same escape codes seem to work on both machines, the Apple version being a serial interface version. They are not related to the IBM Proprinter, despite the name.
Okidata dot-matrix printer. The ML81, 82, 83 and ML181, 182, 183 line of dot-matrix printers from Okidata had their own graphics codes and those are dealt with by this option. The later Okidata ML190 series emulated IBM Graphics Printers so that you would not want to use this option for them but the option for that printer.
Houston Instruments plotter. The Houston Instruments line of plotters were also known as Bausch and Lomb plotters. The code in the programs for these has not been tested recently; I would appreciate anyone who tries it out telling me whether it works. I do not have access to such a plotter myself, and doubt most users will come across one.
Conversion from these formats to others is also possible. There is a free program NetPBM that interconverts many bitmap formats (see above under Rayshade). A more accessible option will be the free image manipulation program GIMP (Gnu Image Manipulation Program) which can read our Postscript, Windows Bitmap (.BMP), PCX, and X Bitmap formats and can write many raster and vector formats.
In the Java GUI version of Drawgram and Drawtree, the graphics capabilities of Java are used for previewing. The programs actually write a Postscript file called JavaPreview.ps, and each time the preview is displayed this is read in and displayed.
Another problem is adding labels (such as vertical scales and branch lengths) to the plots produced by this program. This may require you to use the Postcript, BMP, PICT, Idraw, Xfig, or PCX file format and use a draw or paint program to add them. GIMP and Adobe Illustrator can do this.
I would like to add more built-in fonts. The fontfiles now have recoded versions of the Hershey fonts. They are legally publicly distributable. Most other font families on the market are not public domain and I cannot afford to license them for distribution. Some people have noticed that the Hershey fonts, which are drawn by a series of straight lines, have noticeable angles in what are supposed to be curves, when they are printed on modern laser printers and looked at closely. This is less a problem than one might think since, fortunately, when scientific journals print a tree it is usually shrunk so small that these imperfections (and often the tree itself) are hard to see!
One more font that could be added from the Hershey font collection would be a Greek font. If Greek users would find that useful I could add it, but my impression is that they publish mostly in English anyway.
The C code of these programs consists of two C programs, "drawgram.c" and "drawtree.c". Each of these uses two other pieces of C code "draw.c", "draw2.c", plus a common header file, "draw.h". All of the graphics commands that are common to both programs will be found in "draw.c" and "draw2.c". The following instructions for writing your own code to drive a different kind of printer, plotter, or graphics file format, require you only to make changes in "draw.c" and "draw2.c". The two programs can then be recompiled.
If you want to write code for other printers, plotters, or vector file formats, this is not too hard. The plotter option "U" is provided as a place for you to insert your own code. Chris Meacham's system was to draw everything, including the characters in the names and all curves, by drawing a series of straight lines. Thus you need only master your plotter's commands for drawing straight lines. In function "plotrparms" you must set up the values of variables "xunitspercm" and "yunitspercm", which are the number of units in the x and y directions per centimeter, as well as variables "xsize" and "ysize" which are the size of the plotting area in centimeters in the x direction and the y direction. A variable "penchange" of a user-defined type is set to "yes" or "no" depending on whether the commands to change the pen must be issued when switching between plotting lines and drawing characters. Even though dot-matrix printers do not have pens, penchange should be set to "yes" for them. In function "plot" you must issue commands to draw a line from the current position (which is at (xnow, ynow) in the plotter's units) to the position (xabs, yabs), under the convention that the lower-left corner of the plotting area is (0.0, 0.0). In functions "initplotter" and "finishplotter" you must issue commands to initialize the plotter and to finish plotting, respectively. If the pen is to be changed an appropriate piece of code must be inserted in function "penchange". The code to print the text needs to be added to the "plottext" function.
For dot matrix printers and raster graphics matters are a bit more complex. The functions "plotrparms", "initplotter", "finishplotter" and "plot" still respectively set up the parameters for the plotter, initialize it, finish a plot, and plot one line. But now the plotting consists of drawing dots into a two-dimensional array called "stripe". Once the plot is finished this array is printed out. In most cases the array is not as tall as a full plot: instead it is a rectangular strip across it. When the program has finished drawing in ther strip, it prints it out and then moves down the plot to the next strip. For example, for Hewlett-Packard Laserjets we have defined the strip as 2550 dots wide and 20 dots deep. When the program goes to draw a line, it draws it into the strip and ignores any part of it that falls outside the strip. Thus the program does a complete plotting into the strip, then prints it, then moves down the diagram by (in this case) 20 dots, then does a complete plot into that strip, and so on.
To work with a new raster or dot matrix format, you will have to define the desired width of a strip ("strpwide"), the desired depth ("strpdeep"), and how many lines of bytes must be printed out to print a strip. Procedure "striprint" is the one that prints out a strip, and has special-case code for the different printers and file formats. For file formats, all of which print out a single row of dots at a time, the variable "strpdiv" is not used. The variable "dotmatrix" is set to "true" or "false" in function "plotrparms" according to whether or not "strpdiv" is to be used. Procedure "plotdot" sets a single dot in the array "strip" to 1 at position (xabs, yabs). The coordinates run from 1 at the top of the plot to larger numbers as we proceed down the page. Again, there is special-case code for different printers and file formats in that function. You will probably want to read the code for some of the dot matrix or file format options if you want to write code for one of them. Many of them have provision for printing only part of a line, ignoring parts of it that have no dots to print.
I would be happy to obtain the resulting code from you to consider adding it to this listing so we can cover more kinds of plotters, printers, and file formats.
These pieces of code are to be inserted in the places reserved for the "Y" plotter option. The variables necessary to run this have already been incorporated into the programs.
Calcomp's industrial-strength plotters were once a fixture of University computer centers, but are rarely found now, but just in case you need to use one, this code should work:
A global declaration needed near the front of drawtree.c:
Char cchex[16];
Code to be inserted into function plotrparms:
case 'Y':
plotter = other;
xunitspercm = 39.37;
yunitspercm = 39.37;
xsize = 25.0;
ysize = 25.0;
xposition = 12.5;
yposition = 0.0;
xoption = center;
yoption = above;
rotation = 0.0;
break;
Code to be inserted into function plot:
Declare these variables at the beginning of the function:
long n, inc, xinc, yinc, xlast, ylast, xrel, yrel, xhigh, yhigh, xlow, ylow; Char quadrant;
and insert this into the switch statement:
case other:
if (penstatus == pendown)
putc('H', plotfile);
else
putc('D', plotfile);
xrel = (long)floor(xabs + 0.5) - xnow;
yrel = (long)floor(yabs + 0.5) - ynow;
xnow = (long)floor(xabs + 0.5);
ynow = (long)floor(yabs + 0.5);
if (xrel > 0) {
if (yrel > 0)
quadrant = 'P';
else
quadrant = 'T';
} else if (yrel > 0)
quadrant = 'X';
else
quadrant = '1';
xrel = labs(xrel);
yrel = labs(yrel);
if (xrel > yrel)
n = xrel / 255 + 1;
else
n = yrel / 255 + 1;
xinc = xrel / n;
yinc = yrel / n;
xlast = xrel % n;
ylast = yrel % n;
xhigh = xinc / 16;
yhigh = yinc / 16;
xlow = xinc & 15;
ylow = yinc & 15;
for (i = 1; i <= n; i++)
fprintf(plotfile, "%c%c%c%c%c",
quadrant, cchex[xhigh - 1], cchex[xlow - 1], cchex[yhigh - 1],
cchex[ylow - 1]);
if (xlast != 0 || ylast != 0)
fprintf(plotfile, "%c%c%c%c%c",
quadrant, cchex[-1], cchex[xlast - 1], cchex[-1],
cchex[ylast - 1]);
break;
Code to be inserted into function initplotter:
case other:
cchex[-1] = 'C';
cchex[0] = 'D';
cchex[1] = 'H';
cchex[2] = 'L';
cchex[3] = 'P';
cchex[4] = 'T';
cchex[5] = 'X';
cchex[6] = '1';
cchex[7] = '5';
cchex[8] = '9';
cchex[9] = '/';
cchex[10] = '=';
cchex[11] = '#';
cchex[12] = '"';
cchex[13] = '\'';
cchex[14] = '^';
xnow = 0.0;
ynow = 0.0;
fprintf(plotfile, "CCCCCCCCCC");
break;
Code to be inserted into function finishplotter:
case other:
plot(penup, 0.0, yrange + 50.0);
break;
The Hershey fonts were digitized fonts created by Dr. A. V. Hershey in the late 1960s when he was working at the U. S. Naval Weapons Laboratory. They were published in U. S. National Bureau of Standards Special Publication No. 424, distributed by the U. S. National Technical Information Service. Legally, it is possible to freely distribute these fonts in any encoding system except the original one used by the U. S. National Technical Information Service, provided that you acknowledge that the original fonts were produced by Dr. Hershey and published by NBS. This is a somewhat odd restriction, but convenient for us. Chris Meacham developed the software we use to read the Hershey fonts, and it uses a simple coding system that he developed. The original Hershey fonts were transformed by him into this encoding system. Six of them are distributed with PHYLIP: three Roman fonts, one unserifed and two serifed, two Italic fonts, one unserifed and one serifed, and a Russian Cyrillic font.
Each font file consists of groups of lines, one for each character. Here are the lines for character "h" in the font #1 in this encoding:
Ch 608 21 19 28 -1456 1435 -1445 1748 1949 2249 2448 2545 2535 -12935 |
The group of lines starts with the letter C (for Character). Then follows the character that this font will draw (in this case "h"). It is the byte which, when read by the computer, signals that character. Then there is the number of this character in the original Hershey fonts (608). This is not used by our software.
The Hershey fonts are drawn on a grid of points as a series of lines. The next three numbers (21, 19, and 28) are the height (21), and two widths (19, and 28, which we don't use). Then comes a new line which shows the individual pen moves. When these are negative, they indicate that the pen is to be up when moving; when they are positive, the pen is to be down. They are integers. The last of them is greater than 10,000, and that is the signal to end after that move.
Each number has a final four digits that give the coordinate to which the pen is to move. These are given as (x,y) coordinates. Thus the first number (-1456) indicates the pen is to be up and the plotting is to move to coordinate (14, 56), which is x = 14, y = 56. Then the pen is put down and moved to (14, 35). This draws a line from (14, 56) to (14, 35), in fact the vertical line that forms the back of the "h". Then the pen is picked up and moved to (14, 45). Then there follow a series of moves with pen down to (14, 35), (14, 45), (17, 48), (19, 49), (22, 49), (24, 48), (25, 45), and finally (25, 35). This draws a series of connected line segments that make the arch and right-hand vertical, ending up at the bottom-right of the character. -12935 then signals a pen-up move to (29, 35). This moves to a point where the next character can start, putting in a little "white space".
As you can see, the coding system is quite simple. Does anyone want to draw us some new fonts to add to our repertoire? I have spared you the Gothic, Old English, and Greek Hershey fonts, but perhaps there are some other nice ones people might want to use.
phylip-3.697/doc/drawgram.html 0000644 0047320 0047320 00000050402 12406201172 016000 0 ustar joe felsenst_g
Written by Joseph Felsenstein and James McGill.
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Drawgram interactively plots a cladogram- or phenogram-like rooted tree diagram, with many options including orientation of tree and branches, style of tree, label sizes and angles, tree depth, margin sizes, stem lengths, and placement of nodes in the tree. Particularly if you can use your computer to preview the plot, you can very effectively adjust the details of the plotting to get just the kind of plot you want.
To understand the working of Drawgram you should first read the Tree Drawing Programs web page in this documentation.
Java Interface
All Phylip programs will get Java interfaces in the 4.0 release. But under some operating systems there are currently serious problems with Drawgram, so it has received its Java interface early as part of the 3.695 bug fix release. We do not anticipate changing this Java interface substantially in the 3.7 release, but don't be surprised if we do.
This new Java interface supersedes the old character-mode menu interface. PHYLIP also contains versions of Drawgram and Drawtree that have the character-mode menu interface. We have kept these available because PHYLIP is used in many places as part of pipelines driven by scripts. Since these scripts do not usually invoke the preview mode of Drawgram, we have disabled the previewing of tree plotting in Drawgram in this release. Previewing is available in the version of Drawgram that has the interactive Java interface.
The Java interface is different from the previous character-mode menu interface; it calls the C code of Drawgram, which is in a dynamic library. Thus, after the previewing is done, the code producing final plot file should make plots that are indistinguishable from those produced by previous versions of Drawgram.
The Java Drawgram Interface is a modern GUI. It will run only on a machine that has a recent version of Oracle Java installed. This is not a serious limitation because Java is freeware that is universally available.
When you start the Drawgram Java interface it looks similar to the following, which has been edited to generate the plot which follows:
It has all the usual GUI funtionality: input and output file selectors, drop down menu options, data entry boxes and toggles. "Preview" brings up a nearly WYSIWYG preview window that displays the Postscript plot created by the current settings (the fonts used in the previewing window are not the same, but use Serif, SansSerif, and Monospaced fonts that approximate the PostScript fonts that are used in the output plot):
Each time you select "Preview" another preview window is generated, so that multiple previews can be visible. This allows you to compare various display options. When the plot has been fine tuned, clicking "Create Plot File" writes the Postscript file that generated the last Preview to the plot file specified. Note that if there are multiple preview windows open, the most recent one is the one that shows how the tree in the final plot file will look, since it will be plotted using the most recent settings.
All the functionality in the Java GUI is the same as in the equivalent menu item in the character-mode menu interface. To ease the transition, we have kept the text in the Java GUI as close as possible to the description in the character-mode menu interface. So, for example, "S" in the old interface, which has the description "Tree style", has the counterpart "Tree style" in the new interface. The detailed explanations of each label are found below.
To understand the working of Drawgram and Drawtree, you should first read the Tree Drawing Programs web page in this documentation.
The Command Line Interface gives the user access to a huge collection of both display systems and output formats (some of them are historical curiosities at this point, but they still work so there is no reason to remove them). It can also be driven by scripting because it is a command line interface. But, as most users have little experience with command line systems, it is a bit daunting.
As with Drawtree, to run Drawgram you need a compiled copy of the program, a font file, and a tree file. The tree file has a default name of intree. The font file has a default name of "fontfile". If there is no file of that name, the program will ask you for the name of a font file (we provide ones that have the names font1 through font6). Once you decide on a favorite one of these, you could make a copy of it and call it fontfile, and it will then be used by default. Note that the program will get confused if the input tree file has the number of trees on the first line of the file, so that number may have to be removed.
Once these choices have been made you will see the central menu of the program, which looks like this:
Rooted tree plotting program version 3.695 Here are the settings: 0 Screen type (IBM PC, ANSI): ANSI P Final plotting device: Postscript printer V Previewing device: X Windows display H Tree grows: Horizontally S Tree style: Phenogram B Use branch lengths: (no branch lengths available) L Angle of labels: 90.0 R Scale of branch length: Automatically rescaled D Depth/Breadth of tree: 0.53 T Stem-length/tree-depth: 0.05 C Character ht / tip space: 0.3333 A Ancestral nodes: Centered F Font: Times-Roman M Horizontal margins: 1.65 cm M Vertical margins: 2.16 cm Y to accept these or type the letter for one to change |
These are the settings that control the appearance of the tree, which has already been read in. You can either accept these as is, in which case you would answer Y to the question and press the Return or Enter key, or you can answer N if you want to change one, or simply type the character corresponding to the one you want to change (if you answer N it will just immediately ask you for that number anyway).
For a first run in the Java interface version, you might accept these default values and see what the result looks like.
You can resize the preview window, though you may have to ask the system to redraw the preview to see it at the new window size.
Once you are finished looking at the preview, you will want to specify whether the program should make the final plot or change some of the settings. The possible settings are listed below.
When you are ready to produce the final plot file, you should use the button "Create Plot File" (if you are using the Java interface) or you should type Y (if you are using the character-mode menu). In the Java-interface version, the name of the plot file has been set in the dialog box near the top of the Java window. It defaults to plotfile.ps. In the character-mode menu, the file name defaults to plotfile.
If there is already a file of that name, the program will ask you whether you want to Overwrite the file, Append to the file, or Quit (in the character-mode menu version it also gives the option of writing to a new file whose name you will be asked to supply.
Below I will describe the options one by one; you may prefer to skip reading this unless you are puzzled about one of them.
In spite of the words "cladogram" and "phenogram", there is no implication of the extent to which you consider these diagrams as being genealogies or phenetic clustering diagrams. The names refer to pictorial style, not your own intended final use for the diagram. The six styles can be described as follows (assuming a vertically growing tree):
You should experiment with these and decide which you want -- it depends very much on the effect you want.
Should interior node positions:
be Intermediate between their immediate descendants,
Weighted average of tip positions
Centered among their ultimate descendants
iNnermost of immediate descendants
or so that tree is V-shaped
(type I, W, C, N or V):
|
The five methods (Intermediate, Weighted, Centered, Innermost, and V-shaped) are different horizontal positionings of the interior nodes. It will be helpful to you to try these out and see which you like best. Intermediate places the node halfway between its immediate descendants (horizontally), Weighted places it closer to that descendant who is closer vertically as well, and Centered centers the node below the horizontal positions of the tips that are descended from that node. You may want to choose that option that prevents lines from crossing each other.
V-shaped is another option, one designed, if there are no branch lengths being used, to yield a v-shaped tree of regular appearance. At the moment it can give somewhat wierd trees; we intend to make it better in the next release. With branch lengths it will not necessarily make the tree perfectly V-shaped. "Innermost" is the most unusual option: it chooses a center for the tree, and always places interior nodes below the innermost of their immediate descendants. This leads to a tree that has vertical lines in the center, like a tree with a trunk.
If the tree you are plotting has a full set of lengths, then when it is read in, the node position option is automatically set to "intermediate", which is the setting with the least likelihood of lines in the tree crossing. If it does not have lengths the option is set to "V-shaped". If you change the option which tells the program whether to try to use the branch lengths, then the node position option will automatically be reset to the appropriate one of these defaults. This may be confusing if you do not realise that it is happening.
I recommend that you try all of these options (particularly if you can preview the trees). It is of particular use to try combinations of the style of tree (option S) with the different methods of placing interior nodes (option A). You will find that a wide variety of effects can be achieved.
AfterwordI would appreciate suggestions for improvements in Drawgram, but please be aware that the source code is already very large and I may not be able to implement all suggestions.
phylip-3.697/doc/drawtree.html 0000644 0047320 0047320 00000050551 12406201172 016016 0 ustar joe felsenst_g
Written by Joseph Felsenstein and James McGill.
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Drawtree interactively plots an unrooted tree diagram, with many options including orientation of tree and branches, label sizes and angles, and margin sizes. Particularly if you can use your computer screen to preview the plot, you can very effectively adjust the details of the plotting to get just the kind of plot you want.
To understand the working of Drawtree you should first read the Tree Drawing Programs web page in this documentation.
Java Interface
All Phylip programs will get Java interfaces in the 4.0 release. But under some operating systems there are currently serious problems with Drawtree, so it has received its Java interface early as part of the 3.695 bug fix release. We do not anticipate changing this Java interface substantially in the 4.0 release, but don't be surprised if we do.
This new Java interface supersedes the old character-mode menu interface. PHYLIP also contains versions of Drawgram and Drawtree that have the character-mode menu interface. We have kept these available because PHYLIP is used in many places as part of pipelines driven by scripts. Since these scripts do not usually invoke the preview mode of Drawtree, we have disabled the previewing of tree plotting in Drawtree in this release. Previewing is available in the version of Drawtree that has the interactive Java interface.
The Java interface is different from the previous character-mode menu interface; it calls the C code of Drawgram, which is in a dynamic library. Thus, after the previewing is done, the code producing final plot file should make plots that are indistinguishable from those produced by previous versions of Drawgram.
The Java Drawtree Interface is a modern GUI. It will run only on a machine that has a recent version of Oracle Java installed. This is not a serious limitation because Java is freeware that is universally available.
When you start the Drawtree Java interface it looks similar to the following, which has been edited to generate the plot which follows:
It has all the usual GUI functionality: input and output file selectors, drop down menu options, data entry boxes and toggles. "Preview" brings up a nearly WYSIWYG preview window that displays the Postscript plot created by the current settings:
Each time you select "Preview" another preview window is generated, so that multiple previews can be visible. This allows you to compare various display options. When the plot has been fine tuned, clicking "Create Plot File" writes the Postscript file that generated the last Preview to the plot file specified. Note that if there are multiple preview windows open, the most recent one is the one that shows how the tree in the final plot file will look, since it will be plotted using the most recent settings.
All the functionality in the Java GUI is the same as in the equivalent menu item in the character-mode menu interface. To ease the transition, we have kept the text in the Java GUI as close as possible to the description in the character-mode menu interface. So, for example, "L" in the old interface, which has the helper message "Angle of labels", maps to "Angle of labels" in the new interface. All the detailed explanations of each label are found below.
The Command Line Interface gives the user access to a huge collection of both display systems and output formats (some of them are historical curiosities at this point, but they still work so there is no reason to remove them). It can also be driven by scripting because it is a command line interface. But, as most users have little experience with command line systems, it is a bit daunting.
As with Drawgram, to run Drawtree you need a compiled copy of the program, a font file, and a tree file. The tree file has a default name of intree. The font file has a default name of "fontfile". If there is no file of that name, the program will ask you for the name of a font file (we provide ones that have the names font1 through font5). Once you decide on a favorite one of these, you could make a copy of it and call it fontfile, and it will then be used by default.
Once these choices have been made you will see the central menu of the program, which looks like this:
Unrooted tree plotting program version 3.695 Here are the settings: 0 Screen type (IBM PC, ANSI)? ANSI P Final plotting device: Postscript printer B Use branch lengths: (no branch lengths available) L Angle of labels: branch points to Middle of label R Rotation of tree: 90.0 I Iterate to improve tree: Equal-Daylight algorithm D Try to avoid label overlap? No S Scale of branch length: Automatically rescaled C Relative character height: 0.3333 F Font: Times-Roman M Horizontal margins: 1.65 cm M Vertical margins: 2.16 cm # Page size submenu: one page per tree Y to accept these or type the letter for one to change |
These are the settings that control the appearance of the tree, which has already been read in. You can either accept these as is, in which case you would answer Y to the question and press the Return or Enter key, or you can answer N if you want to change one, or simply type the character corresponding to the one you want to change (if you answer N it will just immediately ask you for that number anyway).
For a first run in the Java interface version you might accept these default values and see what the result looks like.
You can resize the preview window, though you may have to ask the system to redraw the preview to see it at the new window size.
Once you are finished looking at the preview, you will want to specify whether the program should make the final plot or change some of the settings. The possible settings are listed below.
When you are ready to produce the final plot file, you should use the button "Create Plot File" (if you are using the Java interface) or you should type Y (if you are using the character-mode menu). In the Java-interface version, the name of the plot file has been set in the dialog box near the top of the Java window. It defaults to plotfile.ps. In the character-mode menu, the file name defaults to plotfile.
If there is already a file of that name, the program will ask you whether you want to Overwrite the file, Append to the file, or Quit (in the character-mode menu version it also gives the option of writing to a new file whose name you will be asked to supply.
Below I will describe the options one by one; you may prefer to skip reading this unless you are puzzled about one of them.
I recommend that you try all of these options (particularly if you can preview the trees). It is of particular use to try trees with different iteration methods (option I) and with regularization (option G). You will find that a variety of effects can be achieved.
AfterwordI would appreciate suggestions for improvements in Drawtree, but please be aware that the source code is already very large and I may not be able to implement all suggestions.
phylip-3.697/doc/factor.html 0000644 0047320 0047320 00000027161 12406201172 015460 0 ustar joe felsenst_g
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program factors a data set that contains multistate characters, creating a data set consisting entirely of binary (0,1) characters that, in turn, can be used as input to any of the other discrete character programs in this package, except for PARS. Besides this primary function, Factor also provides an easy way of deleting characters from a data set. The input format for Factor is very similar to the input format for the other discrete character programs except for the addition of character-state tree descriptions.
Note that this program has no way of converting an unordered multistate character into binary characters. Fortunately, PARS has joined the package, and it enables unordered multistate characters, in which any state can change to any other in one step, to be analyzed with parsimony.
Factor is really for a different case, that in which there are multiple states related on a "character state tree", which specifies for each state which other states it can change to. That graph of states is assumed to be a tree, with no loops in it.
The first line of the input file should contain the number of species and the number of multistate characters. This first line is followed by the lines describing the character-state trees, one description per line. The species information constitutes the last part of the file. Any number of lines may be used for a single species.
The first line is free format with the number of species first, separated by at least one blank (space) from the number of multistate characters, which in turn is separated by at least one blank from the options, if present.
The options are selected from a menu that looks like this:
Factor -- multistate to binary recoding program, version 3.69 Settings for this run: A put ancestral states in output file? No F put factors information in output file? No 0 Terminal type (IBM PC, ANSI, none)? (none) 1 Print indications of progress of run Yes Are these settings correct? (type Y or the letter for one to change) |
The options particular to this program are:
The character-state trees are described in free format. The character number of the multistate character is given first followed by the description of the tree itself. Each description must be completed on a single line. Each character that is to be factored must have a description, and the characters must be described in the order that they occur in the input, that is, in numerical order.
The tree is described by listing the pairs of character states that are adjacent to each other in the character-state tree. The two character states in each adjacent pair are separated by a colon (":"). If character fifteen has this character state tree for possible states "A", "B", "C", and "D":
A ---- B ---- C
|
|
|
D
then the character-state tree description would be
15 A:B B:C D:B
Note that either symbol may appear first. The ancestral state is identified, if desired, by putting it "adjacent" to a period. If we wanted to root character fifteen at state C:
A <--- B <--- C
|
|
V
D
we could write
15 B:D A:B C:B .:C
Both the order in which the pairs are listed and the order of the symbols in each pair are arbitrary. However, each pair may only appear once in the list. Any symbols may be used for a character state in the input except the character that signals the connection between two states (in the distribution copy this is set to ":"), ".", and, of course, a blank. Blanks are ignored completely in the tree description so that even B:DA:BC:B.:C or B : DA : BC : B. : C would be equivalent to the above example. However, at least one blank must separate the character number from the tree description.
If no description line appears in the input for a particular character, then that character will be omitted from the output. If the character number is given on the line, but no character-state tree is provided, then the symbol for the character in the input will be copied directly to the output without change. This is useful for characters that are already coded "0" and "1". Characters can be deleted from a data set simply by listing only those that are to appear in the output.
The last character-state tree description should be followed by a line containing the number "999". This terminates processing of the trees and indicates the beginning of the species information.
The format for the species information is basically identical to the other discrete character programs. The first ten character positions are allotted to the species name (this value may be changed by altering the value of the constant nmlngth at the beginning of the program). The character states follow and may be continued to as many lines as desired. There is no current method for indicating polymorphisms. It is possible to either put blanks between characters or not.
There is a method for indicating uncertainty about states. There is one character value that stands for "unknown". If this appears in the input data then "?" is written out in all the corresponding positions in the output file. The character value that designates "unknown" is given in the constant unkchar at the beginning of the program, and can be changed by changing that constant. It is set to "?" in the distribution copy.
The first line of output will contain the number of species and the number of binary characters in the factored data set. The factored characters will be written for each species in the format required for input by the other discrete programs in the package. The maximum length of the output lines is 80 characters, but this maximum length can be changed prior to compilation.
If the A (Ancestors) option was chosen, an output file whose default name is ancestors will be written with the ancestors information. If F (Factors) was chosen in the menu, am output file whose default name is factors will be written containing the factors information.
ERRORS
The output should be checked for error messages. Errors will occur in the character-state tree descriptions if the format is incorrect (colons in the wrong place, etc.), if more than one root is specified, if the tree contains loops (and hence is not a tree), and if the tree is not connected, e.g.
A:B B:C D:E
describes
A ---- B ---- C D ---- E
This "tree" is in two unconnected pieces. An error will also occur if a symbol appears in the data set that is not in the tree description for that character. Blanks at the end of lines when the species information is continued to a new line will cause this kind of error.
At the beginning of the program a number of constants are available to be changed to accomodate larger data sets. These are "maxstates", "maxoutput", "sizearray", "factchar" and "unkchar". The constant "maxstates" gives the maximum number of states per character (set at 20 in the distribution copy). The constant "maxoutput" gives the maximum width of a line in the output file (80 in the distribution copy). The constant "sizearray" must be less than the sum of squares of the numbers of states in the characters. It is initially set to set to 2000, so that although 20 states are allowed (at the initial setting of maxstates) per character, there cannot be 20 states in all of 100 characters.
Particularly important constants are "factchar" and "unkchar" which are not numerical values but a character. Initially set to the colon ":", "factchar" is the character that will be used to separate states in the input of character state trees. It can be changed by changing this constant. (We could have used a hyphen ("-") but didn't because that would make the minus-sign ("-") unavailable as a character state in +/- characters). The constant "unkchar" is the character value in the input data that indicates that the state is unknown. It is set to "?" in the distribution copy. If your computer is one that lacks the colon ":" in its character set or uses a nonstandard character code such as EBCDIC, you will want to change the constant "factchar".
The input file for the program has the default file name "infile" and the output file, the one that has the binary character state data, has the name "outfile".
| ----SAMPLE INPUT----- | -----Comments (not part of input file) ----- |
4 6 1 A:B B:C 2 A:B B:. 4 5 0:1 1:2 .:0 6 .:# #:$ #:% 999 Alpha CAW00# Beta BBX01% Gamma ABY12# Epsilon CAZ01$ |
4 species; 6 characters
A ---- B ---- C
B ---> A
Character 3 deleted; 4 unchanged
0 ---> 1 ---> 2
% <--- # ---> $
Signals end of trees
Species information begins
|
| ---SAMPLE OUTPUT----- | -----Comments (not part of output file) ----- |
4 8
Alpha 11100000
Beta 10001001
Gamma 00011100
Epsilon 11101010
|
4 species; 8 factors
Chars. 1 and 2 come from old number 1
Char. 3 comes from old number 2
Char. 4 is old number 4
Chars. 5 and 6 come from old number 5
Chars. 7 and 8 come from old number 6
|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program carries out Fitch-Margoliash, Least Squares, and a number of similar methods as described in the documentation file for distance methods.
The options for Fitch are selected through the menu, which looks like this:
Fitch-Margoliash method version 3.69 Settings for this run: D Method (F-M, Minimum Evolution)? Fitch-Margoliash U Search for best tree? Yes P Power? 2.00000 - Negative branch lengths allowed? No O Outgroup root? No, use as outgroup species 1 L Lower-triangular data matrix? No R Upper-triangular data matrix? No S Subreplicates? No G Global rearrangements? No J Randomize input order of species? No. Use input order M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print out the data at start of run No 2 Print indications of progress of run Yes 3 Print out tree Yes 4 Write out trees onto tree file? Yes Y to accept these or type the letter for one to change
|
Most of the input options (U, P, -, O, L, R, S, J, and M) are as given in the documentation page for distance matrix programs, and their input format is the same as given there. The U (User Tree) option has one additional feature when the N (Lengths) option is used. This menu option will appear only if the U (User Tree) option is selected. If N (Lengths) is set to "Yes" then if any branch in the user tree has a branch length, that branch will not have its length iterated. Thus you can prevent all branches from having their lengths changed by giving them all lengths in the user tree, or hold only one length unchanged by giving only that branch a length (such as, for example, 0.00). You may find program Retree useful for adding and removing branch lengths from a tree. This option can also be used to compute the Average Percent Standard Deviation for a tree obtained from Neighbor, for comparison with trees obtained by Fitch or Kitsch.
The D (methods) option allows choice between the Fitch-Margoliash criterion and the Minimum Evolution method (Kidd and Sgaramella-Zonta, 1971; Rzhetsky and Nei, 1993). Minimum Evolution (not to be confused with parsimony) uses the Fitch-Margoliash criterion to fit branch lengths to each topology, but then chooses topologies based on their total branch length (rather than the goodness of fit sum of squares). There is no constraint on negative branch lengths in the Minimum Evolution method; it sometimes gives rather strange results, as it can like solutions that have large negative branch lengths, as these reduce the total sum of branch lengths!
Another input option available in Fitch that is not available in Kitsch or Neighbor is the G (Global) option. G is the Global search option. This causes, after the last species is added to the tree, each possible group to be removed and re-added. This improves the result, since the position of every species is reconsidered. It approximately triples the run-time of the program. It is not an option in Kitsch because it is the default and is always in force there. The O (Outgroup) option is described in the main documentation file of this package. The O option has no effect if the tree is a user-defined tree (if the U option is in effect). The U (User Tree) option requires an unrooted tree; that is, it requires that the tree have a trifurcation at its base:
((A,B),C,(D,E));
The output consists of an unrooted tree and the lengths of the interior segments. The sum of squares is printed out, and if P = 2.0 Fitch and Margoliash's "average percent standard deviation" is also computed and printed out. This is the sum of squares, divided by N-2, and then square-rooted and then multiplied by 100:
APSD = ( SSQ / (N-2) )1/2 x 100.
where N is the total number of off-diagonal distance measurements that are in the (square) distance matrix. If the S (subreplication) option is in force it is instead the sum of the numbers of replicates in all the non-diagonal cells of the distance matrix. But if the L or R option is also in effect, so that the distance matrix read in is lower- or upper-triangular, then the sum of replicates is only over those cells actually read in. If S is not in force, the number of replicates in each cell is assumed to be 1, so that N is n(n-1), where n is the number of species. The APSD gives an indication of the average percentage error. The number of trees examined is also printed out.
The constants available for modification at the beginning of the program are: "smoothings", which gives the number of passes through the algorithm which adjusts the lengths of the segments of the tree so as to minimize the sum of squares, "delta", which controls the size of improvement in sum of squares that is used to control the number of iterations improving branch lengths, and "epsilonf", which defines a small quantity needed in some of the calculations. There is no feature saving multiple trees tied for best, partly because we do not expect exact ties except in cases where the branch lengths make the nature of the tie obvious, as when a branch is of zero length.
The algorithm can be slow. As the number of species rises, so does the number of distances from each species to the others. The speed of this algorithm will thus rise as the fourth power of the number of species, rather than as the third power as do most of the others. Hence it is expected to get very slow as the number of species is made larger.
7
Bovine 0.0000 1.6866 1.7198 1.6606 1.5243 1.6043 1.5905
Mouse 1.6866 0.0000 1.5232 1.4841 1.4465 1.4389 1.4629
Gibbon 1.7198 1.5232 0.0000 0.7115 0.5958 0.6179 0.5583
Orang 1.6606 1.4841 0.7115 0.0000 0.4631 0.5061 0.4710
Gorilla 1.5243 1.4465 0.5958 0.4631 0.0000 0.3484 0.3083
Chimp 1.6043 1.4389 0.6179 0.5061 0.3484 0.0000 0.2692
Human 1.5905 1.4629 0.5583 0.4710 0.3083 0.2692 0.0000
|
7 Populations
Fitch-Margoliash method version 3.69
__ __ 2
\ \ (Obs - Exp)
Sum of squares = /_ /_ ------------
2
i j Obs
Negative branch lengths not allowed
Name Distances
---- ---------
Bovine 0.00000 1.68660 1.71980 1.66060 1.52430 1.60430
1.59050
Mouse 1.68660 0.00000 1.52320 1.48410 1.44650 1.43890
1.46290
Gibbon 1.71980 1.52320 0.00000 0.71150 0.59580 0.61790
0.55830
Orang 1.66060 1.48410 0.71150 0.00000 0.46310 0.50610
0.47100
Gorilla 1.52430 1.44650 0.59580 0.46310 0.00000 0.34840
0.30830
Chimp 1.60430 1.43890 0.61790 0.50610 0.34840 0.00000
0.26920
Human 1.59050 1.46290 0.55830 0.47100 0.30830 0.26920
0.00000
+---------------------------------------------Mouse
!
! +------Human
! +--5
! +-4 +--------Chimp
! ! !
! +--3 +---------Gorilla
! ! !
1------------------------2 +-----------------Orang
! !
! +---------------------Gibbon
!
+------------------------------------------------------Bovine
remember: this is an unrooted tree!
Sum of squares = 0.01375
Average percent standard deviation = 1.85418
Between And Length
------- --- ------
1 Mouse 0.76985
1 2 0.41983
2 3 0.04986
3 4 0.02121
4 5 0.03695
5 Human 0.11449
5 Chimp 0.15471
4 Gorilla 0.15680
3 Orang 0.29209
2 Gibbon 0.35537
1 Bovine 0.91675
|
phylip-3.697/doc/gendist.html 0000644 0047320 0047320 00000031606 12406201172 015636 0 ustar joe felsenst_g
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved. License terms here.
This program computes any one of three measures of genetic distance from a set of gene frequencies in different populations (or species). The three are Nei's genetic distance (Nei, 1972), Cavalli-Sforza's chord measure (Cavalli- Sforza and Edwards, 1967) and Reynolds, Weir, and Cockerham's (1983) genetic distance. These are written to an output file in a format that can be read by the distance matrix phylogeny programs Fitch and Kitsch.
The three measures have somewhat different assumptions. All assume that all differences between populations arise from genetic drift. Nei's distance is formulated for an infinite isoalleles model of mutation, in which there is a rate of neutral mutation and each mutant is to a completely new allele. It is assumed that all loci have the same rate of neutral mutation, and that the genetic variability initially in the population is at equilibrium between mutation and genetic drift, with the effective population size of each population remaining constant.
Nei's distance is:
\ \
/_ /_ p1mi p2mi
m i
D = - ln ( ------------------------------------- ).
\ \ \ \
[ /_ /_ p1mi2]1/2 [ /_ /_ p2mi2]1/2
m i m i
where m is summed over loci, i over alleles at the m-th locus, and where
p1mi
is the frequency of the i-th allele at the m-th locus in population 1. Subject to the above assumptions, Nei's genetic distance is expected, for a sample of sufficiently many equivalent loci, to rise linearly with time.
The other two genetic distances assume that there is no mutation, and that all gene frequency changes are by genetic drift alone. However they do not assume that population sizes have remained constant and equal in all populations. They cope with changing population size by having expectations that rise linearly not with time, but with the sum over time of 1/N, where N is the effective population size. Thus if population size doubles, genetic drift will be taking place more slowly, and the genetic distance will be expected to be rising only half as fast with respect to time. Both genetic distances are different estimators of the same quantity under the same model.
Cavalli-Sforza's chord distance is given by
\ \ \
D2 = 4 /_ [ 1 - /_ p1mi1/2 p 2mi1/2] / /_ (am - 1)
m i m
where m indexes the loci, where i is summed over the alleles at the m-th locus, and where a is the number of alleles at the m-th locus. It can be shown that this distance always satisfies the triangle inequality. Note that as given here it is divided by the number of degrees of freedom, the sum of the numbers of alleles minus one. The quantity which is expected to rise linearly with amount of genetic drift (sum of 1/N over time) is D squared, the quantity computed above, and that is what is written out into the distance matrix.
Reynolds, Weir, and Cockerham's (1983) genetic distance is
\ \
/_ /_ [ p1mi - p2mi]2
m i
D2 = --------------------------------------
\ \
2 /_ [ 1 - /_ p1mi p2mi ]
m i
where the notation is as before and D2 is the quantity that is expected to rise linearly with cumulated genetic drift.
Having computed one of these genetic distances, one which you feel is appropriate to the biology of the situation, you can use it as the input to the programs Fitch, Kitsch or Neighbor. Keep in mind that the statistical model in those programs implicitly assumes that the distances in the input table have independent errors. For any measure of genetic distance this will not be true, as bursts of random genetic drift, or sampling events in drawing the sample of individuals from each population, cause fluctuations of gene frequency that affect many distances simultaneously. While this is not expected to bias the estimate of the phylogeny, it does mean that the weighing of evidence from all the different distances in the table will not be done with maximal efficiency. One issue is which value of the P (Power) parameter should be used. This depends on how the variance of a distance rises with its expectation. For Cavalli-Sforza's chord distance, and for the Reynolds et. al. distance it can be shown that the variance of the distance will be proportional to the square of its expectation; this suggests a value of 2 for P, which the default value for Fitch and Kitsch (there is no P option in Neighbor).
If you think that the pure genetic drift model is appropriate, and are thus tempted to use the Cavalli-Sforza or Reynolds et. al. distances, you might consider using the maximum likelihood program Contml instead. It will correctly weigh the evidence in that case. Like those genetic distances, it uses approximations that break down as loci start to drift all the way to fixation. Although Nei's distance will not break down in that case, it makes other assumptions about equality of substitution rates at all loci and constancy of population sizes.
The most important thing to remember is that genetic distance is not an abstract, idealized measure of "differentness". It is an estimate of a parameter (time or cumulated inverse effective population size) of the model which is thought to have generated the differences we see. As an estimate, it has statistical properties that can be assessed, and we should never have to choose between genetic distances based on their aesthetic properties, or on the personal prestige of their originators. Considering them as estimates focuses us on the questions which genetic distances are intended to answer, for if there are none there is no reason to compute them. For further perspective on genetic distances, I recommend my own paper evaluating different genetic distances (Felsenstein, 1985c), Reynolds, Weir, and Cockerham (1983), and the material in Nei's book (Nei, 1987).
The input to this program is standard and is as described in the Gene Frequencies and Continuous Characters Programs documentation file above. It consists of the number of populations (or species), the number of loci, and after that a line containing the numbers of alleles at each of the loci. Then the gene frequencies follow in standard format.
The options are selected using a menu:
Genetic Distance Matrix program, version 3.69 Settings for this run: A Input file contains all alleles at each locus? One omitted at each locus N Use Nei genetic distance? Yes C Use Cavalli-Sforza chord measure? No R Use Reynolds genetic distance? No L Form of distance matrix? Square M Analyze multiple data sets? No 0 Terminal type (IBM PC, ANSI, none)? ANSI 1 Print indications of progress of run? Yes Y to accept these or type the letter for one to change |
The A (All alleles) option is described in the Gene Frequencies and Continuous Characters Programs documentation file. As with Contml, it is the signal that all alleles are represented in the gene frequency input, without one being left out per locus. C, N, and R are the signals to use the Cavalli-Sforza, Nei, or Reynolds et. al. genetic distances respectively. The Nei distance is the default, and it will be computed if none of these options is explicitly invoked. The L option is the signal that the distance matrix is to be written out in Lower triangular form. The M option is the usual Multiple Data Sets option, useful for doing bootstrap analyses with the distance matrix programs. It allows multiple data sets, but does not allow multiple sets of weights (since there is no provision for weighting in this program).
The output file simply contains on its first line the number of species (or populations). Each species (or population) starts a new line, with its name printed out first, and then and there are up to nine genetic distances printed on each line, in the standard format used as input by the distance matrix programs. The output, in its default form, is ready to be used in the distance matrix programs.
A constant "epsilong" is available to be changed by the user if the program is recompiled which defines a small quantity that is used when checking whether allele frequencies at a locus sum to more than one: if all alleles are input (option A) and the sum differs from 1 by more than epsilong, or if not all alleles are input and the sum is greater than 1 by more then epsilon, the program will see this as an error and stop. You may find this causes difficulties if you gene frequencies have been rounded. I have tried to keep epsilong from being too small to prevent such problems.
The program is quite fast and the user should effectively never be limited by the amount of time it takes. All that the program has to do is read in the gene frequency data and then, for each pair of species, compute a genetic distance formula for each pair of species. This should require an amount of effort proportional to the total number of alleles over loci, and to the square of the number of populations.
The main change that will be made to this program in the future is to add provisions for taking into account the sample size for each population. The genetic distance formulas have been modified by their inventors to correct for the inaccuracy of the estimate of the genetic distances, which on the whole should artificially increase the distance between populations by a small amount dependent on the sample sizes. The main difficulty with doing this is that I have not yet settled on a format for putting the sample size in the input data along with the gene frequency data for a species or population.
I may also include other distance measures, but only if I think their use is justified. There are many very arbitrary genetic distances, and I am reluctant to include most of them.
5 10
2 2 2 2 2 2 2 2 2 2
European 0.2868 0.5684 0.4422 0.4286 0.3828 0.7285 0.6386 0.0205
0.8055 0.5043
African 0.1356 0.4840 0.0602 0.0397 0.5977 0.9675 0.9511 0.0600
0.7582 0.6207
Chinese 0.1628 0.5958 0.7298 1.0000 0.3811 0.7986 0.7782 0.0726
0.7482 0.7334
American 0.0144 0.6990 0.3280 0.7421 0.6606 0.8603 0.7924 0.0000
0.8086 0.8636
Australian 0.1211 0.2274 0.5821 1.0000 0.2018 0.9000 0.9837 0.0396
0.9097 0.2976
|
5
European 0.000000 0.078002 0.080749 0.066805 0.103014
African 0.078002 0.000000 0.234698 0.104975 0.227281
Chinese 0.080749 0.234698 0.000000 0.053879 0.063275
American 0.066805 0.104975 0.053879 0.000000 0.134756
Australian 0.103014 0.227281 0.063275 0.134756 0.000000
|
phylip-3.697/doc/images/ 0000755 0047320 0047320 00000000000 13212363632 014560 5 ustar joe felsenst_g phylip-3.697/doc/images/DrawGramCat.png 0000644 0047320 0047320 00000140643 12406201357 017431 0 ustar joe felsenst_g ‰PNG
IHDR G û ÷K!ù $iCCPICC Profile 8…UßoÛT>‰oR¤? XG‡ŠÅ¯US[¹ÆI“¥íJ¥éØ*$ä:7‰©Û鶪O{7ü@ÙH§kk?ì<Ê»øÎí¾kktüqóÝ‹mÇ6°nƶÂøد±-ümR;`zŠ–¡Êðv x# =\Ó%
ëoàYÐÚRÚ±£¥êùÐ#&Á?È>ÌÒ¹áЪþ¢þ©n¨_¨Ôß;j„;¦$}*}+ý(}'}/ýLŠtYº"ý$]•¾‘.9»ï½Ÿ%Ø{¯_aÝŠ]hÕkŸ5'SNÊ{äå”ü¼ü²<°¹_ “§ä½ðì öÍ ý ½t
³jMµ{-ñ4%×ÆTÅ„«tYÛŸ“¦R6ÈÆØô#§v\œå–Šx:žŠ'H‰ï‹OÄÇâ3·ž¼ø^ø&°¦õþ“0::àm,L%È3â:qVEô
t›ÐÍ]~ߢI«vÖ6ÊWÙ¯ª¯) |ʸ2]ÕG‡Í4Ïå(6w¸½Â‹£$¾ƒ"ŽèAÞû¾EvÝ mî[D‡ÿÂ;ëVh[¨}íõ¿Ú†ðN|æ3¢‹õº½âç£Hä‘S:°ßû éKâÝt·Ñx€÷UÏ'D; 7ÿ®7;_"ÿÑeó?Y qxl+ @ IDATxì|TE×ÆOHB½„Ы"ˆHQĆ ^åó±V챃¨AáU@DÀ‚€4ÒQzï½·PÈ7Ï$g¹ ›er³û ¿Í;åÌ™ÿ¹÷îafîlب
ë“ 8 ʉãÇ!999åxÊ “O¥I³'Ž28×ògÄQ?ƒœ*7ƒl&“ dÓgHll¬äÉ“GŽ =*… ¶ñˆˆÁ'W®\öfY©fRR’ôêÕKºwï.ùóç÷»©çŸ^7n,×]w$$$øU711Q :$E‹MÓξ}û¤H‘"iÒpräÈûý—½öîÝ+
·ÞzKž~úiÉ›7ïr™p&\kgž?qÈÑr¸–5¾cÇ{À&agý }ò¤”+TXÊ› ¹2
p‚THFe2“q=fFÊúª‡Î#œ:uÊÞÈ8ú
(¶2¾êk^ äxë×¹ÈÕ:8"x럷¶´/8ž-ßYÖŸ¸SžÆõèO}g_õœ}Vκéãþ”I_ÇÛy äxë×¹ÈÕ:8"ÐþÞ,–’¦¬2.á_N äÐþþ±öÆIk:¯ylâO•íë9Þúu.rµŽ¼ÿ3¶Üq3H´k×.‰ˆ2ŽQݘ) )bâÍ’3À‚ 4<<ÜÆ5Íž¤þQàš‡²HSƒœ4òñ¿üO ²5OeëÅ 2RE[9HÓöU®¶ 9§ú¦¯¯r=´]gº¯:(çÌ×öµ=äkšS'¤;ËàÜÒç9õÓ<=z“¯iN™ˆk ãèÔ ùÎ ¶QÝ5Ï)yÊß)KËj]Í£ýS DN†ÊJéóp®×§æéQmœq
kšÊÓ£ÖÁ9âjÄ5
qÚÿôs.…Èé¿ÊÊÉO¹;9kšÓ>â,sZjJ,} Îiÿ3m¦à¯Ï o¬ÁNy:mådîä›>ÝW ”uækûÚ ò5
GýŽq–±Ž?éóœúiž ½É×4‡HÕ:8A Á©¿¦Ù ó‡÷ÿéklŠ+&a“,H®bFŒ ðšÆq®ÆÖ<þ é¸ ´.ÎQ‘Ê@¾³-•¡upÄG/,M‡,œkÐ/_=ײÎ2Î4•©”³œS⪗ƵÎõPNÏ5î ÒµÊA®ê]P 2P ç(«u5
GÕº ¬ê€sÄ![ûˆé?å…úÈÃéОê„r˜ÒP'yhm¢}mCë" íâHÓ~8ãhGë@ äj9=¢
œ§Ú ¤«>(¶•Æ‘*ùÊz ¯´®rr !Ûý‘Ž‘"8 ØÚþÒK/µ#-Ú',V¾ùæ›í4Ú9s¦\tÑE²sçNÛvùòåí¨œ£–-[Ê—_~i§Ï œ#|©;׬ =üÎFŒðÛTh·\¹r²{÷nÙ¿¿u° #"IÂBq8g«V²N t…3 Ñæ¾lÖ¬™u 7½ýQi…‚ƒ …‘åæ´%T_|ñÅR³fM« Fh&Ožlûz¸¦pmÁþ#d€ :ÕeÕfÈG:¬Y³,¿íÛ·Û´xãt-3Ë àXªÍ!S㨋>ÞpÃ
vúî>ã¤bí¸T«VÍN…‚ â°‚öÛ û£,l…ö 'Òxÿ‡ÆýOûŸùý¯Ï܇¸ pîŒ;Ÿ¸wôY‰rZ \õ~²ÇŸ<@pƒ¡"B ”80ëŒöïÚ+Æ‘°¢EŠ˜ ä3q#] ٻÏ8 ûe¿Y¡×ì´
9§_ ú°ÒHÏ_4Vö:,áfî?<¿qŽÌGr›Â #/!B“¼(³À»„ç‹r SÝ kñ±ròà ãÄEIA3=W(waÉž"ïÈÉC²/×~IJ4Î…]¢`¬mº¡o‡àÔ° A >Zm£ d(|”C Èxx¡¼Ê@Yä£
8àƒ<”×2ÈW}Pe‘‡ó åvÐ>Ž8G òñAÐvœòоö º¢^Fö‡ Ô…®Î8ê M´yè®#”Ež:JÈGBú¸ÊÃ2PeðlMSΚ]Ôþ(‹ v(‡~è—ú]T&âβÊiN Qu‡Lmmg—ý¡O…
k‚*V¬hÃñŽÐ kitÊ
úÂÁÖÚà ýÀšpÅTêa
oÁá³<ôòÀéˆcªKC¥J•dË–-vô ›á0àÇI1
#8½xñbiÓ¦•ƒºz #®<•»ö#O¸v`Œ8á }Ä+ü° l 9xË Ñ1Â…€#ÚÇ ®ÈC(‹#>èd íÂŽÎöÑoä£>ÞøáXõúÒë²Uè¢6@ ÀvÆ
vÁ÷c=fGð N# NpC@}½ !zAÕçøh¿ í#àˆsä!hG}µ§ÊÔA{¼ÿùüÇuÃçÿéï(ðÀ½‚`w-Ô›7n&} `fAóÁYr¤Dq‰,RXÂË•‘°â%ñ"É»ÍÃ9<· ‰Ì-›$Hášµ¬`Ü|z“jCz£(YS¶ þSŽE†Kt>ó€Š6¯ææIùµã0h“œ`ó¶ M’‚¦¬>|p3ëŽ `з|±*²a÷‰:)ùÌÜZ8G¹S~Q9ò¾4OHBÒQ³MÀq‰+^Ã>$ÑqôQ¿ø A 8€t Ï=÷œ²Yáòº°æ6eËú3šd²åÉ7@= $ï »ºÙ¢ÓÖ9µ|v)[#KV©¾,Eñæ²,_»víRS±…µËkÐZ;ì5àc¶Ü™Ø¥ÏnfŽ¨nÐ ©œÊ¤z÷ó —Á‘mNéÉ©ÉbÔ>ó䇒ðdˆ'zìÙó ÷LèûÄõô²ñDRë/ç1éû¡ÏãÀ4ý9O“ÜJ ÀÇ ÆCÛʦï_nê/ &í¢‹U(û¾~
%Èâ(\¤¨©ÑÀkWÕ«Ó´5YS†…ƒÐ[Áð mJ®®i£K3ÃÛR]¶ðØr/UÆË‚$ŽIâ-VkÖB šÀLûßf° ždÙúT¶ÌÈh+î–°±Ô{#~Fy–xÙ›¶ì«©ø|ø+*XÆÒš"+¡Œ+
ÞÆ´³º±{œ£Ÿ92ºsçÎéŠÈž|°åÛ&‡?Ñd”÷Qóv%[¬étH§@N·ŸŽxN™ÍÛêì!^œ9s&F &.‚ùp>ïß8ï{ |+y ¼=ü³£,Ç*‰»;;6k£l•ª©4g—¯êm
š2+žæ´´WY‹
š f†·¥ºl±f¥‚NÓ¹”>ت ¹~fd´wKØX’݈ŸQž%^IckR³x3RßZ>}ú˜—¬•É©tžì¢å"v?Û¢ YN¶ô9×9Ø‹-Ò´«!;´3´ŸSc•íòuÁË–&¬,GVýV ÝGqw;Qág7 ýÝ•\«5iË–PÆ(BÛhüiÛZZË·UqÎÉÁc™©Ÿ•uÙU«]ÛÓokâONQN·ŸSýÎÎvy'‚%Ò‚mù÷¿g&QÒÎ<:ùD6¶Ôß%w£Pæ(GÞ~‘ýãxÅOcâQn]‡Ë R;/" ÖÌLÿ4Oòï…7ÎL¢¤ytD6AÀªÖ®…3¼ÇžæÖÀ§:*:è)vˆ E,C€•ª£è&=öøÄÅK8CÛï|Z¶pÛ,áã¸^g‰xÂTlE "íaçGË^ hñ7l@ íu´°‹»,EòWkö øwãÌ$ÑÝÎ<:"› `'q´—ž÷r'PD¾£žn§R\pz
Ðö4z(?`ɵ«ÓÊËÑÝbI;íðˆ`‚€ýð
Ç™o:ö÷HjùÙ‚•¿Ç_z/‚€ àĈ’vâÁÑA@ò7¢¤ó÷øKïA@œQÒN<8"š ‚@þF@”tþ é½ ‚€# JÚ‰GDA@ÈßȬü=þÒû<‚Àî/¾È’k8 ~4é#ãÇ;‚•ðò ¢¤óÝK‡ó"üV§®]»Òë=]œª{ ô´
ôô3!A@È¢¤3†›Ôœ
~™¿è>»^µhkçcbbÔ‹>l-/åA 5¢¤Sã!g‚@®D L™2JA/^ÜéägÙ„A cHàXÆp“Z‚€S!BïÅ5¢XzžwNQz²å”\Ò® %íd£ƒM?ý¨>ÿmø;•t[–/SéÛ~ÿ-Uº½'qq±ŠÏ-›íš-åÉ—Ÿeík¼µe
;ˆ±öŽ\KìV¬XZµj¥Ê:wî Ú·obô6 ___ :tÈ”o”g*dÇ‘lv°‘¢‚@¾D@”´“
{ÄíP|
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program carries out the Fitch-Margoliash and Least Squares methods,
plus a variety of others of the same family, with the assumption that all
tip species are contemporaneous, and that there is an evolutionary clock
(in effect, a molecular clock). This means that branches of the tree cannot
be of arbitrary length, but are constrained so that the total
length from the root of
the tree to any species is the same. The quantity minimized is the same
weighted sum of squares described in the Distance Matrix Methods documentation
file.
The options are set using the menu:
Most of the options are described in the Distance Matrix Programs documentation
file.
The D (methods) option allows choice between the Fitch-Margoliash
criterion and the Minimum Evolution method (Kidd and Sgaramella-Zonta, 1971;
Rzhetsky and Nei, 1993). Minimum Evolution (not to be confused with
parsimony) uses the Fitch-Margoliash criterion to fit branch lengths to each
topology, but then chooses topologies based on their total branch length
(rather than the goodness of fit sum of squares). There is no
constraint on negative branch lengths in the Minimum Evolution method;
it sometimes gives rather strange results, as it can like solutions
that have large negative branch lengths, as these reduce the total
sum of branch lengths!
Note that the User Trees (used by option U) must be
rooted trees (with a bifurcation at their base). If you take a user
tree from Fitch and try to evaluate it in Kitsch, it must first be
rooted. This can be done using Retree. Of the options
available in Fitch, the O option is
not available, as Kitsch estimates a rooted tree which cannot be
rerooted, and the G option is not
available, as global rearrangement is the default condition anyway. It
is also not possible to specify that specific branch lengths of a user tree
be retained when it is read into Kitsch, unless all of them are present. In
that case the tree should be properly clocklike. Readers who wonder why
we have not provided the feature of holding some of the user tree branch
lengths constant while iterating others are invited to tell us how they
would do it. As you consider particular possible patterns of branch
lengths you will find that the matter is not at all simple.
If you use a User Tree (option U) with branch lengths with Kitsch, and the
tree is not clocklike, when two branch lengths give conflicting positions
for a node, Kitsch will use the first of them and ignore the other. Thus
the user tree:
is nonclocklike, so it will be treated as if it were actually the tree:
The input is exactly the same as described in the Distance Matrix Methods
documentation file. The output is a rooted tree, together with the sum of
squares, the number of tree topologies searched, and, if the power P is at
its default value of 2.0, the Average Percent Standard Deviation is also
supplied. The lengths of the branches of the tree are given in a table,
that also shows for each branch the time at the upper end of the
branch. "Time" here really means cumulative branch length from the root, going
upwards (on the printed diagram, rightwards). For each branch, the
"time" given is for the node at the right (upper) end of the branch. It
is important to realize that the branch lengths are not exactly proportional to
the lengths drawn on the printed tree diagram! In particular, short
branches are exaggerated in the length on that diagram so that they are
more visible.
The method may be considered as providing an estimate of the
phylogeny. Alternatively, it can be considered as a phenetic clustering of
the tip species. This method minimizes an objective function, the sum of
squares,
not only setting the levels of the clusters so as to do so, but rearranging
the hierarchy of clusters to try to find alternative clusterings that
give a lower overall sum of squares. When the power option P is set to a
value of P = 0.0, so that we are minimizing a simple sum of squares
of the differences between the observed distance matrix and the expected one,
the method is very close in spirit to Unweighted Pair Group Arithmetic Average
Clustering (UPGMA), also called Average-Linkage Clustering. If the topology of
the tree is fixed and there turn out to be no branches of negative length, its
result should be the same as UPGMA in that case. But since it tries
alternative topologies and (unless
the N option is set) it combines nodes that otherwise could result in a reversal
of levels, it is possible for it to give a different, and better, result than
simple sequential clustering. Of course UPGMA itself is available as an
option in program Neighbor.
The U (User Tree) option requires a bifurcating tree, unlike Fitch, which
requires an unrooted tree with a trifurcation at its base. Thus the tree
shown below would be written:
((D,E),(C,(A,B)));
If a tree with a trifurcation at the base is by mistake fed into the U option
of Kitsch then some of its species (the entire rightmost furc, in fact) will be
ignored and too small a tree read in. This should result in an error message
and the program should stop. It is important to understand the
difference between the User Tree formats for Kitsch and Fitch. You may want
to use Retree to convert a user tree that is suitable for Fitch into one
suitable for Kitsch or vice versa.
An important use of this method will be to do a formal statistical test of
the evolutionary clock hypothesis. This can be done by comparing the sums
of squares achieved by Fitch and by Kitsch, BUT SOME CAVEATS ARE
NECESSARY. First, the assumption is that the observed distances are truly
independent, that no original data item contributes to more than one of them
(not counting the two reciprocal distances from i to j and from j to i). THIS
WILL NOT HOLD IF THE DISTANCES ARE OBTAINED FROM GENE FREQUENCIES, FROM
MORPHOLOGICAL CHARACTERS, OR FROM MOLECULAR SEQUENCES. It may be invalid even
for immunological distances and levels of DNA hybridization, provided that the
use of common standard for all members of a row or column allows an error in
the measurement of the standard to affect all these distances
simultaneously. It will also be invalid if the numbers have been collected in
experimental groups, each measured by taking differences from a common standard
which itself is measured with error. Only if the numbers in different cells
are measured from independent standards can we depend on the statistical
model. The details of the test and the assumptions are discussed in my review
paper on distance methods (Felsenstein, 1984a). For further and sometimes
irrelevant controversy on these matters see the papers by Farris (1981,
1985, 1986) and myself (Felsenstein, 1986, 1988b).
A second caveat is that the distances must be expected to rise linearly with
time, not according to any other curve. Thus it may be necessary to transform
the distances to achieve an expected linearity. If the distances have an upper
limit beyond which they could not go, this is a signal that linearity may
not hold. It is also VERY important to choose the power P at a value
that results in the standard deviation of the variation of the observed from the
expected distances being the P/2-th power of the expected distance.
To carry out the test, fit the same data with both Fitch and Kitsch,
and record the two sums of squares. If the topology has turned out the
same, we have N = n(n-1)/2 distances which have been fit with
2n-3
parameters in Fitch, and with n-1 parameters in Kitsch. Then the
difference between S(K) and S(F) has d1 = n-2
degrees of freedom. It is
statistically independent of the value of S(F), which has
d2 = N-(2n-3)
degrees of freedom. The ratio of mean squares
should, under the
evolutionary clock, have an F distribution with n-2 and
N-(2n-3) degrees of
freedom respectively. The test desired is that the F ratio is in the upper
tail (say the upper 5%) of its distribution. If the S (subreplication)
option is in
effect, the above degrees of freedom must be modified by noting that
N is not n(n-1)/2 but is the sum of the numbers of replicates of all
cells in the distance matrix read in, which may be either square or
triangular. A further explanation of the
statistical test of the clock is given in a paper of mine (Felsenstein, 1986).
The program uses a similar tree construction method to the other programs
in the package and, like them, is not guaranteed to give the best-fitting
tree. The assignment of the branch lengths for a given topology is a
least squares fit, subject to the constraints against negative branch lengths,
and should not be able to be improved upon. Kitsch runs more quickly than
Fitch.
The constant
available for modification at the beginning of the program is
"epsilon", which defines a small quantity needed in
some of the calculations. There is no feature saving multiple trees
tied for best,
because exact ties are not expected, except in cases where it should be
obvious from the tree printed out what is the nature of the tie (as when an
interior branch is of length zero).
address:
PHYLIP, the Phylogeny Inference Package, is a package of programs for
inferring phylogenies (evolutionary trees). It has been distributed since
1980, and has over 30,000 registered users, making it the most widely
distributed package of phylogeny programs. It is available free, from
its web site:
PHYLIP is available as source code in C, and also as executables for
some common computer systems. It can infer phylogenies by parsimony,
compatibility, distance matrix methods, and likelihood. It can also
compute consensus trees, compute distances between trees, draw trees,
resample data sets by bootstrapping or jackknifing, edit trees, and
compute distance matrices. It can handle data that are nucleotide
sequences, protein sequences, gene frequencies, restriction sites,
restriction fragments, distances, discrete characters, and continuous
characters.
The following copyright notice given below is intended to cover all source code, all
documentation, and all executable programs of the PHYLIP package. This is
a "BSD 2-Clause License" which is open source. It is not a GNU license and
does not insist that other materials distributed with PHYLIP be under a similar
license.
© Copyright 1980-2014, Joseph Felsenstein
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
PHYLIP comes with an extensive set of documentation files. These
include the main documentation file (this one), which you should read
fairly completely. In addition there are files for groups of programs,
including ones for the molecular sequence
programs, the distance matrix
programs, the
gene frequency and continuous characters
programs, the discrete characters programs,
and the tree drawing programs. Finally,
each program has its own documentation file. References for the
documentation files are all gathered together in this main documentation
file. A good strategy is to:
There is an excellent guide to using PHYLIP 3.6 also available. It was written by
Jarno Tuimala of the Center for Scientific Computing in Espoo, Finland and is
available as a PDF here. It is also
distributed at the main PHYLIP web site.
Here is a short description of each of the programs. For more detailed
discussion you should definitely read the documentation file for the
individual program and the documentation file for the group of programs
it is in. In this list the name of each program is a link which will
take you to the documentation file for that program. Note that there is no
program in the PHYLIP package called PHYLIP.
This section assumes that you have obtained PHYLIP as compiled executables
(for Windows, Mac OS X, or Linux), or else you have obtained the source code
and compiled it yourself (for Linux, Unix, Mac OS X, or Windows).
For the programs Drawtree and Drawgram you will also
need a recent version of Java installed on your computer to run them interactively.
Note that for machines for
which compiled executables are available, there will usually be no need for
you to have a compiler or compile the programs yourself. This section
describes how to run the programs. Later in this document we will
discuss how to download and install PHYLIP (in case you are
reading this without yet having done that). Normally you will only read
your copy of the documentation files after downloading and installing PHYLIP.
After describing the input files, we will describe how to run most of
the programs on
Windows, Mac OS X, Linux, and Unix systems). After that, we will give
special descriptions of the interactive Java interface for the tree-drawing
programs Drawgram and Drawtree, including how to run these interfaces on
Windows, Mac OS X, and Linux systems. These may require you to
download and install on your computer the most recent version of Oracle
Java, which is available from Oracle at no cost. We describe this below after
discussing input files.
For all of these types of machines, it is
important to have the input files for the programs (typically data files)
prepared in advance. They can be prepared in any editor, but it is important
that they be saved in Text Only ("flat ASCII") format, not in the format that
word processors such as Microsoft Word want to write (in Microsoft Word,
make sure that the data encoding used is "US ASCII", as using any of the
Unicode codings can cause trouble). It is up to you to read
the PHYLIP documentation files which describe the files formats that are
needed. There is a partial description in the next section of this document.
The input files can also be obtained by running a program that
produces output files in PHYLIP format (some of these programs do, and so do
programs by others such as sequence alignment programs such as ClustalW and
sequence format conversion programs such as Readseq). There is not any
input file editor available in any program in PHYLIP (you should not
simply start running one of the programs and then expect to click a mouse
somewhere to start creating a data file).
When they start running, the programs look first for input files with
particular names (such as infile, treefile, intree, or fontfile).
Exactly which file names they look for varies a bit from program to program,
and you should read the documentation file for the particular program to
find out. If you have files with those names the programs will use them
and not ask you for the file name. If they do not find files of those
names, the programs will say that they cannot find a file of that name, and
ask you to type in the file name.
For example, if Dnaml looks
for the file infile and does not find one of that name,
it prints the message:
This does not mean that an error
has occurred. All you need to do is to type in the name of the file.
(Joe, you need to rewrite or eliminate this paragraph, it is too condescending)
The program looks for the input files in the same folder that the
program is in (a folder is the same thing as a "directory"). In Windows,
Mac OS X, Linux, or Unix, if you are asked for the
file name you can type in the path to the file, as part of the name (thus,
if the file is in the folder containing the current folder, you can type in
a file name such as ../myfile.dna). If you do not know what a
"folder" is, or what "above" means, then you are a member of the new
generation who just clicks the mouse and assumes that a list of file names
will magically appear. (Typically members of this generation have no idea
where the files are on their system, and accumulate enormous amounts of
unnecessary clutter in their file systems.) In this case you should ask
someone to explain folders to you.
We have provided a Mac OS X version of the executables, in the form of
"universal binaries" that should run either on PowerMac or Intel iMac
systems (to ensure that they will run on both 32-bit and 64-bit
Mac OS X systems, we have made sure that we compiled the
executables as 32-bit executables). The programs can be
run by clicking on their icons. They open a Terminal window, and the
menu appears in it. Note that after the program is finished, the Terminal
window remains open, and operations can be done in it. You will have to
close the window yourself if you don't want it. The programs can be
terminated by typing control-C (press down the "control" key in the
lower-left corner of the keyboard and type "c").
It is also possible to run the executables from within a
Terminal window by typing the program name, but this is a little harder.
You will find the Terminal utility available in the Utilities folder in the
Applications folder.
You do need to have links made in the exe folder to the
programs. This can be done the first time you need them, by entering
the exe folder and opening a Terminal window, and then typing
source linkmac. This creates the proper links, and thereafter
you do not need to do this again. The programs can be run by typing
their names in a Terminal window whose current working directory is
exe The programs work well this way,
though the programs Drawgram and Drawtree may be slow to
open and close plotting windows.
The programs can be terminated by typing control-C or by
closing the Terminal window by using the red button in the upper-left
corner of the window.
One problem we have often encountered using Mac OS X is that it is possible for
data files to have the wrong kind of characters at the ends of their lines.
They may have carriage-return (ASCII/ISO 13 or control-M) characters at the
ends of their lines when they should instead have the Unix newline character
(ASCII/ISO 10 or control-J) there. This can happen with files transferred
from other operating systems or files produced in some word processors.
It results in segmentation-fault or memory errors. If you encounter these,
check this possibility carefully.
If you normally run Mac OS X applications using open
-a, you may need to use the command lsregister -f -r
/your/path/to/apps. You can find it with the command
locate lsregister.
Type the name of the program
in lower-case letters (such as dnaml). To terminate the program while
it is running, type Control-C (which means to press down on the Ctrl key
while typing the letter C).
On some systems you may need to type ./ before the program name,
so that in the above case it would be ./dnaml. This is mostly
needed if the user's PATH does not include their current directory, something
which is often done as a security precaution.
We no longer produce and distribute Mac OS 8 and Mac OS 9 executables
of the Phylip programs, as we no longer have access to these operating
systems to produce and test them. As a last resort, only
if you do not have access to a system that will run the current
distribution, you have two choices:
With version 3.695 we have released an interactive Java interface for
the tree-drawing programs, Drawgram and Drawtree. The reason
is that the graphic interface language for Mac OS X has changed from the Carbon GUI
to the Cocoa GUI, which would require a lot of rewriting of code.
The alternative X11 (X Windows) GUI machinery on Mac OS X has
been deprecated by Apple, and is showing its age on Linux systems.
Looking at available options, it seemed best to use Java to construct GUI
interfaces, as this could be done in a reasonably compatible way across all
three major platforms. There are disadvantages too -- to get full
compatibility we need to ask users to download the most recent available
Java from its maker, Oracle. That is not difficult but is a tiresome extra
step. Oracle owns Java, and Java is not public-source, but there seems to be
no sign that Oracle is going to make Java runtime machinery unavailable
or charge for it.
Not all Java implementations will run PHYLIP's Drawgram and Drawtree
GUIs. A reasonably compatible Java is distributed with Mac OS X, but no
Java is distributed along with Windows, and the Java distributed with
Linux distributions is unfortunately not compatible enough with our
Java GUI. So for these two platforms you will need to download Oracle Java. We
will give you instructions for that below.
The new GUI for Drawgram and Drawtree is a testbed for a general set of GUI
interfaces for all our programs, which will be present in version 4.0 when
that is distributed, which will be soon. The work you do to put a recent
version of Oracle Java on your system will make using version 4.0 easier.
For people who use Drawgram or Drawtree in a "pipeline" run by shell scripts,
there should be no interruption in your ability to do that.
The current C code for those programs can either be called by the Java GUI
or be run from
a command line or a shellscript (for which see below). Almost all of the
features of Drawgram and Drawtree are available from their character-mode
menu when run that way, except for the interactive previewing of plots. We
hope that the shell scripts will still work and will not need modification
for this version of PHYLIP.
To run the Drawgram or Drawtree programs, you find the Drawgram.jar or
Drawtree.jar files, which are Java Archive files in our folder of
executable programs. You can run them
by clicking on their icons. Detailed instructions for using the
interfaces are given in the general documentation file for tree-drawing
programs draw.html (which you should read),
and the documentation files for the
two programs drawgram.html
and drawtree.html.
To run the interactive interfaces of the tree-drawing programs Drawgram and
Drawtree, you need to have an appropriate version of Java installed on your
computer. If you have Java installed, you should test whether it is an
appropriate version by trying to run Drawgram or Drawtree (for this you
will need an input tree file present as well). Is it likely that you
have a compatible Java on your system?
Double-click on the icon for
the program. A window should open with a menu in it. Further dialog with the
program occurs
by typing on the keyboard in response to what you see in the window. The
programs can be terminated either by typing Control-C (which means to
press down on the Ctrl key while typing the letter C), or by using
the mouse to open the File menu in the upper-left corner of the program's
window area and then select Quit. Other than this, most PHYLIP programs
make no use of the mouse. The tree-drawing programs Drawtree and Drawgram
do allow use of the mouse to select some options.
The programs open a window for their menus. This window may be too small for
your tastes. They can be resized by tugging on the lower-right corner of the
window. In addition, the font may be too small. On most versions of Windows,
you can click on the small C:\ icon
symbol at the upper-left corner of the window, and choose the
Properties menu choice there. One of its tab options allows you
to change the font and size of
the print. I prefer large font sizes such as 16x12.
The programs can also be run in a Command Prompt window under Windows, in much
the same way as they were under the MSDOS operating system, which is what the
Command Prompt window emulates. Command Prompt windows can be open by
choosing that option in the Accessories menu which is in the All Programs menu.
Once in the Command Prompt window, make sure that you are in the
correct folder, using the cd command as needed to find the folder
where the executable PHYLIP programs are. Then type the name of the program
that you want to use in lower-case letters (such as
dnaml). To terminate the program while
it is running, type Control-C (which means to press down on the Ctrl key
while typing the letter C).
In running the programs, you may sometimes want to put them in background
so you can proceed with other work. On systems with a windowing environment
they can be put in their own window, and commands like the Unix and Linux
nice command used to make
them have lower priority so that they do not interfere with interactive
applications in other windows. This part of the discussion will
assume either a Windows system or a Unix or Linux system. I will
note when the commands work on one of these systems but not the other.
Mac OS X is actually Unix (surprise! surprise!) and you can
run PHYLIP programs in background on any Mac OS X system by simply following
the instructions for Unix, using a terminal window to do so if necessary.
(The Terminal utility can be found in the Utilities folder which is
inside the Applications folder).
If there is no windowing environment, or if you want to make PHYLIP programs
part of a larger workflow of some sort, on a Unix or Linux system you will
want to use an
ampersand (&) after the command file name when invoking it to put the
job in the background. You will have to put all the responses to the
interactive menu of the program into a file and tell the background job
to take its input from that file (we cover this below).
On Windows systems there is no & or nice command
but input and output redirection and command files work fine in a Commmand
window. A command file can either be invoked by clicking on its icon or
by typing its name from a Command Prompt window. The a file of commands must
have a name ending in .bat or .cmd, such as
foofile.bat. You can
run the batch file from a Command window by typing its name (such as
foofile) without the .bat.
Here are examples, for the different operating systems:
Here is an example for Windows, Linux, or using a Terminal window of
Mac OS X. Below you will find a separate example for Windows. If you
are using Windows you should read that section instead.
Suppose you want to run Dnaml in a background, taking its
input data from a file called sequences.dat, putting its interactive
output to file called screenout, and using a file called input as
the place to store the interactive input. The file input need only
contain two lines:
which is what you would have typed to run the program interactively, in
response to the program's request for an input file name if it did not
find a file named infile, in response the the menu.
To run the program in background, in Unix or Linux you would simply give the command:
dnaml < input > screenout &
These run the program with input responses coming from input and
interactive output being put into file screenout. The usual output
file and tree file will also be created by this run (keep that in mind
as if you run any other PHYLIP program from the same directory while
this one is running in background you may overwrite the output file from
one program with that from the other!).
If you wanted to give the program lower priority, so that it would
not interfere with other work, and you have Berkeley Unix type job control
facilities in your Unix or Linux (and you usually do), you can use the
nice command:
nice +10 dnaml < input > screenout &
which lowers the priority of the run. To also time the run and put the
timing at the end of screenout, you can do this:
nice +10 ( time dnapars < input ) >& screenout &
which I will not attempt to explain.
On Unix or Linux systems
you may also want to explore putting the interactive output into the
null file /dev/null so as to not be bothered with it (but then you
cannot look at it to see why something went wrong). If you have problems
with creating output files that are too large, you may want to
explore carefully the turning off of options in the programs you run.
If you are doing several runs in one, as for example when you do a
bootstrap analysis using Seqboot, Dnapars (say), and Consense, you
can use an editor to create a "command file" with these commands:
The command file might be named something like
foofile
It must be given
execute permission by using the command chmod +x foofile.
The job that foofile describes
can be run in background on Unix or Linux by giving the command
foofile &
Note that you must also have the interactive input
commands for Seqboot (including the random number seed), Dnapars, and
Consense in the separate files input1, input2, and input3.
If you have a Windows system and want to run Dnaml in a background, taking its
input data from a file called sequences.dat, putting its interactive
output to file called screenout, and using a file called input
as
the place to store the interactive input. The file input need only
contain two lines:
which is what you would have typed to run the program interactively, in
response to the program's request for an input file name if it did not
find a file named infile, in response the the menu.
To run the program in background, you can place the command
dnaml < input > screenout &
in a file called something like foofile.bat. This "batch file" that
has commands and has its name end in .bat or .cmd
can be run simply by double-clicking on the file icon, which will usually
have a picture of a gear. A Command Prompt windows (an MSDOS window) will then
open and the commands in the batch file will be run in it. Alternatively,
you can open a Command Prompt window yourself. It will be found in the
All Programs menu, as one of the options under Accessories. Make sure that
after it opens, you tell it to change its working directory to the one that
has the batch file in it.
The batch file with this command runs the program with input responses coming
from input and
interactive output being put into file screenout. The usual output
file and tree file will also be created by this run (keep that in mind
as, if you run any other PHYLIP program from the same directory while
this one is running in background, you may overwrite the output file from
one program with that from the other!).
Note also that when PHYLIP programs attempt to open a new output file (such as
outfile, outtree, or plotfile, if they see
a file of that name already in existence they will ask you if you want to
overwrite it, and offer alternatives including writing to another file,
appending information to that file, or quitting the program without writing to
he file. This means that in writing batch files it is important to know
whether there will be a prompt of this sort. You must know in advance
whether the file will exist. You may want to put in your batch file a
command that tests for the existence of a pre-existing output file and
if so, removes it, such as these commands in Unix, Linux, or Mac OS X:
You might even want to put in a command that creates a
file of that name, so that you can be sure it is there! Either way,
you will then know whether to put into your file of keyboard responses the
proper response to the inquiry about overwriting that output file.
Offhand, I do not know how to test for the existence of files in Windows, but
I suspect that there is a way.
Making the proper files of keyboard responses for use with command
files is most easily done if you prototype the process by simply
running the program and keeping a careful record of the keyboard
responses that you need to give to get the program to run properly.
Then create a file in an editor and type those keyboard responses into
it. Thus if the program requires that you answer a question about
what to do with the output file with a keyboard response of R,
then wants you to type a menu selection of U (to have it use a User tree),
then wants you to answer Y to end the menu, and another R to tell it to
replace the output file, you would have the file of keyboard responses
be
Since when you run the program interactively, each keyboard
response is ended by pressing the Enter key on your keyboard,
in the file of keyboard responses you must end each line after
typing the appropriate character.
Testing the keyboard responses with an interactive run will
be essential to having batch runs succeed.
The input files for PHYLIP programs must be prepared separately - there is
no data editor within PHYLIP. You can use a word processor (or text
editor) to prepare them yourself, or you can use a program that produces
a PHYLIP-format output.
With the 3.695 release of Phylip we have included a directory called TestData which
contains the data used to generate the examples shown in the individual program html pages
and the output files they produce. Within this TestData directory there is a subdirectory
that has the name of the program (for example contrast) and within that there are the files
contrastinfile.txt, contrastintree.txt and contrastoutfile.txt.
If you look at the Contrast documentation you can see infile,
intree, and outfile mentioned in the example. The testdata/contrast/*.txt
files exactly match those in the example, so if you wish to experiment with Contrast
you have both a good infile and a good intree and the outfile expected
from the example, if you set your conditions to match the example.
Sequence alignment programs such as ClustalW
commonly have an option to produce PHYLIP files as output, and some
other phylogeny programs, such as MacClade and TreeView, are capable of
producing a PHYLIP-format file.
It is very important that the input files be in "Text Only" or "ASCII" format. This means that they contain only printable ASCII/ISO
characters, and not any unprintable characters. Many word processors such
as Microsoft Word save their files in a format that contains unprintable
characters, unless you tell them not to. In the Microsoft Word family of
word processors, the first time you edit a file, when you go to Save
in the File menu,
the file the program will instead do a Save As function, and ask you
in what format you want the file to be written.
For these word processors, the next time you edit the same file, using
Save, the program should use those settings without asking you. If
you have some trouble getting an input file that the programs can read, look
into whether you properly set these options. This can be usually be done by
using the Save As choice in the File menu and making the
right settings.
Text editors such as the vi and emacs editors on
Unix and Linux (and available on Mac OS X too), or the pico
editor that comes with the pine
mailer program, produce their files in Text Only format and should not
cause any trouble.
The format of the input files is discussed below, and you should also
read the other PHYLIP documentation relevant to the particular type of
data that you are using, and the particular programs you want to run, as
there will be more details there.
For most of the PHYLIP programs, information comes from a series of
input files, and ends up in a series of output files:
The programs interact with the user by presenting a menu. Aside from the
user's choices from the menu, they read
all other input from files. These files have default names. The program
will try to find a file of that name - if it does not, it will ask the
user to supply the name of that file.
Input data such as DNA sequences
comes from a file whose default name is infile. If the user
supplies a tree, this is in a file whose default name is intree.
Values of weights for the characters are in weights, and the
tree plotting program need some digitized fonts which are supplied in
fontfile (all these are default names).
For example, if Dnaml looks
for the file infile and does not find one of that name,
it prints the message:
This simply means that it wants you to type in the name of the
input file.
When you run a program, you are in a current folder. If you run it by clicking
on an icon, the folder is the one that has the icon. If you run it by
typing the name of the program, the folder is the current folder when
you do that. The program will look for default files (such as infile
and intree) in that folder. When it writes files, their
default locations are also in the current folder.
The program need not actually be in the current folder. An icon can
sometimes be a link to a program located elsewhere. A program name
typed by you can contain a “path”, so that if you type
/usr/local/phylip/dnaml the program run will be located in
folder /usr/local/phylip. The operating system maintains a default
path for your account, which is a series of names of folders. When you
type the name of a program, the
operating system will look in that series of folders until it finds the
program, and then run it. But in all of these cases, the input and output
files will, by default, be in the current folder, even if the program
is located in some other folder.
Users can change where the input files are, or where the output files
go. If no file called infile is found in the current folder,
you will be asked to type the name of the file. In that case you
can type a filename with a path, such as foobar/mydata, and
in that case the program will look for file mydata in folder
foobar within the current folder. A similar process occurs
when the program cannot find file intree.
When the program starts to write an output file, such as outfile,
a similar series of events happens, with one important difference.
It is when a file outfile already exists in the current folder
that the user will be asked what to do. (In the case of input files,
it was when they did not exist that the user is asked what to do).
You will be given the opportunity to Replace the file, Append to the
file, write to a different File, or Quit. If you choose the response F
you will be asked for the name of the different file, and that is when
you can give a filename with a path, such as foobar/myoutput.out,
and the file will be written in that folder instead of the current folder.
Understanding which folder is the current folder, and whether there are
files named infile, intree, outfile, or
outtree there, is crucial to successfully running PHYLIP
programs, and making sure that they analyze the correct data set and
write their files in the right place.
I have tried to adhere to a rather stereotyped input and output
format. For the parsimony, compatibility and maximum likelihood programs,
excluding the distance matrix methods, the simplest version of the input
data file looks something like this:
The first line of the input file contains the number of species and the
number of characters (in this case sites). These are in free format, separated
by blanks. The information for each species follows, starting with a
ten-character species name (which can include blanks and some punctuation
marks), and continuing with the characters for that species. The name should
be on the same line as the first character of the data for that species.
(I will use the term "species" for the tips of the trees, recognizing
that in some cases these will actually be populations or individual gene
sequences).
The name should be ten characters in length, and either terminated
by a Tab character or filled out to the full
ten characters by blanks if shorter. Any printable ASCII/ISO character is
allowed in the name, except for parentheses ("(" and ")"), square
brackets ("[" and "]"), colon (":"), semicolon (";") and comma (",").
If you forget to extend the names to ten characters in length by blanks,
and do not terminate them with a Tab character,
the program will get out of synchronization with the contents of the data
file, and an error message will result. A Tab character that terminates
a name will not be taken as part of the name that is read; the name will then
automatically be filled with blanks to a total length of 10 characters.
In the
discrete-character programs, DNA sequence programs and protein sequence
programs the characters are each a
single letter or digit, sometimes separated by blanks. In
the continuous-characters programs they are real numbers with decimal points,
separated by blanks:
Latimeria 2.03 3.457 100.2 0.0 -3.7
The conventions about continuing the data beyond one line per species are
different between the molecular sequence programs and the others. The
molecular sequence programs can take the data in "aligned" or "interleaved"
format, in which we first have some lines giving the first part of each of the
sequences, then some
lines giving the next part of each, and so on. Thus the sequences might
look like this:
Note that in these sequences we have a blank every
ten sites to make them easier to read: any such blanks are allowed. The blank
line which separates the two groups of lines (the ones
containing sites 1-20 and ones containing sites 21-39) may or may not
be present. It is important that the number of sites in each
group be the same for all species (i.e., it will not be possible to run
the programs successfully if the first species line contains 20 bases, but
the first line for the second species contains 21 bases).
Alternatively, an option can be selected in the menu to take the data in
"sequential" format, with all of the data for the first species,
then all of the characters for the next species, and so on. This is also
the way that the discrete characters programs and the gene frequencies
and quantitative characters programs want to read the data. They do not
allow the interleaved format.
In the sequential format, the character data can run on to a new line at any
time (except in the middle of a species name or, in the case of continuous
character and distance matrix programs where you cannot go to a new line in
the middle of a real number). Thus it is legal to have:
Archaeopt 001100
or even:
Archaeopt
though note that the full ten characters of the species name must
then be present: in the above case there must be a blank after the "t". In all
cases it is possible to put internal blanks between any of the character
values, so that
Archaeopt 0011001101 0111011100
is allowed.
Note that you can convert molecular sequence data between the interleaved
and the sequential data formats by using the Rewrite option of the J
menu item in Seqboot.
If you make an error in the format of the input file, the programs can
sometimes detect that
they have been fed an illegal character or illegal numerical value and issue
an error message such as BAD CHARACTER STATE:, often printing out the
bad value, and sometimes the number of the species and character in which it
occurred. The program will then stop shortly after. One of the things which
can lead to a bad value is the omission of something earlier in the file, or
the insertion of something superfluous, which cause the reading of the file to
get out of synchronization. The program then starts reading things it
didn't expect, and concludes that they are in error. So if you see this error
message, you may also want
to look for the earlier problem that may have led to the program becoming
confused about what it is reading.
Some options are described below, but you should also read the documentation
for the groups of the programs and for the individual programs.
The menu is straightforward. It typically looks like this (this one is for
Dnapars):
If you want to accept the default settings (they are shown in the above case)
you can simply type Y followed by pressing on the Enter key.
If you want to change any of the options, you should type the letter
shown to the left of its entry in the menu. For example, to set a threshold
type T. Lower-case letters will also work. For many of the options
the program will ask for supplementary information, such as the value of
the threshold.
Note the Terminal type entry, which you will find on all menus. It
allows you to specify which type of terminal your screen is. The options
are an IBM PC screen, an ANSI standard terminal, or none.
Choosing zero (0) toggles
among these three options in cyclical order, changing each time the 0
option is chosen. If one of them is right for your terminal the screen will be
cleared before the menu is displayed. If none works, the none option
should probably be chosen. The programs should start with a terminal option
appropriate for your computer, but if they do not, you can change the
terminal type manually. This is particularly important in program Retree
where a tree is displayed on the screen - if the terminal type is set to the
wrong value, the tree can look very strange.
The other numbered options control which information the program will
display on your screen or on the output files. The option to Print
indications of progress of run will show information such as the names of
the species as they are successively added to the tree, and the
progress of rearrangements. You will usually want to see these as
reassurance that the program is running and to help you estimate how long
it will take. But if you are running the program "in background" as can be
done on multitasking and multiuser systems, and do not have the
program running in its own window, you may want to turn this option off so
that it does not disturb your use of the computer while the program is
running. Note also menu option 3, "Print out tree". This can be useful
when you are running many data sets, and will be using the resulting trees
from the output tree file. It may be helpful to turn off the printing out
of the trees in that case, particularly if those files would be too big.
Most of the programs write their output onto a file called (usually) outfile, and a representation of the trees found onto a file called
outtree.
The exact contents of the output file vary from program to program and also
depend on which menu options you have selected. For many programs, if you
select all possible output information, the output will consist of
(1) the name of the program and its
version number, (2) some of the input information printed out, and (3) a series of
phylogenies, some with associated information indicating how much change
there was in each character or on each part of the tree. A typical rooted tree
looks like this:
The interpretation of the tree is fairly straightforward: it "grows"
from left to right. The numbers at the forks are arbitrary and are used (if
present) merely to identify the forks. For many of the programs the tree
produced is unrooted. Rooted and unrooted trees are printed in nearly the
same form, but the unrooted ones are accompanied by the
warning message:
remember: this is an unrooted tree!
to indicate that this is an unrooted tree and to warn against
taking the position of its root too seriously. (Mathematicians still call
an unrooted tree a tree, though some systematists unfortunately use the term
"network" for an unrooted tree. This conflicts with standard mathematical
usage, which reserves the name "network" for a completely different kind of
graph). The root of this tree could be anywhere, say on the line leading
immediately to Mouse. As an exercise,
see if you can tell whether the following tree is or is not a different
one from the above:
(it is not different). It is important also to realize that the
lengths of the segments of the printed tree may not be significant: some
may actually represent branches of zero length, in the sense that there is no
evidence that
those branches are nonzero in length. Some of the diagrams of trees attempt
to print branches approximately proportional to estimated
branch lengths, while in others the lengths are purely conventional and
are presented just to make the topology visible. You will have to look closely
at the documentation that accompanies each program to see what it presents
and what is known about the lengths of the branches on the tree. The above
tree attempts to represent branch lengths approximately in the diagram. But
even in those cases, some of the smaller branches are likely to be
artificially lengthened to make the tree topology clearer. Here is what
a tree from Dnapars looks like, when no attempt is made to make the
lengths of branches in the diagram proportional to estimated branch
lengths:
When a tree has branch lengths, it will be accompanied by a table showing
for each branch the numbers (or names) of the nodes at each end of the
branch, and the length of that branch. For the first tree shown above,
the corresponding table is:
Ignoring the asterisks and the approximate confidence limits, which will be
described in the documentation file for Dnaml, we can see that the table
gives a more precise idea of what the lengths of all the branches are.
Similar tables exist in distance matrix and likelihood programs, as well
as in the parsimony programs Dnapars and Pars.
Some of the parsimony programs in the package can print out a table
of the number of steps that different characters (or sites) require on
the tree. This table may not be obvious at first. A typical example looks like
this:
The numbers across the top and down the side indicate which site
is being referred to. Thus site 23 is column "3" of row "20"
and has 1 step in this case.
There are many other kinds of information that can appear in the
output file, They vary from program to program, and we leave their
description to the documentation files for the specific programs.
In output from most programs,
a representation of the tree is also written into the tree file
outtree. The tree is specified by nested pairs
of parentheses, enclosing
names and separated by commas. We will describe how this works
below. If there are any blanks in the names,
these must be replaced by the underscore character "_". Trailing blanks
in the name may be omitted. The pattern of the parentheses indicates
the pattern of the tree by having each pair of parentheses enclose all
the members of a monophyletic group. The tree file could look like this:
In this tree the first fork separates the lineage leading to
Mouse and Bovine from the lineage leading to the rest. Within the
latter group there is a fork separating Gibbon from the rest, and so on.
The entire tree is enclosed in an outermost pair of parentheses. The tree ends
with a semicolon. In some programs such as Dnaml, Fitch, and Contml,
the tree will be unrooted. An unrooted tree should have its
bottommost fork have a
three-way split, with three groups separated by two commas:
Here the three groups at the bottom node are A, (B,C,D), and
(E,F). The single three-way split corresponds to one of the interior
nodes of the unrooted tree (it can be any interior node of the tree). The
remaining forks are encountered as you move out from that first node.
In newer programs, some are able to tolerate these other forks being
multifurcations (multi-way splits).
You should check the documentation files
for the particular programs you are using to see in which of these forms
you can expect the user tree to be in. Note that many of the programs
that actually estimate an unrooted tree (such as Dnapars) produce trees in the
treefile in rooted form! This is done for reasons of arbitrary internal bookkeeping. The placement of the root is arbitrary. We are working toward
having all programs be able to read all trees, whether rooted or unrooted,
multifurcating or bifurcating, and having them do the right thing with
them. But this is a long-term goal and it is not yet achieved.
For programs that infer branch lengths, these are given in the trees in the
tree file as real numbers following a colon, and placed immediately
after the group descended from that branch. Here is a typical tree
with branch lengths:
Note that the tree may continue to a new line at any time except in the
middle of a name or the middle of a branch length, although in trees
written to the tree file this will only be done after a comma.
These representations of trees are a subset of the standard adopted
on 24 June 1986 at the annual meetings of the Society for the Study of
Evolution by an informal committee (its final session in Newick's
lobster restaurant - hence its name, the Newick standard)
consisting of Wayne Maddison (author of MacClade), David Swofford (PAUP),
F. James Rohlf (NTSYS-PC), Chris Meacham (COMPROB and the original
PHYLIP tree drawing programs), James Archie,
William H.E. Day, and me. This standard is a generalization of
PHYLIP's format, itself based on a well-known representation of trees in
terms of parenthesis patterns which is due to the famous mathematician
Arthur Cayley, and which has been around for over a century. The
standard is now employed by most phylogeny computer programs but unfortunately
has yet to be decribed in a formal published description. Other
descriptions by me and by Gary Olsen can be accessed using the Web at:
Most of the programs allow various options that alter the amount of
information the program is provided or what is done with the
information. Options are selected in the menu.
A number of the options from the menu, the U (User tree), G (Global),
J (Jumble), O (Outgroup), W (Weights),
T (Threshold), M (multiple data sets), and the tree output options, are used
so widely that it is best to discuss them in this document.
The U (User tree) option. This option toggles between the default
setting, which allows the program to search for the best tree, and the
User tree setting, which reads a tree or trees ("user trees") from the input
tree file and evaluates them. The input tree file's
default name is intree. In many cases the programs will also
tolerate having the trees
be preceded by a line giving the number of trees:
An initial line with the number of trees was formerly
required, but this now can be omitted. Some programs require rooted
trees, some unrooted
trees, and some can handle multifurcating trees. You should read
the documentation for the particular program to find out which it
requires. Program Retree can be used to convert trees among
these forms (on saving a tree from Retree, you are asked whether
you want it to be rooted or unrooted).
In using the user tree option, check the pattern of parentheses
carefully. The programs do not always detect
whether the tree makes sense, and if it does not there will probably be
a crash (hopefully, but not inevitably, with an error message indicating
the nature of the problem). Trees written out by programs are
typically in the proper form.
The G (Global) option. In the programs which construct trees (except for
Neighbor, the "...penny" programs and Clique, and of course
the "...move" programs where you construct the trees yourself),
after all species have been added to the tree a rearrangements phase
ensues. In most of these programs the rearrangements are automatically
global, which in this case means that subtrees will be removed from the tree
and put back on in all possible ways so as to have a better chance of
finding a better tree. Since this can be time consuming (it roughly
triples the time taken for a run) it is left as an option in some of the
programs, specifically Contml, Fitch, Dnaml and Proml. In these programs
the G menu option toggles between the default of local rearrangement and
global rearrangement. The rearrangements are explained more below.
The J (Jumble) option. In most of the tree construction programs
(except for the "...penny" programs and Clique), the exact
details of the search of different trees depend on the order of input of
species. In these programs J option enables you to tell the program to use
a random number
generator to choose the input order of species. This option is toggled on
and off by
selecting option J in the menu. The program will then prompt you for
a "seed" for the random number generator. The seed should be an integer
between 1 and 232-3 (which is 4,294,967,293), and should be
of form 4n+1,
which means that it must give a remainder of 1 when divided by 4. This can be
judged by looking at the last two digits of the number (for example, in the
upper limit given above, the last two digits are 93, which is of form 4n+1. Each different seed
leads to a different sequence of addition of species. By simply changing the
random number seed and re-running the programs one can look for other, and
better trees. If the seed entered is not odd, the program will not proceed,
but will prompt for another seed.
The Jumble option also causes the program to ask you how many times you
want to restart the process. If you answer 10, the program will
try ten different orders of species in constructing the trees, and the
results printed out will reflect this entire search process (that is,
the best trees found among all 10 runs will be printed out, not the
best trees from each individual run).
Some people have asked what are good values of the random number seed.
The random number seed is used to start a process of choosing "random"
(actually pseudorandom) numbers, which behave as if they were
unpredictably randomly chosen between 0 and 232-1 (which is
4,294,967,295). You could put in the number 133 and find that the
next random number was 221,381,825. As they are effectively
unpredictable, there is no such thing as a choice that is better than
any other, provided that the numbers are of the form 4n+1. However
if you re-use a random number seed, the sequence of random numbers
that result will be the same as before, resulting in exactly the same
series of choices, which may not be what you want.
The O (Outgroup) option. This specifies which species is to
have the root of the tree be on the line leading to it. For example, if the
outgroup is a species "Mouse" then the root of the tree will be placed in the
middle of the branch which is connected to this species, with Mouse branching
off on one side of the root and the lineage leading to the rest of the tree
on the other. This option is toggled on and off by choosing O in the
menu (the alphabetic character O, not the digit 0). When it
is on, the program will then prompt for the
number of the outgroup (the species being taken in the numerical order that
they occur in the input file). Responding by typing 6 and then an
Enter character indicates that the sixth species in the data
(the 6th in the first set of data if there are multiple data sets)
is taken as the outgroup. Outgroup-rooting will not be attempted if the
data have already established a root for the tree from some other
consideration, and may not be if it is a user-defined tree,
despite your invoking the option. Thus programs such as Dollop that
produce only rooted trees do not allow the Outgroup option. It is also
not available in Kitsch, Dnamlk, Promlk or Clique. When it is used, the tree as
printed out is still listed as being an
unrooted tree, though the outgroup is connected to the bottommost node
so that it is easy to visually convert the tree into rooted form.
The T (Threshold) option. This sets a threshold forn the
parsimony programs such that if the
number of steps counted in a character is higher than the threshold, it
will be taken to be the threshold value rather than the actual number of
steps. The default is a threshold so high that it will never be
surpassed (in which case the steps whill simply be counted). The T
menu option toggles on and off asking the user to
supply a threshold. The use of thresholds to obtain methods intermediate
between parsimony and compatibility methods is described in my 1981b paper.
When the T option is in force, the program
will prompt for the numerical threshold value. This will be a positive
real number greater than 1. In programs Mix, Move, Penny, Protpars,
Dnapars, Dnamove, and Dnapenny, do not use threshold values less
than or equal to 1.0, as they have no meaning and lead to a tree which
depends only on considerations such as the input order of species and not at
all on the character state data! In programs Dollop, Dolmove, and Dolpenny
the threshold should never be 0.0 or less, for the same
reason. The T option is an
important and underutilized one: it is, for example, the only way in this
package (except for program Dnacomp) to do a compatibility analysis when there
are missing data. It is a method of de-weighting characters that evolve
rapidly. I wish more people were aware of its properties.
The M (Multiple data sets) option. In menu programs there is an
M menu
option which allows one to toggle on the multiple data sets option. The
program will ask you how many data sets it should expect. The data sets
have the same format as the first data set. Here is a (very small) input file
with two five-species data sets:
The main use of this option will be to allow all of the methods in these
programs to be bootstrapped. Using the program Seqboot one can take any
DNA, protein, restriction sites, gene frequency or binary character data set and
make multiple data sets by bootstrapping. Trees can be produced for all of
these using the M option. They will be written on the tree output file if
that option is left in force. Then the program Consense can be used with
that tree file as its input file. The result is a majority rule consensus
tree which can be used to make confidence intervals. The present version
of the package allows, with the use of Seqboot and Consense and the M option,
bootstrapping of many of the methods in the package.
Programs Dnaml, Dnapars and Pars can also take multiple weights
instead of multiple data sets. They can then do bootstrapping by
reading in one data set, together with a file of weights that show how
the characters (or sites) are reweighted in each bootstrap sample. Thus a
site that is omitted in a bootstrap sample has effectively been given
weight 0, while a site that has been duplicated has effectively been
given weight 2. Seqboot has a menu selection to produce the file of
weights information automatically, instead of producing a file of
multiple data sets. It can be renamed and used as the input weights file.
The W (Weights) option. This signals the program that, in
addition to the data set, you want to read in a series of weights that
tell how many times each character is to be counted. If the weight
for a character is zero (0) then that character is in effect to
be omitted when the tree is evaluated. If it is (1) the
character is to be counted once. Some programs allow weights greater than
1 as well. These have the effect that the character is counted as
if it were present that many times, so that a weight of 4 means that the
character is counted 4 times.
The values 0-9 give weights 0 through 9, and the
values A-Z give weights 10 through 35. By use of the weights we can
give overwhelming weight to some characters, and drop others from the
analysis. In the molecular sequence programs only two values of the
weights, 0 or 1 are allowed.
The weights are used to analyze subsets of the characters, and also can be
used for resampling of the data as in bootstrap and jackknife resampling.
For those programs that allow weights to be greater than 1, they can also
be used to emphasize information from some characters more strongly than
others. Of course, you must have some rationale for doing this.
The weights are provided as a sequence of digits. Thus they might be
10011111100010100011110001100
The weights are to be provided in an input file
whose default name is weights. The weights in it are
a simple string of digits. Blanks in the weightfile are skipped over and
ignored, and the weights can continue to a new line. In programs such as
Seqboot
that can also output a file of weights, the input weights have a default
file name of inweights, and the output file name has a default
file name of outweights.
Weights can be used to analyze different subsets of characters (by weighting
the rest as zero). Alternatively, in the discrete characters programs
they can be used to force a certain
group to appear on the phylogeny (in effect confining consideration to only
phylogenies containing that group). This is done by adding an imaginary
character that has 1's for the members of the group, and 0's
for all the
other species. That imaginary character is then given the highest weight
possible: the result will be that any phylogeny that does not contain that
group will be penalized by such a heavy amount that it will not (except in
the most unusual circumstances) be considered. Of course, the new character
brings extra steps to the tree, but the number of these can be calculated
in advance and subtracted out of the total when reporting the results. This
use of weights is an important one, and one sadly ignored
by many users who could profit from it. In the case of molecular sequences
we cannot use weights this way, so that to force a given group to appear we
have to add a large extra segment of sites to the molecule, with (say) A's
for that group and C's for every other species.
The option to write out the trees into a tree file. This specifies that you
want the program to write
out the tree not only on its usual output, but also onto a file in
nested-parenthesis notation (as described above). This option is sufficiently
useful that it is turned on by default in all programs that allow it. You
can optionally turn it off if you wish, by typing the appropriate number
from the menu (it varies from program to program). This option is useful for
creating tree files that can be directly read into the programs, including
the consensus tree and tree distance programs, and the tree plotting programs.
The output tree file has a default name of outtree.
The (0) terminal type option . (This is the digit 0, not
the alphabetic character O). The program will default to
one particular assumption about your terminal (ANSI in the case of Linux,
Unix, or Mac OS X, and IBM PC in the case of
Windows). You can
alternatively select it to be either an IBM PC, or nothing.
This affects the ability of the programs to clear the screen when they
display their menus, and the graphics characters used to display trees
in the programs Dnamove, Move, Dolmove, and Retree. In the case of Windows,
the screen will clear properly with either the IBM PC or the ANSI settings,
but the graphics characters needed by Move, Dnamove, Dolmove, or Retree
will display correctly only with the IBM PC setting.
All of the programs except Factor, Dnadist, Gendist, Dnainvar, Seqboot,
Contrast, Retree, and the plotting and
consensus tree programs act to construct an estimate of a phylogeny. Move,
Dolmove, and Dnamove let you construct it yourself by hand. All of
the rest but Neighbor, the "...penny" programs and Clique make use of
a common approach involving additions and rearrangements. They are
trying to minimize or maximize some quantity over the space of all
possible evolutionary trees. Each program contains a part that, given
the topology of the tree, evaluates the quantity that is being minimized
or maximized. The straightforward approach would be to evaluate all
possible tree topologies one after another and pick the one which,
according to the criterion being used, is best. This would not be
possible for more than a small number of species, since the number of
possible tree topologies is enormous. A review of the literature on the
counting of evolutionary trees will be found one of my papers
(Felsenstein, 1978a) and in my book (Felsenstein, 2004, chapter 3).
Since we cannot search all topologies, these programs are not
guaranteed to always find the best tree, although they seem to do quite
well in practice. The strategy they employ is as follows: the species
are taken in the order in which they appear in the input file. The
first two (in some programs the first three) are taken and a tree
constructed containing only those. There is only one possible topology for
this tree. Then the next species is taken, and we consider where it
might be added to the tree. If the initial tree is (say) a rooted tree
with two species and we want the resulting three-species tree to be a
bifurcating tree, there are only three places where we could add the
third species. Each of these is tried, and each time the resulting tree is
evaluated according to the criterion. The best one is chosen to be the
basis for further operations. Now we consider adding the fourth
species, again at each of the five possible places that would result in
a bifurcating tree. Again, the best of these is accepted. This is
usually known as the Sequential Addition strategy.
The process continues in this manner, with one important exception. After
each species is added, and before the next
is added, a number of rearrangements of the tree are tried, in an effort
to improve it. The algorithms move through the tree, making all
possible local rearrangements of the tree. A local rearrangement involves an
internal segment of the tree in the following manner. Each internal
segment of the tree is of this form (where T1, T2, and T3 are subtrees
- parts of the tree that can contain further forks and tips):
the segment we are discussing being indicated by the asterisks. A local
rearrangement consists of switching the subtrees T1 and T3 or T2 and T3,
so as to obtain one of the following:
Each time a local rearrangement is successful in finding a better tree,
the new arrangement is accepted. The phase of local rearrangements does
not end until the program can traverse the entire tree, attempting local
rearrangements, without finding any that improve the tree.
This strategy of adding species and making local rearrangements will look
at about (n-1)x(2n-3) different topologies, though if
rearrangements are frequently successful the number may be larger. I
have been describing the strategy when rooted trees are being
considered. For unrooted trees there is a precisely similar strategy,
though the first tree constructed may be a three-species tree and the
rearrangements may not start until after the addition of the fifth
species.
These local rearrangements have come to be called Nearest Neighbor
Interchanges (NNIs) in the phylogeny literature.
Though we are not guaranteed to have found the best tree topology,
we are guaranteed that no nearby topology (i. e. none accessible by a
single local rearrangement) is better. In this sense we have reached a
local optimum of our criterion. Note that the whole process is
dependent on the order in which the species are present in the input
file. We can try to find a different and better solution by reordering
the species in the input file and running the program again (or, more
easily, by using the J option). If none of
these attempts finds a better solution, then we have some indication
that we may have found the best topology, though we can never be certain
of this.
Note also that a new topology is never accepted unless it is better
than the previous one, so that the rearrangement process can never fall
into an endless loop. This is also the way ties in our criterion are
resolved, namely by sticking with the tree found first. However, the tree
construction programs other than Clique, Contml, Fitch,
and Dnaml do keep a record of all trees found that are tied with the best one
found. This gives you some immediate idea of which parts of the tree can be
altered without affecting the quality of the result.
A feature of most of the programs, such as Protpars, Dnapars,
Dnacomp, Dnaml, Dnamlk, Restml, Kitsch, Fitch, Contml, Mix, and Dollop,
is "global" optimization of the tree. In four of these (Contml,
Fitch, Dnaml and Dnamlk) this is an option, G. In the others it
automatically applies. When
it is present there is an additional stage to the search for the best tree.
Each possible subtree is removed from the tree from the tree and added back in
all possible places. This process continues until all subtrees can be removed
and added again without any improvement in the tree. The purpose of this
extra rearrangement is to make it less likely that one or more a species gets
"stuck" in a suboptimal region of the space of all possible trees. The use of
global optimization results in approximately a tripling (3 x ) of the run-time,
which is why I have left it as an option in some of the slower programs.
What PHYLIP calls "global" rearrangements are more properly called
SPR (subtree pruning and regrafting) by Swofford et. al. (1996) as distinct
from the NNI (nearest neighbor interchange) rearrangements that PHYLIP
also uses, and the TBR (tree bisection and reconnection) rearrangements
that it does not use. My book (Felsenstein, 2004, chapter 4) contains
a review of work on these and other rearrangements and search methods.
The programs doing global optimization print out a dot "." after each group is
removed and re-added to the tree, to give the user some sign that the
rearrangements are proceeding. A new line of dots is started whenever a new
round of global rearrangements is started following an improvement in the
tree. On the line before the dots are printed there is printed a bar of
the form "!---------------!" to show how many dots
to expect. The dots will
not be printed out at a uniform rate, but the later dots, which represent
removal of larger groups from the tree and trying them consequently in fewer
places, will print out more quickly. With some compilers each row of dots may
not be printed out until it is complete.
It should be noted that Penny, Dolpenny, Dnapenny and Clique use a more
sophisticated strategy of "depth-first search" with a "branch and bound"
search method that guarantees that all
of the best trees will be found. In the case
of Penny, Dolpenny and Dnapenny there can be a considerable sacrifice of
computer time if the number of species is greater than about ten: it is a
matter for you to consider whether it is worth it for you to guarantee finding
all the most parsimonious trees, and that depends on how much free computer
time you have! Clique finds all largest cliques, and does so without undue
burning of computer time. Although all of these problems that have been
investigated fall into the
category of "NP-hard" problems that in effect do not have a rapid solution,
the cases that cause this trouble for the largest-cliques algorithm in
Clique apparently are not biologically realistic and do not occur in actual
data.
As just mentioned, for most of these programs the search depends on the order
in which the species are entered into the tree. Using the J (Jumble)
option you can supply a random number seed which will allow the program to put
the species in in a random order. Jumbling can be
done multiple times. For example, if you tell the program to do it
10 times, it will go through the tree-building process 10 times, each with a
different random order of adding species. It will keep a record of the trees
tied for best over the whole process. In other words, it does not just
record the best trees from each of the 10 runs, but records the best ones
overall. Of course this is slow, taking 10 times longer than a single run.
But it does give us a much greater chance of finding all of the most
parsimonious trees. In the terminology of Maddison (1991) it
can find different "islands" of trees. The present algorithms do not
guarantee us to find all trees in a given "island" from a single run, so
multiple runs also help explore those "islands" that are found.
For the parsimony and compatibility programs, one can have a perfect tie
between two or more trees. In these programs these trees are all
saved. For the newer parsimony programs such as Dnapars and Pars,
global rearrangement is carried out on all of these tied trees. This can
be turned off in the menu.
For trees with criteria which are real numbers, such as the distance
matrix programs Fitch and Kitsch, and the likelihood programs Dnaml,
Dnamlk, Contml, and Restml, it is difficult to get an exact tie between
trees. Consequently these programs save only the single best tree
(even though the others may be only a tiny bit worse).
In practice, it is advisable to use the Jumble option to evaluate many
different orderings of the input species. It is advisable to use the
Jumble option and specify that it be done many times (as many as
different orderings
of the input species). (This is usually not necessary when bootstrapping,
though the programs will then default to doing it once to avoid artifacts
caused by the order in which species are added to the tree.)
People who want a magic "black box" program whose results they do
not have to question (or think about) often are upset that these
programs give results that are dependent on the order in which the species
are entered in the data. To me this property is an advantage, for it
permits you to try different searches for better trees, simply by
varying the input order of species. If you do not use the multiple Jumble
option, but do multiple individual runs instead, you
can easily decide which to pay most attention to - the one or ones that
are best according to the criterion employed (for example, with parsimony,
the one out of the runs that results in the tree with the fewest changes).
In practice, in a single run, it usually seems best to put species that are
likely to be sources of confusion in the topology last, as by the time they are
added the arrangement of the earlier species will have stabilized into a
good configuration, and then the last few species will by fitted into
that topology. There will be less chance this way of a poor initial
topology that would affect all subsequent parts of the search. However,
a variety of arrangements of the input order of species should be tried,
as can be done if the J option is used,
and no species should be kept in a fixed place in the order of input.
Note that the results of the "...penny" programs and Clique
are not sensitive to the input order of species, and Neighbor is only
slightly sensistive to it, so that multiple Jumbling is not possible
with those programs. Note also that with global search, which
is standard in many programs and in others is an
option, each group (including
each individual species) will be removed and re-added in all possible
positions, so that a species causing confusion will have more chance of moving
to a new location than it would without global rearrangement.
An innovative search strategy was developed by Kevin Nixon (1999). If one
uses a manual rearrangement program such as Dnamove, Move, or Dolmove, and
look at the distribution of characters on the trees, you will see some
characters whose distributions appear to recommend alternative groupings.
One would want a program that automatically found such alternative
suggestions and used them to rearrange the tree so as to explore trees that
had those groups. Nixon had the idea of using resampling methods to
do this. Using either bootstrap or jackknife sampling, one can make data
sets that emphasize randomly sampled subsets of characters. We
then search for trees that fit those data sets. After finding them, we
revert to the initial data set and then search using those trees as
starting points. This sampling allows us to explore parts of tree space
recommended by particular subsets of characters. (This is not exactly
Nixon's original strategy, which started the searches for each resampled
data set from the best tree found so far. For each resampled data set we
instead start from scratch, doing sequential addition of taxa.)
Nixon's method has proven to be very effective in searching for most
parsimonious trees -- it is currently the state of the art for that.
Nixon called his method the "parsimony ratchet", but actually it can be
applied straightforwardly to any method of phylogeny inference that has an
optimality criterion, including likelihood and least squares distance methods.
Starting with version 3.7, PHYLIP programs have the ability to search by
rearranging a tree supplied to them by the user. This makes it possible to
implement our variant of Nixon's strategy. You need to do so in multiple steps:
There is some more information on how this may be done in the documentation
files for Seqboot and for the individual tree inference programs.
Probably the most important thing to keep in mind while running any of the
parsimony or compatibility programs is not
to overinterpret the result. Some users treat the set of most parsimonious
trees as if it were a confidence interval. If a group appears in all of the
most parsimonious trees then they treat it as well established. Unfortunately
the confidence interval on phylogenies appears to be much
larger than the set of all most parsimonious trees (Felsenstein, 1985b).
Likewise, variation of result among different methods will not be a good
indicator of the size of the confidence interval. Consider a simple data set
in which, out of 100 binary characters, 51 recommend the unrooted tree
((A,B),(C,D)) and 49 the tree ((A,D),(B,C)). Many different
methods will all give the same result on
such a data set: they will estimate the tree as ((A,B),(C,D)).
Nevertheless it is
clear that the 51:49 margin by which this tree is favored is not statistically
significantly different from 50:50. So consistency among different methods
is a poor guide to statistical significance.
C compilers differ in efficiency of the code they generate,
and some deal with some features of the language better than with
others. Thus a program which is unusually fast on one computer may be
unusually slow on another. Nevertheless, as a rough guide to relative
execution speeds, I have tested the programs on three data sets, each of
which has 10 species and 40 characters. The first is an imaginary one
in which all characters are compatible - ("The Willi Hennig Memorial
Data Set" as J. S. Farris once called ones like it). The second is the binary
recoded form of the fossil horses data set of Camin and Sokal (1965).
The third data set has data that is completely random: 10 species and 20
characters that have a 50% chance that each character state is 0 or
1 (or A or G). The data sets thus range from a completely
compatible one in which there is no homoplasy (paralellism or convergence),
through the horses data set, which requires 29 steps where the possible
minimum number would be 20, to the random data set, which requires 49 steps.
We can thus see how this increasing messiness of the data affects running
times. The three data sets have all had 20 sites of A's added to the
end of each sequence, so as to prevent likelihood or distance matrix programs
from having infinite branch lengths (the test data sets used for timing
previous versions of PHYLIP were the same except that they lacked these
20 extra sites).
Here are the nucleotide sequence versions of the three data sets:
Here are the timings of many of the version 3.6 programs on these three data
sets as run after being compiled by Gnu C (version 3.2) and run on an
AMD Athlon XP 2200+ computer under Linux.
In all cases the programs were run under the default options with optimized
compiler switches (-03 -fomit-frame-pointer), except as
specified here.
The data sets used for the discrete characters programs have 0's and 1's
instead of A's and C's. For Contml the A's and C's
were made into 0.0's and 1.0's and considered as 40 2-allele loci.
For the distance programs 10 x 10 distance matrices were
computed from the three data sets.
For the restriction sites programs A and C were changed into
+ and -. It does not
make much sense to benchmark Move, Dolmove, or Dnamove, although when there
are many characters and many species the response time after each
alteration of the tree should be proportional to the product of the number of
species and the number of characters.
For Dnaml, Dnamlk, and Dnadist the frequencies of the four bases were
set to be equal rather than determined empirically as is the default. For
Restml the number of enzymes was set to 1.
In most cases, the benchmark was made more accurate by analyzing 100 data
sets using the M (Multiple data sets) option and dividing the resulting
time by 100. Times were determined as user times using the Linux time
command. Several patterns will be apparent from this. The algorithms (Mix,
Dollop, Contml, Fitch, Kitsch, Protpars, Dnapars, Dnacomp, and
Dnaml, Dnamlk, Restml) that use the above-described addition strategy have
run times that do not depend strongly on the messiness of the data. The only
exception to this is that if a data set such as the Random data requires
extra rounds of global rearrangements it takes longer. The
programs differ greatly in run time: the protein likelihood programs
Proml and Promlk were very slow, and the other likelihood programs
Restml, Dnaml and
Contml are slower than the rest of the programs. The protein sequence parsimony
program, which has to do a considerable amount of bookkeeping to keep track of
which amino acids can mutate to each other, is also relatively slow.
Another class of algorithms includes Penny, Dolpenny, Dnapenny and Clique.
These are branch-and-bound methods: in principle they should have execution
times that rise exponentially with the number of species and/or
characters, and they might be much more sensitive to messy data. This is
apparent with Penny, Dolpenny, and Dnapenny, which go from being reasonably
fast with clean data to very slow with messy data. Dolpenny is particularly
slow on messy data - this is because this algorithm cannot make use of some of
the lower-bound calculations that are possible with Dnapenny and Penny. Clique
is very fast on all
data sets. Although in theory it should bog down if the number of cliques in
the data is very large, that does not happen with random data, which in
fact has few cliques and those small ones. Apparently the "worst-case"
data sets that cause exponential run time are much rarer for Clique than for
the other branch-and-bound methods.
Neighbor is quite fast compared to Fitch and Kitsch, and should make it
possible to run much larger cases, although the results are expected to be
a bit rougher than with those programs.
How will the speed depend on the number of species and the number
of characters? For the sequential-addition algorithms, the speed should
be proportional to somewhere between the cube of the number of species and
the square of the number of species, and to the number
of characters. Thus a case that has, instead of 10 species and 20
characters, 20 species and 50 characters would take (in the cubic case)
2 x 2 x 2 x 2.5 = 20
times as long. This implies that cases with more than 20 species will
be slow, and cases with more than 40 species very slow. This places a
premium on working on small subproblems rather than just dumping a whole
large data set into the programs.
An exception to these rules will be some of the DNA programs that use an
aliasing device to save execution time. In these programs execution time
will not necessarily increase proportional to the number of sites,
as sites that show the same pattern of nucleotides will be detected
as identical and the calculations for them will be done only once, which does
not lead to more execution time. This is particularly
likely to happen with few species and many sites, or with data sets that have
small amounts of evolutionary divergence.
For programs Fitch and Kitsch, the distance matrix is square, so
that when we double the number of species we also double the number of
"characters", so that running times will go up as the fourth power of
the number of species rather than the third power. Thus a 20-species
case with Fitch is expected to run sixteen times more slowly than a 10-species
case.
For programs like Penny and Clique the run times will rise faster
than the cube of the number of species (in fact, they can rise faster
than any power since these algorithms are not guaranteed to work in
polynomial time). In practice, Penny will frequently bog down above 11
species, while Clique easily deals with larger numbers.
For Neighbor the speed should vary only as the cube of the number of
species, so a case twice as large will take only eight times as long. This
will make it an attractive alternative to Fitch and Kitsch for large data
sets.
Suggestion: If you are unsure of how long a program will take, try it first on
a few species, then work your way up until you get a feel for the speed
and for what size programs you can afford to run.
Execution time is not the most important criterion for a program,
particularly as computer time gets much cheaper than your time or a
programmer's time. With workstations on which background jobs can be run
all night, execution speed is not overwhelmingly relevant. Some of us have been
conditioned by an earlier era of computing to consider execution speed
paramount. But ease of use, ease of adaptation to your computer system,
and ease of modification are much more important in practice, and in
these respects I think these programs are adequate. Only if you are
engaged in 1960's style mainframe computing, or if you have very large
amounts of data is minimization of execution
time paramount. If you spent six months getting your data, it may not be
overwhelmingly important whether your run takes 10 seconds or 10 hours.
Nevertheless it would have been nice to have made the programs
faster. The present speeds are a compromise between speed and
effectiveness: by making them slower and trying more rearrangements in the
trees, or by enumerating all possible trees, I could have made the programs
more likely to find the best tree. By trying fewer rearrangements I
could have speeded them up, but at the cost of finding worse trees. I
could also have speeded them up by writing critical sections in assembly
language, but this would have sacrificed ease of distribution to new
computer systems. There are also some options included in these programs that
make it
harder to adopt some of the economies of bookkeeping that make other programs
faster. However to some extent I have simply made the decision not to spend
time trying to speed up program bookkeeping when there were new likelihood and
statistical methods to be developed.
It is interesting to compare different machines using Dnapars as the
standard task. One can rate a machine on the Dnapars benchmark by summing the
times for all three of the data sets. Here are relative total timings over
all three data sets (done with various versions of Dnapars) for some machines,
taking an AMD Athlon 1.2 GHz computer running Linux with gcc as the
standard. Benchmarks from versions 3.4 and 3.5 of the program are
also included (respectively the Pascal and C versions whose timings are in
parentheses). They are compared only with each other and are scaled to the
rest of the timings using the joint runs on the 386SX and the Pentium MMX 266.
This use of separate standards is necessary not
because of different languages but because different versions of the package
are being compared. Thus, the "Time" is the ratio of the Total to that for
the Pentium, adjusted by the scalings of machines using 3.4 and 3.5 when
appropriate. The Relative Speed is the reciprocal of the Time. For the
moment these benchmarks are for version 3.6; they will be updated when 3.7
is fully released.
This benchmark not only reflects integer performance of these machines
(as Dnapars has few floating-point operations) but also the efficiency
of the compilers. Some of the machines (the DEC 3000/400 AXP
and the IBM RS/6000, in particular) are much faster than this benchmark
would indicate. The numerical programs benchmark below gives them a
fairer test. The Compaq/Digital Alpha 500au times are exaggerated because,
although their compiles are optimized for that processor, some of the Pentium
compiles are not similarly optimized.
Note that parallel machines like the Sequent and the SGI PowerChallenge are not
really as slow as indicated by the data here, as these runs did nothing to take
advantage of their parallelism.
These benchmarks have now extended over 22 years (1986-2008), and in the Dnapars
benchmark they extend over a range of over 54,000-fold in speed!
The experience of our laboratory, which seems typical, is that
computer power grows by a factor of about 1.85 per year. This is
roughly consistent with these benchmarks.
For a picture of speeds for a more numerically intensive program,
here are benchmarks using Dnaml, with an AMD Athlon 1.2 GHz Linux system
as the standard. Some of the timings, the ones in parentheses, are
using PHYLIP version 3.5, and those are compared to that version run on
the Pentium 266. Runs using the PHYLIP 3.4 Pascal version are adjusted
using the 386SX timings where both were run. Numbers are
total run times (total user time in the case of Unix) over all three data sets.
As before, the parallel machines such as the Convex and the SGI PowerChallenge
were only run using one processor, which does not take into account the
gain that could be obtained by parallelizing the programs. The speed of the
Compaq/Digital Alpha 500au is exaggerated because it was compiled in a way
optimized for its processor, while some of the Pentium compiles were not.
You are invited to send me figures for your machine for
inclusion in future tables. Use the data sets above and compute the total
times for Dnapars and for Dnaml for the three data sets (setting the
frequencies of the four bases to 0.25 each for the Dnaml runs). Be sure to
tell me the name and version of your compiler, and the version of PHYLIP you
tested.
If the times are too small to be measured accurately, obtain the times
for 10 or 100 data sets (the Multiple data sets option) and divide by 10 or
100.
In the sections following you will find instructions on how to adapt the
programs to different computers and compilers. The programs should compile
without alteration on most versions of C. They use the "malloc" library
or "calloc" function to allocate memory so that the upper limits on how many
species or how many sites or characters they can run is set by the system memory
available to that memory-allocation function.
In the document file for each program, I have supplied a small
input example, and the output it produces, to help you check whether the
programs are running properly.
If you have not been able to get executables for PHYLIP, you should be
able to make your own. This can be easy under Linux and Unix, but more
difficult if you have a Macintosh or a Windows system. If you have the
latter, we strongly recommend you download and use the Macintosh and
Windows executables that we distribute. If you do that, you will not need
to have any compiler or to do any compiling. I get a certain number of
inquiries each year from confused users who are not sure what a compiler
is but think they need one. After downloading the executables they
contact me and complain that they did not find a compiler included in the
package, and would I please e-mail them the compiler. What they really
need to do is use the executables and forget about compiling them.
Some users may also need to compile the programs in order to modify them.
The instructions below will help with this.
I will discuss how to compile PHYLIP using one of a number of widely-used
compilers. After these I will comment on compiling PHYLIP on other, less
widely-used systems.
For Unix and Linux (which is Unix in all important functional respects, if
not in all legal respects) you must compile PHYLIP yourself.
This is usually easy to do.
Unix (and Linux)
systems generally have a C compiler and have the make utility. We
distribute with the PHYLIP source code a Unix-compatible Makefile.
We use GNU's
make utility,
which might be installed on your system as "make" or as
"gmake".
However, note that some popular Linux distributions do not include a C
compiler in their default configuration. For example, in RedHat Linux version
8, the "Personal Workstation" installation that is the default does not
include the C compiler or the X Windows libraries needed to compile
PHYLIP. These are available, and can be loaded from the CDROMs in the
distribution. The following instructions assume that you have the
C compiler and X libraries. If you cannot easily configure your system to
include them, you should look into using the RedHat RPM binary
distribution, mentioned on the PHYLIP 3.6 web page.
As is mentioned below (under Macintoshes) the Mac OS X operating system is
a Unix, and if the X windows windowing system is installed, these Unix
instructions will work for it.
After you have finished unpacking the Documentation and Source Code
archive, you will find that you have created a folder phylip-3.6
in which there are three
folders, called exe, src, and doc.
There is also an HTML web page, phylip.html. The exe
folder
will be empty, src contains the source code files, including the
Makefile. Directory doc contains the documentation files.
Enter the src folder. Before you compile, you will want to
look at the Makefile and see whether you want to alter the compilation
command. We have the default C compiler flags set with no flags. If you
have modified the programs, you might want to use the debugging flags
"-g". On the other hand, if you are trying to make a fast executable using
the GCC compiler, you may want to use the one which is "An optimized one
for gcc". In either case, remove the "#" before that CFLAGS command, and
place it before the CFLAGS command that was previously in use.
There are careful instructions on this in the Makefile.
Once you have set up the CFLAGS and DFLAGS statements to be the way you
want, to compile all the programs just type:
make install
You will then see the compiling commands as they happen, with
occasional warning messages. If these are warnings, rather than errors,
they are not too serious. A typical warning would be like this:
dnaml.c:1204: warning: static declaration for re_move follows non-static
After a time the compiler will finish compiling. If you have done a
make install the system will then move the executables into the
exe folder and also save space by erasing all the relocatable
object files that were produced in the process. You should be left with
useable executables in the exe folder, and the src
folder should be as before. To run the executables, go into the
exe folder and type the program name (say dnaml, which
you may or may not have to precede by a dot and a slash./).
The names of the
executables will be the same as the names of the C programs, but without the
.c suffix. Thus dnaml.c compiles to make an executable called dnaml.
Our two tree-drawing programs, Drawgram and Drawtree, require an
X Windows installation including the Athena Widgets. These are provided
with most X Windows installations.
If you see messages that the compilation could not find "Xlib.h" and other,
similar functions, this means that some parts of the X Windows development
environment is not installed on your system, or is not installed in the
default location.
Similarly, if you get error messages saying that some files with "Xaw"
in the name cannot be found, this means that the Athena Widgets
are not installed on your system, or are not installed in the
default location.
In either case, you will need to make sure that they are installed properly.
If they are there but not found during the compile, change the
DFLAGS and DLIBS variables in the Makefile to
point to the locations of the header files and libraries, respectively.
Another is that the usual Linux C compiler is the Gnu GCC compiler.
In some Linux systems it is not invoked by the command cc but
by gcc. You would then need to edit the Makefile to reflect this
(see below for comments on that process).
A typical Unix or Linux installation would put the directory phylip-3.6
in /usr/local. The name of the executables directory EXEDIR
could be changed to be /usr/local/bin, so that the make install
command puts the executables there. If the users have /usr/local/bin
in their paths, the programs would be found when their names are typed.
The font files font1 through font6 could also be
placed there. A batch script containing the lines
could be used to establish links in the user's working directory so that
Drawtree and Drawgram would find these font files when users
type a name such as font1 when the program asks
them for a font file name. The
documentation web pages are in subdirectory doc of the
main PHYLIP directory, except for one, phylip.html which is
in the main PHYLIP directory. It has a table of all of the documentation
pages, including this one. If users create a bookmark to that page
it can be used to access all of the other documentation pages.
To compile just one program, such as Dnaml, type:
make dnaml
After this compilation, dnaml will be in the src
subdirectory. So will some relocatable object code files that
were used to create the executable. These have names ending in
.o - they can safely be deleted.
If you have problems with the compilation command, you can edit the
Makefile. It has careful explanations at its front of how you
might want to do so. For example, you might want to change the C
compiler name cc to the name of the Gnu C compiler, gcc.
This can be done by removing the comment character # from the
front of one line, and placing it at the front of a nearby line.
How to do so should be clear from the material at the beginning of the
Makefile. We have included sample lines for using the gcc
compiler and for using the Cygwin Gnu C++ environment on Windows, as
well as the default of cc.
We have encountered some problems with the Gnu C Compiler (gcc)
on 64-bit Itanium processors when compiled with the the -O 3
optimization level, in our code for generating random numbers.
Some older C compilers (notably the Berkeley C compiler which is
included free with some Sun systems) do not adhere to the ANSI C
standard (because they were written before it was set down).
They have trouble with the function prototypes which are in
our programs. We have included an #ifndef preprocessor
command to eliminate the problem, if you use the switch -DOLDC
when compiling. Thus with these compilers you need only use this in
your C flags (in the Makefile) and compilers such as Berkeley C
will cause no trouble.
We distribute Windows executables, and most likely you can use these and
do not need to recompile them. The following instructions will only be
necessary if you want to modify the programs and need to recompile them.
They are given for several different compilers available on Windows systems.
Another major compiler is Intel compiler -- we do not have information yet
on how to use it, but expect that PHYLIP will compile on it.
Cygnus Solutions (now a part of Red Hat, Inc.) has adapted the Gnu C compiler
to Windows systems and
provided an environment, CygWin, which mimics Unix for compiling.
Currently, this is the compiler that we use to prepare the Windows
executables.
Cygwin is available for purchase, and they also make it
available to be downloaded for free. The download is large. To get it, go
to the Cygwin web site at
When installing Cygwin it is important to install gcc and make. During
the course of the setup program Setup will ask you to select packages.
Expand the Devel Category by clicking on it. Scroll down to gcc and
check if the "New" column says "Skip". If it does,
click on "skip". "Skip" will change to the current version of gcc.
Scroll down to the make package, and if it has "Skip"
click on "Skip". These two
programs are nessessary to install phylip.
Once you have
installed the free CygWin environment and the associated Gnu C compiler
on your Windows system, compiling PHYLIP is closely similar to
what one does for Unix or Linux:
dnaml ICON "dna.ico"
We have provided a folder icons in the src
folder, containing a full set of icons and a full set of resource
files (*.rc) so you will not have to do this yourself.
Compiling with Microsoft Visual C++
We have had success in the past compiling PHYLIP with Microsoft Visual
C++ (the compiler in the Microsoft .NET package), although the
Windows executables that we distribute are built
using the Cygwin GCC compiler. The following instructions are the ones we
have used for Visual C++ for the .NET 2008 version with Visual C++ version 9.0.
Microsoft also makes a free download version of their C++ compiler
from 2005 available as Visual C++ Express Edition. That version has a somewhat
different content, and these instructions will not work with it. If you
figure out how to get the compiler and Makefiles to work together, please
let us know -- we don't have the energy to figure this out for all possible
configurations of the Microsoft C++ compiler.
The instructions use the nmake command that uses a Makefile which is
called Makefile.msvc in our distribution.
At the end of this section we have some comments on how to
compile the programs with Visual C++ version 7.0, which also has a somewhat
different file folder structure.
With Microsoft Visual C++, you can compile using a Makefile. We have supplied
this in the source code distrubution as Makefile.msvc.
You may wish to preserve the Unix Makefile by renaming Makefile to,
say, Makefile.unix, then make a copy of Makefile.msvc
and call it Makefile. (You may have to change your Windows desktop
settings to make the three-letter extensions visible, or you could use
the RENAME command in the Command tool).
Once you have set MSVC, type
The Makefile has some paths in it, which are, I hope, in
the correct form for Visual C++ 9.0 on your system.
If not, the statement
If instead you have an earlier version of Visual Studio .NET which
has the Visual C++ 7.0 compiler, you should proceed as above, but
instead, set MSVC to
C:\Program Files\Microsoft Visual Studio .NET,
and then type
You will also need to edit the line in the Makefile that
defines the variable MSVCPATH. You should change this to
Compiling with Borland C++
Borland C++ can be downloaded for free. It is a compiler released
in 2000, and which is now owned by Embarcadero Technologies, Inc.
(see their site
http://www.codegear.com/downloads/free/cppbuilder). To download
it you need to register with them.
It has a somewhat restrictive license, so we cannot use it for the
widely-distributed executables.
You should download the compiler as it includes all the utilities needed
to compile phylip. It can compile using a Makefile. We have supplied
this in the source code distribution as Makefile.bcc. You will need to
preserve the Unix Makefile by renaming it to, say, Makefile.unix, then
make a copy of Makefile.bcc and call it Makefile. The
Makefile is invoked using the make command.
You will first need to create an ilink32.cfg and a bcc32.cfg
file and put the files into the src folder. These files are text files
and their contents are described in the readme.txt that comes with the
Borland tools. If the Borland tools are in the default location the
contents of ilink32.cfg would be.
and the contents of bcc32.cfg
These files can be created in a text editor such as Notepad or Wordpad.
To invoke the make command you will first need to open a command
prompt window. Then set the path appropriately. To set the path, type
Where "Path" is where Borland is installed, such as C:\Borland\BCC55.
Then type
If you simply type make you will get a list of possible make commands.
For example, to compile a single program such as Dnaml but not install
it, type make dnaml. To compile and install all programs type
make install. We have supplied all the the support files and icons
needed for
the compilations. They are in folder bcc of the main PHYLIP
ource code folder.
We have had to supply a complete second set of the resource files with
names *.brc because Borland resource files have a minor
incompatibility with Microsoft Visual C++ resource files.
Compiling with GCC on Mac OS X with our Makefile
The executables distributed by us for Mac OS X are currently compiled
using the GCC compiler that is distributed with Mac OS X. You may not need
to recompile them, unless you want to make changes in the programs.
We are distributing 32-bit "universal binaries" that work on both PowerMac and
Intel iMac.
You may not need to recompile unless you need to make a version of the
executables more closely adapted to your system, or unless you want to
modify the programs. One reason to recompile might be if you want
64-bit executables, which you might need to address large amounts of
memory.
If you do want to recompile, conder the following:
make install -f Makefile.osx
make dnaml.install
Compiling with GCC on Mac OS X with X Windows
On Mac OS X systems you can also use the GCC compiler and X Windows to
compile a version of the executables that runs from the command line
in native mode.
To do that, you must have the GCC compiler and the X11 windows
development kit materials installed.
X Windows is an optional install present on the Mac OS X for version
10.3 (Panther) and 10.4 (Tiger) distribution disks, and part
of the default distribution for 10.5 (Leopard) on.
You can search for the latest X11 release for your system by
looking at results after
searching for "x11"
on the Apple Downloads page.
It is easy to download and install on a Mac OS X system.
If you have the GCC compiler and the X11 libraries installed, you can use a
Terminal window (which you
will find available in the Utilities folder in the Applications folder) and
compile PHYLIP by treating it as a Unix or Linux application and following the
instructions given above under "Unix and Linux". Basically you just get
into the folder that contains the PHYLIP source code and type
make install
This uses the ordinary Unix/Linux Makefile, which works
in creating programs using X11 for
Mac OS X with the gcc compiler. Note that to run the
programs drawgram and drawtree that actually use the X Windows, you
will need to
What about the Metrowerks Codewarrior compiler?
We previously also supported the Metrowerks Codewarrior compiler, for
both Mac OS 9 and Mac OS X (and even for producing Windows executables).
Codewarrior required that one maintain "projects" for each program, and we
distributed the projects as well as the source code. As Metrowerks was
bought out by Freescale and has retargeted its compilers for building
embedded applications, we are ceasing this rather cumbersome support. That
means that we do not at present give you a way to recompile our programs
if you have Mac OS 9. We may not make a set of executables for Mac OS 9
ourself. If you absolutely need to obtain compilation support routines and
projects for Metrowerks Codewarrior, contact us and we will send you what we
have. Of course, for Mac OS X the GCC compiler is available, and we describe
above how to compile the programs with it.
VMS VAX systems have almost disappeared, so
we have not tried to compile version 3.695 on an OpenVMS system. The
following instructions should work.
On the OpenVMS operating system with DEC VAX VMS C the programs will compile
without alteration. The commands for compiling a typical program
(Dnapars, which depends on the separately compiled files phylip.c
and seq.c) are:
$ DEFINE LNK$LIBRARY SYS$LIBRARY:VAXCRTL
Once you use this $ DEFINE statement during a given interactive session,
you need not repeat it again as the symbol LNK$LIBRARY is thereafter
properly defined. The compilation process leaves a file DNAPARS.OBJ
in your directory: this can
be discarded. The executable program is named DNAPARS.EXE. To run the program
one then uses the command:
$ R DNAPARS
The compiler defaults to the filenames INFILE., OUTFILE., and
TREEFILE..
If the input file INFILE. does not exist the program will prompt you to
type in its name. Note that some commands on VMS such as TYPE OUTFILE
will fail because the name of the file that it will attempt to type out will be not
OUTFILE. but OUTFILE.LIS. To get it to type the write file you
would have to instead issue the command TYPE OUTFILE..
When you are
using the interactive previewing feature of Drawgram (or Drawtree) on
a Tektronix or DEC ReGIS compatible terminal, you will want before
running the program to have issued the command:
$ SET TERM/NOWRAP/ESCAPE
so that you do not run into trouble from the VMS line length limit of
255 characters or the filtering of escape characters.
To know which files to compile together, look at the entries in the
Makefile.
As parallel computers become more common, the issue of how to compile
PHYLIP for them has become more pressing. People have been compiling
PHYLIP for vector machines and parallel machines for many years. We
have not made a version for parallel machines because there is still
no standard parallel programming environment on such machines (or rather,
there are many standards, so that one cannot find one that makes
a parallel execution version of PHYLIP widely distributable). However
symmetric multiprocessing using the
MPI Message Passing Interface is spreading rapidly, and we will
probably support it in future versions of PHYLIP.
Although the underlying algorithms of most programs,
which treat sites independently, should be amenable to vector and
parallel processors,
there are details of the code which might best be changed.
In certain of the programs (Dnaml, Dnamlk,
Proml, Promlk) I have put a special
comment statement next to the loops in the program where
the program will spend most of its time, and which are the places
most likely to benefit from parallelization. This comment statement is:
If you succeed in making a parallel version of PHYLIP we would like to
know how you did it. In particular, if you can prepare a web page which
describes how to do it for your computer system, we would like to use
material from it
in our PHYLIP web pages. Please e-mail it to me. We hope to
have a set of pages that give detailed instructions on how to make parallel
version of PHYLIP on various kinds of machines. Alternatively, if we
were given your modified version of the program we might be able to
figure out how to make modifications to our source code to allow
users to compile the program in a way which makes those modifications.
As you can see from the variety of different systems on which these
programs have been successfully run, there are no serious
incompatibility problems with most computer systems. PHYLIP in various
past Pascal versions has also been compiled on 8080 and Z80 CP/M Systems, Apple
II systems running UCSD Pascal, a variety of minicomputer systems such as
DEC PDP-11's and HP 1000's, on 1970's era mainframes such as CDC
Cyber systems, and so on. In a later era
it was also compiled on IBM 370 mainframes, and of course on DOS and
Windows systems and on Macintosh systems.
We have gradually
accumulated experience on a wider variety of C compilers. If you succeed in
compiling the C version of PHYLIP on a different machine or a different
compiler, I would like to
hear the details so that I can consider including the instructions in a future version
of this manual.
The ONLY reason you should do this is if you want to add or modify
functionality on the Java interface. In all other cases, the .jar files that
already exist in the javajars folder will run on your Mac / MS /
Linux / Unix system and you should not be here.
Welcome to a fairly complex process. Unless you are an experienced object
oriented programmer, you will find Java has a steep learning curve and will
cause you headaches.
The general overview is that there is a Java interface that gathers and
validates input from the user, there is a call from the Java code to a dynamic
C library that contains the Phylip functionality, and there is feedback to the
user from the Java interface as to the status of the underlying C code.
Because one has two very different kinds of software running, the feedback is
not as elegant as one would expect from a single integrated environment.
Now for the specifics. We have developed these Java interfaces using the Eclipse environment (available from
www.eclipse.org). Go there and download the version of the Java
development environment appropriate to your operating system.
In the distribution there is a javasrc folder which contains folders
that match the programs. These folders contain the program's Java interfaces.
For example folder drawgram contains DrawgramInterface.java
and DrawgramUserInterface.java. The former does the interaction with
the compiled C library, the latter contains the user interface.
If you want to modify the Java interface for Drawgram, open the Eclipse
Java development environment, create a project called Phylip3.695, create a
folder under it called src. Under that create a project called
drawgram. Now import the two Drawgram associated java files
(DrawgramUserInterface.java and DrawgramInterface.java) into
that project. You will also need to create a project called util and
import all the items in the javasrc/util directory. Open
DrawgramUserInterface.java with the Eclipse WindowBuilderEditor and
you can edit it however you want. Remember that you'll need to add
ActionListeners (described in Java manuals) to anything that changes things on
the screen. There are plenty of examples of them in
DrawgramUserInterface.java, for example, TreeGrowToggle
which handles the toggling "Tree grows:" between "Horizontal" and "Vertical"
using Radio buttons. Most of the pieces you'll need are in the existing code.
You can clone them and edit to fit. Beyond that, "Google is your friend".
Once you have added new functionality or changed existing functionality in the
user interface, you will need to pass the information it collects from the
user to the underlying C code. This is a bit tricky because C and Java are
very different kinds of languages. Luckily Sun provided the Java Native Access
/ Java Native Interface (JNA/JNI) interface package to take care of it. We
used JNA (which calls JNI) because it is simpler to use and our needs were
basic enough we could live within its confines. In order to use it you will
also need to get two public jars off the web (do a Google search for these as
they keep moving around):
JNA passes everything via an enormous list of variables. This is simple to
program but very hard to keep track of, as you have to keep things exactly
parallel in the Java and C code and there is no debugger that will help you.
We have found it best to build a public class in Java that contains everything
that is going to the C code and create an instance of it when the user is
finished with data entry and decides to execute the process (in the
Drawgram case, selects Preview). We then copy all the data
from the screen into the members of the class, and pass these directly into
the JNA call to the underlying C code (look in DrawgramInterface.java
for an example).
In the underlying C code (which must be compiled as a library so that Java can
access it), there is an entry point that is the name of the program (for
example the function drawgram in drawgram.c) containing as
arguments every one of the variables that were passed by the Java interface,
in the same order. If you have weird bugs, most likely you messed this up.
Make a copy of the Java class definition, paste it into the C code and check
everything. Another wrinkle that can bite you is that booleans come though as
integers and Java and C do not agree as to what that means. False is 0 in both
languages. True is "not 0" in Java and often set to all bits on (which is a
very big negative number in C). C often has problems with this. Each compiler
is different and there are environment variables that effect this also. It is
safest to explicitly fix things before you execute any C code. There are a lot
of other odd quirks, but you have two working examples (Drawgram.c
and Drawtree.c), so you can probably figure them out.
Feedback from C to Java can be difficult. In Drawgram and Drawtree it is
fairly easy, as the plotting is done (to the file JavaPreview.ps in
case you need to know) and the program returns. The Java interface waits until
the C code completes and returns, then reads JavaPreview.ps and
displays the preview. In cases where one needs progress indicators, one needs
to multithread the Java code and display a continually updating progress
file. Phylip 3.695 has no need of multithreading but it will be implemented
in Phylip 4.0.
This set of Frequently Asked Questions, and their answers, is from the
PHYLIP web site. A more up-to-date version can be found there, at:
At the upper-undergraduate level:
and as graduate-level texts:
For more mathematically-oriented readers, there is the book
Best of all is of course my own book on phylogenies, which
covers the subject for many data types, at a
graduate course level:
There are also some recent books that take a more practical hands-on approach,
and give some detailed information on how to use programs, including PHYLIP
programs. These include:
In addition, one of these three review articles may help:
A useful article introducing the inference of phylogenies at a more
elementary level is:
I have already mentioned above that there is an excellent guide to using PHYLIP 3.6
for molecular analyses available. It is by Jarno Tuimala:
Felsenstein, J. 2009. PHYLIP (Phylogeny Inference Package) version 3.695.
Distributed by the author. Department of Genome Sciences, University of
Washington, Seattle.
or if the editor for whom you are writing insists that the citation must be to
a printed publication, you could cite a notice for version 3.2 published in
Cladistics:
Felsenstein, J. 1989. PHYLIP - Phylogeny Inference Package (Version 3.2).
Cladistics 5: 164-166.
(This citation has been so commonly made that this is the most-cited paper
ever in the journal Cladistics, I am the most-cited author ever in that
journal, and these citations are responsible for more than 15% of the
impact factor of that journal!).
For a while a printed version of the PHYLIP documentation was available and one
could cite that. This is no longer true. Other than that, this is difficult,
because I have never written a paper announcing PHYLIP! My 1985b paper in
Evolution on the bootstrap method contains a
one-paragraph Appendix describing the availability of this package, and that
can also be cited as a reference for the package, although it was
distributed since 1980 while the bootstrap paper is 1985. A paper on PHYLIP
is needed mostly to give people something to cite, as word-of-mouth, references
in other people's papers, and electronic newsgroup postings have spread the
word about PHYLIP's existence quite effectively.
(The following four questions, once
common, have finally disappeared, I am pleased to report. I include them to
give you some idea of what kinds of requests I had to cope with.)
Version 3.6 has many new features:
There are many more, lesser features added as well.
Version 3.7 has some new features:
There are some obvious deficiencies in this version. Some of these
holes will be filled in the next few releases (leading to version
4.0). They include:
There will also be many future developments in the programs that treat
continuously-measured data (quantitative characters) and morphological
or behavioral data with discrete states, as I have new ideas for
analyzing these data in ways that connect to within-species
quantitative genetic analyses. This will compete with parsimony
analysis.
Here are some comments people have made in print about PHYLIP. Explanatory
material in square brackets is my own. They fall naturally into three groups:
(note also W. Fink's critical remarks (1986) on version 2.8 of PHYLIP).
In the documentation files that follow I frequently refer to papers
in the literature. In order to centralize the references they are given
in this section. If you want to find further papers beyond these, my
book (Felsenstein, 2004) lists more than 1,000 further references.
Adams, E. N. 1972. Consensus techniques and the comparison of
taxonomic trees. Systematic Zoology 21: 390-397.
Adams, E. N. 1986. N-trees as nestings: complexity, similarity, and
consensus. Journal of Classification 3: 299-317.
Archie, J. W. 1989. A randomization test for phylogenetic information in
systematic data. Systematic Zoology 38: 239-252.
Backeljau, T., L. De Bruyn, H. De Wolf, K. Jordaens, S. Van Dongen,
and B. Winnepenninckx.
1996. Multiple UPGMA and neighbor-joining trees and the performance of some
computer packages. Molecular Biology and Evolution 13: 309–313.
Barry, D., and J. A. Hartigan. 1987. Statistical analysis of hominoid
molecular evolution. Statistical Science 2: 191-210.
Baum, B. R. 1989. PHYLIP: Phylogeny Inference Package. Version 3.2. (Software
review). Quarterly Review of Biology 64: 539-541.
Bourque, M. 1978. Arbres de Steiner et reseaux dont certains sommets sont
à localisation variable. Ph. D. Dissertation, Université de
Montréal, Quebec.
Bron, C., and J. Kerbosch. 1973. Algorithm 457: Finding all cliques
of an undirected graph. Communications of the Association for Computing Machinery 16: 575-577.
Camin, J. H., and R. R. Sokal. 1965. A method for deducing branching
sequences in phylogeny. Evolution 19: 311-326.
Carpenter, J. 1987a. A report on the Society for the Study of Evolution
workshop "Computer Programs for Inferring Phylogenies". Cladistics 3:
363-375.
Carpenter, J. 1987b. Cladistics of cladists. Cladistics 3: 363-375.
Cavalli-Sforza, L. L., and A. W. F. Edwards. 1967. Phylogenetic
analysis: models and estimation procedures. Evolution 32: 550-570
(also American Journal of Human Genetics 19: 233-257).
Cavender, J. A. and J. Felsenstein. 1987. Invariants of phylogenies in a
simple case with discrete states. Journal of Classification 4: 57-71.
Churchill, G.A. 1989. Stochastic models for heterogeneous DNA sequences.
Bulletin of Mathematical Biology 51: 79-94.
Conn, E. E. and P. K. Stumpf. 1963. Outlines of Biochemistry. John Wiley
and Sons, New York.
Day, W. H. E. 1983. Computationally difficult parsimony problems in
phylogenetic systematics. Journal of Theoretical Biology 103:
429-438.
Dayhoff, M. O. and R. V. Eck. 1968. Atlas of Protein Sequence
and Structure 1967-1968. National Biomedical Research Foundation,
Silver Spring, Maryland.
Dayhoff, M. O., R. M. Schwartz, and B. C. Orcutt. 1979. A model of
evolutionary change in proteins. pp. 345-352 in Atlas of
Protein Sequence and Structure, volume 5, supplement 3, 1978, ed.
M. O. Dayhoff. National Biomedical Research Foundation, Silver Spring, Maryland
.
Dayhoff, M. O. 1979. Atlas of Protein Sequence and Structure, Volume 5,
Supplement 3, 1978. National Biomedical Research Foundation, Washington, D.C.
DeBry, R. W. and N. A. Slade. 1985. Cladistic analysis of restriction
endonuclease cleavage maps within a maximum-likelihood framework.
Systematic Zoology 34: 21-34.
Dempster, A. P., N. M. Laird, and D. B. Rubin. 1977. Maximum
likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39: 1-38.
Eck, R. V., and M. O. Dayhoff. 1966. Atlas of Protein Sequence and
Structure 1966. National Biomedical Research Foundation, Silver
Spring, Maryland.
Edwards, A. W. F., and L. L. Cavalli-Sforza. 1964. Reconstruction of
evolutionary trees. pp. 67-76 in Phenetic and Phylogenetic
Classification, ed. V. H. Heywood and J. McNeill. Systematics
Association Volume No. 6. Systematics Association, London.
Estabrook, G. F., C. S. Johnson, Jr., and F. R. McMorris. 1976a. A
mathematical foundation for the analysis of character
compatibility. Mathematical Biosciences 23: 181-187.
Estabrook, G. F., C. S. Johnson, Jr., and F. R. McMorris. 1976b. An
algebraic analysis of cladistic characters. Discrete Mathematics 16: 141-147.
Estabrook, G. F., F. R. McMorris, and C. A. Meacham. 1985. Comparison of
undirected phylogenetic trees based on subtrees of four evolutionary units.
Systematic Zoology 34: 193-200.
Faith, D. P. 1990. Chance marsupial relationships. Nature 345: 393-394.
Faith, D. P. and P. S. Cranston. 1991. Could a cladogram this short have
arisen by chance alone?: On permutation tests for cladistic
structure. Cladistics 7: 1-28.
Farris, J. S. 1977. Phylogenetic analysis under Dollo's Law. Systematic Zoology 26: 77-88.
Farris, J. S. 1978a. Inferring phylogenetic trees from chromosome
inversion data. Systematic Zoology 27: 275-284.
Farris, J. S. 1981. Distance data in phylogenetic analysis. pp. 3-23
in Advances in Cladistics: Proceedings of the first meeting of the
Willi Hennig Society, ed. V. A. Funk and D. R. Brooks. New York
Botanical Garden, Bronx, New York.
Farris, J. S. 1983. The logical basis of phylogenetic analysis. pp. 1-47
in Advances in Cladistics, Volume 2, Proceedings of the Second Meeting of
the Willi Hennig Society. ed. Norman I. Platnick and V. A. Funk. Columbia
University Press, New York.
Farris, J. S. 1985. Distance data revisited. Cladistics 1: 67-85.
Farris, J. S. 1986. Distances and statistics. Cladistics 2: 144-157.
Farris, J. S. [“T. N. Nayenizgani”]. 1990. The systematics association
enters its golden years (review of Prospects in Systematics, ed. D.
Hawksworth). Cladistics 6: 307-314.
Farris, J. S., V. A. Albert, M. K&aauml;llersj&oauml;, D.
Lipscomb, and A. G. Kluge. 1996. Parsimony jackknifing outperforms
neighbor-joining. Cladistics 12: 99-124.
Felsenstein, J. 1973a. Maximum likelihood and minimum-steps methods
for estimating evolutionary trees from data on discrete characters.
Systematic Zoology 22: 240-249.
Felsenstein, J. 1973b. Maximum-likelihood estimation of evolutionary
trees from continuous characters. American Journal of Human Genetics 25:
471-492.
Felsenstein, J. 1978a. The number of evolutionary trees. Systematic Zoology 27: 27-33.
Felsenstein, J. 1978b. Cases in which parsimony and compatibility
methods will be positively misleading. Systematic Zoology 27:
401-410.
Felsenstein, J. 1979. Alternative methods of phylogenetic inference
and their interrelationship. Systematic Zoology 28: 49-62.
Felsenstein, J. 1981a. Evolutionary trees from DNA sequences: a
maximum likelihood approach. Journal of Molecular Evolution 17: 368-376.
Felsenstein, J. 1981b. A likelihood approach to character weighting
and what it tells us about parsimony and compatibility. Biological Journal of the Linnean Society 16: 183-196.
Felsenstein, J. 1981c. Evolutionary trees from gene frequencies and
quantitative characters: finding maximum likelihood estimates.
Evolution 35: 1229-1242.
Felsenstein, J. 1982. Numerical methods for inferring evolutionary
trees. Quarterly Review of Biology 57: 379-404.
Felsenstein, J. 1983b. Parsimony in systematics: biological and
statistical issues. Annual Review of Ecology and Systematics 14: 313-333.
Felsenstein, J. 1984a. Distance methods for inferring phylogenies: a
justification. Evolution 38: 16-24.
Felsenstein, J. 1984b. The statistical approach to inferring
evolutionary trees and what it tells us about parsimony and
compatibility. pp. 169-191 in: Cladistics: Perspectives in the
Reconstruction of Evolutionary History, edited by T. Duncan and T. F.
Stuessy. Columbia University Press, New York.
Felsenstein, J. 1985a. Confidence limits on phylogenies with a molecular
clock. Systematic Zoology 34: 152-161.
Felsenstein, J. 1985b. Confidence limits on phylogenies: an approach
using the bootstrap. Evolution 39: 783-791.
Felsenstein, J. 1985c. Phylogenies from gene frequencies: a statistical
problem. Systematic Zoology 34: 300-311.
Felsenstein, J. 1985d. Phylogenies and the comparative method. American Naturalist 125: 1-12.
Felsenstein, J. 1986. Distance methods: a reply to Farris. Cladistics 2:
130-144.
Felsenstein, J. and E. Sober. 1986. Parsimony and likelihood: an
exchange. Systematic Zoology 35: 617-626.
Felsenstein, J. 1988a. Phylogenies and quantitative characters. Annual Review of Ecology and Systematics 19: 445-471.
Felsenstein, J. 1988b. Phylogenies from molecular sequences: inference and
reliability. Annual Review of Genetics 22: 521-565.
Felsenstein, J. 1992. Phylogenies from restriction sites, a
maximum likelihood approach. Evolution 46: 159-173.
Felsenstein, J. and G. A. Churchill. 1996.
A hidden Markov model approach to variation among sites in rate of evolution
Molecular Biology and Evolution 13: 93-104.
Felsenstein, J. 2004. Inferring Phylogenies. Sinauer Associates,
Sunderland, Massachusetts.
Felsenstein, J. 2005. Using the threshold model of quantitative genetics
for inferences within and between species. Philosophical Transactions
of the Royal Society of London, Series B 360 1427-1434.
Felsenstein, J. 2008. Comparative methods with sampling error and within-species
variation: contrasts revisited and revised. American Naturalist 171: 713-725.
Fink, W. L. 1986. Microcomputers and phylogenetic analysis. Science 234: 1135-1139.
Fitch, W. M., and E. Markowitz. 1970. An improved method for determining
codon variability in a gene and its application to the rate of fixation of
mutations in evolution. Biochemical Genetics 4: 579-593.
Fitch, W. M., and E. Margoliash. 1967. Construction of phylogenetic
trees. Science 155: 279-284.
Fitch, W. M. 1971. Toward defining the course of evolution: minimum
change for a specified tree topology. Systematic Zoology 20: 406-416.
Fitch, W. M. 1975. Toward finding the tree of maximum parsimony. pp. 189-230
in Proceedings of the Eighth International Conference on Numerical Taxonomy,
ed. G. F. Estabrook. W. H. Freeman, San Francisco.
Fitch, W. M. and E. Markowitz. 1970. An improved method for determining
codon variability and its application to the rate of fixation of mutations
in evolution. Biochemical Genetics 4: 579-593.
George, D. G., L. T. Hunt, and W. C. Barker. 1988. Current methods in
sequence comparison and analysis. pp. 127-149 in Macromolecular Sequencing
and Synthesis, ed. D. H. Schlesinger. Alan R. Liss, New York.
Gilmour, R. 2000. Taxonomic markup language: applying XML to systematic data.
Bioinformatics 16: 406-407.
Goldman, N., and Z. Yang. 1994.
A codon-based model of nucleotide substitution for protein-coding DNA
sequences.
Molecular Biology and Evolution 11: 725-736.
Goldstein, D. B., A. Ru&iiacute;z-Linares, M. Feldman, and L. L. Cavalli-Sforza.
1995. Genetic absolute dating based on microsatellites and the origin of
modern humans. Proceedings of the National Academy of Sciences USA
92: 6720-6727.
Gomberg, D. 1968. "Bayesian" post-diction in an evolution process.
unpublished manuscript, University of Pavia, Italy.
Graham, R. L., and L. R. Foulds. 1982. Unlikelihood that minimal
phylogenies for a realistic biological study can be constructed in
reasonable computational time. Mathematical Biosciences 60: 133-142.
Hasegawa, M. and T. Yano. 1984a. Maximum likelihood method of phylogenetic
inference from DNA sequence data. Bulletin of the Biometric Society of Japan No. 5: 1-7.
Hasegawa, M. and T. Yano. 1984b. Phylogeny and classification of
Hominoidea as inferred from DNA sequence data. Proceedings of the Japan Academy 60 B: 389-392.
Hasegawa, M., Y. Iida, T. Yano, F. Takaiwa, and M. Iwabuchi. 1985a.
Phylogenetic relationships among eukaryotic kingdoms as inferred from
ribosomal RNA sequences. Journal of Molecular Evolution 22: 32-38.
Hasegawa, M., H. Kishino, and T. Yano. 1985b. Dating of the human-ape
splitting by a molecular clock of mitochondrial DNA. Journal of Molecular
Evolution 22: 160-174.
Hendy, M. D., and D. Penny. 1982. Branch and bound algorithms to
determine minimal evolutionary trees. Mathematical Biosciences 59: 277-290.
Higgins, D. G. and P. M. Sharp. 1989. Fast and sensitive
multiple sequence alignments on a microcomputer. Computer Applications in the Biological Sciences (CABIOS) 5: 151-153.
Hochbaum, D. S. and A. Pathria. 1997. Path costs in evolutionary
tree reconstruction. Journal of Computational Biology 4: 163-175.
Holmquist, R., M. M. Miyamoto, and M. Goodman. 1988. Higher-primate
phylogeny - why can't we decide? Molecular Biology and Evolution 5: 201-216.
Inger, R. F. 1967. The development of a phylogeny of frogs.
Evolution 21: 369-384.
Jin, L. and M. Nei. 1990. Limitations of the evolutionary parsimony method
of phylogenetic analysis. Molecular Biology and Evolution 7: 82-102.
Jones, D. T., W. R. Taylor and J. M. Thornton. 1992. The rapid generation of
mutation data matrices from protein sequences. Computer Applications
in the Biosciences (CABIOS) 8: 275-282.
Jukes, T. H. and C. R. Cantor. 1969. Evolution of protein molecules. pp.
21-132 in Mammalian Protein Metabolism, ed. H. N. Munro. Academic Press, New
York.
Kidd, K. K. and L. A. Sgaramella-Zonta. 1971. Phylogenetic analysis: concepts
and methods. American Journal of Human Genetics 23: 235-252.
Kim, J. and M. A. Burgman. 1988. Accuracy of phylogenetic-estimation
methods using simulated allele-frequency data. Evolution 42: 596-602.
Kimura, M. 1980. A simple model for estimating evolutionary rates of base
substitutions through comparative studies of nucleotide sequences. Journal of Molecular Evolution 16: 111-120.
Kimura, M. 1983. The Neutral Theory of Molecular Evolution. Cambridge
University Press, Cambridge.
Kishino, H. and M. Hasegawa. 1989. Evaluation of the maximum likelihood
estimate of the evolutionary tree topologies from DNA sequence data, and the
branching order in Hominoidea. Journal of Molecular Evolution 29: 170-179.
Kluge, A. G., and J. S. Farris. 1969. Quantitative phyletics and the
evolution of anurans. Systematic Zoology 18: 1-32.
Kosiol, C., and N. Goldman. 2005. Different versions of the Dayhoff rate
matrix. Molecular Biology and Evolution 22: 193-199.
Kuhner, M. K. and J. Felsenstein. 1994. A simulation comparison of
phylogeny algorithms under equal and unequal evolutionary rates.
Molecular Biology and Evolution 11: 459-468 (Erratum 12: 525 1995).
Künsch, H. R. 1989. The jackknife and the bootstrap for general stationary
observations. Annals of Statistics 17: 1217-1241.
Lake, J. A. 1987. A rate-independent technique for analysis of nucleic acid
sequences: evolutionary parsimony. Molecular Biology and Evolution 4: 167-191.
Lake, J. A. 1994. Reconstructing evolutionary trees from DNA and protein
sequences: paralinear distances.
Proceedings of the Natonal Academy of Sciences, USA 91: 1455-1459.
Le Quesne, W. J. 1969. A method of selection of characters in
numerical taxonomy. Systematic Zoology 18: 201-205.
Le Quesne, W. J. 1974. The uniquely evolved character concept and its
cladistic application. Systematic Zoology 23: 513-517.
Lewis, H. R., and C. H. Papadimitriou. 1978. The efficiency of
algorithms. Scientific American 238: 96-109 (January issue)
Lockhart, P. J., M. A. Steel, M. D. Hendy, and D. Penny. 1994.
Recovering evolutionary trees under a more realistic model of sequence
evolution. Molecular Biology and Evolution 11: 605-612.
Luckow, M. and D. Pimentel. 1985. An empirical comparison of
numerical Wagner computer programs. Cladistics 1: 47-66.
Lynch, M. 1990. Methods for the analysis of comparative data in evolutionary
biology. Evolution 45: 1065-1080.
Maddison, D. R. 1991. The discovery and importance of multiple islands of
most-parsimonious trees. Systematic Zoology 40: 315-328.
Margush, T. and F. R. McMorris. 1981. Consensus n-trees. Bulletin of Mathematical Biology 43: 239-244.
Muse, S. V. and B. S. Gaut. 1994.
A likelihood approach for comparing synonymous and nonsynonymous nucleotide
substitution rates, with application to the chloroplast genome.
Molecular Biology and Evolution 11: 715-724,
Nelson, G. 1979. Cladistic analysis and synthesis: principles and definitions,
with a historical note on Adanson's Familles des Plantes
(1763-1764). Systematic Zoology 28: 1-21.
Nei, M. 1972. Genetic distance between populations. American Naturalist 106: 283-292.
Nei, M. and W.-H. Li. 1979. Mathematical model for studying genetic variation
in terms of restriction endonucleases. Proceedings of the National Academy of Sciences, USA 76: 5269-5273.
Nei, M. and T. Gojobori. 1986. Simple methods for estimating the numbers of
synonymous and nonsynonymous nucleotide substitutions. Molecular Biology and
Evolution 3: 418-426.
Nielsen, R., and Z. Yang. 1998. Likelihood models for detecting positively
selected amino acid sites and applications to the HIV-1 envelope gene.
Genetics 148: 929-936.
Nixon, K. C. 1999. The parsimony ratchet, a new method for rapid parsimony
analysis. Cladistics 15: 407-414.
Page, R. D. M. 1989. Comments on component-compatibility in historical
biogeography. Cladistics 5: 167-182.
Penny, D. and M. D. Hendy. 1985. Testing methods of evolutionary tree
construction. Cladistics 1: 266-278.
Platnick, N. 1987. An empirical comparison of microcomputer parsimony
programs. Cladistics 3: 121-144.
Platnick, N. 1989. An empirical comparison of microcomputer parsimony
programs. II. Cladistics 5: 145-161.
Reynolds, J. B., B. S. Weir, and C. C. Cockerham. 1983. Estimation of the
coancestry coefficient: basis for a short-term genetic
distance. Genetics 105: 767-779.
Robinson, D. F. and L. R. Foulds. 1979. Comparison of weighted
labelled trees. pp. 119-126 in Combinatorial Mathematics VI. Proceedings
of the Sixth Australian Conference on Combinatorial Mathematics, Armidale,
Australia, August, 1978, ed. A. F. Horadam and W. D. Wallis. Lecture Notes in Mathematics, No. 748. Springer-Verlag, Berlin.
Robinson, D. F. and L. R. Foulds. 1981. Comparison of phylogenetic trees.
Mathematical Biosciences 53: 131-147.
Rohlf, F. J. and M. C. Wooten. 1988. Evaluation of the restricted maximum
likelihood method for estimating phylogenetic trees using simulated allele-
frequency data. Evolution 42: 581-595.
Rzhetsky, A., and M. Nei. 1992. Statistical properties of the ordinary
least-squares, generalized least-squares, and minimum-evolution methods
of phylogenetic inference. Journal of Molecular Evolution 35:
367-375 .
Saitou, N., and M. Nei. 1987. The neighbor-joining method: a new method for
reconstructing phylogenetic trees. Molecular Biology and Evolution 4: 406-425.
Sanderson, M. J. 1990. Flexible phylogeny reconstruction: a review of
phylogenetic inference packages using parsimony. Systematic Zoology 39: 414-420.
Sankoff, D. D., C. Morel, R. J. Cedergren. 1973. Evolution of 5S RNA and
the nonrandomness of base replacement. Nature New Biology 245: 232-234.
Shimodaira, H. and M. Hasegawa. 1999. Multiple comparisons of log-likelihoods
with applications to phylogenetic inference. Molecular Biology and
Evolution 16: 1114-1116.
Shimodaira, H. 2002. An approximately unbiased test of phylogenetic
tree selection. Systematic Biology 51: 492-508.
Sokal, R. R. and P. H. A. Sneath. 1963. Principles of Numerical Taxonomy.
W. H. Freeman, San Francisco.
Smouse, P. E. and W.-H. Li. 1987. Likelihood analysis of mitochondrial
restriction-cleavage patterns for the human-chimpanzee-gorilla trichotomy.
Evolution 41: 1162-1176.
Sober, E. 1983a. Parsimony in systematics: philosophical issues. Annual Review of Ecology and Systematics 14: 335-357.
Sober, E. 1983b. A likelihood justification of parsimony. Cladistics 1: 209-233.
Sober, E. 1988. Reconstructing the Past: Parsimony, Evolution,
and Inference. MIT Press, Cambridge, Massachusetts.
Sokal, R. R., and P. H. A. Sneath. 1963. Principles of Numerical
Taxonomy. W. H. Freeman, San Francisco.
Steel, M. A., P. J. Lockhart, and D. Penny. 1993. Confidence in evolutionary trees from biological sequence data. Nature 364: 440-442.
Steel, M. A. 1994. Recovering a tree from the Markov leaf colourations
it generates under a Markov model. Applied Mathematics Letters
7: 19-23.
Studier, J. A. and K. J. Keppler. 1988. A note on the neighbor-joining
algorithm of Saitou and Nei. Molecular Biology and Evolution 5: 729-731.
Swofford, D. L. and G. J. Olsen. 1990. Phylogeny reconstruction. Chapter
11, pages 411-501 in Molecular Systematics, ed. D. M. Hillis and C. Moritz.
Sinauer Associates, Sunderland, Massachusetts.
Swofford, D. L., G. J. Olsen, P. J. Waddell, and D. M. Hillis. 1996.
Phylogenetic inference. pp. 407-514 in Molecular Systematics, 2nd ed.,
ed. D. M. Hillis, C. Moritz, and B. K. Mable. Sinauer Associates, Sunderland,
Massachusetts.
Templeton, A. R. 1983. Phylogenetic inference from restriction endonuclease
cleavage site maps with particular reference to the evolution of humans and the
apes. Evolution 37: 221-244.
Thompson, E. A. 1975. Human Evolutionary Trees. Cambridge University
Press, Cambridge.
Veerassamy, S., A. Smith and E. R. M. Tillier. 2003.
A transition probability model for amino acid substitutions from Blocks.
Journal of Computational Biology 10: 997-1010.
Wright, S. 1934. An analysis of variability in number of digits in an inbred
strain of guinea pigs. Genetics 19: 506-536.
Wu, C. F. J. 1986. Jackknife, bootstrap and other resampling plans in
regression analysis. Annals of Statistics 14: 1261-1295.
Yang, Z. 1993. Maximum-likelihood estimation of phylogeny from DNA sequences
when substitution rates differ over sites. Molecular Biology and
Evolution 10: 1396-1401.
Yang, Z. 1994. Maximum likelihood phylogenetic estimation from DNA sequences
with variable rates over sites: approximate methods. Journal of Molecular
Evolution 39: 306-314.
Yang, Z. 1995. A space-time process model for the evolution of DNA sequences.
Genetics 139: 993-1005.
Yang, Z. 1998. Likelihood ratio tests for detecting positive selection and
application to primate lysozyme evolution. Molecular Biology and
Evolution15: 568-573.
Yang, Z., and R. Nielsen. 1998. Synonymous and nonsynonymous rate variation in
nuclear genes of mammals. Journal of Molecular Evolution 46: 409-418.
Yang, Z. 2006. Computational Molecular Evolution. Oxford University
Press, Oxford.
Zharkikh, A. and W.-H. Li. 1995. Estimation of confidence in phylogeny:
the complete-and-partial bootstrap technique. Molecular Biology and
Evolution 4: 44-63.
Over the years various granting agencies have contributed to the
support of the PHYLIP project (at first without knowing it). They are:
However, starting in April, 2009 there is no grant support for
PHYLIP.
I am particularly grateful to past program administrators William Moore,
Irene Eckstrand, Peter Arzberger, and Conrad Istock, who have
gone beyond the call of duty to make sure that PHYLIP continued.
Booby prizes for funding are awarded to:
The original Camin-Sokal parsimony program and the polymorphism parsimony
program were written by me in 1977 and 1978. They were Pascal versions of
earlier FORTRAN programs I wrote in 1966 and 1967 using the same algorithm to
infer phylogenies under the Camin-Sokal and polymorphism parsimony
criteria. Harvey Motulsky worked for me as a programmer in 1971 and wrote
FORTRAN programs to carry out the Camin-Sokal, Dollo, and polymorphism
methods (he is better-known these days as the author of the scientific
data analysis package GraphPad). But most of the early work on PHYLIP other
than my own was by Jerry
Shurman and Mark Moehring. Jerry Shurman worked for me in the summers of
1979 and 1980, and Mark Moehring worked for me in the summers of 1980 and
1981. Both wrote original versions of many of the other programs, based on
the original versions of my Camin-Sokal parsimony program and my polymorphism
parsimony program. These
formed the basis of Version 1 of the Package, first distributed in October,
1980.
Version 2, released in the spring of 1982, involved a fairly complete rewrite
by me of many of those programs. Hisashi Horino for
version 3.3 reworked some parts of the programs Clique and Consense
to make their output more comprehensible, and has added some code to the
tree-drawing programs Drawgram and Drawtree as well. He also worked on
some of the Drawtree and Drawgram driver code.
Later programmers Akiko Fuseki, Sean Lamont,
Andrew Keeffe, Daniel Yek, Dan Fineman, Patrick Colacurcio,
Mike Palczewski, Doug Buxton, Ian Robertson, Marissa LaMadrid, Eric Rynes,
and Elizabeth Walkup gave
me substantial help with the 3.6 releases, and their excellent work is
greatly appreciated. Akiko, in over 10 years of excellent work, did much of the
hard work of adding
new features and changing old ones in the 3.4 and 3.5 releases,
centralized many of the C routines in support files, and is responsible for the
new versions of Dnapars and Pars. Andrew
prepared the Macintosh version, wrote Retree, added the ray-tracing
and PICT code to the Draw... programs and has since done much other work. Sean
was central to the conversion to
C, and tested it extensively. Mike Palczewski reorganized the code and
centralized routines, bringing us closer to object-oriented structure.
My (then) postdoctoral fellow
Mary Kuhner and her associate Jon Yamato created Neighbor, the
neighbor-joining and UPGMA program, for the current release, for which I am
also grateful (Naruya Saitou and Li Jin kindly encouraged us to use some of the
code from their own implementation of this method). Lucas Mix created
the protein likelihood programs Protml and Protmlk. Elisabeth Tillier
provided the code for her PMB amino acid model. My current programmers
Jim McGill and Bob Giansiracusa have made a great contribution to
getting the current version working.
I am very grateful to over 400
users for algorithmic suggestions, complaints about features (or lack of
features), and information about the behavior of their operating systems
and compilers. A list of some of their names will be found at
the
credits page on the PHYLIP web site which is at
A major contribution to this package has been made by others
writing programs or parts of programs. Chris Meacham contributed the
important program Factor, long demanded by users, and the even more
important ones PLOTREE and PLOTGRAM. Important parts of the code in
Drawgram and Drawtree were taken over from those two programs.
Kent Fiala wrote
function "reroot" to do outgroup-rooting, which was an essential part of many
programs in earlier versions. Someone at the Western Australia Institute of
Technology suggested the name PHYLIP (by writing it the label on the
outside of a magnetic tape). Probably it was the late Julian Ford
(I've lost the relevant letter).
The distribution of the package also owes much to Buz Wilson and Willem Ellis,
who put a lot of effort into the early distributions of the PCDOS and
Macintosh versions respectively. Christopher Meacham and Tom Duncan for three
versions distributed a printed version of these documentation files (they could
not continue to do so), and I am
very grateful to them for those efforts. William H.E. Day and F. James Rohlf
were very helpful in setting up the listserver news bulletin service which
succeeded the PHYLIP newsletter for a time.
I also wish to thank the people who have made computer resources available to
me, mostly in the loan of use of microcomputers. These include Jeremy
Field, Clem Furlong, Rick Garber, Dan Jacobson, Rochelle Kochin, Monty Slatkin,
Jim Archie, Jim Thomas, and George Gilchrist.
I should also note the computers used to develop this package:
These include a CDC 6400, two DECSystem 1090s, my trusty old SOL-20, my
old Osborne-1, a VAX 11/780, a VAX 8600, a MicroVAX I, a DECstation
3100, my old Toshiba 1100+, my
DECstation 5000/200, a DECstation 5000/125, a Compudyne 486DX/33, a
Trinity Genesis 386SX, a Zenith Z386, a Mac Classic, a DEC Alphastation 400
4/233, a Pentium 120, a Pentium 200, a PowerMac 6100, and a Macintosh G3.
(One of the reasons
we have been successful in achieving compatibility between different computer
systems is that I have had to run them myself under so many different operating
systems and compilers).
A comprehensive list of phylogeny programs is maintained at the PHYLIP
web site on the Phylogeny Programs pages:
Here we will simply mention some of the major general-purpose programs. For
many more and much more, see those web pages.
PAUP* A comprehensive program with parsimony, likelihood, and
distance matrix methods. It competes with PHYLIP to be responsible for
the most trees published. Written by David Swofford, now of Duke
University and distributed by
Sinauer Associates of Sunderland, Massachusetts.
It is described in
a web page.
at http://www.sinauer.com/detail.php?id=8060.
Current prices are $100 for the Macintosh version, $85 for the
Windows version, and $150 for Unix versions for many kinds of workstations.
MrBayes The leading program for Bayesian inference of
phylogenies. It uses Markov Chain Monte Carlo inference to assess
support for clades and to infer posterior distrubutions of parameters.
Produced by John Huelsenbeck and Fredrik Ronquist, it is available at
its web site at
MEGA A program by Sudhir Kumar of Arizona State University
(written together with Koichiro Tamura, Joel Dudley and Masatoshi Nei).
It can carry out parsimony and distance matrix methods
for DNA sequence data. Version 4 for Windows, Macintosh, and Linux
can be downloaded from
the MEGA web site
at http://www.megasoftware.net.
PAML Ziheng Yang of the Department of Genetics and Biometry at
University College, London has written this package of programs to
carry out likelihood analysis of DNA and protein sequence data.
It is one of the only packages able to use the codon model for protein
sequence data which takes the genetic code reasonably fully into account.
PAML is particularly strong in the options for coping with variability of rates
of evolution from site to site, though it is less able than some other
packages to search effectively for the best tree. It is available as
C source code and as Mac OS X and Windows executables from its web site at
http://abacus.gene.ucl.ac.uk/software/paml.html.
Phyml
Stephane Guindon, currently of the University of Auckland, New Zealand,
has written Phyml, a fast likelihood program for molecular sequence data
It is available as binaries from
its web page at the ATGC site
at the Université de Montpellier in
France. Source code for Phyml, including later developments of the program,
are available at its site at Google Code.
RAxML Alexandros Stamatakis, of the Exelexis Lab at the
Technische Universität München has written RAxML, a very fast
likelihood program for molecular sequences. It is available from
the
Exelexis Lab software web page. Source code is available too. RAxML
seems to be the fastest implementation of likelihood for molecular data.
TNT This program, by Pablo Goloboff, J. S. Farris, and Kevin Nixon,
is for searching large data sets for most parsimonious trees.
The authors are respectively at the Instituto Miguel Lillo in Tucumán,
Argentina, the Naturhistoriska Riksmuseet in Stockholm, Sweden, and the
Hortorium, Cornell University, Ithaca, New York.
TNT is described
as faster than other methods, though not faster than NONA for small to
medium data sets. It is distributed as
Windows, Linux, and Mac OS X executables (the latter two
require the PVM Parallel Virtual Machine library to be installed).
The program and some support files including documentation are
available from its download
area at http://www.zmuc.dk/public/phylogeny/tnt
(see the ReadMe! web page there).
It is free, provided you agree to a license with some reasonable limitations.
DAMBE A package written by Xuhua Xia of the
Department of Biology of the University of Ottawa.
Its initials stand for Data Analysis in Molecular Biology and Evolution.
DAMBE is a general-purpose package for DNA and protein sequence phylogenies.
It can read and
convert a number of file formats, and has many features for
descriptive statistics, and can compute a number of commonly-used
distance matrix measures and infer phylogenies by parsimony, distance,
or likelihood methods, including bootstrapping and jackknifing. There are
a number of kinds of statistical tests of trees available and it
can also display phylogenies. DAMBE includes a copy of ClustalW as well;
DAMBE consists of Windows executables. It is available from its
web site at
These are only a few of the over 380 different phylogeny packages that
are now available (as of July, 2010 - the number keeps increasing). The
others are described (and web links provided) at my
Phylogeny Programs web pages at the address given above.
Simply let me know of any problems you have had adapting the
programs to your computer. I can often make "transparent" changes that, by
making the code avoid the wilder, woolier, and less standard parts of
C, not only help others who have your machine but even improve the
chance of the programs functioning on new machines. I would like fairly
detailed information on what gave trouble, on what operating system,
machine, and (if relevant) compiler, and what had to be done to make the
programs work. I am sometimes able to do some over-the-telephone
trouble-shooting, particularly
if I don't have to pay for the call, but electronic mail is a the best
way for me to be asked about problems, as you can include your
input and output files so I can see what is going on (please do not
send them as Attachments, but as part of the body of a message). I'd really
like these programs to be
able to run with only routine changes on absolutely everything, down to
and possibly including the Amana Touchmatic Radarange Microwave Oven
which was an Intel 8080 system (in fact, early versions of this package did
run successfully on Intel 8080 systems running the CP/M operating system).
A PalmPilot version was contemplated too.
I would also like to know timings of programs from the package, when
run on the three test input files provided above, for various computer and
compiler combinations, so that I can provide this information in the
section on speeds of this document.
For the phylogeny plotting programs Drawgram and Drawtree,
I am particularly interested in knowing what has to be done
to adapt them for other graphic file formats.
You can also be helpful to PHYLIP users in your part of the world by
helping them get the latest version of PHYLIP from our web site
and by helping them with any
problems they may have in getting PHYLIP working on their data.
Your help is appreciated. I am always happy to hear suggestions
for features and programs that ought to be incorporated in the package,
but please do not be upset if I turn out to have already considered the
particular possibility you suggest and decided against it.
Read The (documentation) Files Meticulously ("RTFM"). If that doesn't solve the
problem, please check the Frequently Asked Questions web page at the
PHYLIP web site:
and the PHYLIP Bugs web page at that site:
If none of these answers your question, get in touch with me. My email address
is given below. If you do ask about a problem, please specify the program
name, version of the package, computer operating system, and
send me your data file so I can test the problem. Also it will help if you
have the relevant output and documentation files so that you
can refer to them in any correspondence. I can also be reached by telephone
by calling me in my office:
+1-(206)-543-0150, or at home: +1-(206)-526-9057 (how's that for user
support!). If I cannot be reached at either place, a message can be left at
the office of
the Department of Genome Sciences, +1-(206)-221-7377 but I prefer strongly that I not
call you, as in any phone consultation the least you can do is pay the phone
bill. Better yet, use email.
Particularly if you are in a part of the world distant from me, you may also
want to try to get in touch with other users of PHYLIP nearby. I can also,
if requested, provide a list of nearby users.
Electronic mail addresses: joe (at) gs.washington.edu
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Mix is a general parsimony program which carries out the Wagner and
Camin-Sokal parsimony methods in mixture, where each character can have
its method specified separately. The program defaults to carrying out Wagner
parsimony.
The Camin-Sokal parsimony method explains the data by assuming that
changes 0 --> 1 are allowed but not changes 1 --> 0. Wagner parsimony
allows both kinds of changes. (This under the assumption that 0 is the
ancestral state, though the program allows reassignment of the ancestral
state, in which case we must reverse the state numbers 0 and 1
throughout this discussion). The criterion is to find the tree which
requires the minimum number of changes. The Camin-Sokal method is due
to Camin and Sokal (1965) and the Wagner method to Eck and Dayhoff
(1966) and to Kluge and Farris (1969).
Here are the assumptions of these two methods:
That these are the assumptions of parsimony methods has been documented
in a series of papers of mine: (1973a, 1978b, 1979, 1981b,
1983b, 1988b). For an opposing view arguing that the parsimony methods
make no substantive
assumptions such as these, see the papers by Farris (1983) and Sober (1983a,
1983b), but also read the exchange between Felsenstein and Sober (1986).
The input for Mix is the standard input for discrete characters
programs, described above in the documentation file for the
discrete-characters programs. States "?", "P", and "B" are allowed.
The options are selected using a menu:
The options U, X, J, O, T, A, and M are the usual User Tree, miXed
methods, Jumble, Outgroup,
Ancestral States, and Multiple Data Sets options, described either
in the main documentation file or in the Discrete Characters Programs
documentation file. The
user-defined trees supplied if you use the U option must be given as rooted
trees with two-way splits (bifurcations). The O option is acted upon only if
the final tree is unrooted and is not a user-defined tree. One of the
important uses of the the O option is to root the tree so that if there are
any characters in which the ancestral states have not been specified, the
program will print out a table showing which ancestral states require the
fewest steps. Note that when any of the characters has Camin-Sokal parsimony
assumed for it, the tree is rooted and the O option will have no effect.
The option P toggles between the Camin-Sokal parsimony criterion
and the default Wagner parsimony criterion. Option X invokes
mixed-method parsimony. If the A option is invoked, the ancestor is not
to be counted as one of the species.
The F (Factors)
option is not available in this program, as it would have no effect on
the result even if that information were provided in the input file.
Output is standard: a list of equally parsimonious trees, which will be printed
as rooted or unrooted depending on which is appropriate, and, if the
user chooses, a table of the
number of changes of state required in each character. If the Wagner option is
in force for a character, it may not be possible to unambiguously locate the
places on the tree where the changes occur, as there may be multiple
possibilities. If the user selects menu option 5, a table is printed out
after each tree, showing for each
branch whether there are known to be changes in the branch, and what the states
are inferred to have been at the top end of the branch. If the inferred state
is a "?" there will be multiple equally-parsimonious assignments of states; the
user must work these out for themselves by hand.
If the Camin-Sokal parsimony method
is invoked and the Ancestors option is also used, then the program will
infer, for any character whose ancestral state is unknown ("?") whether the
ancestral state 0 or 1 will give the fewest state changes. If these are
tied, then it may not be possible for the program to infer the
state in the internal nodes, and these will all be printed as ".". If this
has happened and you want to know more about the states at the internal
nodes, you will find helpful to use Move to display the tree and examine
its interior states, as the algorithm in Move shows all that can be known
in this case about the interior states, including where there is and is not
amibiguity. The algorithm in Mix gives up more easily on displaying these
states.
If the A option is not used, then the program will assume 0 as the
ancestral state for those characters following the Camin-Sokal method,
and will assume that the ancestral state is unknown for those characters
following Wagner parsimony. If any characters have unknown ancestral
states, and if the resulting tree is rooted (even by outgroup),
a table will also be printed out
showing the best guesses of which are the ancestral states in each
character. You will find it useful to understand the difference between
the Camin-Sokal parsimony criterion with unknown ancestral state and the Wagner
parsimony criterion.
If the U (User Tree) option is used and more than one tree is supplied, the
program also performs a statistical test of each of these trees against the
best tree. This test, which is a version of the test proposed by
Alan Templeton (1983) and evaluated in a test case by me (1985a). It is
closely parallel to a test using log likelihood differences
invented by Kishino and Hasegawa (1989), and uses the mean and variance of
step differences between trees, taken across characters. If the mean
is more than 1.96 standard deviations different then the trees are declared
significantly different. The program
prints out a table of the steps for each tree, the differences of
each from the highest one, the variance of that quantity as determined by
the step differences at individual characters, and a conclusion as to
whether that tree is or is not significantly worse than the best one. It
is important to understand that the test assumes that all the binary
characters are evolving independently, which is unlikely to be true for
many suites of morphological characters.
If there are more than two trees, the test done is an extension of
the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out
that a correction for the number of trees was necessary, and they
introduced a resampling method to make this correction. In the version
used here the variances and covariances of the sums of steps across
characters are computed for all pairs of trees. To test whether the
difference between each tree and the best one is larger than could have
been expected if they all had the same expected number of steps,
numbers of steps for all trees are sampled with these covariances and equal
means (Shimodaira and Hasegawa's "least favorable hypothesis"),
and a P value is computed from the fraction of times the difference between
the tree's value and the lowest number of steps exceeds that actually
observed. Note that this sampling needs random numbers, and so the
program will prompt the user for a random number seed if one has not
already been supplied. With the two-tree KHT test no random numbers
are used.
In either the KHT or the SH test the program
prints out a table of the number of steps for each tree, the differences of
each from the lowest one, the variance of that quantity as determined by
the differences of the numbers of steps at individual characters,
and a conclusion as to
whether that tree is or is not significantly worse than the best one.
If option 6 is left in its default state the trees
found will be written to a tree file, so that they are available to be used
in other programs. If the program finds multiple
trees tied for best, all of these are written out onto the output tree
file. Each is followed by a numerical weight in square brackets (such as
[0.25000]). This is needed when we use the trees to make a consensus
tree of the results of bootstrapping or jackknifing, to avoid overrepresenting
replicates that find many tied trees.
At the beginning of the program is a constant, maxtrees,
the maximum number of trees which the program will store for output.
The program is descended from earlier programs Sokal and Wagner which have
long since been removed from the PHYLIP package, since Mix has all their
capabilites and more.
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Move is an interactive parsimony program, inspired by Wayne Maddison and
David Maddison's marvellous program MacClade, which was written for
Macintosh computers. Move reads in a data set which is prepared in almost the
same format as one for the mixed method parsimony program Mix. It allows
the user to choose an initial tree, and displays this tree on the screen. The
user can look at different characters and the way their states are
distributed on that tree, given the most parsimonious reconstruction of state
changes for that particular tree. The user then can specify how the tree is to
be rearraranged, rerooted or written out to a file. By looking at different
rearrangements of the tree the user can manually search for the most
parsimonious tree, and can get a feel for how different characters are affected by changes in the tree topology.
This program is compatible with fewer computer systems than the other
programs in PHYLIP. It can be adapted to MSDOS systems or to
any system whose screen or terminals emulate DEC VT100
terminals (such as Telnet programs for logging in to remote computers over a
TCP/IP network,
VT100-compatible windows in the X windowing system, and any
terminal compatible with ANSI standard terminals).
For other screen types, there is a generic option which does
not make use of screen graphics characters to display the character
states. This will be less effective, as the states will be less
easy to see when displayed.
The input data file is set up almost identically to the data files for
Mix.
The user interaction starts with the program presenting a menu. The
menu looks like this:
The P (Parsimony method) option selects among Wagner parsimony
and Camin-Sokal parsimony. If X (miXed methods) is selected the
P menu item disappears, as it is then irrelevant.
The X (miXed methods), A (Ancestors), F (Factors),
O (Outgroup), T (Threshold), and 0 (Graphics type) options are the usual
ones and are described in the main documentation page and in the
discrete characters program documentation page.
The U (initial tree) option allows the user to choose whether
the initial tree is to be arbitrary, interactively specified by the user, or
read from a tree file. Typing U causes the program to change among the
three possibilities in turn. I
would recommend that for a first run, you allow the tree to be set up
arbitrarily (the default), as the "specify" choice is difficult
to use and the "user tree" choice requires that you have available a tree file
with the tree topology of the initial tree.
Its default name is intree. The program will ask you for its name if
it looks for the input tree file and does not find one of this name.
If you wish to set up some
particular tree you can also do that by the rearrangement commands specified
below.
The T (threshold) option allows a continuum of methods between
parsimony and compatibility. Thresholds less than or equal to 1.0 do not
have any
meaning and should not be used: they will result in a tree dependent only on
the input order of species and not at all on the data!
The usual W (Weights) option is available in Move. It allows integer
weights up to 36, using the symbols 0-9 and A-Z. Increased weight on a step
increases both the number of parsimony steps on the character and the
contribution it makes to the number of compatibilities.
The F (Factors)
option is available in this program. It is only used to inform the program
which
groups of characters are to be counted together in computing the number of
characters compatible with the tree. Thus if three binary characters are all
factors of the same multistate character, the multistate character will
be counted as compatible with the tree only if all three factors are compatible
with it.
The S (Screen width) option allows the width in characters of the
display to be adjusted when more then 80 characters can be displayed on
the user's screen.
The L (screen Lines) option allows the user to change the height of the
screen (in lines of characters) that is assumed to be available on the
display. This may be particularly helpful when displaying large trees
on terminals that have more than 24 lines per screen, or on workstation
or X-terminal screens that can emulate the ANSI terminals with more than
24 lines.
After the initial menu is displayed and the choices are made,
the program then sets up an initial tree and displays it. Below it will be a
one-line menu of possible commands, which looks like this:
If you type H or ? you will get a single screen showing a description of each
of these commands in a few words. Here are slightly more detailed
descriptions:
As we have seen, the initial menu of the program allows you to choose
among three screen types (PCDOS, Ansi, and none).
If you want to avoid having to make this choice every time, you can change
some of the constants in the file phylip.h to have the terminal type
initialize itself in the proper way, and recompile. We have tried to
have the default values be correct for PC, Macintosh, and Unix
screens. If the setting is "none" (which is necessary on
Macintosh MacOS 9 screens), the special graphics
characters will not be used to indicate nucleotide states, but only letters
will be used for the four nucleotides. This is less easy to look at.
The constants that need attention are ANSICRT and IBMCRT.
Currently these are both set to "false" on Macintosh MacOS 9 systems,
to "true" on MacOS X and on Unix/Linux
systems, and IBMCRT is set to "true" on Windows systems. If your system
has an ANSI compatible terminal, you might want to find the
definition of ANSICRT in phylip.h and set it to "true", and
IBMCRT to "false".
Move uses as its numerical criterion the Wagner and
Camin-Sokal parsimony methods in mixture, where each character can have
its method specified separately. The program defaults to carrying out Wagner
parsimony.
The Camin-Sokal parsimony method explains the data by assuming that changes 0
--> 1 are allowed but not changes 1 --> 0. Wagner parsimony allows both kinds
of changes. (This is under the assumption that 0 is the ancestral state, though
the program allows reassignment of the ancestral state, in which case we must
reverse the state numbers 0 and 1 throughout this discussion). The criterion
is to find the tree which requires the minimum number of changes. The Camin-
Sokal method is due to Camin and Sokal (1965) and the Wagner method to Eck and
Dayhoff (1966) and to Kluge and Farris (1969).
Here are the assumptions of these two methods:
That these are the assumptions of parsimony methods has been documented
in a series of papers of mine: (1973a, 1978b, 1979, 1981b,
1983b, 1988b). For an opposing view arguing that the parsimony methods
make no substantive
assumptions such as these, see the papers by Farris (1983) and Sober (1983a,
1983b), but also read the exchange between Felsenstein and Sober (1986).
Below is a test data set, but we cannot show the
output it generates because of the interactive nature of the program.
© Copyright 1991-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program implements the Neighbor-Joining method of Saitou and Nei (1987)
and the UPGMA method of clustering. The program was written by Mary Kuhner
and Jon Yamato, using some code from program Fitch. An important part of the
code was translated
from FORTRAN code from the neighbor-joining program written by Naruya Saitou
and by Li Jin, and is used with the kind permission of Drs. Saitou and Jin.
Neighbor constructs a tree by successive clustering of lineages, setting
branch lengths as the lineages join. The tree is not rearranged
thereafter. The tree does not assume an evolutionary clock, so that it
is in effect an unrooted tree. It should be somewhat similar to the tree
obtained by Fitch. The program cannot evaluate a User tree, nor can it prevent
branch lengths from becoming negative. However the algorithm is far faster
than Fitch or Kitsch. This will make it particularly effective in their place
for large studies or for bootstrap or jackknife resampling studies which
require runs on multiple data sets.
The UPGMA option constructs a tree by successive (agglomerative) clustering
using an average-linkage method of clustering. It has some relationship
to Kitsch, in that when the tree topology turns out the same, the
branch lengths with UPGMA will turn out to be the same as with the P = 0
option of Kitsch.
The options for Neighbor are selected through the menu, which looks like
this:
Most of the input options
(L, R, S, J, and M) are as given in the Distance Matrix Programs
documentation file,
and their input format is the same as given there.
The O (Outgroup) option
is described in the main
documentation file of this package. It is not available when the
UPGMA option is selected. The Jumble option (J) does not allow
multiple jumbles (as most of the other programs that have it do),
as there is no objective way of choosing which of the multiple
results is best, there being no explicit criterion for optimality of the tree.
An important use of the Jumble option is in the use of Neighbor with
bootstrap samples. Backeljau et al. (1996)
and Farris et al. (1996)
point out that when there are ties in the distance matrix, Neighbor will
resolve them in a way dependent on the order of species in the input file. If
we have many bootstrap samples from a data set, and run Neighbor on them, we
can then get apparent strong support for one resolution of a multifurcation,
purely as an artifact of the order of species in the input file. By using a
random order of input species for each bootstrap, the problem disappears, as
Farris et al. (1996) acknowledge. Neighbor therefore has the Jumble option
turned on whenever multiple distance matrices (the M option) is activated.
Only one Jumble needs to be done per data set in that case.
Option N chooses between the Neighbor-Joining and UPGMA methods. Option
S is the usual Subreplication option. Here, however, it is present only
to allow Neighbor to read the input data: the number of replicates is
actually ignored, even though it is read in. Note that this means that
one cannot use it to have missing data in the input file, if Neighbor is
to be used.
The output consists of an tree (rooted if UPGMA, unrooted if Neighbor-Joining)
and the lengths of the
interior segments. The Average Percent Standard Deviation is not
computed or printed out. If the tree found by Neighbor is fed into Fitch
as a User Tree, it will compute this quantity if one also selects the
N option of Fitch to ensure that none of the branch lengths is re-estimated.
As Neighbor runs it prints out an account of the successive clustering
levels, if you allow it to. This is mostly for reassurance and can be
suppressed using menu option 2. In this printout of cluster levels
the word "OTU" refers to a tip species, and the word "NODE" to an
interior node of the resulting tree.
The constants available for modification at the beginning of the
program are "namelength" which gives the length of a
species name, and the usual boolean
constants that initialize the terminal type. There is no feature saving
multiple trees tied for best,
partly because we do not expect exact ties except in cases where the branch
lengths make the nature of the tie obvious, as when a branch is of zero
length.
The major advantage of Neighbor is its speed: it requires a time only
proportional to the cube of the number of species. It is significantly
faster than version 3.5 of this program. By contrast Fitch
and Kitsch require a time that rises as the fourth power of the number
of species. Thus Neighbor is well-suited to bootstrapping studies and
to analysis of very large trees. Our simulation studies (Kuhner
and Felsenstein, 1994) show that, contrary to statements in the
literature by others, Neighbor does not get as accurate an estimate of
the phylogeny as does Fitch. However it does nearly as well, and in
view of its speed this will make it a quite useful program.
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Pars is a general parsimony program which carries out the Wagner
parsimony method with multiple states. Wagner parsimony
allows changes among all states. The criterion is to find the tree which
requires the minimum number of changes.
The Wagner method was originated by Eck and Dayhoff (1966) and by Kluge and
Farris (1969). Here are its assumptions:
That these are the assumptions of parsimony methods has been documented
in a series of papers of mine: (1973a, 1978b, 1979, 1981b,
1983b, 1988b). For an opposing view arguing that the parsimony methods
make no substantive
assumptions such as these, see the papers by Farris (1983) and Sober (1983a,
1983b), but also read the exchange between Felsenstein and Sober (1986).
The input for Pars is the standard input for discrete characters
programs, described above in the documentation file for the
discrete-characters programs, except that multiple states (up to 8 of them)
are allowed. Any characters other than "?" are allowed as states, up to a
maximum of 8 states. In fact, one can
use different symbols in different columns of the data matrix,
although it is rather unlikely that you would want to do that.
The symbols you can use are:
Pars can handle both bifurcating and multifurcating trees. In doing its
search for most parsimonious trees, it adds species not only by creating new
forks in the middle of existing branches, but it also tries putting them at
the end of new branches which are added to existing forks. Thus it searches
among both bifurcating and multifurcating trees. If a branch in a tree
does not have any characters which might change in that branch in the most
parsimonious tree, it does not save that tree. Thus in any tree that
results, a branch exists only if some character has a most parsimonious
reconstruction that would involve change in that branch.
It also saves a number of trees tied for best (you can alter the number
it saves using the V option in the menu). When rearranging trees, it
tries rearrangements of all of the saved trees. This makes the algorithm
slower than earlier programs such as Mix.
The options are selected using a menu:
The Weights (W) option
takes the weights from a file whose default name is "weights". The weights
follow the format described in the main documentation file, with integer
weights from 0 to 35 allowed by using the characters 0, 1, 2, ..., 9 and
A, B, ... Z.
The User tree (option U) is read from a file whose default name is
intree.
The trees can be multifurcating. They must be preceded in the file by a
line giving the number of trees in the file.
The options J, O, T, and M are the usual Jumble, Outgroup,
Threshold parsimony, and Multiple Data Sets options,
described either
in the main documentation file or in the Discrete Characters Programs
documentation file.
The S (search) option controls how, and how much, rearrangement is done on the
tied trees that are saved by the program. If the "More thorough search"
option (the default) is chosen, the program will save multiple tied trees,
without collapsing internal branches that have no evidence of change on them.
It will subsequently rearrange on all parts of each of those trees.
If the "Less thorough search" option is chosen, before saving,
the program will collapse
all branches that have no evidence that there is any change on that branch.
This leads to less attempted rearrangement. If the "Rearrange on one
best tree" option is chosen, only the first of the tied trees is used for
rearrangement. This is faster but less thorough. If your trees are likely
to have large multifurcations, do not use the default "More thorough search"
option as it could result in too large a number of trees being saved.
The M (multiple data sets option) will ask you whether you want to
use multiple sets of weights (from the weights file) or multiple data sets.
The ability to use a single data set with multiple weights means that
much less disk space will be used for this input data. The bootstrapping
and jackknifing tool Seqboot has the ability to create a weights file with
multiple weights.
The O (outgroup) option will have no effect if the U (user-defined tree)
option is in effect.
The T (threshold) option allows a continuum of methods
between parsimony and compatibility. Thresholds less than or equal to 1.0 do
not have any meaning and should
not be used: they will result in a tree dependent only on the input
order of species and not at all on the data!
Output is standard: if option 1 is toggled on, the data is printed out,
with the convention that "." means "the same as in the first species".
Then comes a list of equally parsimonious trees.
Each tree has branch lengths. These are computed using an algorithm
published by Hochbaum and Pathria (1997) which I first heard of from
Wayne Maddison who invented it independently of them. This algorithm
averages the number of reconstructed changes of state over all sites
over all possible most parsimonious placements of the changes of state
among branches. Note that it does not correct in any way for multiple
changes that overlay each other.
If option 2 is
toggled on a table of the
number of changes of state required in each character is also
printed. If option 5 is toggled
on, a table is printed
out after each tree, showing for each branch whether there are known to be
changes in the branch, and what the states are inferred to have been at the
top end of the branch. This is a reconstruction of the ancestral sequences
in the tree. If you choose option 5, a menu item D appears which gives you
the opportunity to turn off dot-differencing so that complete ancestral
sequences are shown. If the inferred state is a "?",
there will be multiple
equally-parsimonious assignments of states; the user must work these out for
themselves by hand.
If option 6 is left in its default state the trees
found will be written to a tree file, so that they are available to be used
in other programs. If the program finds multiple
trees tied for best, all of these are written out onto the output tree
file. Each is followed by a numerical weight in square brackets (such as
[0.25000]). This is needed when we use the trees to make a consensus
tree of the results of bootstrapping or jackknifing, to avoid overrepresenting
replicates that find many tied trees.
If the U (User Tree) option is used and more than one tree is supplied, the
program also performs a statistical test of each of these trees against the
best tree. This test is a version of the test proposed by
Alan Templeton (1983), evaluated in a test case by me (1985a). It is
closely parallel to a test using log likelihood differences
due to Kishino and Hasegawa (1989), and
uses the mean and variance of
step differences between trees, taken across sites. If the mean
is more than 1.96 standard deviations different then the trees are declared
significantly different. The program
prints out a table of the steps for each tree, the differences of
each from the best one, the variance of that quantity as determined by
the step differences at individual sites, and a conclusion as to
whether that tree is or is not significantly worse than the best one.
It is important to understand that the test assumes that all the discrete
characters are evolving independently, which is unlikely to be true for
many suites of morphological characters.
If there are more than two trees, the test done is an extension of
the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out
that a correction for the number of trees was necessary, and they
introduced a resampling method to make this correction. In the version
used here the variances and covariances of the sums of steps across
characters are computed for all pairs of trees. To test whether the
difference between each tree and the best one is larger than could have
been expected if they all had the same expected number of steps,
numbers of steps for all trees are sampled with these covariances and equal
means (Shimodaira and Hasegawa's "least favorable hypothesis"),
and a P value is computed from the fraction of times the difference between
the tree's value and the lowest number of steps exceeds that actually
observed. Note that this sampling needs random numbers, and so the
program will prompt the user for a random number seed if one has not
already been supplied. With the two-tree KHT test no random numbers
are used.
In either the KHT or the SH test the program
prints out a table of the number of steps for each tree, the differences of
each from the lowest one, the variance of that quantity as determined by
the differences of the numbers of steps at individual characters,
and a conclusion as to
whether that tree is or is not significantly worse than the best one.
Option 6 in the menu controls whether the tree estimated by the program
is written onto a tree file. The default name of this output tree file
is "outtree". If the U option is in effect, all the user-defined
trees are written to the output tree file.
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program implements the maximum likelihood method for protein
amino acid sequences.
It uses the either the Jones-Taylor-Thornton or the Dayhoff
probability model of change between amino acids.
The assumptions of these present models are:
Note the assumption that we are looking at all positions, including those
that have not changed at all. It is important not to restrict attention
to some positions based on whether or not they have changed; doing that
would bias branch lengths by making them too long, and that in turn
would cause the method to misinterpret the meaning of those positions that
had changed.
This program uses a Hidden Markov Model (HMM)
method of inferring different rates of evolution at different amino acid
positions. This
was described in a paper by me and Gary Churchill (1996). It allows us to
specify to the program that there will be
a number of different possible evolutionary rates, what the prior
probabilities of occurrence of each is, and what the average length of a
patch of positions all having the same rate is. The rates can also be chosen
by the program to approximate a Gamma distribution of rates, or a
Gamma distribution plus a class of invariant positions. The program computes the
the likelihood by summing it over all possible assignments of rates to positions,
weighting each by its prior probability of occurrence.
For example, if we have used the C and A options (described below) to specify
that there are three possible rates of evolution, 1.0, 2.4, and 0.0,
that the prior probabilities of a position having these rates are 0.4, 0.3, and
0.3, and that the average patch length (number of consecutive positions
with the same rate) is 2.0, the program will sum the likelihood over
all possibilities, but giving less weight to those that (say) assign all
positions to rate 2.4, or that fail to have consecutive positions that have the
same rate.
The Hidden Markov Model framework for rate variation among positions
was independently developed by Yang (1993, 1994, 1995). We have
implemented a general scheme for a Hidden Markov Model of
rates; we allow the rates and their prior probabilities to be specified
arbitrarily by the user, or by a discrete approximation to a Gamma
distribution of rates (Yang, 1995), or by a mixture of a Gamma
distribution and a class of invariant positions.
This feature effectively removes the artificial assumption that all positions
have the same rate, and also means that we need not know in advance the
identities of the positions that have a particular rate of evolution.
Another layer of rate variation also is available. The user can assign
categories of rates to each positions (for example, we might want
amino acid positions in the active site of a protein to change more slowly
than other positions. This is done with the categories input file and the
C option. We then specify (using the menu) the relative rates of evolution of
amino acid positions
in the different categories. For example, we might specify that positions
in the active site
evolve at relative rates of 0.2 compared to 1.0 at other positions. If we
are assuming that a particular position maintains a cysteine bridge to another,
we may want to put it in a category of positions (including perhaps the
initial position of the protein sequence which maintains methionine) which
changes at a rate of 0.0.
If both user-assigned rate categories and Hidden Markov Model rates
are allowed, the program assumes that the
actual rate at a position is the product of the user-assigned category rate
and the Hidden Markov Model regional rate. (This may not always make
perfect biological sense: it would be more natural to assume some upper
bound to the rate, as we have discussed in the Felsenstein and Churchill
paper). Nevertheless you may want to use both types of rate variation.
Subject to these assumptions, the program is a
correct maximum likelihood method. The
input is fairly standard, with one addition. As usual the first line of the
file gives the number of species and the number of amino acid positions.
Next come the species data. Each
sequence starts on a new line, has a ten-character species name
that must be blank-filled to be of that length, followed immediately
by the species data in the one-letter amino acid code. The sequences must
either be in the "interleaved" or "sequential" formats
described in the Molecular Sequence Programs document. The I option
selects between them. The sequences can have internal
blanks in the sequence but there must be no extra blanks at the end of the
terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
The user either types "Y" (followed, of course, by a carriage-return)
if the settings shown are to be accepted, or the letter or digit corresponding
to an option that is to be changed.
The options U, W, J, O, M, and 0 are the usual ones. They are described in the
main documentation file of this package. Option I is the same as in
other molecular sequence programs and is described in the documentation file
for the sequence programs.
The P option toggles between three models of amino acid change. One
is the Jones-Taylor-Thornton model, another the PMB (Probability
Matrix from Blocks) model of Veerassamy, Smith and Tillier (2003),
another the DCMut model (Kosiol and Goldman, 2005) based on the
the Dayhoff PAM matrix
model. These are all based on Margaret Dayhoff's (Dayhoff and Eck, 1968;
Dayhoff et. al., 1979) method of empirical tabulation of changes of
amino acid sequences, and conversion of these to a probability
model of amino acid change which is used to make a transition probability
matrix which allows prediction of the probability of changing from any
one amino acid to any other, and also predicts equilibrium amino acid
composition.
The default method is that of Jones,
Taylor, and Thornton (1992). This is similar to the Dayhoff
PAM model, except that it is based on a recounting of the number of
observed changes in amino acids, using a much larger sample of protein
sequences than did Dayhoff. Because its sample is so much larger this
model is to be preferred over the original Dayhoff PAM model.
The PMB model was recently derived from the Blocks database of
conserved protein motifs, and is described in a paper by
Veerassamy, Smith and Tillier (2003).
The Dayhoff model uses the DCMut version (Kosiol and Goldman, 2005)
of Margaret Dayhoff's PAM matrix.
The R (Hidden Markov Model rates) option allows the user to
approximate a Gamma distribution of rates among positions, or a
Gamma distribution plus a class of invariant positions, or to specify how
many categories of
substitution rates there will be in a Hidden Markov Model of rate
variation, and what are the rates and probabilities
for each. By repeatedly selecting the R option one toggles among
no rate variation, the Gamma, Gamma+I, and general HMM possibilities.
If you choose Gamma or Gamma+I the program will ask how many rate
categories you want. If you have chosen Gamma+I, keep in mind that
one rate category will be set aside for the invariant class and only
the remaining ones used to approximate the Gamma distribution.
For the approximation we do not use the quantile method of Yang (1995)
but instead use a quadrature method using generalized Laguerre
polynomials. This should give a good approximation to the Gamma
distribution with as few as 5 or 6 categories.
In the Gamma and Gamma+I cases, the user will be
asked to supply the coefficient of variation of the rate of substitution
among positions. This is different from the parameters used by Nei and Jin
(1990) but
related to them: their parameter a is also known as "alpha",
the shape parameter of the Gamma distribution. It is
related to the coefficient of variation by
CV = 1 / a1/2
or
a = 1 / (CV)2
(their parameter b is absorbed here by the requirement that time is scaled so
that the mean rate of evolution is 1 per unit time, which means that a = b).
As we consider cases in which the rates are less variable we should set a
larger and larger, as CV gets smaller and smaller.
If the user instead chooses the general Hidden Markov Model option,
they are first asked how many HMM rate categories there
will be (for the moment there is an upper limit of 9,
which should not be restrictive). Then
the program asks for the rates for each category. These rates are
only meaningful relative to each other, so that rates 1.0, 2.0, and 2.4
have the exact same effect as rates 2.0, 4.0, and 4.8. Note that an
HMM rate category
can have rate of change 0, so that this allows us to take into account that
there may be a category of amino acid positions that are invariant. Note that
the run time
of the program will be proportional to the number of HMM rate categories:
twice as
many categories means twice as long a run. Finally the program will ask for
the probabilities of a random amino acid position falling into each of these
regional rate categories. These probabilities must be nonnegative and sum to
1. Default
for the program is one category, with rate 1.0 and probability 1.0 (actually
the rate does not matter in that case).
If more than one HMM rate category is specified, then another
option, A, becomes
visible in the menu. This allows us to specify that we want to assume that
positions that have the same HMM rate category are expected to be clustered
so that there is autocorrelation of rates. The
program asks for the value of the average patch length. This is an expected
length of patches that have the same rate. If it is 1, the rates of
successive positions will be independent. If it is, say, 10.25, then the
chance of change to a new rate will be 1/10.25 after every position. However
the "new rate" is randomly drawn from the mix of rates, and hence could
even be the same. So the actual observed length of patches with the same
rate will be a bit larger than 10.25. Note below that if you choose
multiple patches, there will be an estimate in the output file as to
which combination of rate categories contributed most to the likelihood.
Note that the autocorrelation scheme we use is somewhat different
from Yang's (1995) autocorrelated Gamma distribution. I am unsure
whether this difference is of any importance -- our scheme is chosen
for the ease with which it can be implemented.
The C option allows user-defined rate categories. The user is prompted
for the number of user-defined rates, and for the rates themselves,
which cannot be negative but can be zero. These numbers, which must be
nonnegative (some could be 0),
are defined relative to each other, so that if rates for three categories
are set to 1 : 3 : 2.5 this would have the same meaning as setting them
to 2 : 6 : 5.
The assignment of rates to amino acid positions
is then made by reading a file whose default name is "categories".
It should contain a string of digits 1 through 9. A new line or a blank
can occur after any character in this string. Thus the categories file
might look like this:
With the current options R, A, and C the program has a good
ability to infer different rates at different positions and estimate
phylogenies under a more realistic model. Note that Likelihood Ratio
Tests can be used to test whether one combination of rates is
significantly better than another, provided one rate scheme represents
a restriction of another with fewer parameters. The number of parameters
needed for rate variation is the number of regional rate categories, plus
the number of user-defined rate categories less 2, plus one if the
regional rate categories have a nonzero autocorrelation.
The G (global search) option causes, after the last species is added to
the tree, each possible group to be removed and re-added. This improves the
result, since the position of every species is reconsidered. It
approximately triples the run-time of the program.
The User tree (option U) is read from a file whose default name is
intree. The trees can be multifurcating. They must be
preceded in the file by a line giving the number of trees in the file.
If the U (user tree) option is chosen another option appears in
the menu, the L option. If it is selected,
it signals the program that it
should take any branch lengths that are in the user tree and
simply evaluate the likelihood of that tree, without further altering
those branch lengths. This means that if some branches have lengths
and others do not, the program will estimate the lengths of those that
do not have lengths given in the user tree. Note that the program Retree
can be used to add and remove lengths from a tree.
The U option can read a multifurcating tree. This allows us to
test the hypothesis that a certain branch has zero length (we can also
do this by using Retree to set the length of that branch to 0.0 when
it is present in the tree). By
doing a series of runs with different specified lengths for a branch we
can plot a likelihood curve for its branch length while allowing all
other branches to adjust their lengths to it. If all branches have
lengths specified, none of them will be iterated. This is useful to allow
a tree produced by another method to have its likelihood
evaluated. The L option has no effect and does not appear in the
menu if the U option is not used.
The W (Weights) option is invoked in the usual way, with only weights 0
and 1 allowed. It selects a set of positions to be analyzed, ignoring the
others. The positions selected are those with weight 1. If the W option is
not invoked, all positions are analyzed.
The Weights (W) option
takes the weights from a file whose default name is "weights". The weights
follow the format described in the main documentation file.
The M (multiple data sets) option will ask you whether you want to
use multiple sets of weights (from the weights file) or multiple data sets
from the input file.
The ability to use a single data set with multiple weights means that
much less disk space will be used for this input data. The bootstrapping
and jackknifing tool Seqboot has the ability to create a weights file with
multiple weights. Note also that when we use multiple weights for
bootstrapping we can also then maintain different rate categories for
different sites in a meaningful way. If you use the multiple
data sets option rather than multiple weights, you should not at the
same time use the user-defined rate categories option (option C), because
the user-defined rate categories could then be associated with the
wrong sites. This is not a concern when the M option is used
by using multiple weights.
The algorithm used for searching among trees uses
a technique invented by David Swofford
and J. S. Rogers. This involves not iterating most branch lengths on most
trees when searching among tree topologies, This is of necessity a
"quick-and-dirty" search but it saves much time. There is a menu option
(option S) which can turn off this search and revert to the earlier
search method which iterated branch lengths in all topologies. This will
be substantially slower but will also be a bit more likely to find the
tree topology of highest likelihood. If the Swofford/Rogers search
finds the best tree topology, the branch lengths inferred will
be almost precisely the same as they would be with the more thorough
search, as the maximization of likelihood with respect to branch
lengths for the final tree is not different in the two kinds of search.
The output starts by giving the number of species and the number of amino acid
positions.
If the R (HMM rates) option is used a table of the relative rates of
expected substitution at each category of positions is printed, as well
as the probabilities of each of those rates.
There then follow the data sequences, if the user has selected the menu
option to print them, with the sequences printed in
groups of ten amino acids. The
trees found are printed as an unrooted
tree topology (possibly rooted by outgroup if so requested). The
internal nodes are numbered arbitrarily for the sake of
identification. The number of trees evaluated so far and the log
likelihood of the tree are also given. Note that the trees printed out
have a trifurcation at the base. The branch lengths in the diagram are
roughly proportional to the estimated branch lengths, except that very short
branches are printed out at least three characters in length so that the
connections can be seen. The unit of branch length is the expected
fraction of amino acids changed (so that 1.0 is 100 PAMs).
A table is printed
showing the length of each tree segment (in units of expected amino acid
substitutions per position), as well as (very) rough confidence
limits on their lengths. If a confidence limit is
negative, this indicates that rearrangement of the tree in that region
is not excluded, while if both limits are positive, rearrangement is
still not necessarily excluded because the variance calculation on which
the confidence limits are based results in an underestimate, which makes
the confidence limits too narrow.
In addition to the confidence limits,
the program performs a crude Likelihood Ratio Test (LRT) for each
branch of the tree. The program computes the ratio of likelihoods with and
without this branch length forced to zero length. This done by comparing the
likelihoods changing only that branch length. A truly correct LRT would
force that branch length to zero and also allow the other branch lengths to
adjust to that. The result would be a likelihood ratio closer to 1. Therefore
the present LRT will err on the side of being too significant. YOU ARE
WARNED AGAINST TAKING IT TOO SERIOUSLY. If you want to get a better
likelihood curve for a branch length you can do multiple runs with
different prespecified lengths for that branch, as discussed above in the
discussion of the L option.
One should also
realize that if you are looking not at a previously-chosen branch but at all
branches, that you are seeing the results of multiple tests. With 20 tests,
one is expected to reach significance at the P = .05 level purely by
chance. You should therefore use a much more conservative significance level,
such as .05 divided by the number of tests. The significance of these tests
is shown by printing asterisks next to
the confidence interval on each branch length. It is important to keep
in mind that both the confidence limits and the tests
are very rough and approximate, and probably indicate more significance than
they should. Nevertheless, maximum likelihood is one of the few methods that
can give you any indication of its own error; most other methods simply fail to
warn the user that there is any error! (In fact, whole philosophical schools
of taxonomists exist whose main point seems to be that there isn't any
error, that the "most parsimonious" tree is the best tree by definition and
that's that).
The log likelihood printed out with the final tree can be used to perform
various likelihood ratio tests. One can, for example, compare runs with
different values of the relative rate of change in the active site and in
the rest of the protein to determine
which value is the maximum likelihood estimate, and what is the allowable range
of values (using a likelihood ratio test, which you will find described in
mathematical statistics books). One could also estimate the base frequencies
in the same way. Both of these, particularly the latter, require multiple runs
of the program to evaluate different possible values, and this might get
expensive.
If the U (User Tree) option is used and more than one tree is supplied,
and the program is not told to assume autocorrelation between the
rates at different amino acid positions, the
program also performs a statistical test of each of these trees against the
one with highest likelihood. If there are two user trees, the test
done is one which is due to Kishino and Hasegawa (1989), a version
of a test originally introduced by Templeton (1983). In this
implementation it uses the mean and variance of
log-likelihood differences between trees, taken across amino acid
positions. If the two
trees' means are more than 1.96 standard deviations different
then the trees are
declared significantly different. This use of the empirical variance of
log-likelihood differences is more robust and nonparametric than the
classical likelihood ratio test, and may to some extent compensate for the
any lack of realism in the model underlying this program.
If there are more than two trees, the test done is an extension of
the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out
that a correction for the number of trees was necessary, and they
introduced a resampling method to make this correction. In the version
used here the variances and covariances of the sum of log likelihoods across
amino acid positions are computed for all pairs of trees. To test whether the
difference between each tree and the best one is larger than could have
been expected if they all had the same expected log-likelihood,
log-likelihoods for all trees are sampled with these covariances and equal
means (Shimodaira and Hasegawa's "least favorable hypothesis"),
and a P value is computed from the fraction of times the difference between
the tree's value and the highest log-likelihood exceeds that actually
observed. Note that this sampling needs random numbers, and so the
program will prompt the user for a random number seed if one has not
already been supplied. With the two-tree KHT test no random numbers
are used.
In either the KHT or the SH test the program
prints out a table of the log-likelihoods of each tree, the differences of
each from the highest one, the variance of that quantity as determined by
the log-likelihood differences at individual sites, and a conclusion as to
whether that tree is or is not significantly worse than the best one. However
the test is not available if we assume that there
is autocorrelation of rates at neighboring positions (option A) and is not
done in those cases.
The branch lengths printed out are scaled in terms of 100 times the
expected numbers of
amino acid substitutions, scaled so that the average rate of
change, averaged over all the positions analyzed, is set to 100.0,
if there are multiple categories of positions. This means that whether or not
there are multiple categories of positions, the expected percentage of change
for very small branches is equal to the branch length. Of course,
when a branch is twice as
long this does not mean that there will be twice as much net change expected
along it, since some of the changes occur in the same position and overlie or
even reverse each
other.
underlying numbers of changes. That means that a branch of length 26
is 26 times as long as one which would show a 1% difference between
the amino acid sequences at the beginning and end of the branch, but we
would not expect the sequences at the beginning and end of the branch to be
26% different, as there would be some overlaying of changes.
Confidence limits on the branch lengths are
also given. Of course a
negative value of the branch length is meaningless, and a confidence
limit overlapping zero simply means that the branch length is not necessarily
significantly different from zero. Because of limitations of the numerical
algorithm, branch length estimates of zero will often print out as small
numbers such as 0.00001. If you see a branch length that small, it is really
estimated to be of zero length.
Another possible source of confusion is the existence of negative values for
the log likelihood. This is not really a problem; the log likelihood is not a
probability but the logarithm of a probability. When it is
negative it simply means that the corresponding probability is less
than one (since we are seeing its logarithm). The log likelihood is
maximized by being made more positive: -30.23 is worse than -29.14.
At the end of the output, if the R option is in effect with multiple
HMM rates, the program will print a list of what amino acid position
categories contributed the most to the final likelihood. This combination of
HMM rate categories need not have contributed a majority of the likelihood,
just a plurality. Still, it will be helpful as a view of where the
program infers that the higher and lower rates are. Note that the
use in this calculations of the prior probabilities of different rates,
and the average patch length, gives this inference a "smoothed"
appearance: some other combination of rates might make a greater
contribution to the likelihood, but be discounted because it conflicts
with this prior information. See the example output below to see
what this printout of rate categories looks like.
A second list will also be printed out, showing for each position which
rate accounted for the highest fraction of the likelihood. If the fraction
of the likelihood accounted for is less than 95%, a dot is printed instead.
Option 3 in the menu controls whether the tree is printed out into
the output file. This is on by default, and usually you will want to
leave it this way. However for runs with multiple data sets such as
bootstrapping runs, you will primarily be interested in the trees
which are written onto the output tree file, rather than the trees
printed on the output file. To keep the output file from becoming too
large, it may be wisest to use option 3 to prevent trees being
printed onto the output file.
Option 4 in the menu controls whether the tree estimated by the program
is written onto a tree file. The default name of this output tree file
is "outtree". If the U option is in effect, all the user-defined
trees are written to the output tree file.
Option 5 in the menu controls whether ancestral states are estimated
at each node in the tree. If it is in effect, a table of ancestral
sequences is printed out (including the sequences in the tip species which
are the input sequences).
The symbol printed out is for the amino acid which accounts for the
largest fraction of the likelihood at that position.
In that table, if a position has an amino acid which
accounts for more than 95% of the likelihood, its symbol printed in capital
letters (W rather than w). One limitation of the current
version of the program is that when there are multiple HMM rates
(option R) the reconstructed amino acids are based on only the single
assignment of rates to positions which accounts for the largest amount of the
likelihood. Thus the assessment of 95% of the likelihood, in tabulating
the ancestral states, refers to 95% of the likelihood that is accounted
for by that particular combination of rates.
The constants defined at the beginning of the program include "maxtrees",
the maximum number of user trees that can be processed. It is small (100)
at present to save some further memory but the cost of increasing it
is not very great. Other constants
include "maxcategories", the maximum number of position
categories, "namelength", the length of species names in
characters, and three others, "smoothings", "iterations", and "epsilon", that
help "tune" the algorithm and define the compromise between execution speed and
the quality of the branch lengths found by iteratively maximizing the
likelihood. Reducing iterations and smoothings, and increasing epsilon, will
result in faster execution but a worse result. These values
will not usually have to be changed.
The program spends most of its time doing real arithmetic.
The algorithm, with separate and independent computations
occurring for each pattern, lends itself readily to parallel processing.
This program is derived in version 3.6 by Lucas Mix from Dnaml,
with which it shares
many of its data structures and much of its strategy.
(Note that although these may look like DNA sequences, they are being
treated as protein sequences consisting entirely of alanine, cystine,
glycine, and threonine).
(It was run with HMM rates having gamma-distributed rates
approximated by 5 rate categories,
with coefficient of variation of rates 1.0, and with patch length
parameter = 1.5. Two user-defined rate categories were used, one for
the first 6 positions, the other for the last 7, with rates 1.0 : 2.0.
Weights were used, with sites 1 and 13 given weight 0, and all others
weight 1.)
© Copyright 2000-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program implements the maximum likelihood method for protein amino
acid sequences under the constraint that the trees estimated must be
consistent with a molecular clock. The molecular clock is the
assumption that the tips of the tree are all equidistant, in branch
length, from its root. This program is indirectly related to Proml.
It uses the Dayhoff probability model of change between amino acids.
Its algorithmic details are not yet published, but many of them are
similar to Dnamlk.
The assumptions of the model are:
Note the assumption that we are looking at all positions, including those
that have not changed at all. It is important not to restrict attention
to some positions based on whether or not they have changed; doing that
would bias branch lengths by making them too long, and that in turn
would cause the method to misinterpret the meaning of those positions that
had changed.
This program uses a Hidden Markov Model (HMM)
method of inferring different rates of evolution at different amino acid
positions. This
was described in a paper by me and Gary Churchill (1996). It allows us to
specify to the program that there will be
a number of different possible evolutionary rates, what the prior
probabilities of occurrence of each is, and what the average length of a
patch of positions all having the same rate. The rates can also be chosen
by the program to approximate a Gamma distribution of rates, or a
Gamma distribution plus a class of invariant positions. The program computes the
likelihood by summing it over all possible assignments of rates to positions,
weighting each by its prior probability of occurrence.
For example, if we have used the C and A options (described below) to specify
that there are three possible rates of evolution, 1.0, 2.4, and 0.0,
that the prior probabilities of a position having these rates are 0.4, 0.3, and
0.3, and that the average patch length (number of consecutive positions
with the same rate) is 2.0, the program will sum the likelihood over
all possibilities, but giving less weight to those that (say) assign all
positions to rate 2.4, or that fail to have consecutive positions that have the
same rate.
The Hidden Markov Model framework for rate variation among positions
was independently developed by Yang (1993, 1994, 1995). We have
implemented a general scheme for a Hidden Markov Model of
rates; we allow the rates and their prior probabilities to be specified
arbitrarily by the user, or by a discrete approximation to a Gamma
distribution of rates (Yang, 1995), or by a mixture of a Gamma
distribution and a class of invariant positions.
This feature effectively removes the artificial assumption that all positions
have the same rate, and also means that we need not know in advance the
identities of the positions that have a particular rate of evolution.
Another layer of rate variation also is available. The user can assign
categories of rates to each positions (for example, we might want
amino acid positions in the active site of a protein to change more slowly
than other positions. This is done with the categories input file and the
C option. We then specify (using the menu) the relative rates of evolution of
amino acid positions
in the different categories. For example, we might specify that positions
in the active site
evolve at relative rates of 0.2 compared to 1.0 at other positions. If we
are assuming that a particular position maintains a cysteine bridge to another,
we may want to put it in a category of positions (including perhaps the
initial position of the protein sequence which maintains methionine) which
changes at a rate of 0.0.
If both user-assigned rate categories and Hidden Markov Model rates
are allowed, the program assumes that the
actual rate at a position is the product of the user-assigned category rate
and the Hidden Markov Model regional rate. (This may not always make
perfect biological sense: it would be more natural to assume some upper
bound to the rate, as we have discussed in the Felsenstein and Churchill
paper). Nevertheless you may want to use both types of rate variation.
Subject to these assumptions, the program is a
correct maximum likelihood method. The
input is fairly standard, with one addition. As usual the first line of the
file gives the number of species and the number of amino acid positions.
Next come the species data. Each
sequence starts on a new line, has a ten-character species name
that must be blank-filled to be of that length, followed immediately
by the species data in the one-letter amino acid code. The sequences must
either be in the "interleaved" or "sequential" formats
described in the Molecular Sequence Programs document. The I option
selects between them. The sequences can have internal
blanks in the sequence but there must be no extra blanks at the end of the
terminated line. Note that a blank is not a valid symbol for a deletion.
The options are selected using an interactive menu. The menu looks like this:
The user either types "Y" (followed, of course, by a carriage-return)
if the settings shown are to be accepted, or the letter or digit corresponding
to an option that is to be changed.
The options U, W, J, O, M, and 0 are the usual ones. They are described in the
main documentation file of this package. Option I is the same as in
other molecular sequence programs and is described in the documentation file
for the sequence programs.
The P option toggles between three models of amino acid change. One
is the Jones-Taylor-Thornton model, another the PMB (Probability
Matrix from Blocks) model of Veerassamy, Smith and Tillier (2003),
another the DCMut model (Kosiol and Goldman, 2005) based on the
the Dayhoff PAM matrix
model. These are all based on Margaret Dayhoff's (Dayhoff and Eck, 1968;
Dayhoff et. al., 1979) method of empirical tabulation of changes of
amino acid sequences, and conversion of these to a probability
model of amino acid change which is used to make a transition probability
matrix which allows prediction of the probability of changing from any
one amino acid to any other, and also predicts equilibrium amino acid
composition.
The R (Hidden Markov Model rates) option allows the user to
approximate a Gamma distribution of rates among positions, or a
Gamma distribution plus a class of invariant positions, or to specify how
many categories of
substitution rates there will be in a Hidden Markov Model of rate
variation, and what are the rates and probabilities
for each. By repeatedly selecting the R option one toggles among
no rate variation, the Gamma, Gamma+I, and general HMM possibilities.
If you choose Gamma or Gamma+I the program will ask how many rate
categories you want. If you have chosen Gamma+I, keep in mind that
one rate category will be set aside for the invariant class and only
the remaining ones used to approximate the Gamma distribution.
For the approximation we do not use the quantile method of Yang (1995)
but instead use a quadrature method using generalized Laguerre
polynomials. This should give a good approximation to the Gamma
distribution with as few as 5 or 6 categories.
In the Gamma and Gamma+I cases, the user will be
asked to supply the coefficient of variation of the rate of substitution
among positions. This is different from the parameters used by Nei and Jin
(1990) but
related to them: their parameter a is also known as "alpha",
the shape parameter of the Gamma distribution. It is
related to the coefficient of variation by
CV = 1 / a1/2
or
a = 1 / (CV)2
(their parameter b is absorbed here by the requirement that time is scaled so
that the mean rate of evolution is 1 per unit time, which means that a = b).
As we consider cases in which the rates are less variable we should set a
larger and larger, as CV gets smaller and smaller.
If the user instead chooses the general Hidden Markov Model option,
they are first asked how many HMM rate categories there
will be (for the moment there is an upper limit of 9,
which should not be restrictive). Then
the program asks for the rates for each category. These rates are
only meaningful relative to each other, so that rates 1.0, 2.0, and 2.4
have the exact same effect as rates 2.0, 4.0, and 4.8. Note that an
HMM rate category
can have rate of change 0, so that this allows us to take into account that
there may be a category of amino acid positions that are invariant. Note that
the run time
of the program will be proportional to the number of HMM rate categories:
twice as
many categories means twice as long a run. Finally the program will ask for
the probabilities of a random amino acid position falling into each of these
regional rate categories. These probabilities must be nonnegative and sum to
1. Default
for the program is one category, with rate 1.0 and probability 1.0 (actually
the rate does not matter in that case).
If more than one HMM rate category is specified, then another
option, A, becomes
visible in the menu. This allows us to specify that we want to assume that
positions that have the same HMM rate category are expected to be clustered
so that there is autocorrelation of rates. The
program asks for the value of the average patch length. This is an expected
length of patches that have the same rate. If it is 1, the rates of
successive positions will be independent. If it is, say, 10.25, then the
chance of change to a new rate will be 1/10.25 after every position. However
the "new rate" is randomly drawn from the mix of rates, and hence could
even be the same. So the actual observed length of patches with the same
rate will be a bit larger than 10.25. Note below that if you choose
multiple patches, there will be an estimate in the output file as to
which combination of rate categories contributed most to the likelihood.
Note that the autocorrelation scheme we use is somewhat different
from Yang's (1995) autocorrelated Gamma distribution. I am unsure
whether this difference is of any importance -- our scheme is chosen
for the ease with which it can be implemented.
The C option allows user-defined rate categories. The user is prompted
for the number of user-defined rates, and for the rates themselves,
which cannot be negative but can be zero. These numbers, which must be
nonnegative (some could be 0),
are defined relative to each other, so that if rates for three categories
are set to 1 : 3 : 2.5 this would have the same meaning as setting them
to 2 : 6 : 5.
The assignment of rates to amino acid positions
is then made by reading a file whose default name is "categories".
It should contain a string of digits 1 through 9. A new line or a blank
can occur after any character in this string. Thus the categories file
might look like this:
With the current options R, A, and C the program has a good
ability to infer different rates at different positions and estimate
phylogenies under a more realistic model. Note that Likelihood Ratio
Tests can be used to test whether one combination of rates is
significantly better than another, provided one rate scheme represents
a restriction of another with fewer parameters. The number of parameters
needed for rate variation is the number of regional rate categories, plus
the number of user-defined rate categories less 2, plus one if the
regional rate categories have a nonzero autocorrelation.
The G (global search) option causes, after the last species is added to
the tree, each possible group to be removed and re-added. This improves the
result, since the position of every species is reconsidered. It
approximately triples the run-time of the program.
The User tree (option U) is read from a file whose default name is
intree. The trees can be multifurcating. This allows us to test the
hypothesis that a given branch has zero length.
If the U (user tree) option is chosen another option appears in
the menu, the L option. If it is selected,
it signals the program that it
should take any branch lengths that are in the user tree and
simply evaluate the likelihood of that tree, without further altering
those branch lengths. In the case of a clock, if some branches have lengths
and others do not, the program does not estimate the lengths of those that
do not have lengths given in the user tree. If any of the branches
do not have lengths, the program re-estimates the lengths of all of them.
This is done because estimating some and not others is hard in the
case of a clock.
The W (Weights) option is invoked in the usual way, with only weights 0
and 1 allowed. It selects a set of positions to be analyzed, ignoring the
others. The positions selected are those with weight 1. If the W option is
not invoked, all positions are analyzed.
The Weights (W) option
takes the weights from a file whose default name is "weights". The weights
follow the format described in the main documentation file.
The M (multiple data sets) option will ask you whether you want to
use multiple sets of weights (from the weights file) or multiple data sets
from the input file.
The ability to use a single data set with multiple weights means that
much less disk space will be used for this input data. The bootstrapping
and jackknifing tool Seqboot has the ability to create a weights file with
multiple weights. Note also that when we use multiple weights for
bootstrapping we can also then maintain different rate categories for
different sites in a meaningful way. If you use the multiple
data sets option rather than multiple weights, you should not at the
same time use the user-defined rate categories option (option C), because
the user-defined rate categories could then be associated with the
wrong sites. This is not a concern when the M option is used
by using multiple weights.
The algorithm used for searching among trees is faster than it was in
version 3.5, thanks to using a technique invented by David Swofford
and J. S. Rogers. This involves not iterating most branch lengths on most
trees when searching among tree topologies, This is of necessity a
"quick-and-dirty" search but it saves much time.
The output starts by giving the number of species, the number of amino
acid positions.
If the R (HMM rates) option is used a table of the relative rates of
expected substitution at each category of positions is printed, as well
as the probabilities of each of those rates.
There then follow the data sequences, if the user has selected the menu
option to print them out, with the base sequences printed in
groups of ten amino acids. The
trees found are printed as a rooted
tree topology. The
internal nodes are numbered arbitrarily for the sake of
identification. The number of trees evaluated so far and the log
likelihood of the tree are also given. The branch lengths in the diagram are
roughly proportional to the estimated branch lengths, except that very short
branches are printed out at least three characters in length so that the
connections can be seen. The unit of branch length is the expected
fraction of amino acids changed (so that 1.0 is 100 PAMs).
A table is printed
showing the length of each tree segment, and the time (in units of
expected amino acid substitutions per position) of each fork in the tree,
measured from the root of the tree. I have not attempted to include
code for approximate confidence limits on branch points, as I have done
for branch lengths in Proml, both because of the extreme crudeness of
that test, and because the variation of times for different forks would be
highly correlated.
The log likelihood printed out with the final tree can be used to perform
various likelihood ratio tests. One can, for example, compare runs with
different values of the relative rate of change in the active site and in
the rest of the protein to determine
which value is the maximum likelihood estimate, and what is the allowable range
of values (using a likelihood ratio test, which you will find described in
mathematical statistics books). One could also estimate the base frequencies
in the same way. Both of these, particularly the latter, require multiple runs
of the program to evaluate different possible values, and this might get
expensive.
This program makes possible a (reasonably) legitimate
statistical test of the molecular clock. To do such a test, run Proml
and Promlk on the same data. If the trees obtained are of the same
topology (when considered as unrooted), it is legitimate to compare
their likelihoods by the likelihood ratio test. In Proml the likelihood
has been computed by estimating 2n-3 branch lengths, if there are n tips
on the tree. In Promlk it has been computed by estimating n-1 branching
times (in effect, n-1 branch lengths). The difference in the number of
parameters is (2n-3)-(n-1) = n-2. To perform the test take the
difference in log likelihoods between the two runs (Proml should be the
higher of the two, barring numerical iteration difficulties) and double
it. Look this up on a chi-square distribution with n-2 degrees of
freedom. If the result is significant, the log likelihood has been
significantly increased by allowing all 2n-3 branch lengths to be
estimated instead of just n-1, and molecular clock may be rejected.
If the U (User Tree) option is used and more than one tree is supplied,
and the program is not told to assume autocorrelation between the
rates at different amino acid positions, the
program also performs a statistical test of each of these trees against the
one with highest likelihood. If there are two user trees, the test
done is one which is due to Kishino and Hasegawa (1989), a version
of a test originally introduced by Templeton (1983). In this
implementation it uses the mean and variance of
log-likelihood differences between trees, taken across amino acid
positions. If the two
trees' means are more than 1.96 standard deviations different
then the trees are
declared significantly different. This use of the empirical variance of
log-likelihood differences is more robust and nonparametric than the
classical likelihood ratio test, and may to some extent compensate for the
any lack of realism in the model underlying this program.
If there are more than two trees, the test done is an extension of
the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out
that a correction for the number of trees was necessary, and they
introduced a resampling method to make this correction. In the version
used here the variances and covariances of the sum of log likelihoods across
amino acid positions are computed for all pairs of trees. To test whether the
difference between each tree and the best one is larger than could have
been expected if they all had the same expected log-likelihood,
log-likelihoods for all trees are sampled with these covariances and equal
means (Shimodaira and Hasegawa's "least favorable hypothesis"),
and a P value is computed from the fraction of times the difference between
the tree's value and the highest log-likelihood exceeds that actually
observed. Note that this sampling needs random numbers, and so the
program will prompt the user for a random number seed if one has not
already been supplied. With the two-tree KHT test no random numbers
are used.
In either the KHT or the SH test the program
prints out a table of the log-likelihoods of each tree, the differences of
each from the highest one, the variance of that quantity as determined by
the log-likelihood differences at individual sites, and a conclusion as to
whether that tree is or is not significantly worse than the best one. However
the test is not available if we assume that there
is autocorrelation of rates at neighboring positions (option A) and is not
done in those cases.
The branch lengths printed out are scaled in terms of 100 times the
expected numbers of
amino acid substitutions, scaled so that the average rate of
change, averaged over all the positions analyzed, is set to 100.0,
if there are multiple categories of positions. This means that whether or not
there are multiple categories of positions, the expected percentage of change
for very small branches is equal to the branch length. Of course,
when a branch is twice as
long this does not mean that there will be twice as much net change expected
along it, since some of the changes occur in the same position and overlie or
even reverse each
other.
underlying numbers of changes. That means that a branch of length 26
is 26 times as long as one which would show a 1% difference between
the amino acid sequences at the beginning and end of the branch, but we
would not expect the sequences at the beginning and end of the branch to be
26% different, as there would be some overlaying of changes.
Because of limitations of the numerical
algorithm, branch length estimates of zero will often print out as small
numbers such as 0.00001. If you see a branch length that small, it is really
estimated to be of zero length.
Another possible source of confusion is the existence of negative values for
the log likelihood. This is not really a problem; the log likelihood is not a
probability but the logarithm of a probability. When it is
negative it simply means that the corresponding probability is less
than one (since we are seeing its logarithm). The log likelihood is
maximized by being made more positive: -30.23 is worse than -29.14.
At the end of the output, if the R option is in effect with multiple
HMM rates, the program will print a list of what amino acid position
categories contributed the most to the final likelihood. This combination of
HMM rate categories need not have contributed a majority of the likelihood,
just a plurality. Still, it will be helpful as a view of where the
program infers that the higher and lower rates are. Note that the
use in this calculations of the prior probabilities of different rates,
and the average patch length, gives this inference a "smoothed"
appearance: some other combination of rates might make a greater
contribution to the likelihood, but be discounted because it conflicts
with this prior information. See the example output below to see
what this printout of rate categories looks like.
A second list will also be printed out, showing for each position which
rate accounted for the highest fraction of the likelihood. If the fraction
of the likelihood accounted for is less than 95%, a dot is printed instead.
Option 3 in the menu controls whether the tree is printed out into
the output file. This is on by default, and usually you will want to
leave it this way. However for runs with multiple data sets such as
bootstrapping runs, you will primarily be interested in the trees
which are written onto the output tree file, rather than the trees
printed on the output file. To keep the output file from becoming too
large, it may be wisest to use option 3 to prevent trees being
printed onto the output file.
Option 4 in the menu controls whether the tree estimated by the program
is written onto a tree file. The default name of this output tree file
is "outtree". If the U option is in effect, all the user-defined
trees are written to the output tree file.
Option 5 in the menu controls whether ancestral states are estimated
at each node in the tree. If it is in effect, a table of ancestral
sequences is printed out (including the sequences in the tip species which
are the input sequences).
The symbol printed out is for the amino acid which accounts for the
largest fraction of the likelihood at that position.
In that table, if a position has an amino acid which
accounts for more than 95% of the likelihood, its symbol printed in capital
letters (W rather than w). One limitation of the current
version of the program is that when there are multiple HMM rates
(option R) the reconstructed amino acids are based on only the single
assignment of rates to positions which accounts for the largest amount of the
likelihood. Thus the assessment of 95% of the likelihood, in tabulating
the ancestral states, refers to 95% of the likelihood that is accounted
for by that particular combination of rates.
The constants defined at the beginning of the program include "maxtrees",
the maximum number of user trees that can be processed. It is small (100)
at present to save some further memory but the cost of increasing it
is not very great. Other constants
include "maxcategories", the maximum number of position
categories, "namelength", the length of species names in
characters, and three others, "smoothings", "iterations", and "epsilon", that
help "tune" the algorithm and define the compromise between execution speed and
the quality of the branch lengths found by iteratively maximizing the
likelihood. Reducing iterations and smoothings, and increasing epsilon, will
result in faster execution but a worse result. These values
will not usually have to be changed.
The program spends most of its time doing real arithmetic.
The algorithm, with separate and independent computations
occurring for each pattern, lends itself readily to parallel processing.
This program was developed in version 3.6 by Lucas Mix by combining code
from Dnamlk and from Proml.
(It was run with HMM rates having gamma-distributed rates
approximated by 5 rate categories,
with coefficient of variation of rates 1.0, and with patch length
parameter = 1.5. Two user-defined rate categories were used, one for
the first 6 positions, the other for the last 7, with rates 1.0 : 2.0.
Weights were used, with sites 1 and 13 given weight 0, and all others
weight 1.)
© Copyright 1983, 2000-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program uses protein sequences to compute a distance matrix, under
four different models of amino acid replacement. It can also
compute a table of similarity between the amino acid sequences.
The distance for each
pair of species estimates the total branch length between the two species, and
can be used in the distance matrix programs Fitch, Kitsch or Neighbor. This
is an alternative to using the sequence data itself in the
parsimony program Protpars.
The program reads in protein sequences and writes an output file containing
the distance matrix or similarity table. The five models of amino acid
substitution are
one which is based on the Jones, Taylor and Thornton (1992) model of
amino acid change, the PMB model (Veerassamy, Smith and Tillier, 2003)
which is derived from the Blocks database of conserved protein motifs,
the DCMut model (Kosiol and Goldman, 2005) based on the PAM matrices of
Margaret Dayhoff,
one due to
Kimura (1983, p.75) which approximates it based simply on the fraction of
similar amino acids, and one based on a model in which the amino acids are
divided up into groups, with change occurring based on the genetic code
but with greater difficulty of changing between groups. The program correctly
takes into account a variety of sequence ambiguities.
The five methods are:
(1) The Dayhoff PAM matrix. This uses the DCMut model (Kosiol and Goldman,
2005) which is a version of the PAM model of Margaret Dayhoff.
The PAM model is an empirical one that
scales probabilities of change from one amino acid to another in
terms of a unit which is an expected 1% change between two amino acid
sequences. The PAM 001 matrix is used to make a transition probability
matrix which allows prediction of the probability of changing from any
one amino acid to any other, and also predicts equilibrium amino acid
composition. The program assumes that these probabilities are correct
and bases its computations of distance on them. The distance that is
computed is scaled in units of expected fraction of amino acids
changed. This is a unit such that 1.0 is 100 PAM's.
(2) The Jones-Taylor-Thornton model. This is similar to the Dayhoff
PAM model, except that it is based on a recounting of the number of
observed changes in amino acids by Jones, Taylor, and Thornton (1992).
They used a much larger sample of protein sequences than did Dayhoff.
The distance is scaled in units of the expected fraction of amino acids
changed (100 PAM's). Because its sample is so much larger this
model is to be preferred over the original Dayhoff PAM model. It
is the default model in this program.
(3) The PMB (Probability Matrix from Blocks) model. This is derived using the
Blocks database of conserved protein motifs. It is described in
a paper by Veerassamy, Smith and Tillier (2003). Elisabeth Tillier
kindly made the matrices available for this model.
(4) Kimura's distance. This is a rough-and-ready distance formula for
approximating PAM distance by simply measuring the fraction of amino
acids, p, that differs between two sequences and computing the
distance as (Kimura, 1983)
D = - loge ( 1 - p - 0.2 p2 ).
This is very quick to do but has some obvious limitations. It does not
take into account which amino acids differ or to what amino acids
they change, so some information is lost. The units of the distance
measure are the fraction of amino acids differing, as also in the case of
the PAM distance. If the fraction of amino acids differing gets larger
than about 0.8541 the distance becomes infinite. Note that this can happen
with bootstrapped sequences even when the original sequences are below
this level of difference.
(5) The Categories distance. This is my own concoction. I imagined
a nucleotide sequence changing according to Kimura's 2-parameter model,
with the exception that some changes of amino acids are less likely than
others. The amino acids are grouped into a series of categories. Any
base change that does not change which category the amino acid is in is
allowed, but if an amino acid changes category this is allowed only a
certain fraction of the time. The fraction is called the "ease" and
there is a parameter for it, which is 1.0 when all changes are allowed
and near 0.0 when changes between categories are nearly impossible.
In this option I have allowed the user to select the Transition/Transversion
ratio, which of several genetic codes to use, and which categorization
of amino acids to use. There are three of them, a somewhat random sample:
Interestingly enough, all of them are consistent with the same linear
ordering of amino acids, which they divide up in different ways. For the
Categories model I have set as default the George/Hunt/Barker classification
with the "ease" parameter set to 0.457 which is approximately the value
implied by the empirical rates in the Dayhoff PAM matrix.
The method uses, as I have noted, Kimura's (1980) 2-parameter model of DNA
change. The Kimura "2-parameter" model allows
for a difference between transition and transversion rates. Its transition
probability matrix for a short interval of time is:
where a is u dt, the product of the rate of transitions per unit time and dt
is the length dt of the time interval, and b is v dt, the product of half the
rate of transversions (i.e., the rate of a specific transversion)
and the length dt of the time interval.
Each distance that is calculated is an estimate, from that particular pair of
species, of the divergence time between those two species. The Kimura
distance is straightforward to compute. The other two are considerably
slower, and they look at all positions, and find that distance which
makes the likelihood highest. This likelihood is in effect the length of
the internal branch in a two-species tree that connects these two
species. Its likelihood is just the product, under the model, of the
probabilities of each position having the (one or) two amino acids that
are actually found. This is fairly slow to compute.
The computation proceeds from an eigenanalysis (spectral decomposition)
of the transition probability matrix. In the case of the PAM 001 matrix
the eigenvalues and eigenvectors are precomputed and are hard-coded
into the program in over 400 statements. In the case of the Categories
model the program computes the eigenvalues and eigenvectors itself, which
will add a delay. But the delay is independent of the number of species
as the calculation is done only once, at the outset.
The actual algorithm for estimating the distance is in both cases a
bisection algorithm which tries to find the point at which the derivative
of the likelihood is zero. Some of the kinds of ambiguous amino acids
like "glx" are correctly taken into account. However, gaps are treated
as if they are unkown nucleotides, which means those positions get dropped
from that particular comparison. However, they are not dropped from the
whole analysis. You need not eliminate regions containing gaps, as long
as you are reasonably sure of the alignment there.
Note that there is an
assumption that we are looking at all positions, including those
that have not changed at all. It is important not to restrict attention
to some positions based on whether or not they have changed; doing that
would bias the distances by making them too large, and that in turn
would cause the distances
to misinterpret the meaning of those positions that
had changed.
The program can now correct distances for unequal rates of change at different
amino acid positions. This correction, which was introduced for DNA
sequences by Jin and Nei (1990), assumes that the distribution of rates
of change among amino acid positions follows a Gamma distribution. The
user is asked for the value of a parameter that determines the amount of
variation of rates among amino acid positions. Instead of the more
widely-known coefficient alpha, Protdist uses the coefficient of variation
(ratio of the standard deviation to the mean) of rates among amino acid
positions. So if there is 20% variation in rates, the CV is
is 0.20. The square of the C.V. is also the reciprocal of the
better-known "shape parameter", alpha, of the Gamma
distribution, so in this case the shape parameter alpha = 1/(0.20*0.20)
= 25. If you want to achieve a particular value of alpha, such as 10,
you will want to use a CV of 1/sqrt(10) = 1/3.162 = 0.3162.
In addition to the five distance calculations, the program can also
compute a table of similarities between amino acid sequences. These values
are the fractions of amino acid positions identical between the sequences.
The diagonal values are 1.0000. No attempt is made to count similarity
of nonidentical amino acids, so that no credit is given for having
(for example) different hydrophobic amino acids at the corresponding
positions in the two sequences. This option has been requested by many
users, who need it for descriptive purposes. It is not intended that
the table be used for inferring the tree.
Input is fairly standard, with one addition. As usual the first line of the
file gives the number of species and the number of sites. There follows the
character W if the Weights option is being used.
Next come the species data. Each
sequence starts on a new line, has a ten-character species name
that must be blank-filled to be of that length, followed immediately
by the species data in the one-letter code. The sequences must either
be in the "interleaved" or "sequential" formats
described in the Molecular Sequence Programs document. The I option
selects between them. The sequences can have internal
blanks in the sequence but there must be no extra blanks at the end of the
terminated line. Note that a blank is not a valid symbol for a deletion.
After that are the lines (if any) containing the information for the
W option, as described below.
The options are selected using an interactive menu. The menu looks like this:
The user either types "Y" (followed, of course, by a carriage-return)
if the settings shown are to be accepted, or the letter or digit corresponding
to an option that is to be changed.
The P option selects one of the five distance methods, or the
similarity table. It toggles among these
six methods. The default method, if none is specified, is the
Jones-Taylor-Thornton model. If the Categories distance is selected
another menu option, T, will appear allowing the user
to supply the Transition/Transversion ratio that should be assumed
at the underlying DNA level, and another one, C, which allows the
user to select among various nuclear and mitochondrial genetic codes.
The transition/transversion ratio can be any number from 0.5 upwards.
The G option chooses Gamma distributed rates of evolution across amino
acid psoitions. The program will prompt you for the Coefficient of Variation
of rates. As is noted above, this is 1/sqrt(alpha) if alpha is the more
familiar "shape coefficient" of the Gamma distribution. If the G option
is not selected, the program defaults to having no variation of rates
among sites.
The C option allows user-defined rate categories. The user is prompted
for the number of user-defined rates, and for the rates themselves,
which cannot be negative but can be zero. These numbers, which must be
nonnegative (some could be 0),
are defined relative to each other, so that if rates for three categories
are set to 1 : 3 : 2.5 this would have the same meaning as setting them
to 2 : 6 : 5.
The assignment of rates to
sites is then made by reading a file whose default name is "categories".
It should contain a string of digits 1 through 9. A new line or a blank
can occur after any character in this string. Thus the categories file
might look like this:
If both user-assigned rate categories and Gamma-distributed rates
are allowed, the program assumes that the
actual rate at a site is the product of the user-assigned category rate
and the Gamma-distributed rate. This allows you to specify that
certain sites have higher or lower rates of change while also allowing the
program to allow variation of rates in addition to that.
The M (multiple data sets) option will ask you whether you want to
use multiple sets of weights (from the weights file) or multiple data sets
from the input file.
The ability to use a single data set with multiple weights means that
much less disk space will be used for this input data. The bootstrapping
and jackknifing tool Seqboot has the ability to create a weights file with
multiple weights. Note also that when we use multiple weights for
bootstrapping we can also then maintain different rate categories for
different sites in a meaningful way. If you use the multiple
data sets option rather than multiple weights, you should not at the
same time use the user-defined rate categories option (option C), because
the user-defined rate categories could then be associated with the
wrong sites. This is not a concern when the M option is used
by using multiple weights.
Option 0 is the usual one. It is described in the
main documentation file of this package. Option I is the same as in
other molecular sequence programs and is described in the documentation file
for the sequence programs.
The W (Weights) option is invoked in the usual way, with only weights 0
and 1 allowed. It selects a set of sites to be analyzed, ignoring the
others. The sites selected are those with weight 1. If the W option is
not invoked, all sites are analyzed.
As the
distances are computed, the program prints on your screen or terminal
the names of the species in turn,
followed by one dot (".") for each other species for which the distance to
that species has been computed. Thus if there are ten species, the first
species name is printed out, followed by one dot, then on the next line
the next species name is printed out followed by two dots, then the
next followed by three dots, and so on. The pattern of dots should form
a triangle. When the distance matrix has been written out to the output
file, the user is notified of that.
The output file contains on its first line the number of species. The
distance matrix is then printed in standard
form, with each species starting on a new line with the species name, followed
by the distances to the species in order. These continue onto a new line
after every nine distances. The distance matrix is square
with zero distances on the diagonal. In general the format of the distance
matrix is such that it can serve as input to any of the distance matrix
programs.
If the similarity table is selected, the table that is produced is not
in a format that can be used as input to the distance matrix programs.
It has a heading, and the species names are also put at the tops of the
columns of the table (or rather, the first 8 characters of each species
name is there, the other two characters omitted to save space). There
is not an option to put the table into a format that can be read by
the distance matrix programs, nor is there one to make it into a table
of fractions of difference by subtracting the similarity values from 1.
This is done deliberately to make it more difficult to
use these values to construct trees. The similarity values are
not corrected for multiple changes, and their use to construct trees
(even after converting them to fractions of difference) would be
wrong, as it would lead to severe conflict between the distant
pairs of sequences and the close pairs of sequences.
If the option to print out the data is selected, the output file will
precede the data by more complete information on the input and the menu
selections. The output file begins by giving the number of species and the
number of characters, and the identity of the distance measure that is
being used.
In the Categories model of substitution,
the distances printed out are scaled in terms of expected numbers of
substitutions, counting both transitions and transversions but not
replacements of a base by itself, and scaled so that the average rate of
change is set to 1.0. For the Dayhoff PAM and Kimura models the
distance are scaled in terms of the expected numbers of amino acid
substitutions per site. Of course, when a branch is twice as
long this does not mean that there will be twice as much net change expected
along it, since some of the changes may occur in the same site and overlie or
even reverse each
other. The branch lengths estimates here are in terms of the expected
underlying numbers of changes. That means that a branch of length 0.26
is 26 times as long as one which would show a 1% difference between
the protein (or nucleotide) sequences at the beginning and end of the
branch. But we
would not expect the sequences at the beginning and end of the branch to be
26% different, as there would be some overlaying of changes.
One problem that can arise is that two or more of the species can be so
dissimilar that the distance between them would have to be infinite, as
the likelihood rises indefinitely as the estimated divergence time
increases. For example, with the Kimura model, if the two sequences
differ in 85.41% or more of their positions then the estimate of divergence
time would be infinite. Since there is no way to represent an infinite
distance in the output file, the program regards this as an error, issues a
warning message indicating which pair of species are causing the problem, and
computes a distance of -1.0.
The constants that are available to be changed by the user at the beginning
of the program include
"namelength", the length of species names in
characters, and "epsilon", a parameter which controls the accuracy of the
results of the iterations which estimate the distances. Making "epsilon"
smaller will increase run times but result in more decimal places of
accuracy. This should not be necessary.
The program spends most of its time doing real arithmetic. Any software or
hardware changes that speed up that arithmetic will speed it up by a nearly
proportional amount.
(Note that although these may look like DNA sequences, they are being
treated as protein sequences consisting entirely of alanine, cystine,
glycine, and threonine).
(Note that when the numerical options are not on, the output file produced is
in the correct format to be used as an input file in the distance matrix
programs).
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program infers an unrooted phylogeny from protein sequences, using a
new method intermediate between the approaches of Eck and Dayhoff (1966) and
Fitch (1971). Eck and Dayhoff (1966) allowed any amino acid to change to
any other, and counted the number of such changes needed to evolve the
protein sequences on each given phylogeny. This has the problem that it
allows replacements which are not consistent with the genetic code, counting
them equally with replacements that are consistent. Fitch, on the other hand,
counted the minimum number of nucleotide substitutions that would be
needed to achieve the given protein sequences. This counts silent
changes equally with those that change the amino acid.
The present method insists that any changes of amino acid be consistent
with the genetic code so that, for example, lysine is allowed to change
to methionine but not to proline. However, changes between two amino acids
via a third are allowed and counted as two changes if each of the two
replacements is individually allowed. This sometimes allows changes that
at first sight you would think should be outlawed. Thus we can change from
phenylalanine to glutamine via leucine in two steps
total. Consulting the genetic code, you will find that there is a leucine
codon one step away from a phenylalanine codon, and a leucine codon one
step away from glutamine. But they are not the same leucine codon. It
actually takes three base substitutions to get from either of the
phenylalanine codons TTT and TTC to either of the glutamine codons
CAA or CAG. Why then does this program count only two? The answer
is that recent DNA sequence comparisons seem to show that synonymous
changes are considerably faster and easier than ones that change the
amino acid. We are assuming that, in effect, synonymous changes occur
so much more readily that they need not be counted. Thus, in the chain
of changes TTT (Phe) --> CTT (Leu) --> CTA (Leu) --> CAA (Glu), the middle
one is not counted because it does not change the amino acid (leucine).
To maintain consistency with the genetic code, it is necessary for the
program internally to treat serine as two separate states (ser1 and ser2)
since the two groups of serine codons are not adjacent in the
code. Changes to the state "deletion" are counted as three steps to prevent the
algorithm from assuming unnecessary deletions. The state "unknown" is
simply taken to mean that the amino acid, which has not been determined,
will in each part of a tree that is evaluated be assumed to be whichever one
causes the fewest steps.
The assumptions of this method (which has not been described in the
literature), are thus something like this:
That these are the assumptions of parsimony methods has been documented
in a series of papers of mine: (1973a, 1978b, 1979, 1981b, 1983b, 1988b). For
an opposing view arguing that the parsimony methods make no substantive
assumptions such as these, see the papers by Farris (1983) and Sober (1983a,
1983b, 1988), but also read the exchange between Felsenstein and Sober (1986).
The input for the program is fairly standard. The first line contains the
number of species and the number of amino acid positions (counting any
stop codons that you want to include).
Next come the species data. Each
sequence starts on a new line, has a ten-character species name
that must be blank-filled to be of that length, followed immediately
by the species data in the one-letter code. The sequences must either
be in the "interleaved" or "sequential" formats
described in the Molecular Sequence Programs document. The I option
selects between them. The sequences can have internal
blanks in the sequence but there must be no extra blanks at the end of the
terminated line. Note that a blank is not a valid symbol for a deletion.
The protein sequences are given by the one-letter codes
described in the Molecular Sequence Programs documentation file. Note that
if two polypeptide chains are being used that are of different lengths
owing to one terminating before the other, they should be coded as (say)
since after the stop codon we do not definitely know that
there has been a deletion, and do not know what amino acid would
have been there. If DNA studies tell us that there is
DNA sequence in that region, then we could use "X" rather than "?". Note
that "X" means an unknown amino acid, but definitely an amino acid,
while "?" could mean either that or a deletion. The distinction is often
significant in regions where there are deletions: one may want to encode
a six-base deletion as "-?????" since that way the program will only count
one deletion, not six deletion events, when the deletion arises. However,
if there are overlapping deletions it may not be so easy to know what
coding is correct.
One will usually want to
use "?" after a stop codon, if one does not know what amino acid is there. If
the DNA sequence has been observed there, one probably ought to resist
putting in the amino acids that this DNA would code for, and one should use
"X" instead, because under the assumptions implicit in this parsimony
method, changes to any noncoding sequence are much easier than
changes in a coding region that change the amino acid, so that they
shouldn't be counted anyway!
The form of this information
is the standard one described in the main documentation file. For the U option
the tree
provided must be a rooted bifurcating tree, with the root placed anywhere
you want, since that root placement does not affect anything.
The options are selected using an interactive menu. The menu looks like this:
The user either types "Y" (followed, of course, by a carriage-return)
if the settings shown are to be accepted, or the letter or digit corresponding
to an option that is to be changed.
The options U, J, O, T, W, M, and 0 are the usual ones. They are described in
the main documentation file of this package. Option I is the same as in
other molecular sequence programs and is described in the documentation file
for the sequence programs. Option C allows the user to select among various
nuclear and mitochondrial genetic codes. There is no provision for coping
with data where different genetic codes have been used in different
organisms.
In the U (User tree) option, the trees should
not be preceded by a line with the number of trees on it.
Output is standard: if option 1 is toggled on, the data is printed out,
with the convention that "." means "the same as in the first species".
Then comes a list of equally parsimonious trees, and (if option 2 is
toggled on) a table of the
number of changes of state required in each position. If option 5 is toggled
on, a table is printed
out after each tree, showing for each branch whether there are known to be
changes in the branch, and what the states are inferred to have been at the
top end of the branch. This is a reconstruction of the ancestral sequences
in the tree. If you choose option 5, a menu item "." appears which gives you
the opportunity to turn off dot-differencing so that complete ancestral
sequences are shown. If the inferred state is a "?" there will be multiple
equally-parsimonious assignments of states; the user must work these out for
themselves by hand. If option 6 is left in its default state the trees
found will be written to a tree file, so that they are available to be used
in other programs. If the program finds multiple
trees tied for best, all of these are written out onto the output tree
file. Each is followed by a numerical weight in square brackets (such as
[0.25000]). This is needed when we use the trees to make a consensus
tree of the results of bootstrapping or jackknifing, to avoid overrepresenting
replicates that find many tied trees.
If the U (User Tree) option is used and more than one tree is supplied, the
program also performs a statistical test of each of these trees against the
best tree. This test is a version of the test proposed by
Alan Templeton (1983), and evaluated in a test case by me (1985a). It is
closely parallel to a test using log likelihood differences
due to Kishino and Hasegawa (1989), and uses the mean
and variance of
step differences between trees, taken across positions. If the mean
is more than 1.96 standard deviations different then the trees are declared
significantly different. The program
prints out a table of the steps for each tree, the differences of
each from the best one, the variance of that quantity as determined by
the step differences at individual positions, and a conclusion as to
whether that tree is or is not significantly worse than the best one.
The program is derived from Mix but has had some rather elaborate
bookkeeping using sets of bits installed. It is not a very fast
program but is speeded up substantially over version 3.2.
© Copyright 2000-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Restdist reads the same restriction sites format as Restml and
computes a restriction sites distance. It can also compute a restriction
fragments distance. The original restriction fragments and restriction
sites distance methods were introduced by Nei and Li (1979). Their original
method for restriction fragments is
also available in this program, although its default methods
are my modifications of the original Nei and Li methods.
These two distances assume that the restriction sites are accidental byproducts
of random change of nucleotide sequences. For my restriction sites distance
the DNA sequences are assumed to be changing according to the Kimura
2-parameter model of DNA change (Kimura, 1980). The user can set the
transition/transversion rate for the model. For my restriction fragments
distance there is
an implicit assumption of a Jukes-Cantor (1969) model of change.
The user can also set the
parameter of a correction for unequal rates of evolution between sites in
the DNA sequences, using a Gamma distribution of rates among sites.
The Jukes-Cantor model is also implicit in the restriction fragments
distance of Nei and Li(1979). It
does not allow us to correct for a Gamma distribution of rates among
sites.
The restriction sites distances use data coded for the presence or absence of
individual restriction sites (usually as + and - or 0 and 1). My
distance is based on the proportion, out of all sites observed in one species
or the other, which are present in both species. This is done to correct for
the ascertainment of sites, for the fact that we are not aware of many sites
because they do not appear in any species.
My distance starts by computing from the particular pair of species the fraction
If each restriction site is s nucleotides long, the probability
that a restriction site is present in the other species, given that it is
present in a species, is
The distance is then computed by finding the branch length of a two-species
tree (connecting these two species with a single branch) such
that Q equals the s-th root of f. For this the
program computes Q for various values of branch length, iterating them
by a Newton-Raphson algorithm until the two quantities are equal.
The resulting distance should be numerically close to the original
restriction sites distance of Nei and Li (1979) when divergence is small.
Theirs computes the probability of retention of a site in a way that assumes
that the site is present in the common ancestor of the two species. Ours
does not make this assumption. It is inspired by theirs,
but differs in this detail. Their distance also assumes a Jukes-Cantor (1969)
model of base change, and does not allow for transitions being more frequent
than transversions. In this sense mine generalizes theirs somewhat.
Their distance does include, as mine does as well, a correction for
Gamma distribution of rate of change among nucleotide sites.
I have made their original distance available here (option N).
For restriction fragments data we use a different distance. If we
average over all restriction fragment lengths, each at its own
expected frequency, the probability that the fragment will still be
in existence after a certain amount of branch length, we must take into
account the probability that the two restriction sites at the ends of
the fragment do not mutate, and the probability that no new
restriction site occurs within the fragment in that amount of branch
length. The result for a restriction site length of s is:
Alternatively, if we use the Nei and Li (1979) restriction fragments
distancen (available in this program using
menu option N), this involves solving for g in the nonlinear
equation
Comparing these two restriction fragments distances in a case
where their underlying DNA model is the same (which is when the
transition/transversion ratio of the modified model is set to
0.5), you will find
that they are very close to each other, differing very little at
small distances, with the modified distance becoming smaller than
the Nei/Li distance at larger distances. It will therefore matter
very little which one you use.
Although these distances are designed for restriction sites
and restriction fragments data, they can be applied to RAPD and
AFLP data as well. RAPD (Randomly Amplified Polymorphic DNA)
and AFLP (Amplified Fragment Length Polymorphism) data consist
of presence or absence of individual bands on a gel. The bands
are segments of DNA with PCR primers at each end. These
primers are defined sequences of known length (often about
10 nucleotides each). For AFLPs the relevant length is the primer
length, plus three nucleotides. Mutation in these sequences makes them no
longer be primers, just as in the case of restriction sites.
Thus a pair of 10-nucleotide primers will behave much the same
as a 20-nucleotide restriction site, for RAPDs (26 for AFLPs). You can use
the restriction sites distance as the distance between RAPD or AFLP patterns
if you set the proper value for the total length of the site to the
total length of the primers (plus 6 in the case of AFLPs).
Of course there are many possible sources of noise in these data,
including confusing fragments of similar length for each other
and having primers near each other in the genome, and these are
not taken into account in the statistical model used here.
The input is fairly standard, with one addition. As usual the first line of
the file gives the number of species and the number of sites, but there is also
a third number, which is the number of different restriction enzymes that were
used to detect the restriction sites. Thus a data set with 10 species and
35 different sites, representing digestion with 4 different enzymes, would
have the first line of the data file look like this:
The site data are in standard form. Each species starts with a species name
whose maximum length is given by the constant "nmlngth"
(whose value in the
program as distributed is 10 characters). The name should, as usual, be padded
out to that length with blanks if necessary. The sites data then follows, one
character per site (any blanks will
be skipped and ignored). Like the DNA and protein sequence data, the
restriction sites data may be either in the "interleaved" form or the
"sequential" form. Note that if you are analyzing restriction sites
data with the programs Dollop or Mix or other discrete character
programs, at the moment those programs do not use the "aligned" or
"interleaved" data format. Therefore you may want to avoid that format
when you have restriction sites data that you will want to feed into
those programs.
The presence of a site is indicated by a "+" and the absence by a "-". I have
also allowed the use of "1" and "0" as synonyms for "+" and "-", for
compatibility with Mix and Dollop which do not allow "+" and "-". If the
presence of
the site is unknown (for example, if the DNA containing it has been deleted so
that one
does not know whether it would have contained the site) then the state "?" can
be used to indicate that the state of this site is unknown.
The options are selected using an interactive menu. The menu looks like this:
The user either types "Y" (followed, of course, by a carriage-return)
if the settings shown are to be accepted, or the letter or digit corresponding
to an option that is to be changed.
The R option toggles between a restriction sites distance, which
is the default setting, and a restriction fragments distance. In
both cases, another option appears, the N (Nei/Li) option.
This allows the user to choose the original Nei and Li (1979)
restriction sites or fragments distances rather than my modified versions,
which are the defaults.
If the G (Gamma distribution) option is selected, the user will be
asked to supply the coefficient of variation of the rate of substitution
among sites. This is different from the parameters used by Nei and Jin, who
introduced Gamma distribution of rates in DNA distances, but
related to their parameters: their parameter a is also known as
"alpha", the shape parameter of the Gamma distribution. It is
related to the coefficient of variation by
CV = 1 / a1/2
or
a = 1 / (CV)2
(their parameter b is absorbed here by the requirement that time is scaled so
that the mean rate of evolution is 1 per unit time, which means that a = b).
As we consider cases in which the rates are less variable we should set a
larger and larger, as CV gets smaller and smaller.
The Gamma distribution option is not available when using the
original Nei/Li restriction fragments distance.
The T option is the Transition/transversion option. The user is prompted for
a real number greater than 0.0, as the expected ratio of transitions to
transversions. Note
that this is the resulting expected ratio of transitions to transversions.
The default value of the T parameter if you do not use the T
option is 2.0. The T option is not available when you choose the original
Nei/Li restriction fragment distance, which assumes a Jukes-Cantor (1969)
model of DNA change, for which the transition/transversion ratio is
in effect fixed at 0.5.
The S option selects the site length. This is set to a default
value of 6. It can be set to any positive integer. While in
the Restml program there is an upper limit on the restriction
site length (set by memory limitations), in Restdist there is
no effective limit on the size of the restriction sites. A value
of 20, which might be appropriate in many cases for RAPD or AFLP
data, is typically not practical in Restml, but it is useable in
Restdist.
Option L specifies that the output file will have a square matrix
of distances. It can be used to change to lower-triangular
data matrices. This will usually not be
necessary, but if the distance matrices are going to be very
large, this alternative can reduce their size by half. The
programs which are to use them should then of course be informed
that they can expect lower-triangular distance matrices.
The M, I, and 0 options are the usual Multiple data set,
Interleaved input, and screen terminal type options. These are
described in the main documentation file.
Option 1 specifies that the input data will be written out
on the output file before the distances. This is off by
default. If it is done, it will make the output file unusable
as input to our distance matrix programs.
Option 2 turns off or on the indications of the progress of the run.
The program prints out a row of dots (".") indicating the
calculation of individual distances. Since the distance matrix
is symmetrical, the program only computes the distances for the
upper triangle of the distance matrix, and then duplicates
the distance to the other corner of the matrix. Thus the rows of
dots start out at full length, and then get shorter and shorter.
The output file contains on its first line the number of species. The
distance matrix is then printed in standard
form, with each species starting on a new line with the species name, followed
by the distances to the species in order. These continue onto a new line
after every nine distances. If the L option is used, the matrix of distances
is in lower triangular form, so that only the distances to the other species
that precede each species are printed. Otherwise the distance matrix is square
with zero distances on the diagonal. In general the format of the distance
matrix is such that it can serve as input to any of the distance matrix
programs.
If the option to print out the data is selected, the output file will
precede the data by more complete information on the input and the menu
selections. The output file begins by giving the number of species and the
number of characters.
The distances printed out are scaled in terms of expected numbers of
substitutions per DNA site, counting both transitions and transversions but not
replacements of a base by itself, and scaled so that the average rate of
change, averaged over all sites analyzed, is set to 1.0. Thus when the
G option is used, the rate of change at one site may be higher than
at another, but their mean is expected to be 1.
The constants available to be changed are "initialv" and
"iterationsr". The constant "initialv" is the starting
value of the distance in the iterations. This will typically not need to
be changed. The constant "iterationsr" is the number of
times that the Newton-Raphson method which is used to solve the
equations for the distances is iterated. The program can be
speeded up by reducing the number of iterations from the default
value of 20, but at the possible risk of computing the distance
less accurately.
(Note that when the options for displaying the input data are turned off,
the output is in a form suitable for use as an input file in the distance
matrix programs).
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program implements a maximum likelihood method for restriction sites
data (not restriction fragment data). This program is one of the slowest
programs in this package, and can be very tedious to run. It is
possible to have the program search for the maximum likelihood tree.
It will be more practical for some users (those that do not have
fast machines) to use the U (User Tree)
option, which takes less run time, optimizing branch lengths and computing
likelihoods for particular tree topologies suggested by the user. The
model used here is essentially identical to that used by Smouse and Li (1987)
who give explicit expressions for computing the likelihood for three-species
trees. It does not place prior probabilities on trees as they do. The present
program extends their approach to multiple species by
a technique which, while it does not give explicit expressions for likelihoods,
does enable their computation and the iterative improvement of branch
lengths. It also allows for multiple restriction enzymes. The algorithm
has been described in a paper (Felsenstein, 1992). Another relevant
paper is that of DeBry and Slade (1985).
The assumptions of the present model are:
Note that if the existing base is, say, an A, the chance of it being
replaced by a G is 1/3, and so is the chance that it is replaced by
a T. This means that there can be no difference in the (expected)
rate of transitions and transversions. Users who are upset at this
might ponder the fact that a version allowing different rates of
transitions and transversions would run an estimated 16 times
slower. If it also allowed for unequal frequencies of the four bases,
it would run about 300,000 times slower! For the moment, until a better
method is available, I guess I'll stick with this one!
Subject to these assumptions, the program is an approximately
correct maximum likelihood method. The
input is fairly standard, with one addition. As usual the first line of the
file gives the number of species and the number of sites, but there is also
a third number, which is the number of different restriction enzymes that were
used to detect the restriction sites. Thus a data set with 10 species and
35 different sites, representing digestion with 4 different enzymes, would
have the first line of the data file look like this:
The site data are in standard form. Each species starts with a species name
whose maximum length is given by the constant "nmlngth"
(whose value in the
program as distributed is 10 characters). The name should, as usual, be padded
out to that length with blanks if necessary. The sites data then follows, one
character per site (any blanks will
be skipped and ignored). Like the DNA and protein sequence data, the
restriction sites data may be either in the "interleaved" form or the
"sequential" form. Note that if you are analyzing restriction sites
data with the programs Dollop or Mix or other discrete character
programs, at the moment those programs do not use the "aligned" or
"interleaved" data format. Therefore you may want to avoid that format
when you have restriction sites data that you will want to feed into
those programs.
The presence of a site is indicated by a "+" and the absence by a "-". I have
also allowed the use of "1" and "0" as synonyms for "+" and "-", for
compatibility with Mix and Dollop which do not allow "+" and "-". If the
presence of
the site is unknown (for example, if the DNA containing it has been deleted so
that one
does not know whether it would have contained the site) then the state "?" can
be used to indicate that the state of this site is unknown.
User-defined trees may follow the
data in the usual way. The trees must be unrooted, which means that at their
base they must have a trifurcation.
The options are selected by a menu, which looks like this:
The U, J, O, M, and 0 options are the usual ones, described in the main
documentation file. The user trees for option U are read from a file whose
default name is intree. The I option selects between Interleaved and
Sequential input data formats, and is described in the documentation file for
the molecular sequences programs.
The G
(global search) option causes, after the last species is added to the tree,
each possible group to be removed and re-added. This improves the result,
since the position of every species is reconsidered. It approximately triples
the run-time of the program.
The two options specific to this program are the A, and L options. The L
(Length) option allows the user to specify the length in bases of the
restriction sites. At the
moment the program assumes that all sites have the same length (for example,
that all enzymes are 6-base-cutters). The default value for this parameter is
6, which will be used if the L option is not invoked. A desirable future
development for the package would be allowing the L parameter to be different
for every site. It would also be desirable to allow for ambiguities in the
recognition site, since some enzymes recognize 2 or 4 sequences. Both of these
would require fairly complicated programming or else slower execution times.
The A (All) option specifies that all sites are detected, even those for which
all of the species have the recognition sequence absent (character state
"-"). The default condition is that it is assumed that such sites will not
occur in the data. The likelihood computed when the A option is not used is
the probability of the pattern of sites given that tree and conditional on the
pattern not being all absences. This will be realistic for most data, except
for cases in which the data are extracted from sites data for a larger number
of species, in which case some of the site positions could have all absences in
the subset of species. In such cases an effective way of analyzing the data
would be to omit those sites and not use the A option, as such positions, even
if not absolutely excluded, are nevertheless less likely than random to have
been incorporated in the data set.
The W (Weights) option, which is invoked in the input file rather than in
the menu, allows the user to select a subset of sites to
be analyzed. It is invoked in the usual way, except that only weights
0 and 1 are allowed. If the W option is not used, all sites will be
analyzed. If the Weights option is used, there must be a W in the first
line of the input file.
The output starts by giving the number of species, and the number of sites. If
the default condition is used instead of the A option the program states that
it is assuming that sites absent in all species have been omitted. The value
of the site length (6 bases, for example) is also given.
If option 2 (print out data) has been selected,
there then follow the restriction site sequences, printed in
groups of ten sites. The trees found are printed as an unrooted
tree topology (possibly rooted by outgroup if so requested). The
internal nodes are numbered arbitrarily for the sake of
identification. The number of trees evaluated so far and the log
likelihood of the tree are also given.
A table is printed
showing the length of each tree segment, as well as (very) rough confidence
limits on the length. As with Dnaml, if a confidence limit is
negative, this indicates that rearrangement of the tree in that region
is not excluded, while if both limits are positive, rearrangement is
still not necessarily excluded because the variance calculation on which
the confidence limits are based results in an underestimate, which makes
the confidence limits too narrow.
In addition to the confidence limits,
the program performs a crude Likelihood Ratio Test (LRT) for each
branch of the tree. The program computes the ratio of likelihoods with and
without this branch length forced to zero length. This done by comparing the
likelihoods changing only that branch length. A truly correct LRT would
force that branch length to zero and also allow the other branch lengths to
adjust to that. The result would be a likelihood ratio closer to 1. Therefore
the present LRT will err on the side of being too significant.
One should also
realize that if you are looking not at a previously-chosen branch but at all
branches, that you are seeing the results of multiple tests. With 20 tests,
one is expected to reach significance at the P = .05 level purely by
chance. You should therefore use a much more conservative significance level,
such as .05 divided by the number of tests. The significance of these tests
is shown by printing asterisks next to
the confidence interval on each branch length. It is important to keep
in mind that both the confidence limits and the tests
are very rough and approximate, and probably indicate more significance than
they should. Nevertheless, maximum likelihood is one of the few methods that
can give you any indication of its own error; most other methods simply fail to
warn the user that there is any error! (In fact, whole philosophical schools
of taxonomists exist whose main point seems to be that there isn't any
error, that the "most parsimonious" tree is the best tree by definition and
that's that).
The log likelihood printed out with the final tree can be used to perform
various likelihood ratio tests. Remember that testing one tree topology
against another is not a simple matter, because two different tree topologies
are not hypotheses that are
nested one within the other. If the trees differ by only one branch
swap, it seems to be conservative to test the difference between their
likelihoods with one degree of freedom, but other than that little is known and
more work on this is needed.
If the U (User Tree) option is used and more than one tree is supplied,
and the program is not told to assume autocorrelation between the
rates at different sites, the
program also performs a statistical test of each of these trees against the
one with highest likelihood. If there are two user trees, the test
done is one which is due to Kishino and Hasegawa (1989), a version
of a test originally introduced by Templeton (1983). In this
implementation it uses the mean and variance of
log-likelihood differences between trees, taken across sites. If the two
trees' means are more than 1.96 standard deviations different
then the trees are
declared significantly different. This use of the empirical variance of
log-likelihood differences is more robust and nonparametric than the
classical likelihood ratio test, and may to some extent compensate for the
any lack of realism in the model underlying this program.
If there are more than two trees, the test done is an extension of
the KHT test, due to Shimodaira and Hasegawa (1999). They pointed out
that a correction for the number of trees was necessary, and they
introduced a resampling method to make this correction. In the version
used here the variances and covariances of the sum of log likelihoods across
sites are computed for all pairs of trees. To test whether the
difference between each tree and the best one is larger than could have
been expected if they all had the same expected log-likelihood,
log-likelihoods for all trees are sampled with these covariances and equal
means (Shimodaira and Hasegawa's "least favorable hypothesis"),
and a P value is computed from the fraction of times the difference between
the tree's value and the highest log-likelihood exceeds that actually
observed. Note that this sampling needs random numbers, and so the
program will prompt the user for a random number seed if one has not
already been supplied. With the two-tree KHT test no random numbers
are used.
In either the KHT or the SH test the program
prints out a table of the log-likelihoods of each tree, the differences of
each from the highest one, the variance of that quantity as determined by
the log-likelihood differences at individual sites, and a conclusion as to
whether that tree is or is not significantly worse than the best one.
The branch lengths printed out are scaled in terms of expected numbers of
base substitutions, not counting replacements of a base by itself. Of course,
when a branch is twice as long this does not mean that there will be twice as
much net change expected along it, since some of the changes occur in the same
site and overlie or even reverse each other. Confidence limits on the branch
lengths are also given. Of course a negative value of the branch length is
meaningless, and a confidence limit overlapping zero simply means that the
branch length is not necessarily significantly different from zero. Because of
limitations of the numerical algorithm, branch length estimates of zero will
often print out as small numbers such as 0.00001. If you see a branch length
that small, it is really estimated to be of zero length.
Another possible source of confusion is the existence of negative values for
the log likelihood. This is not really a problem; the log likelihood is not a
probability but the logarithm of a probability, and since probabilities never
exceed 1.0 this logarithm will typically be negative. The log likelihood is
maximized by being made more positive: -30.23 is worse than -29.14. The log
likelihood will not always be negative since a combinatorial constant has been
left out of the expression for the likelihood. This does not affect the tree
found or the likelihood ratios (or log likelihood differences) between trees.
The program uses a Newton-Raphson algorithm to update one branch length at a
time. This is faster than the EM algorithm which was described in my
paper on restriction sites maximum likelihood (Felsenstein, 1992). The
likelihood that is being
maximized is the same one used by Smouse and Li (1987) extended for multiple
species.
moving down on the likelihood surface. You may have to "tune" the value of
extrapol to suit your data.
The constants include "maxcutter" (set in phylip.h),
the maximum length of an enzyme
recognition site. The memory used by the program will be approximately
proportional to this value, which is 8 in the distribution copy.
The program also uses constants
"iterations" and "smoothings", and decreasing "epsilon". Reducing
"iterations" and "smoothings" or increasing "epsilon"
will result in faster execution but a worse result. These values will
not usually have to be changed.
The program spends most of its time doing real arithmetic. The algorithm, with
separate and independent computations
occurring at each site, lends itself readily to parallel processing.
A feature of the algorithm is that it saves time by recognizing sites at which
the pattern of presence/absence is the same, and does that computation only
once. Thus if we have only four species but a large number of sites, there are
only about (ignoring ambiguous bases) 16 different patterns of presence/absence
(2 x 2 x 2 x 2) that can occur. The program automatically counts
occurrences of each and does the computation for each pattern only once, so
that it only needs to do as much computation as would be needed with at most
16 sites, even though the number of sites is actually much larger. Thus
the program will run very effectively with few species and many sites.
This program was developed by modifying Dnaml version 3.1 and also adding
some of the modifications that were added to Dnaml version 3.2, with which
it shares many of its data structures and much of its strategy. Version
3.6 changed from EM iterations of branch lengths, which involved arbitrary
extrapolation factors, to the Newton-Raphson algorithm, which improved the
speed of the program (though only from "very slow" to "slow").
There are a number of obvious directions in which the program needs to be
modified in the future. Extension to allow for different rates of transition
and transversion is straightforward, but would slow down the program
considerably, as I have mentioned above. I have not included in the program
any provision for saving and printing out
multiple trees tied for highest likelihood, in part because an exact tie is
unlikely.
© Copyright 1993-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Retree is a tree editor. It reads in a tree,
or allows the user to construct one, and
displays this tree on the screen. The
user then can specify how the tree is to
be rearranged, rerooted or written out to a file.
The input trees are in one file (with default file name
intree), the
output trees are written into another (outtree). The user
can reroot, flip branches, change names of species, change or remove
branch lengths, and move around to look at various parts of the tree if it is
too large to fit on the screen. The trees can be multifurcating at any
level, although the user is warned that many PHYLIP programs still cannot
handle multifurcations above the root, or even at the root.
A major use for this program will be to change rootedness of trees so that
a rooted tree derived from one program can be fed in as an unrooted tree to
another (you are asked about this when you give the command to write out
the tree onto the tree output file). It will also be useful for specifying
the length of a branch in
a tree where you want a program like Dnaml, Dnamlk, Fitch, or Contml to
hold that branch length constant (see the L suboption of the User Tree
option in those programs). It will also be useful for changing the order
of species for purely cosmetic reasons for Drawgram and Drawtree, including
using the Midpoint method of rooting the tree. It can also be used to write out
a tree file in the Nexus format used by Paup and MacClade or in our XML tree
file format.
This program uses graphic characters that show the tree to best
advantage on some computer systems.
Its graphic characters will work best on MSDOS systems or MSDOS windows in
Windows, and to
any system whose screen or terminals emulate ANSI standard terminals
such as old Digitial VT100 terminals,
Telnet programs,
or VT100-compatible windows in the X windowing system.
For any other screen types, (such as Macintosh windows) there is a generic
option which does
not make use of screen graphics characters. The program will work well
in those cases, but the tree it displays will look a bit uglier.
As we will see below, the initial menu of the program allows you to choose
among three screen types (PCDOS, Ansi, and none).
If you want to avoid having to make this choice every time, you can change
some of the constants in the file phylip.h to have the terminal type
initialize itself in the proper way, and recompile. We have tried to
have the default values be correct for PC, Macintosh, and Unix
screens. If the setting is "none" (which is necessary on
Macintosh MacOS 9 screens), the special graphics
characters will not be used to indicate nucleotide states, but only letters
will be used for the four nucleotides. This is less easy to look at.
The constants that need attention are ANSICRT and IBMCRT.
Currently these are both set to "false" on Macintosh MacOS 9 systems,
to "true" on MacOS X and on Unix/Linux
systems, and IBMCRT is set to "true" on Windows systems. If your system
has an ANSI compatible terminal, you might want to find the
definition of ANSICRT in phylip.h and set it to "true", and
IBMCRT to "false".
The user interaction starts with the program presenting a menu. The
menu looks like this:
The 0 (Graphics type) option is the usual
one and is described in the main documentation file. The U (initial tree)
option allows the user to choose whether
the initial tree is to be arbitrary, interactively specified by the user, or
read from a tree file. Typing U causes the program to change among the
three possibilities in turn. Usually we will want to use a User Tree from
a file. It requires that you have available a tree file
with the tree topology of the initial tree. If you wish to set up some other
particular tree you can either use the "specify" choice in the initial tree
option (which is somewhat clumsy to use) or rearrange a User Tree of an
arbitrary tree into the shape you want by using
the rearrangement commands given below.
The L (screen Lines) option allows the user to change the height of the
screen (in lines of characters) that is assumed to be available on the
display. This may be particularly helpful when displaying large trees
on displays that have more than 24 lines per screen, or on workstation
or X-terminal screens that can emulate the ANSI terminals with more than
24 lines.
The N (output file format) option allows the user to specify that the tree files that
are written by the program will be in one of three formats:
The XML tree file format is fairly simple. The tree file, which
may have multiple trees, is enclosed in a pair of
<PHYLOGENIES> ... </PHYLOGENIES> tags.
Each tree is included in tags
<PHYLOGENY> ... </PHYLOGENY>. Each branch of the tree is enclosed in a pair of tags
<CLADE> ... </CLADE>, which enclose the branch and all its descendants.
If the branch has a length, this is given by the LENGTH attribute of the
CLADE tag, so that the pair of tags looks like this:
A tip of the tree is at the end of a branch (and hence that
branch is enclosed in a pair of
<CLADE> ... </CLADE> tags). Its name is enclosed by <NAME> ... </NAME>
tags. Here is an XML tree:
The indentation is for readability but is not part of the XML tree
standard, which ignores that kind of white space.
What programs can read an XML tree? None right now, not even PHYLIP programs!
But soon our lab's LAMARC package will have programs that can read an XML tree.
XML is rapidly becoming the standard for representing and interchanging
complex data -- it is time to have an XML tree standard. Certain extensions
are obvious (to represent the bootstrap proportion for a branch, use
BOOTP=0.83 in the CLADE tag, for example).
There are other proposals for an XML tree standard. They have
many similarities to this one, but are not identical to it. At the
moment there is no mechanism in place for deciding between them other
than seeing which get widely used. Here are links to other
proposals:
The W (screen and window Width) option specifies the width in characters
of the area which the trees will be plotted to fit into. This is by
default 80 characters so that they will fit on a normal width terminal. The
actual width of the display on the terminal (normally 80 characters) will
be regarded as a window displaying part of the tree. Thus you could
set the "plotting area" to 132 characters, and inform the program that
the screen width is 80 characters. Then the program will display only part
of the tree at any one time. Below we will show how to move the "window"
and see other parts of the tree.
After the initial menu is displayed and the choices are made,
the program then sets up an initial tree and displays it. Below it will be a
one-line menu of possible commands. Here is what the tree and the menu
look like (this is the tree specified by the example input tree given at
the bottom of this page, as it displays when the terminal type is "none"):
The tree that was read in had no branch lengths on its branches. The
absence of a branch length is indicated by drawing the branch with ">"
characters (>>>>>>>). When branches have branch lengths, they are
drawn with "-" characters (-------) and their
lengths on the screen are approximately proportional to the branch length.
If you type "?" you will get a single screen showing a description of each
of these commands in a few words. Here are slightly more detailed
descriptions of the commands:
The program was written by Andrew Keeffe, using some code from Dnamove, which
he also wrote.
Below is a test tree file. We have already showed (above), what the
resulting tree display looks like when the terminal type is "none".
For ANSI or IBM PC screens it will look better, using the graphics characters
of those screens, which we do not attempt to show here.
© Copyright 1991-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
Seqboot is a general bootstrapping and data set translation tool. It is intended to allow you to
generate multiple data sets that are resampled versions of the input data
set. Since almost all programs in the package can analyze these multiple
data sets, this allows almost anything in this package to be bootstrapped,
jackknifed, or permuted. Seqboot can handle molecular sequences,
binary characters, restriction sites, or gene frequencies. It
can also convert data sets between Sequential and Interleaved
format, and into the NEXUS format or into a new XML sequence alignment format.
To carry out a bootstrap (or jackknife, or permutation test) with some method
in the package, you may need to use three programs. First, you need to run
Seqboot to take the original data set and produce a large number of
bootstrapped or jackknifed data
sets (somewhere between 100 and 1000 is usually adequate).
Then you need to find the phylogeny estimate for
each of these, using the particular method of interest. For example, if
you were using Dnapars you would first run Seqboot and make a file with 100
bootstrapped data sets. Then you would give this file the proper name to
have it be the input file for Dnapars. Running Dnapars with the M (Multiple
Data Sets) menu choice and informing it to expect 100 data sets, you
would generate a big output file as well as a treefile with the trees from
the 100 data sets. This treefile could be renamed so that it would serve
as the input for Consense. When Consense is run the majority rule consensus
tree will result, showing the outcome of the analysis.
This may sound tedious, but the run of Consense is fast, and that of
Seqboot is fairly fast, so that it will not actually take any longer than
a run of a single bootstrap program with the same original data and the same
number of replicates. This is not very hard and allows bootstrapping or
jackknifing on many of the methods in
this package. The same steps are necessary with all of them. Doing things
this way some of the intermediate files (the tree file from the Dnapars
run, for example) can be used to summarize the results of the bootstrap in
other ways than the majority rule consensus method does.
If you are using the Distance Matrix programs, you will have to add one extra
step to this, calculating distance matrices from each of the replicate data
sets, using Dnadist or Gendist. So (for example) you would run Seqboot, then
run Dnadist using the output of Seqboot as its input, then run (say) Neighbor
using the output of Dnadist as its input, and then run Consense using the
tree file from Neighbor as its input.
The resampling methods available are:
The data input file is of standard form for molecular sequences (either in
interleaved or sequential form), restriction sites, gene frequencies, or
binary morphological characters.
When the program runs it first asks you for a random number seed. This should
be an integer greater than zero (and probably less than 32767) and which is
of the form 4n+1, that is, it leaves a remainder of 1 when divided by 4. This
can be judged by looking at the last two digits of the integer (for instance
7651 is not of form 4n+1 as 51, when divided by 4, leaves the remainder 3).
The random number seed is used to start the random number generator.
If the randum number seed is not odd, the program will request it again.
Any odd number can be used, but may result in a random number sequence that
repeats itself after less than the full one billion numbers. Usually this
is not a problem. As the random numbers appear to be unpredictable,
there is no such thing as a "good" seed -- the numbers produced from one
seed are statistically indistinguishable from those produced by another, and
it is not true that the numbers produced from one seed (say 4533) are similar
to those produced from a nearby seed (say 4537).
Then the program shows you a menu to allow you to choose options. The menu
looks like this:
The user selects options by typing one of the letters in the left column,
and continues to do so until all options are correctly set. Then the
program can be run by typing Y.
It is important to select the correct data type (the D selection). Each
time D is typed the program will change data type, proceeding successively
through Molecular Sequences, Discrete Morphological Characters, Restriction
Sites, and Gene Frequencies. Some of these will cause additional entries
to appear in the menu. If Molecular Sequences or Restriction Sites settings
are chosen the I (Interleaved)
option appears in the menu (and as Molecular Sequences are also the default,
it therefore appears in the first menu). It is the usual
I option discussed in the Molecular Sequences document file and in the main
documentation files for the package, and is on by default.
If the Restriction Sites option is chosen the menu option E appears, which
asks whether the input file contains a third number on the first line of
the file, for the number of restriction enzymes used to detect these sites.
This is necessary because data sets for Restml need this third number, but
other programs do not, and Seqboot needs to know what to expect.
If the Gene Frequencies option is chosen a menu option A appears which allows
the user to specify that all alleles at each locus are in the input file.
The default setting is that one allele is absent at each locus.
Note that for sampling methods such as the bootstrap and jackknife,
whole loci are sampled, not individual alleles.
The J option allows the user to select Bootstrapping, Delete-Half-Jackknifing,
the Archie-Faith permutation of species within characters, permutation of
character order, shuffling character order separately within each species,
or Rewriting. It changes
successively among these each time J is typed.
The P menu option appears if the data are molecular
sequences and the J option is used to choose the Rewrite option. It
gives you the choice between our normal PHYLIP format or a new (and
nonstandard) XML sequence alignment format. This encloses the alignment
between <ALIGNMENT> ... </ALIGNMENT> tags. Each sequence
between <SEQUENCE> ... </SEQUENCE> tags, has a TYPE
attribute of the sequence which is either "dna", "rna" or "protein".
This is set by default to "dna" but can be changed by the
user in an S Sequence type menu option. Each
sequence has its name, enclosed between <NAME> ... </NAME>
tags, and the data itself, enclosed between <DATA> ... </DATA>
tags. The XML option is not available unless the data are molecular
sequences. It is a new format -- no programs yet read it.
In other cases the P menu option does not appear and the
PHYLIP output format is assumed. Here is a simple example of this
XML sequence alignment format, for the (silly) data set used in
our main documentation file:
For the gene frequencies and restriction sites data types, this Rewrite
option does not change the data set. The option will be useful mostly to
write the data out in a standard format, in cases where the input file
is messy-looking.
The B option selects the Block Bootstrap. When you select option B the program
will ask you to enter the block length. When the block length is 1,
this means that we are doing regular bootstrapping rather than
block-bootstrapping.
The % option allows the user control over what fraction of the characters
are sampled in the bootstrap and jackknife methods. Normally the
bootstrap samples a number of times equal to the number of characters,
and the jackknife samples half that number. This option permits you to
specify a smaller fraction of characters to be sampled. Note that doing
so is "statistically incorrect", but it is available here for whatever other
purposes you may have in mind. Note that the fraction you will be asked to
enter is the fraction of characters sampled, not the fraction left out.
If you specify 100 as the fraction of sites retained and are using
the jackknife, the data set
will simply be rewritten. Note (as mentioned below) that this can be
used together with the W (Weights) option to rewrite a data set while
omitting a particular set of sites.
The R option allows the user to set the number of replicate data sets.
This defaults to 100. Most statisticians would be happiest with 1000 to
10,000 replicates in a bootstrap, but 100 gives a rough picture. You
will have to decide this based on how long a running time of the
tree programs you are willing to tolerate. (The time needed to do the
sampling in this program is not much of an issue).
The W (Weights) option allows weights to be read
from a file whose default name is "weights". The weights
follow the format described in the main documentation file.
Weights can only be 0 or 1, and act to select
the characters (or sites) that will be used in the resampling, the others
being ignored and always omitted from the output data sets.
If you use W together with the S (just weights)
option, you write a file of weights (whose default name is "outweights").
In that file, any character whose original weight is 0 will have weight
0, the other weights varying according to the resampling. Note that if
you write out data sets rather than weights (not using the S option),
this output weights file is not written, as the characters are
written different numbers of times in the data output file.
Note that with restriction sites, the weights are not used by
some of the programs. Writing out files of weights will not be
helpful with those programs. For the moment, with all gene frequencies
programs the weights are also not used.
Note that it is possible to use Seqboot to rewrite a data set while
omitting certain sites. This can be done, not with the rewrite choice in
option J, but with its jackknife choice. Choose the delete-half
jackknife, but then use the % option to set the fraction of sites
sampled to 100%. Also use the W option to read a set of weights that
select which sites to retain (those with weights 1 instead of 0).
Use the R option to set the number of replicates to 1. The program will
write one data set, with all the sites that have weights 1, in order.
The C (Categories) option can be used with molecular sequence programs to
allow assignment of sites or amino acid positions to user-defined rate
categories. The assignment of rates to
sites is then made by reading a file whose default name is "categories".
It should contain a string of digits 1 through 9. A new line or a blank
can occur after any character in this string. Thus the categories file
might look like this:
The only use of the Categories information in Seqboot is that they
are sampled along with the sites (or amino acid positions) and are
written out onto a file whose default name is "outcategories",
which has one set of categories information for each bootstrap
or jackknife replicate.
In the discrete characters data type, three more options appear in the
menu. These are the N (aNcestors), X (miXture of methods), and F (Factors)
options. They
may be useful with the program Mix, which allows input of ancestors
information and information specifying the mixture of parsimony methods
to be used. Factors information is also read and used by
programs Move, Dolmove, and Clique, in calculating how many multistate
characters are compatible with a tree.
The mixture, ancestors, and factors information for the characters
are specified in input files whose default names are
"ancestors", "mixture", and "factors". Seqboot produces output files that
properly
reflect what the resampling implies for these files. The corresponding
output files have default file names "outancestors", "outmixture", and
"outfactors".
For futher description of the mixture, ancestors, and factors file
formats and contents
see the Discrete Characters Programs documentation file.
The S option is a particularly important one. It is used whether to
produce multiple output files or multiple weights. If your
data set is large, a file with (say) 1000 such data sets can be very
large and may use up too much space on your system. If you choose
the S option, the program will instead produce a weights file with
multiple sets of weights. The default name of this file is "outweights".
Except for some programs that cannot handle multiple sets of
weights,
PHYLIP programs have an M (multiple data sets) option that asks the
user whether to use multiple data sets or multiple sets of weights.
If the latter is selected when running those programs, they
read one data set, but analyze it multiple times, each time reading a new
set of weights. As both bootstrapping and jackknifing can be thought of
as reweighting the characters, this accomplishes the same thing (the
multiple weights option is not available for the various kinds of permutation).
As the file with multiple sets of weights is much smaller than a file with
multiple data sets, this can be an attractive way to save file space.
When multiple sets of weights are chosen, they reflect the sampling as
well as any set of weights that was read in, so that you can use
Seqboot's W option as well.
The 0 (Terminal type) option is the usual one.
Often runs of distance programs, or of phylogeny programs, on large numbers
of bootstrap replicates are very time-consuming. If you have multiple
computers, you can save time by splitting up these runs among multiple
machines. For example, if you have 1000 replicate data sets (or weights) from
bootstrapping, you could divide these into ten files of 100 data sets (or
you could simply use Seqboot ten times with different random number seeds). If
these are run on ten separate computers, the execution time is speeded up
by as much as a factor of 10. Each input file of 100 data sets results in
an output tree file. These can be concatenated end-to-end using a word
processor program or using a command such as the Unix/Linux cat
command. Make sure that these files are not turned into Microsoft Word
format when this is done. The consensus tree program Consense will
hande the concatenated tree file properly.
If a distance matrix method is being used, you can also produce the distance
matrices on different machines, and concatenate them end-to-end to
produce an input file of distance matrices for Fitch, Kitsch, or Neighbor.
This is particularly relevant for Neighbor, which in most cases makes trees more
quickly than the distance matrices can be produced.
The data files read by Seqboot are the standard ones for the various kinds of
data. For molecular sequences the sequences may be either interleaved or
sequential, and similarly for restriction sites. Restriction sites data
may either have or not have the third argument, the number of restriction
enzymes used. Discrete morphological
characters are always assumed to be in sequential format. Gene frequencies
data start with the number of species and the number of loci, and then
follow that by a line with the number of alleles at each locus. The data for
each locus may either have one entry for each allele, or omit one allele at
each locus. The details of the formats are given in the main documentation
file, and in the documentation files for the groups of programs.
The output file will contain the data sets generated by the resampling
process. Note that, when Gene Frequencies data is used or when
Discrete Morphological characters with the Factors option are used,
the number of characters in each data set may vary. It may also vary
if there are an odd number of characters or sites and the Delete-Half-Jackknife
resampling method is used, for then there will be a 50% chance of choosing
(n+1)/2 characters and a 50% chance of choosing (n-1)/2 characters.
The Factors option causes the characters to be resampled together. If
(say) three adjacent characters all have the same factor, so
that they all are understood to be recoding one multistate character, they
will be resampled together as a group.
The numerical options 1 and 2 in the menu also affect the output file.
If 1 is chosen (it is off by default) the program will print the original
input data set on the output file before the resampled data sets. I cannot
actually see why anyone would want to do this. Option 2 toggles the
feature (on by default) that prints out up to 20 times during the resampling
process a notification that the program has completed a certain number of
data sets. Thus if 100 resampled data sets are being produced, every 5
data sets a line is printed saying which data set has just been completed.
This option should be turned off if the program is running in background and
silence is desirable. At the end of execution the program will always (whatever
the setting of option 2) print
a couple of lines saying that output has been written to the output file.
The program runs moderately quickly, though more slowly when the Permutation
resampling method is used than with the others.
(If Replicates are set to 10 and seed to 4333)
phylip-3.697/doc/sequence.html 0000644 0047320 0047320 00000036766 12406201173 016026 0 ustar joe felsenst_g
© Copyright 1986-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
These programs estimate phylogenies from protein
sequence or nucleic acid sequence data. Protpars uses a parsimony method
intermediate between Eck and Dayhoff's
method (1966) of allowing transitions between all amino acids and counting
those, and Fitch's (1971) method of counting the number of nucleotide changes
that would be needed to evolve the protein sequence. Dnapars uses the
parsimony method allowing changes between all bases
and counting the number of those. Dnamove is an interactive parsimony
program allowing the user to rearrange trees by hand and see where
character states change. Dnapenny
uses the branch-and-bound method to search for all most
parsimonious trees in the nucleic acid sequence case. Dnacomp
adapts to nucleotide sequences the compatibility (largest clique)
approach. Dnainvar does not directly estimate a phylogeny, but computes Lake's
(1987) and Cavender's (Cavender and Felsenstein, 1987) phylogenetic invariants,
which are quantities whose values depend on the phylogeny. Dnaml does a
maximum likelihood estimate of the phylogeny (Felsenstein, 1981a). Dnamlk
is similar to Dnaml but assumes a molecular clock. Dnadist
computes distance measures between pairs of species from nucleotide sequences,
distances that can then be used by the distance matrix programs Fitch and
Kitsch. Restml does a maximum likelihood estimate from restriction
sites data. Seqboot allows you to read in a data set and then produce
multiple data sets from it by bootstrapping, delete-half jackknifing, or
by permuting within sites. This
then allows most of these methods to be bootstrapped or jackknifed, and
for the Permutation Tail Probability Test of Archie (1989) and Faith and
Cranston (1991) to be carried out.
The input and output format for Restml is described in
its document files. In general its input format is similar to
those described here, except that the one-letter codes for restriction sites
is specific to that program and is described in that document file. Since
the input formats for the eight DNA sequence and two protein sequence
programs apply to more than one program, they are described here. Their
input formats are standard, making use of the IUPAC standards.
The sequences can continue over multiple lines; when this is done the
sequences must be either in "interleaved" format, similar to the
output of alignment programs, or "sequential" format. These are
described in the main documentation file. In sequential format all
of one sequence is given, possibly on multiple lines, before the next starts.
In interleaved format the first part of the file should contain the first
part of each of the sequences, then optionally a line containing nothing
but a carriage-return character, then the second part of each sequence,
and so on. Only the first parts of the sequences should be preceded by
names. Here is a hypothetical example of interleaved format:
while in sequential format the same sequences would be:
Note, of course, that a portion of a sequence like this:
300 AAGCGTGAAC GTTGTACTAA TRCAG
is perfectly legal, assuming that the species name has gone before, and is
filled out to full length by blanks. The above
digits and blanks will be ignored, the sequence being taken as starting
at the first base symbol (in this case an A). This should enable you to
use output from many multiple-sequence alignment programs with only
minimal editing.
The input format for the DNA sequence programs is
standard: the data have A's, G's, C's and T's (or U's). The first line of the
input file contains the number of species and the number of sites. As
with the other programs, options information may follow this. Following this,
each species starts on a new line. The first 10
characters of that line are the species name. There then follows
the base sequence of that species, each character
being one of the letters A, B, C, D, G, H, K, M, N, O, R, S, T, U, V,
W, X, Y, ?, or - (a period was also previously allowed but it is no longer
allowed, because it sometimes is used in different senses in other
programs). Blanks will be ignored, and so will numerical
digits. This allows GENBANK and EMBL sequence entries to be read with
minimum editing.
These characters can be either upper or lower case. The algorithms
convert all input characters to upper case (which is how they
are treated). The characters constitute the IUPAC (IUB) nucleic acid code
plus some slight
extensions. They enable input of nucleic acid sequences taking full account
of any ambiguities in the sequence.
The input for the protein sequence programs is fairly standard. The first
line contains the
number of species and the number of amino acid positions (counting any
stop codons that you want to include). These are followed on the same line
by the options. The only options which
need information in the input file are U (User Tree) and W (Weights). They are
as described in the main documentation file. If the W (Weights) option is
used there must be a W in the first line of the input file.
Next come the species data. Each
sequence starts on a new line, has a ten-character species name
that must be blank-filled to be of that length, followed immediately
by the species data in the one-letter code. The sequences must either
be in the "interleaved" or "sequential" formats. The I option
selects between them. The sequences can have internal
blanks in the sequence but there must be no extra blanks at the end of the
terminated line. Note that a blank is not a valid symbol for a deletion.
The protein sequences are given by the one-letter code used by
the late Margaret Dayhoff's group in the Atlas of Protein Sequences,
and consistent with the IUB standard abbreviations.
In the present version it is:
where "nonsense", and "unknown" mean respectively a nonsense (chain
termination) codon and an amino acid whose identity has not been
determined. The state "asx" means "either asn or asp",
and the state "glx" means "either gln or glu" and the state "deletion"
means that alignment studies indicate a deletion has happened in the
ancestry of this position, so that it is no longer present. Note that
if two polypeptide chains are being used that are of different length
owing to one terminating before the other, they can be coded as (say)
Here are the same one-letter codes tabulated the other way 'round:
The programs allow options chosen from their menus. Many of these are as described in the
main documentation file, particularly the options J, O, U, T, W,
and Y. (Although T has a different meaning in the programs Dnaml and
Dnadist than in the others).
The U option indicates that
user-defined trees are provided at the end of the input file. This
happens in the usual way, except that for Protpars, Dnapars, Dnacomp, and
Dnamlk, the trees must be strictly
bifurcating, containing only two-way splits, e. g.: ((A,B),(C,(D,E)));. For
Dnaml and Restml it must have a trifurcation at its base,
e. g.: ((A,B),C,(D,E));. The
root of the tree may in those cases be placed arbitrarily, since the trees
needed are actually unrooted, though they look different when printed out. The
program Retree should enable you to reroot the trees without having to
hand-edit or retype them. For
Dnamove the U option is not available (although
there is an equivalent feature which uses rooted user trees).
A feature of the nucleotide sequence programs other than Dnamove
is that they save time and computer memory space by recognizing sites
at which the pattern of bases is the same, and doing their computation only
once. Thus if we have only four species but a large number of sites, there
are (ignoring ambiguous bases) only about 256 different patterns of
nucleotides (4 x 4 x 4 x 4) that can occur. The programs automatically
count how many occurrences there are of each and then only needs to do as much
computation as would be
needed with 256 sites, even though the number of sites is actually much
larger. If there are ambiguities (such as Y or R nucleotides), these are also
handled correctly, and do not cause trouble. The programs store the full
sequences but reserve other space for bookkeeping only for the distinct
patterns. This saves space. Thus the programs will run very effectively
with few species and many sites. On larger numbers of species,
if rates of evolution are small, many of the sites will be invariant
(such as having all A's) and thus will mostly have one of four patterns. The
programs will in this way automatically avoid doing duplicate
computations for such sites.
phylip-3.697/doc/treedist.html 0000644 0047320 0047320 00000050612 12406201173 016023 0 ustar joe felsenst_g
© Copyright 2000-2014 by Joseph Felsenstein. All rights reserved.
License terms here.
This program computes distances between trees. Two distances are computed,
the Branch Score Distance of Kuhner and Felsenstein (1994), and
the more widely known Symmetric Difference of Robinson and Foulds (1981).
The Branch Score Distance uses branch lengths, and can only be calculated
when the trees have lengths on all branches. The Symmetric Difference
does not use branch length information, only the tree topologies. It
must also be borne in mind that neither distance has any immediate
statistical interpretation -- we cannot say whether a larger distance is
significantly larger than a smaller one.
These distances are computed by considering all possible branches that could
exist on the
the two trees. Each branch divides the set of species into two groups --
the ones connected to one end of the branch and the ones connected to the
other. This makes a partition of the full set of species.
The following tree (in Newick notation)
In the case of the Branch Score distance, each partition that does exist on
a tree also has a branch length associated with it. Thus if the tree is
Note that the tree is being treated as unrooted here, so that the branch
lengths on either side of the rootmost node are summed up to get a branch
length of 0.4.
The Branch Score Distance imagines us as having made a list of all
possible partitions, the ones shown above and also all 7 other possible
partitions, which correspond to branches that are not found in this tree.
These are assigned branch lengths of 0.
For two trees, we imagine constructing these lists, and then summing the
squared differences between the branch lengths. Thus if both trees have
branches {A, D | B, C, E}, the sum
contains the square of the difference between the branch lengths. If one
tree has the branch and the other doesn't, it contains the square of the
difference between the branch length and zero (in other words, the
square of that branch length). If both trees do not have a particular
branch, nothing is added to the sum because the difference is then between
0 and 0.
The Branch Score Distance takes this sum of squared differences and
computes its square root. Note that it has some desirable properties.
When small branches differ in tree topology, it is not very big.
When branches are both present but differ in length, it is affected.
The Symmetric Difference is simply a count of how many partitions there are,
among the two trees, that are on one tree and not on the other. In the
example above there are two partitions, {A, C | B, D, E} and {A, D | B, C, E},
each of which is present on only one of the two trees. The Symmetric
Difference between the two trees is therefore 2. When the two trees are
fully resolved bifurcating trees, their symmetric distance must be an even
number; it can range from 0 to twice the number of internal branches, so
that for n species it can be as large as 2n-6 (for 3 species or more).
Note the relationship between the two distances. If all branches in the
two trees have length 1.0, the Branch Score Distance is the square root of the
Symmetric Difference, as each branch that is present in one but not in the
other results in 1.0 being added to the sum of squared differences.
We have assumed that nothing is lost if the trees are treated as unrooted trees.
It is easy to define a counterpart to the Branch Score Distance and one to
the Symmetric Difference for these rooted trees.
Each branch then defines a set of species, namely the clade defined by that
branch. Thus if the first of the two trees above were considered as a rooted
tree it would define the three clades {A, C}, {B, D, E}, and {B, E}. The
Branch Score Distance is computed from the branch lengths for all possible
sets of species, with 0 put for each set that does not occur on that tree.
The table above will be nearly the same, but with two entries instead of
one for the sets on either side of the root, {A C D} and {B E}.
The
Symmetric Difference between two rooted trees is simply the count of the number
of clades that are defined by one but not by the other. For the second tree
the clades would be {A, D}, {B, C, E}, and {B, E}. The Symmetric Difference
between these two rooted trees would then be 4.
Although the examples we have discussed have involved fully
bifurcating trees, the input trees can have multifurcations.
This does not cause any complication for the Branch Score Distance.
For the Symmetric Difference, it can lead to distances that are odd numbers.
However, note one strong restriction. The trees should all have the same
list of species. If you use one set of species in the first two trees,
and another in the second two, and choose distances for adjacent pairs,
the distances will be incorrect and will depend on the order of these
pairs in the input tree file, in odd ways.
The program reads one or two input tree files. If there is one input tree
file, its default name is intree. If there are two their default
names are intree and intree2. The tree files may either
have the number of trees on their first line, or not. If the number of
trees is given, it is actually ignored and all trees in the tree file
are considered, even if there are more trees than indicated by the number.
There is no maximum number of trees that can be processed but, if you
feed in too many, there may be an error message about running out of
memory. The problem is particularly acute if you choose the option to
examine all possible pairs of trees in an input tree file, or all possible
pairs of trees one from each of two input tree files. Thus if there are
1,000 trees in the input tree file, keep in mind that all possible
pairs means 1,000,000 pairs to be examined!
Earlier versions of this program had an option to change the value of
the variable maxgrp to overcome a hard-wired
limit on the number of species the program can handle.
This feature has been removed as it is no longer necessary.
The options are selected from a menu, which looks like this:
The D option chooses which distance measure to use. The Branch Score
Distance is the default. If it is in force, and any of the trees which are
read in have even one branch that fails to have a length, the program will
terminate with an error. If the Symmetric Difference option is chosen,
no check of branch lengths is made.
The O option allows you to root the trees using an outgroup. It is specified
by giving its number, where the species are numbered in the order they
appear in the first tree. Outgroup-rooting all the trees does not
affect the distances if the trees are treated as unrooted, and if it is done
and trees are
treated as rooted, the distances turn out to be the same as the unrooted
ones. Thus it is unlikely that you will find this option of interest.
The R option controls whether the Symmetric Distance that is computed is
to treat the trees as unrooted or rooted. Unrooted is the default.
The terminal type (0) and progress (1) options do not need description here.
Option 2 controls how many tree files are read in, which trees are to
be compared, and how the output is to be presented. It causes
another menu to appear:
Option A computes the distances between successive pairs of trees in the
tree input file -- between trees 1 and 2, trees 3 and 4, trees
5 and 6, and so on. If there are an odd number of trees in the input tree
file the last tree will be ignored and a warning message printed to
remind the user that nothing was done with it.
Option P computes distances between all pairs of trees in the input tree
file. Thus with 10 trees 10 x 10 = 100 distances will be computed,
including distances between each tree and itself.
Option C takes input from two tree files and computes distances between
corresponding members of the two tree files. Thus distances will be
computed between tree 1 of the first tree file and tree 1 of the second one,
between tree 2 of the first file and tree 2 of the second one, and so on.
If the number of trees in the two files differs, the extra trees in the
file that has more of them are ignored and a warning is printed out.
Option L computes distances between all pairs of trees, where one tree is
taken from one tree file and the other from the other tree file. Thus if
the first tree file has 7 trees and the second has 5 trees, 7 x 5 = 35
different distances will be computed.
If option 2 is not selected, the program defaults to looking at one tree
file and computing distances of adjacent pairs (so that option A is
the default).
The results of the analysis are written onto an output file whose
default file name is outfile.
If any of the four types of analysis are selected, the program asks the
user how they want the results presented. Here is that menu for options
P or L:
The Full matrix (choice F) is a table showing all distances. It is
written onto the output file. The table is presented as groups of
10 columns. Here is the Full matrix for the 12 trees in the input
tree file which is given as an example at the end of this page.
The Full matrix is only available for analyses P and L (not for A or C).
Option V (Verbose) writes one distance per line. The Verbose
output is the default. Here it is for the example data set given below:
Option S (Sparse or terse) is similar except that all that is
given on each line are the numbers of the two trees and the distance,
separated by blanks. This may be a convenient format if you want to
write a program to read these numbers in, and you want to spare yourself
the effort of having the program wade through the words on each line
in the Verbose output.
The first four lines of the Sparse output are titles that your program would
want to skip past. Here is the Sparse output for the example trees.
Treedist was originally written by Dan Fineman, with fixes by Doug Buxton.
We also hope in the future to compute a distance based on
quartets shared and not shared by trees (implicit in the work of Estabrook, McMorris, and
Meacham, 1985). We will also implement the tree distance of Robinson and
Foulds (1979), which is like the Branch Score Distance but uses absolute
values of differences between branch lengths rather than sums of squares of
differences.
The output from the setting in the D menu choice of the Symmetric Difference
for this test set is given above (it is the
Verbose output example).
This data set is the first part of the previous one, but with branch
lengths on the trees, to serve as an example for the Branch Score distance.
This was run using the default Branch Score distance, and asking in
option 2 for the P (all pairs in file) setting and the F (Full matrix output)
setting.
version 3.696
Kitsch -- Fitch-Margoliash and Least Squares Methods
with Evolutionary Clock
Fitch-Margoliash method with contemporary tips, version 3.696
Settings for this run:
D Method (F-M, Minimum Evolution)? Fitch-Margoliash
U Search for best tree? Yes
P Power? 2.00000
- Negative branch lengths allowed? No
L Lower-triangular data matrix? No
R Upper-triangular data matrix? No
S Subreplicates? No
J Randomize input order of species? No. Use input order
M Analyze multiple data sets? No
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Write out trees onto tree file? Yes
Y to accept these or type the letter for one to change
((A:0.1,B:0.2):0.4,(C:0.06,D:0.01):43);
((A:0.1,B:0.1):0.4,(C:0.06,D:0.06):44);
[S(K)-S(F)]/d1
----------------
S(F)/d2
TEST DATA SET
7
Bovine 0.0000 1.6866 1.7198 1.6606 1.5243 1.6043 1.5905
Mouse 1.6866 0.0000 1.5232 1.4841 1.4465 1.4389 1.4629
Gibbon 1.7198 1.5232 0.0000 0.7115 0.5958 0.6179 0.5583
Orang 1.6606 1.4841 0.7115 0.0000 0.4631 0.5061 0.4710
Gorilla 1.5243 1.4465 0.5958 0.4631 0.0000 0.3484 0.3083
Chimp 1.6043 1.4389 0.6179 0.5061 0.3484 0.0000 0.2692
Human 1.5905 1.4629 0.5583 0.4710 0.3083 0.2692 0.0000
TEST SET OUTPUT FILE (with all numerical options on)
phylip-3.697/doc/main.html 0000644 0047320 0047320 00001115507 13212365300 015131 0 ustar joe felsenst_g
7 Populations
Fitch-Margoliash method with contemporary tips, version 3.69
__ __ 2
\ \ (Obs - Exp)
Sum of squares = /_ /_ ------------
2
i j Obs
negative branch lengths not allowed
Name Distances
---- ---------
Bovine 0.00000 1.68660 1.71980 1.66060 1.52430 1.60430
1.59050
Mouse 1.68660 0.00000 1.52320 1.48410 1.44650 1.43890
1.46290
Gibbon 1.71980 1.52320 0.00000 0.71150 0.59580 0.61790
0.55830
Orang 1.66060 1.48410 0.71150 0.00000 0.46310 0.50610
0.47100
Gorilla 1.52430 1.44650 0.59580 0.46310 0.00000 0.34840
0.30830
Chimp 1.60430 1.43890 0.61790 0.50610 0.34840 0.00000
0.26920
Human 1.59050 1.46290 0.55830 0.47100 0.30830 0.26920
0.00000
+-------Human
+-6
+----5 +-------Chimp
! !
+---4 +---------Gorilla
! !
+------------------------3 +--------------Orang
! !
+----2 +------------------Gibbon
! !
--1 +-------------------------------------------Mouse
!
+------------------------------------------------Bovine
Sum of squares = 0.107
Average percent standard deviation = 5.16213
From To Length Height
---- -- ------ ------
6 Human 0.13460 0.81285
5 6 0.02836 0.67825
6 Chimp 0.13460 0.81285
4 5 0.07638 0.64990
5 Gorilla 0.16296 0.81285
3 4 0.06639 0.57352
4 Orang 0.23933 0.81285
2 3 0.42923 0.50713
3 Gibbon 0.30572 0.81285
1 2 0.07790 0.07790
2 Mouse 0.73495 0.81285
1 Bovine 0.81285 0.81285
PHYLIP
Phylogeny Inference Package
Version 3.697
December, 2017
by Joseph Felsenstein
Department of Genome Sciences and Department of Biology
University of Washington
Department of Genome Sciences
Box 355065
Seattle, WA 98195-5065
USA
E-mail address: joe (at) gs.washington.edu
Contents of This Document
A Brief Description of the Programs
Copyright Notice for PHYLIP
The Documentation Files and How to Read Them
What The Programs Do
Running the Programs
A word about input files
Installing a recent version of Oracle Java
Running the programs on a Windows machine
Running the programs on a Macintosh with Mac OS X
Running the programs on a Unix or Linux system
Running the programs on a Macintosh with Mac OS 8 or 9 (deprecated)
Running the programs in MSDOS
Running the Drawgram and Drawtree Java interfaces
Running the Drawgram and Drawtree Java GUI interfaces in Windows
Running the programs in background or under control of a command file
An example (Unix, Linux or Mac OS X)
Subtleties (in Unix, Linux, or Mac OS X)
An example (Windows)
Testing for existence of files
Prototyping keyboard response files
Preparing Input Files
Input and output files
Where the files are
Data file format
The Menu
The Output File
The Tree File
The Options and How To Invoke Them
Common options in the menu
The U (User tree) option
The G (Global) option
The J (Jumble) option
The O (Outgroup) option
The T (Threshold) option
The M (Multiple data sets) option
The W (Weights) option
The option to write out the trees into a tree file
The (0) terminal type option
The Algorithm for Constructing Trees
Local rearrangements
Global rearrangements
Multiple jumbles
Saving multiple tied trees
Strategy for finding the best tree
A Warning on Interpreting Results
Relative Speed of Different Programs and Machines
Relative speed of the different programs
Speed with different numbers of species
Relative speed of different machines
General Comments on Adapting the Package to Different Computer Systems
Compiling the programs
Unix and Linux
On Windows systems
Compiling with Cygnus Gnu C++
Compiling with Microsoft Visual C++
Macintosh
Compiling with GCC on Mac OS X with our Makefile
Compiling with GCC on Mac OS X with X Windows
What about
the Metrowerks Codewarrior compiler?
VMS VAX systems
Parallel computers
Other computer systems
Compiling the Java interfaces
Frequently Asked Questions
How to make it do various things
Background information needed:
Questions about distribution and citation:
Questions about documentation
Additional Frequently Asked Questions, or: "Why didn't it occur to you to ...
(Fortunately) obsolete questions
New Features in This Version
Coming Attractions, Future Plans
Endorsements
From the pages of Cladistics
... in the pages of other journals:
... and in the comments made by users when they register:
References for the Documentation Files
Credits
Other Phylogeny Programs Available Elsewhere
PAUP*
MrBayes
MEGA
PAML
Phyml
RAxML
TNT
DAMBE
How You Can Help Me
In Case of Trouble
A Brief Description of the Programs
Copyright Notice for PHYLIP
All rights reserved.
The Documentation Files and How to Read Them
What The Programs Do
Running the Programs
A word about input files.
dnaml: can't find input file "infile"
Please enter a new file name>
Running the programs on a Macintosh with Mac OS X
Running the programs on a Unix or Linux system.
Running the programs on a Macintosh with Mac OS 8 or 9 (deprecated)
Once you have the executables, you may follow the directions below.
Running the Drawgram and Drawtree Java interfaces
Running the Drawgram and Drawtree Java GUI interfaces in Windows
Installing a recent version of Oracle Java
Once a useable version of Java is installed, you do not have to repeat the
installation every time you run one of the programs Drawgram or Drawtree.
Running the programs on a Windows machine.
Running the programs in background or under control of a command file
An example (Unix, Linux or Mac OS X)
sequences.dat
Y
Subtleties (in Unix, Linux, or Mac OS X)
seqboot < input1 > screenout
mv outfile infile
dnapars < input2 >> screenout
mv outtree intree
consense < input3 >> screenout
An example (Windows)
sequences.dat
Y
Testing for existence of files
if test -e fubarfile
then
rm fubarfile
fi
Prototyping keyboard response files
R
U
Y
R
Preparing Input Files
Input and output files
-------------------
| |
infile ---------> | |
| |
intree ---------> | | -----------> outfile
| |
weights --------> | program | -----------> outtree
| |
categories -----> | | -----------> plotfile
| |
fontfile -------> | |
| |
-------------------
dnaml: can't find input file "infile"
Please enter a new file name>
Where the files are
Data file format
6 13
Archaeopt CGATGCTTAC CGC
HesperorniCGTTACTCGT TGT
BaluchitheTAATGTTAAT TGT
B. virginiTAATGTTCGT TGT
BrontosaurCAAAACCCAT CAT
B.subtilisGGCAGCCAAT CAC
6 39
Archaeopt CGATGCTTAC CGCCGATGCT
HesperorniCGTTACTCGT TGTCGTTACT
BaluchitheTAATGTTAAT TGTTAATGTT
B. virginiTAATGTTCGT TGTTAATGTT
BrontosaurCAAAACCCAT CATCAAAACC
B.subtilisGGCAGCCAAT CACGGCAGCC
TACCGCCGAT GCTTACCGC
CGTTGTCGTT ACTCGTTGT
AATTGTTAAT GTTAATTGT
CGTTGTTAAT GTTCGTTGT
CATCATCAAA ACCCATCAT
AATCACGGCA GCCAATCAC
1101
0011001101
The Menu
DNA parsimony algorithm, version 3.695
Setting for this run:
U Search for best tree? Yes
S Search option? More thorough search
V Number of trees to save? 10000
J Randomize input order of sequences? No. Use input order
O Outgroup root? No, use as outgroup species 1
T Use Threshold parsimony? No, use ordinary parsimony
N Use Transversion parsimony? No, count all steps
W Sites weighted? No
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Print out steps in each site No
5 Print sequences at all nodes of tree No
6 Write out trees onto tree file? Yes
Y to accept these or type the letter for one to change
The Output File
+-------------------Gibbon
+----------------------------2
! ! +------------------Orang
! +------4
! ! +---------Gorilla
+-----3 +--6
! ! ! +---------Chimp
! ! +----5
--1 ! +-----Human
! !
! +-----------------------------------------------Mouse
!
+------------------------------------------------Bovine
+-----------------------------------------------Mouse
!
+---------4 +------------------Orang
! ! +------3
! ! ! ! +---------Chimp
---6 +----------------------------1 ! +----2
! ! +--5 +-----Human
! ! !
! ! +---------Gorilla
! !
! +-------------------Gibbon
!
+-------------------------------------------Bovine
remember: this is an unrooted tree!
+--Human
+--5
+--4 +--Chimp
! !
+--3 +-----Gorilla
! !
+--2 +--------Orang
! !
+--1 +-----------Gibbon
! !
--6 +--------------Mouse
!
+-----------------Bovine
remember: this is an unrooted tree!
Between And Length Approx. Confidence Limits
------- --- ------ ------- ---------- ------
1 Bovine 0.90216 ( 0.50346, 1.30086) **
1 Mouse 0.79240 ( 0.42191, 1.16297) **
1 2 0.48553 ( 0.16602, 0.80496) **
2 3 0.12113 ( zero, 0.24676) *
3 4 0.04895 ( zero, 0.12668)
4 5 0.07459 ( 0.00735, 0.14180) **
5 Human 0.10563 ( 0.04234, 0.16889) **
5 Chimp 0.17158 ( 0.09765, 0.24553) **
4 Gorilla 0.15266 ( 0.07468, 0.23069) **
3 Orang 0.30368 ( 0.18735, 0.41999) **
2 Gibbon 0.33636 ( 0.19264, 0.48009) **
* = significantly positive, P < 0.05
** = significantly positive, P < 0.01
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 2 2 2 2 1 1 2 2 1
10! 1 2 3 1 1 1 1 1 1 2
20! 1 2 2 1 2 2 1 1 1 2
30! 1 2 1 1 1 2 1 3 1 1
40! 1
The Tree File
((Mouse,Bovine),(Gibbon,(Orang,(Gorilla,(Chimp,Human)))));
(A,(B,(C,D)),(E,F));
((cat:47.14069,(weasel:18.87953,((dog:25.46154,(raccoon:19.19959,
bear:6.80041):0.84600):3.87382,(sea_lion:11.99700,
seal:12.00300):7.52973):2.09461):20.59201):25.0,monkey:75.85931);
The Options and How To Invoke Them
Common options in the menu
((Alligator,Bear),((Cow,(Dog,Elephant)),Ferret));
((Alligator,Bear),(((Cow,Dog),Elephant),Ferret));
((Alligator,Bear),((Cow,Dog),(Elephant,Ferret)));
5 6
Alpha CCACCA
Beta CCAAAA
Gamma CAACCA
Delta AACAAC
Epsilon AACCCA
5 6
Alpha CACACA
Beta CCAACC
Gamma CAACAC
Delta GCCTGG
Epsilon TGCAAT
The Algorithm for Constructing Trees
Local rearrangements
T1 T2 T3
\ / /
\ / /
\ / /
\/ /
* /
* /
* /
* /
*
!
!
T3 T2 T1 T1 T3 T2
\ / / \ / /
\ / / \ / /
\ / / \ / /
\ / / \ / /
\ / \ /
\ / \ /
\ / \ /
\ / \ /
! !
! !
! !
Global rearrangements
Multiple jumbles
Saving multiple tied trees
Strategy for finding the best tree
Nixon's search strategy
It will not necessarily be fast to do this, as the last step may be slow.
But the resampling will cause emphasis on different sets of characters in
the initial searches, allowing the process to explore regions of tree
space not usually examined by conventional rearrangement strategies.
A Warning on Interpreting Results
Relative Speed of Different
Programs and MachinesRelative speed of the different programs
10 40
A CACACACAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAA
B CACACAACAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAA
C CACAACAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAA
D CAACAAAACAAAAAAAAACAAAAAAAAAAAAAAAAAAAAA
E CAACAAAAACAAAAAAAACAAAAAAAAAAAAAAAAAAAAA
F ACAAAAAAAACACACAAAACAAAAAAAAAAAAAAAAAAAA
G ACAAAAAAAACACAACAAACAAAAAAAAAAAAAAAAAAAA
H ACAAAAAAAACAACAAAAACAAAAAAAAAAAAAAAAAAAA
I ACAAAAAAAAACAAAACAACAAAAAAAAAAAAAAAAAAAA
J ACAAAAAAAAACAAAAACACAAAAAAAAAAAAAAAAAAAA
10 40
MesohippusAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
HypohippusAAACCCCCCCAAAAAAAAACAAAAAAAAAAAAAAAAAAAA
ArchaeohipCAAAAAAAAAAAAAAAACACAAAAAAAAAAAAAAAAAAAA
ParahippusCAAACAACAACAAAAAAAACAAAAAAAAAAAAAAAAAAAA
MerychippuCCAACCACCACCCCACACCCAAAAAAAAAAAAAAAAAAAA
M. secunduCCAACCACCACCCACACCCCAAAAAAAAAAAAAAAAAAAA
Nannipus CCAACCACAACCCCACACCCAAAAAAAAAAAAAAAAAAAA
NeohippariCCAACCCCCCCCCCACACCCAAAAAAAAAAAAAAAAAAAA
Calippus CCAACCACAACCCACACCCCAAAAAAAAAAAAAAAAAAAA
PliohippusCCCACCCCCCCCCACACCCCAAAAAAAAAAAAAAAAAAAA
10 40
A CACACAACCAAACAAACCACAAAAAAAAAAAAAAAAAAAA
B AAACCACACACACAAACCCAAAAAAAAAAAAAAAAAAAAA
C ACAAAACCAAACCACCCACAAAAAAAAAAAAAAAAAAAAA
D AAAAACACAACACACCAAACAAAAAAAAAAAAAAAAAAAA
E AAACAACCACACACAACCAAAAAAAAAAAAAAAAAAAAAA
F CCCAAACACCCCCAAAAAACAAAAAAAAAAAAAAAAAAAA
G ACACCCCCACACCCACCAACAAAAAAAAAAAAAAAAAAAA
H AAAACAACAACCACCCCACCAAAAAAAAAAAAAAAAAAAA
I ACACAACAACACAAACAACCAAAAAAAAAAAAAAAAAAAA
J CCAAAAACACCCAACCCAACAAAAAAAAAAAAAAAAAAAA
Hennigian Data
Horses Data
Random Data
Protpars
0.00500
0.00670
0.01289
Dnapars
0.01050
0.00940
0.00980
Dnapenny
0.01400
0.00860
1.71100
Dnacomp
0.00240
0.00250
0.00590
Dnaml
0.17749
0.23970
0.21350
Dnamlk
0.21740
0.19450
0.24400
Proml
1.3527
3.2085
2.0055
Promlk
3.3567
8.6078
4.4886
Dnainvar
0.00020
0.00020
0.00020
Dnadist
0.00140
0.00080
0.00150
Protdist
0.09220
0.09210
0.09310
Restml
0.14560
0.28810
0.21540
Restdist
0.00110
0.00090
0.00080
Fitch
0.00760
0.01280
0.00880
Kitsch
0.00180
0.00260
0.00280
Neighbor
0.00020
0.00050
0.00050
Contml
0.01310
0.01500
0.01780
Gendist
0.00070
0.00070
0.00070
Pars
0.00780
0.00610
0.02930
Mix
0.00360
0.00410
0.00610
Penny
0.00190
0.00470
0.8060
Dollop
0.00480
0.00450
0.00820
Dolpenny
0.00200
0.01060
1.1270
Clique
0.00100
0.00070
0.00130
Speed with different numbers of species
Relative speed of different machines
Machine
Operating
SystemCompiler
Total
Time
Relative
SpeedToshiba T1100+
MSDOS
Turbo Pascal 3.01A
(269)
10542
0.00009486
Apple Mac Plus
Mac OS
Lightspeed Pascal 2
(175.84)
6891
0.00014511
Toshiba T1100+
MSDOS
Turbo Pascal 5.0
(162)
6349
0.00015750
Macintosh Classic
Mac OS
Think Pascal 3
(160)
6271
0.00015947
Macintosh Classic
Mac OS
Think C
(43.0)
4771
0.0002096
IBM PS2/60
MSDOS
Turbo Pascal 5.0
(58.76)
2303
0.0004343
80286 (12 Mhz)
MSDOS
Turbo Pascal 5.0
(47.09)
1845.4
0.0005419
Apple Mac IIcx
Mac OS
Think Pascal 3
(42)
1645.5
0.0006077
Apple Mac SE/30
Mac OS
Think Pascal 3
(42)
1645.6
0.0006077
Apple Mac IIcx
Mac OS
Lightspeed Pascal 2
(39.84)
1561.6
0.0006404
Apple Mac IIcx
Mac OS
Lightspeed Pascal 2#
(39.69)
1555.0
0.00006431
Zenith Z386 (16MHz)
MSDOS
Turbo Pascal 5.0
(38.27)
1539.0
0.0006498
Macintosh SE/30
Mac OS
Think C
(13.6)
1508.4
0.0006630
386SX (16 MHz)
MSDOS
Turbo Pascal 6.0
(34)
1333.6
0.0007498
386SX (16 MHz)
MSDOS
Microsoft Quick C
(12.01)
1333.6
0.0007499
Sequent-S81
DYNIX
Silicon Valley Pascal
(13.0)
509.0
0.0019646
VAX 11/785
Unix
Berkeley Pascal
(11.9)
466.3
0.002144
80486-33
MSDOS
Turbo Pascal 6.0
(11.46)
449.0
0.02227
Sun 3/60
SunOS
Sun C
(3.93)
435.7
0.002295
NeXT Cube (68030)
Mach
Gnu C
(2.608)
289.3
0.003456
Sequent S-81
DYNIX
Sequent Symmetry C
(2.604)
288.9
0.003461
VAXstation 3500
Unix
Berkeley Pascal
(7.3)
286.5
0.003491
Sequent S-81
DYNIX
Berkeley Pascal
(5.6)
219.5
0.004557
Unisys 7000/40
Unix
Berkeley Pascal
(5.24)
205.3
0.004870
VAX 8600
VMS
DEC VAX Pascal
(3.96)
155.23
0.006442
Sun SPARC IPX
SunOS
Gnu C version 2.1
(1.28)
142.04
0.007040
VAX 6000-530
VMS
DEC C
(0.858)
95.14
0.010511
VAXstation 4000
VMS
DEC C
(0.809)
89.81
0.011135
IBM RS/6000 540
AIX
XLP Pascal
(2.276)
89.14
0.011219
NeXTstation(040/25)
Mach
Gnu C
(0.75)
83.15
0.012027
Sun SPARC IPX
SunOS
Sun C
(0.68)
75.43
0.01326
486DX (33 MHz)
Linux
Gnu C #
(0.63)
69.95
0.01430
Sun SPARCstation-1
Unix
Sun Pascal
(1.7)
66.62
0.01501
DECstation 5000/200
Unix
DEC Ultrix C
(0.45)
49.97
0.02001
Sun SPARC 1+
SunOS
Sun C
(0.40)
44.37
0.02254
DECstation 3100
Unix
DEC Ultrix Pascal
(0.77)
30.11
0.03321
IBM 3090-300E
AIX
Metaware High C
(0.27)
29.98
0.03336
DECstation 5000/125
Unix
DEC Ultrix C
(0.267)
29.58
0.03381
DECstation 5000/200
Unix
DEC Ultrix C
(0.256)
28.38
0.03524
Sun SPARC 4/50
SunOS
Sun C
(0.249)
27.62
0.03621
DEC 3000/400 AXP
Unix
DEC C
(0.224)
24.85
0.04024
DECstation 5000/240
Unix
DEC Ultrix C
(0.1889)
20.96
0.04771
SGI Iris R4000
Unix
SGI C
(0.184)
20.41
0.04898
IBM 3090-300E
VM
Pascal VS
(0.464)
18.12
0.05519
DECstation 5000/200
Unix
DEC Ultrix Pascal
(0.39)
15.188
0.06583
Pentium 120
Linux
Gnu C
1.848
11.953
0.08366
Pentium Pro 180
Linux
Gnu C
1.009
6.527
0.1532
Pentium 266 MMX
Linux
Gnu C (PHYLIP 3.5)
(0.054)
5.996
0.1668
Pentium 266 MMX
Linux
Gnu C
0.927
5.996
0.1668
Pentium 200
Linux
Gnu C
0.853
5.517
0.1812
SGI PowerChallenge
Irix
Gnu C
0.844
5.459
0.1832
DEC Alpha 400 4/233
DUNIX
Digital C (cc -fast)
0.730
4.722
0.2118
Pentium II 500
Linux
Gnu C
0.368
2.380
0.4201
Dual 448/633 MHz Pentiums
Linux
gcc
0.3069
1.985
0.5037
Sun Ultra 10
Solaris 8
gcc
0.25848
1.672
0.5981
Macintosh G3 300 MHz
Mac OS X
Gnu C (-O 3)
0.2330
1.5071
0.6635
Compaq/Digital Alpha 500au
DUNIX
Digital C (cc -fast)
0.167
1.080
0.9257
AMD Athlon 1.2 GHz
Linux
gcc
0.1546
1.0
1.0
Intel Pentium 4 2.26 GHz
Windows XP
Cygwin gcc
0.1078
0.6973
1.434
Pentium 4 1700 MHz
Linux
Gnu C
0.10730
0.6940
1.441
SGI Fuel R16000/700MHz
IRIX 6.5.30
MipsPro 7.4.4
0.09
0.58
1.72
Macintosh G4 1.2GHz
Mac OS X
Gnu C (-O 3)
0.0582
0.3765
2.656
AMD Athlon 2800 2.1 GHz
Linux
gcc (-O 3)
0.0455
0.2943
3.398
iMac 2 Ghz Intel Core Duo
Mac OS X
gcc (-O 3)
0.0300
0.1940
5.153
Machine
Operating
SystemCompiler
Seconds
Time
Relative
Speed386SX 16 Mhz
PCDOS
Turbo Pascal 6
(7826)
1027.55
0.0009732
386SX 16 Mhz
PCDOS
Quick C
(6549.79)
1027.55
0.0009732
Compudyne 486DX/33
Linux
Gnu C
(1599.9)
251.0
0.003984
SUN Sparcstation 1+
SunOS
Sun C
(1402.8)
220.1
0.004543
Everex STEP 386/20
PCDOS
Turbo Pascal 5.5
(1440.8)
189.17
0.005286
486DX/33
PCDOS
Turbo C++
(1107.2)
173.70
0.005757
Compudyne 486DX/33
PCDOS
Waterloo C/386
(1045.78)
164.07
0.006094
Sun SPARCstation IPX
SunOS
Gnu C
(960.2)
150.64
0.006638
NeXTstation(68040/25)
Mach
Gnu C
(916.6)
143.80
0.006954
486DX/33
PCDOS
Waterloo C/386
(861.0)
135.08
0.007403
Sun SPARCstation IPX
SunOS
Sun C
(787.7)
123.58
0.008091
486DX/33
PCDOS
Gnu C
(650.9)
102.12
0.009792
VAX 6000-530
VMS
DEC C
(637.0)
99.94
0.01001
DECstation 5000/200
Unix
DEC Ultrix RISC C
(423.3)
66.41
0.01506
IBM 3090-300E
AIX
Metaware High C
(201.8)
31.65
0.03159
Convex C240/1024
Unix
C
(101.6)
15.940
0.06274
DEC 3000/400 AXP
Unix
DEC C
(98.29)
15.42
0.06485
Pentium 120
Linux
Gnu C
25.26
19.230
0.05200
Pentium Pro 180
Linux
Gnu C
18.88
14.372
0.06957
Pentium 200
Linux
Gnu C
16.51
12.569
0.07956
SGI PowerChallenge
IRIX
Gnu C
12.446
9.475
0.10554
DEC Alpha 400 4/233
Linux
Gnu C (cc -fast)
8.0418
6.122
0.16335
Pentium MMX 266
Linux
Gnu C (PHYLIP 3.5)
(36.15)
5.671
0.17632
Pentium MMX 266
Linux
Gnu C
7.45
5.671
0.17632
Pentium II 500
Linux
Gnu C
6.02
4.583
0.2182
Dual 448/633 MHz Pentiums
Linux
Gnu C
3.7225
2.834
0.3529
Sun Ultra 10
Solaris 8
Gnu C
3.7101
2.824
0.3541
Pentium 4 1.7 GHz
Linux
Gnu C
2.0668
1.5734
0.6356
Macintosh G3 300 MHz
Mac OS X
Gnu C (-O 3)
1.805
1.3741
0.7278
Intel Pentium 4 2.26 GHz
Windows XP
Cygwin gcc
1.55457
1.1834
0.8450
AMD Athlon 1.2 GHz
Linux
Gnu C
1.3136
1.0
1.0
Compaq/Digital Alpha 500au
Linux
Gnu C (cc -fast)
0.9383
0.7143
1.4000
Macintosh G4 1.2 GHz
Mac OS X
Gnu C (-O 3)
0.7080
0.5390
1.8554
SGI Fuel R16000/700Mhz
IRIX 6.5.30
MipsPro 7.4.4
0.55
0.41
2.43
AMD Athlon 2800 2.1 GHz
Linux
gcc (-O 3)
0.3065
0.2333
4.286
iMac 2 Ghz Intel Core Duo
Mac OS X
gcc (-O 3)
0.2535
0.1930
5.182
General Comments on Adapting
the Package to Different Computer Systems
ln -s /usr/local/bin/font1 font1
ln -s /usr/local/bin/font2 font2
ln -s /usr/local/bin/font3 font3
ln -s /usr/local/bin/font4 font4
ln -s /usr/local/bin/font5 font5
ln -s /usr/local/bin/font6 font6
http://www.cygwin.com and follow the
instructions there. To download it you need to download
their setup.exe program and then it will download the rest
when it is run. You will need a lot of disk space for it (about a gigabyte).
set MSVC=Path
where Path is where Microsoft Visual Studio is installed.
On our Windows XP system, "Path" is
C:\Program Files\Microsoft Visual Studio 9.0
PATH=%PATH%;%MSVC%\VC\bin;%MSVC%\Common7\IDE
(The "7" is correct here; it is not a typo.)
MSVCPATH="C:\Program Files\Microsoft Visual Studio 9.0\VC"
will need to be changed so that
it points to wherever Microsoft Visual Studio is installed, followed by
\VC (for Visual Studio 9.0).
PATH=%PATH%;%MSVC%\Vc7\bin;%MSVC%\Common7\IDE
MSVCPATH="C:\Program Files\Microsoft Visual Studio .NET\Vc7"
If this does not work with your Visual C++ 7.0 compiler,
then the most likely reason is that your installation
was not placed into the folder C:\Program Files,
or has a name that is not exactly identical to
Microsoft Visual Studio .NET. In that case,
you will need to find the correct path to the Visual C++
7.0 installation on your system, and supply this in the
MSVC variable above, and also in the Makefile. (Note
that in the Makefile, you will need to follow this path
with \Vc7.)
-L"c:\Borland\Bcc55\lib"
-I"c:\Borland\Bcc55\include"
-L"c:\Borland\Bcc55\lib"
set BORLAND=Path
PATH=%PATH%;%BORLAND%\Bin
$ CC DNAPARS.C
$ CC PHYLIP.C
$ CC SEQ.C
$ LINK DNAPARS,PHYLIP,SEQ
/* parallelize here */
In particular
within these innermost loops of the programs there are often scalar quantities
that are used for temporary bookkeeping. These quantities, such as
sum1, sum2, zz, z1, yy, y1, aa, bb, cc, sum, and denom in procedure makenewv
of Dnaml and similar quantities in procedure nuview) are there to
minimize the number of array references. For vectorizing and parallelizing
compilers it will
be better to replace them by arrays so that processing can occur
simultaneously.
Problems that are encountered
How to make it do various things
It's not a feature but is not too hard to do in many of the programs. In
parsimony programs like Mix, for which the W (Weights) and A (Ancestral states)
options are available, and weights can be larger than 1, all you need to do is:
(a)
In Mix, make up an extra character with states 0 for all the outgroups
and 1 for all the ingroups. If using
Dnapars the ingroup can have (say)
G and the outgroup A.(b)
Assign this character an enormous weight
(such as Z for 35) using the W
option,
all other characters getting weight 1, or whatever weight they had
before.(c)
If it is available, Use the A (Ancestral
states) option to designate that
for that new character the state found in the
outgroup is the ancestral
state.(d) In Mix do not use the O (Outgroup) option.
(e)
After the tree is found, the designated
ingroup should have been held
together by the fake character. The tree will be
rooted somewhere in the
outgroup (the program may or may not have a preference for one place in
the outgroup over another).
Make sure that you subtract from the total
number of steps on the tree all steps in the new character.In programs like Dnapars, you cannot use this method as weights of sites
cannot be greater than 1. But you do an analogous trick, by adding a
largish number of extra sites to the data, with one nucleotide state ("A")
for the ingroup and another ("G") for the outgroup. You will then have to
use Retree to manually reroot the tree in the desired place.
For restriction sites (rather than fragments) life is a bit
easier: they evolve nearly independently so bootstrapping is possible
and Restml can be used, as well as restriction sites distances
computed in Restdist. Also directionality of change
is less ambiguous when parsimony is used. A more complete tour of the
issues for restriction sites and restriction fragments is given in chapter
15 of my book (Felsenstein, 2004).
Background information needed:
"How do I use the programs? I can't find any documentation!"
and it is
available as a PDF here.
Questions about distribution and citation:
Questions about documentation
Additional Frequently Asked Questions, or:
"Why didn't it occur to you to ...
(Fortunately) obsolete questions
"Why didn't it occur to you to ...
New Features in This Version
Coming Attractions, Future Plans
Endorsements
From the pages of Cladistics:
"Under no circumstances can we recommend PHYLIP/WAG [their name for the
Wagner parsimony option of Mix]."
Luckow, M. and R. A. Pimentel (1985)
"PHYLIP has not proven very effective in implementing parsimony (Luckow and
Pimentel, 1985)."
J. Carpenter (1987a)
"... PHYLIP. This is the computer program where every newsletter concerning
it is mostly bug-catching, some of which have been put there by previous
corrections. As Platnick (1987) documents, through dint of much labor useful
results may be attained with this program, but I would suggest an
easier way: FORMAT b:"
J. Carpenter (1987b)
"PHYLIP is bug-infested and both less effective and orders of
magnitude slower than other programs ...."
"T. N. Nayenizgani" [J. S. Farris] (1990)
"Hennig86 [by J. S. Farris] provides such substantial improvements over
previously available programs (for both mainframes and microcomputers) that
it should now become the tool of choice for practising systematists."
N. Platnick (1989)
... in the pages of other journals:
"The availability, within PHYLIP of distance, compatibility, maximum likelihood,
and generalized `invariants' algorithms (Cavender and Felsenstein, 1987) sets
it apart from other packages .... One of the strengths of PHYLIP is its
documentation ...."
Michael J. Sanderson (1990)
(Sanderson also criticizes PHYLIP for slowness and inflexibility of its
parsimony algorithms, and compliments other packages on their strengths).
"This package of programs has gradually become a basic necessity to anyone
working seriously on various aspects of phylogenetic inference .... The package
includes more programs than any other known phylogeny package. But it is not
just a collection of cladistic and related programs. The package has great
value added to the whole, and for this it is unique and of extreme
importance .... its various strengths are in the great array of methods
provided ...."
Bernard R. Baum (1989)
... and in the comments made by users when they register:
"a program on phylogeny --
PHYLOGENY INTERFERENCE PACKAGE (PHYLIP). We would therefore like to ask ..."
[names withheld] (in 1994)
"I am struglling with your clever programs."
[name withheld] (in 1995)
"I'm famously computer illiterate - I look forward to many frustrating hours trying to run this program"
Desmond Maxwell (in 1998)
"I am a brave man. PHYLIP is a brave program. We'll do fine together."
Christopher Winchell (in 2000)
"The Mahabarata of phylogenetics looks better than ever."
Ross Crozier (in 2001)
"I love phylip. Tastes great and less filling!"
Byron Adams (in 2002)
References for the Documentation Files
Years
Agency
Grant or Contract Number
2005-2009
NIH NIGMS
R01 GM071639
2003-2007
NIH NIGMS
R01 GM51929-05 (PI: Mary Kuhner)
1999-2003
NSF
BIR-9527687
1999-2002
NIH NIGMS
R01 GM51929-04
1999-2001
NIH NIMH
R01 HG01989-01
1995-1999
NIH NIGMS
R01 GM51929-01
1992-1995
National Science Foundation
DEB-9207558
1992-1994
NIH NIGMS Shannon Award
2 R55 GM41716-04
1989-1992
NIH NIGMS
1 R01-GM41716-01
1990-1992
National Science Foundation
BSR-8918333
1987-1990
National Science Foundation
BSR-8614807
1979-1987
U.S. Department of Energy
DE-AM06-76RLO2225 TA DE-AT06-76EV71005
http://evolution.gs.washington.edu/phylip/credits.html
Other Phylogeny Programs Available Elsewhere
http://mrbayes.net as a Mac OS X or Windows
executable, or in source code in C.
http://dambe.bio.uottawa.ca/dambe.asp.
How You Can Help Me
In Case of Trouble
Joe Felsenstein
Department of Genome Sciences
University of Washington
Box 355065
Seattle, Washington 98195-5065, U.S.A.
phylip-3.697/doc/mix.html 0000644 0047320 0047320 00000034657 12406201173 015010 0 ustar joe felsenst_g
version 3.696
Mix - Mixed method discrete characters parsimony
INPUT FORMAT
Mixed parsimony algorithm, version 3.69
Settings for this run:
U Search for best tree? Yes
X Use Mixed method? No
P Parsimony method? Wagner
J Randomize input order of species? No. Use input order
O Outgroup root? No, use as outgroup species 1
T Use Threshold parsimony? No, use ordinary parsimony
A Use ancestral states in input file? No
W Sites weighted? No
M Analyze multiple data sets? No
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Print out steps in each character No
5 Print states at all nodes of tree No
6 Write out trees onto tree file? Yes
Are these settings correct? (type Y or the letter for one to change)
OUTPUT FORMAT
TEST DATA SET
5 6
Alpha 110110
Beta 110000
Gamma 100110
Delta 001001
Epsilon 001110
TEST SET OUTPUT (with all numerical options on)
phylip-3.697/doc/move.html 0000644 0047320 0047320 00000040455 12406201173 015152 0 ustar joe felsenst_g
Mixed parsimony algorithm, version 3.69
5 species, 6 characters
Wagner parsimony method
Name Characters
---- ----------
Alpha 11011 0
Beta 11000 0
Gamma 10011 0
Delta 00100 1
Epsilon 00111 0
4 trees in all found
+--Epsilon
+-----4
! +--Gamma
+--2
! ! +--Delta
--1 +-----3
! +--Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 9.000
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 2 2 2 1 1 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 1?011 0
1 2 no ..... .
2 4 maybe .0... .
4 Epsilon yes 0.1.. .
4 Gamma no ..... .
2 3 yes ...00 .
3 Delta yes 001.. 1
3 Beta maybe .1... .
1 Alpha maybe .1... .
+--------Gamma
!
+--2 +--Epsilon
! ! +--4
! +--3 +--Delta
--1 !
! +-----Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 9.000
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 1 2 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 1?011 0
1 2 no ..... .
2 Gamma maybe .0... .
2 3 maybe ...?? .
3 4 yes 001.. .
4 Epsilon maybe ...11 .
4 Delta yes ...00 1
3 Beta maybe .1.00 .
1 Alpha maybe .1... .
+--------Epsilon
+--4
! ! +-----Gamma
! +--2
--1 ! +--Delta
! +--3
! +--Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 9.000
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 2 2 2 1 1 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 1?011 0
1 4 maybe .0... .
4 Epsilon yes 0.1.. .
4 2 no ..... .
2 Gamma no ..... .
2 3 yes ...00 .
3 Delta yes 0.1.. 1
3 Beta yes .1... .
1 Alpha maybe .1... .
+--------Gamma
+--2
! ! +-----Epsilon
! +--4
--1 ! +--Delta
! +--3
! +--Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 9.000
steps in each character:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 2 2 2 1 1 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 1?011 0
1 2 maybe .0... .
2 Gamma no ..... .
2 4 maybe ?.?.. .
4 Epsilon maybe 0.1.. .
4 3 yes ...00 .
3 Delta yes 0.1.. 1
3 Beta yes 110.. .
1 Alpha maybe .1... .
version 3.696
Move - Interactive mixed method parsimony
Interactive mixed parsimony algorithm, version 3.696
Settings for this run:
X Use Mixed method? No
P Parsimony method? Wagner
A Use ancestral states? No
F Use factors information? No
O Outgroup root? No, use as outgroup species 1
W Sites weighted? No
T Use Threshold parsimony? No, use ordinary parsimony
U Initial tree (arbitrary, user, specify)? Arbitrary
0 Graphics type (IBM PC, ANSI, none)? ANSI
S Width of terminal screen? 80
L Number of lines on screen? 24
Are these settings correct? (type Y or the letter for one to change)
NEXT? (Options: R # + - S . T U W O F C H ? X Q) (H or ? for Help)
ADAPTING THE PROGRAM TO YOUR COMPUTER AND TO YOUR TERMINAL
MORE ABOUT THE PARSIMONY CRITERION
TEST DATA SET
phylip-3.697/doc/neighbor.html 0000644 0047320 0047320 00000021622 12406201173 015774 0 ustar joe felsenst_g
5 6
Alpha 110110
Beta 110000
Gamma 100110
Delta 001001
Epsilon 001110
version 3.696
Neighbor -- Neighbor-Joining and UPGMA methods
Neighbor-Joining/UPGMA method version 3.69
Settings for this run:
N Neighbor-joining or UPGMA tree? Neighbor-joining
O Outgroup root? No, use as outgroup species 1
L Lower-triangular data matrix? No
R Upper-triangular data matrix? No
S Subreplicates? No
J Randomize input order of species? No. Use input order
M Analyze multiple data sets? No
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Write out trees onto tree file? Yes
Y to accept these or type the letter for one to change
TEST DATA SET
7
Bovine 0.0000 1.6866 1.7198 1.6606 1.5243 1.6043 1.5905
Mouse 1.6866 0.0000 1.5232 1.4841 1.4465 1.4389 1.4629
Gibbon 1.7198 1.5232 0.0000 0.7115 0.5958 0.6179 0.5583
Orang 1.6606 1.4841 0.7115 0.0000 0.4631 0.5061 0.4710
Gorilla 1.5243 1.4465 0.5958 0.4631 0.0000 0.3484 0.3083
Chimp 1.6043 1.4389 0.6179 0.5061 0.3484 0.0000 0.2692
Human 1.5905 1.4629 0.5583 0.4710 0.3083 0.2692 0.0000
OUTPUT FROM TEST DATA SET (with all numerical options on)
phylip-3.697/doc/pars.html 0000644 0047320 0047320 00000032715 12406201173 015151 0 ustar joe felsenst_g
7 Populations
Neighbor-Joining/UPGMA method version 3.69
Neighbor-joining method
Negative branch lengths allowed
Name Distances
---- ---------
Bovine 0.00000 1.68660 1.71980 1.66060 1.52430 1.60430
1.59050
Mouse 1.68660 0.00000 1.52320 1.48410 1.44650 1.43890
1.46290
Gibbon 1.71980 1.52320 0.00000 0.71150 0.59580 0.61790
0.55830
Orang 1.66060 1.48410 0.71150 0.00000 0.46310 0.50610
0.47100
Gorilla 1.52430 1.44650 0.59580 0.46310 0.00000 0.34840
0.30830
Chimp 1.60430 1.43890 0.61790 0.50610 0.34840 0.00000
0.26920
Human 1.59050 1.46290 0.55830 0.47100 0.30830 0.26920
0.00000
+---------------------------------------------Mouse
!
! +---------------------Gibbon
1------------------------2
! ! +----------------Orang
! +--5
! ! +--------Gorilla
! +-4
! ! +--------Chimp
! +-3
! +------Human
!
+------------------------------------------------------Bovine
remember: this is an unrooted tree!
Between And Length
------- --- ------
1 Mouse 0.76891
1 2 0.42027
2 Gibbon 0.35793
2 5 0.04648
5 Orang 0.28469
5 4 0.02696
4 Gorilla 0.15393
4 3 0.03982
3 Chimp 0.15167
3 Human 0.11753
1 Bovine 0.91769
version 3.696
Pars - Discrete character parsimony
INPUT FORMAT
But note that these do not include blank (" "). Blanks in the
input data are simply skipped by the program, so that they can be used to
make characters into groups for ease of viewing.
The "?" (question mark) symbol has special meaning. It is allowed in the
input but is not available as the symbol of a state. Rather, it means that
the state is unknown.
(of these, probably only + and - will be of interest to most users).
Discrete character parsimony algorithm, version 3.69
Setting for this run:
U Search for best tree? Yes
S Search option? More thorough search
V Number of trees to save? 100
J Randomize input order of species? No. Use input order
O Outgroup root? No, use as outgroup species 1
T Use Threshold parsimony? No, use ordinary parsimony
W Sites weighted? No
M Analyze multiple data sets? No
I Input species interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Print out steps in each site No
5 Print character at all nodes of tree No
6 Write out trees onto tree file? Yes
Y to accept these or type the letter for one to change
OUTPUT FORMAT
TEST DATA SET
5 6
Alpha 110110
Beta 110000
Gamma 100110
Delta 001001
Epsilon 001110
TEST SET OUTPUT (with all numerical options on)
phylip-3.697/doc/phylip.gif 0000644 0047320 0047320 00000001636 12406201173 015310 0 ustar joe felsenst_g GIF87a€ € ñ ÿÿQQûÿÿÿ, € € þœ©Ëí£œ´Ú+Þû†
G≦Î&ò jí‹1Í÷ î|Äâ p â £“¦ÉE…ͧSH
Ú ³z
OJ—é¨äˆ×™nÚ üžÙtÆ´ŒÈë|Å=¯$—5HXÒòèUÈÈxˆ˜¤è%ØXùˆ'©¹©v‰É)èù‰•9šÚ9g
©ú:ÔŠùš+ëJ«¹w{”«(È›BöKÊ"<Œú+(€¬òUì"à<ÚÊ\ͤª½ÝÓ-*× N$ lnT9X¾nDÙ &?HMŸ¯¿ÏßïÏ6ˆBÀ1Y(Ø«3°!‚ % ¤“0BÄ '<XQ! þ#\\3ñAÈ !|3rDI‘+ œ “’ÁK’$Þyli%æ‚U lºÌ‚Ï!Î':Øói4 Òò–æ ÚàèÏN§öDU%U™E¹ÖÄÚ°‚;öXÜ
1+Ív1È š eWsµ²-eQž]5 êrQËò{m ð{0EK ÊUü7lcG Æ›X2S (¯5 'b©'<ÛA{ÙìaÈH‹0íGíÌÌW'ðäûšu_¥&E£Žpô)Í Æ¬›6¸ƒ“¯6nõXËÅ¿–†.‘÷Yë¨o'îVwpÊò ;*ä²U{·k ˜{nðÓÇNn³óèÂþ
è±ßÜ<˜”À\
wR`x…Ì'_/Ò§à~d¥§ßƒª‰0ÛpmÉ`Þ‚ÖH¸vÞ• ƒZP "Ñ‚Ú™(âu'ª âi5Ƈ[1z"77Z4ãã!ác2JÇ!…‹mèƒw)°ÅÙ1)Ãy3( $•W¾˜Œ‘"©"–6j™dƒ rydŽg’¹%šzYž”ÀYÙ™œîX&žbÙy ”O˜×%›ºIĘ]`a¢„a^~)š¨£E.º'¤‘ÒÑ¡¤ÇYú Œh:&§öñé}¢VªŽ…¤JשøµR*¢®ÂêS´Ê
g?±V§æ?›ÖÆ ¾Ææá£ý
Kz,€KöŠlj`þÚl°Jå'æ>¨VUµ³ m¨¢vkÁ¬à†{길Bj®†¦‹^¢ì¶b®ïJ{ì¼íÊkï²õæK.¾ü¦öoþœ,³÷+ìÁÆr«ð” 7Œã¾<ñ¹W 1ÇøðÆ¡
q¶‹ü/ÉüšìC ; phylip-3.697/doc/proml.html 0000644 0047320 0047320 00000103143 12406201173 015327 0 ustar joe felsenst_g
Discrete character parsimony algorithm, version 3.69
5 species, 6 sites
Name Sequences
---- ---------
Alpha 110110
Beta ...00.
Gamma .0....
Delta 001001
Epsilon 001...
One most parsimonious tree found:
+Epsilon
+----------------3
+--------2 +-------------------------Delta
| |
| +Gamma
|
1----------------Beta
|
+Alpha
requires a total of 8.000
between and length
------- --- ------
1 2 1.00
2 3 2.00
3 Epsilon 0.00
3 Delta 3.00
2 Gamma 0.00
1 Beta 2.00
1 Alpha 0.00
steps in each site:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0| 1 1 1 2 2 1
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 110110
1 2 yes .0....
2 3 yes 0.1...
3 Epsilon no ......
3 Delta yes ...001
2 Gamma no ......
1 Beta yes ...00.
1 Alpha no ......
version 3.696
Proml -- Protein Maximum Likelihood program
INPUT FORMAT AND OPTIONS
Amino acid sequence Maximum Likelihood method, version 3.696
Settings for this run:
U Search for best tree? Yes
P JTT, PMB or PAM probability model? Jones-Taylor-Thornton
C One category of sites? Yes
R Rate variation among sites? constant rate of change
W Sites weighted? No
S Speedier but rougher analysis? Yes
G Global rearrangements? No
J Randomize input order of sequences? No. Use input order
O Outgroup root? No, use as outgroup species 1
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Write out trees onto tree file? Yes
5 Reconstruct hypothetical sequences? No
Y to accept these or type the letter for one to change
122231111122411155
1155333333444
OUTPUT FORMAT
PROGRAM CONSTANTS
PAST AND FUTURE OF THE PROGRAM
TEST DATA SET
5 13
Alpha AACGTGGCCAAAT
Beta AAGGTCGCCAAAC
Gamma CATTTCGTCACAA
Delta GGTATTTCGGCCT
Epsilon GGGATCTCGGCCC
CONTENTS OF OUTPUT FILE (with all numerical options on)
phylip-3.697/doc/promlk.html 0000644 0047320 0047320 00000075335 12406201173 015515 0 ustar joe felsenst_g
Amino acid sequence Maximum Likelihood method, version 3.69
5 species, 13 sites
Site categories are:
1111112222 222
Sites are weighted as follows:
01111 11111 110
Jones-Taylor-Thornton model of amino acid change
Name Sequences
---- ---------
Alpha AACGTGGCCA AAT
Beta ..G..C.... ..C
Gamma C.TT.C.T.. C.A
Delta GGTA.TT.GG CC.
Epsilon GGGA.CT.GG CCC
Discrete approximation to gamma distributed rates
Coefficient of variation of rates = 1.000000 (alpha = 1.000000)
States in HMM Rate of change Probability
1 0.264 0.522
2 1.413 0.399
3 3.596 0.076
4 7.086 0.0036
5 12.641 0.000023
Expected length of a patch of sites having the same rate = 1.500
Site category Rate of change
1 1.000
2 2.000
+Beta
|
| +Epsilon
| +-------------------------------------3
1-------2 +--------Delta
| |
| +----------Gamma
|
+------Alpha
remember: this is an unrooted tree!
Ln Likelihood = -104.53314
Between And Length Approx. Confidence Limits
------- --- ------ ------- ---------- ------
1 Alpha 0.46548 ( zero, 1.16234) **
1 Beta 0.00010 ( zero, 0.56371)
1 2 0.53585 ( zero, 1.53611) *
2 3 2.52202 ( zero, 5.51952) **
3 Epsilon 0.00010 ( zero, 0.70102)
3 Delta 0.56179 ( zero, 1.37921) **
2 Gamma 0.72465 ( zero, 1.87900) **
* = significantly positive, P < 0.05
** = significantly positive, P < 0.01
Combination of categories that contributes the most to the likelihood:
1122111111 111
Most probable category at each site if > 0.95 probability ("." otherwise)
....1....1 1..
Probable sequences at interior nodes:
node Reconstructed sequence (caps if > 0.95)
1 .AGGTCGCCA AA.
Beta AAGGTCGCCA AAC
2 .AggTCGCCA CA.
3 .GGATCTCGG CC.
Epsilon GGGATCTCGG CCC
Delta GGTATTTCGG CCT
Gamma CATTTCGTCA CAA
Alpha AACGTGGCCA AAT
version 3.696
Promlk -- Protein maximum likelihood program
with molecular clock
INPUT FORMAT AND OPTIONS
Amino acid sequence
Maximum Likelihood method with molecular clock, version 3.69
Settings for this run:
U Search for best tree? Yes
P JTT, PMB or PAM probability model? Jones-Taylor-Thornton
C One category of substitution rates? Yes
R Rate variation among sites? constant rate of change
G Global rearrangements? No
W Sites weighted? No
J Randomize input order of sequences? No. Use input order
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Write out trees onto tree file? Yes
5 Reconstruct hypothetical sequences? No
Are these settings correct? (type Y or the letter for one to change)
122231111122411155
1155333333444
OUTPUT FORMAT
PROGRAM CONSTANTS
PAST AND FUTURE OF THE PROGRAM
TEST DATA SET
5 13
Alpha AACGTGGCCAAAT
Beta AAGGTCGCCAAAC
Gamma CATTTCGTCACAA
Delta GGTATTTCGGCCT
Epsilon GGGATCTCGGCCC
CONTENTS OF OUTPUT FILE (with all numerical options on)
phylip-3.697/doc/protdist.html 0000644 0047320 0047320 00000052445 12406201173 016056 0 ustar joe felsenst_g
Amino acid sequence Maximum Likelihood method, version 3.69
5 species, 13 sites
Site categories are:
1111112222 222
Sites are weighted as follows:
01111 11111 110
Jones-Taylor-Thornton model of amino acid change
Name Sequences
---- ---------
Alpha AACGTGGCCA AAT
Beta ..G..C.... ..C
Gamma C.TT.C.T.. C.A
Delta GGTA.TT.GG CC.
Epsilon GGGA.CT.GG CCC
Discrete approximation to gamma distributed rates
Coefficient of variation of rates = 1.000000 (alpha = 1.000000)
States in HMM Rate of change Probability
1 0.264 0.522
2 1.413 0.399
3 3.596 0.076
4 7.086 0.0036
5 12.641 0.000023
Expected length of a patch of sites having the same rate = 1.500
Site category Rate of change
1 1.000
2 2.000
+Beta
|
| +Epsilon
| +-------------------------------------3
1-------2 +--------Delta
| |
| +----------Gamma
|
+------Alpha
remember: this is an unrooted tree!
Ln Likelihood = -104.53314
Between And Length Approx. Confidence Limits
------- --- ------ ------- ---------- ------
1 Alpha 0.46548 ( zero, 1.16234) **
1 Beta 0.00010 ( zero, 0.56371)
1 2 0.53585 ( zero, 1.53611) *
2 3 2.52202 ( zero, 5.51952) **
3 Epsilon 0.00010 ( zero, 0.70102)
3 Delta 0.56179 ( zero, 1.37921) **
2 Gamma 0.72465 ( zero, 1.87900) **
* = significantly positive, P < 0.05
** = significantly positive, P < 0.01
Combination of categories that contributes the most to the likelihood:
1122111111 111
Most probable category at each site if > 0.95 probability ("." otherwise)
....1....1 1..
Probable sequences at interior nodes:
node Reconstructed sequence (caps if > 0.95)
1 .AGGTCGCCA AA.
Beta AAGGTCGCCA AAC
2 .AggTCGCCA CA.
3 .GGATCTCGG CC.
Epsilon GGGATCTCGG CCC
Delta GGTATTTCGG CCT
Gamma CATTTCGTCA CAA
Alpha AACGTGGCCA AAT
version 3.696
Protdist -- Program to compute distance matrix
from protein sequences
To: A G C T
---------------------------------
A | 1-a-2b a b b
From: G | a 1-a-2b b b
C | b b 1-a-2b a
T | b b a 1-a-2b
INPUT FORMAT AND OPTIONS
Protein distance algorithm, version 3.69
Settings for this run:
P Use JTT, PMB, PAM, Kimura, categories model? Jones-Taylor-Thornton matrix
G Gamma distribution of rates among positions? No
C One category of substitution rates? Yes
W Use weights for positions? No
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
Are these settings correct? (type Y or the letter for one to change)
122231111122411155
1155333333444
OUTPUT FORMAT
PROGRAM CONSTANTS
TEST DATA SET
5 13
Alpha AACGTGGCCACAT
Beta AAGGTCGCCACAC
Gamma CAGTTCGCCACAA
Delta GAGATTTCCGCCT
Epsilon GAGATCTCCGCCC
CONTENTS OF OUTPUT FILE (with all numerical options on )
phylip-3.697/doc/protpars.html 0000644 0047320 0047320 00000034720 12406201173 016054 0 ustar joe felsenst_g
Jones-Taylor-Thornton model distance
Name Sequences
---- ---------
Alpha AACGTGGCCA CAT
Beta ..G..C.... ..C
Gamma C.GT.C.... ..A
Delta G.GA.TT..G .C.
Epsilon G.GA.CT..G .CC
Alpha 0.000000 0.331834 0.628142 1.036660 1.365098
Beta 0.331834 0.000000 0.377406 1.102689 0.682218
Gamma 0.628142 0.377406 0.000000 0.979550 0.866781
Delta 1.036660 1.102689 0.979550 0.000000 0.227515
Epsilon 1.365098 0.682218 0.866781 0.227515 0.000000
version 3.696
Protpars -- Protein Sequence Parsimony Method
HIINMA*????
HIPNMGVWABT
Protein parsimony algorithm, version 3.69
Setting for this run:
U Search for best tree? Yes
J Randomize input order of sequences? No. Use input order
O Outgroup root? No, use as outgroup species 1
T Use Threshold parsimony? No, use ordinary parsimony
C Use which genetic code? Universal
W Sites weighted? No
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Print out steps in each site No
5 Print sequences at all nodes of tree No
6 Write out trees onto tree file? Yes
Are these settings correct? (type Y or the letter for one to change)
TEST DATA SET
5 10
Alpha ABCDEFGHIK
Beta AB--EFGHIK
Gamma ?BCDSFG*??
Delta CIKDEFGHIK
Epsilon DIKDEFGHIK
CONTENTS OF OUTPUT FILE (with all numerical options on)
phylip-3.697/doc/restdist.html 0000644 0047320 0047320 00000042403 12406201173 016040 0 ustar joe felsenst_g
Protein parsimony algorithm, version 3.69
5 species, 10 sites
Name Sequences
---- ---------
Alpha ABCDEFGHIK
Beta ..--......
Gamma ?...S..*??
Delta CIK.......
Epsilon DIK.......
3 trees in all found
+--------Gamma
!
+--2 +--Epsilon
! ! +--4
! +--3 +--Delta
1 !
! +-----Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 16.000
steps in each position:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 3 1 5 3 2 0 0 2 0
10! 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 ANCDEFGHIK
1 2 no ..........
2 Gamma yes ?B..S..*??
2 3 yes ..?.......
3 4 yes ?IK.......
4 Epsilon maybe D.........
4 Delta yes C.........
3 Beta yes .B--......
1 Alpha maybe .B........
+--Epsilon
+--4
+--3 +--Delta
! !
+--2 +-----Gamma
! !
1 +--------Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 16.000
steps in each position:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 3 1 5 3 2 0 0 2 0
10! 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 ANCDEFGHIK
1 2 no ..........
2 3 maybe ?.........
3 4 yes .IK.......
4 Epsilon maybe D.........
4 Delta yes C.........
3 Gamma yes ?B..S..*??
2 Beta yes .B--......
1 Alpha maybe .B........
+--Epsilon
+-----4
! +--Delta
+--3
! ! +--Gamma
1 +-----2
! +--Beta
!
+-----------Alpha
remember: this is an unrooted tree!
requires a total of 16.000
steps in each position:
0 1 2 3 4 5 6 7 8 9
*-----------------------------------------
0! 3 1 5 3 2 0 0 2 0
10! 0
From To Any Steps? State at upper node
( . means same as in the node below it on tree)
1 ANCDEFGHIK
1 3 no ..........
3 4 yes ?IK.......
4 Epsilon maybe D.........
4 Delta yes C.........
3 2 no ..........
2 Gamma yes ?B..S..*??
2 Beta yes .B--......
1 Alpha maybe .B........
version 3.696
Restdist -- Program to compute distance matrix
from restriction sites or fragmentsRestriction Sites Distance
n++
f = ---------------------
n++ + 1/2 (n+- + n-+)
where n++ is the number of sites contained in both species,
n+- is the number of sites contained in the first of the
two species but not in the second, and n-+ is the
number of sites contained in the second of the two species but not in the
first. This is the fraction of sites that are present in one species which are
present in both. Since the number of sites present in the two species will
often differ, the denominator is the average of the number of sites found in
the two species.
Qs,
where Q is the probability that a nucleotide has no net change as one
goes from the one species to the other. It may have changed in between; we
are interested in the probability that that nucleotide site is the same base
in both species, irrespective of what has happened in between.
Restriction Fragments Distance
Q2s
f = --------
2 - Qs
(The details of the derivation are given in my book
Inferring Phylogenies (Felsenstein, 2004).)
Given the observed fraction of restriction sites retained, f,
we can solve a quadratic equation from the above expression for
Qs. That makes it easy to obtain a value of Q,
and the branch length can then be estimated by adjusting it so the
probability of a base not changing is equal to that value.
g = [ f (3 - 2g) ]1/4
and then the distance is given by
d = - (2/r) loge(g)
where r is the length of the restriction site.
A Comment About RAPDs and AFLPs
INPUT FORMAT AND OPTIONS
10 35 4
Restriction site or fragment distances, version 3.69
Settings for this run:
R Restriction sites or fragments? Sites
N Original or modified Nei/Li model? Modified
G Gamma distribution of rates among sites? No
T Transition/transversion ratio? 2.000000
S Site length? 6.0
L Form of distance matrix? Square
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run? No
2 Print indications of progress of run? Yes
Y to accept these or type the letter for one to change
OUTPUT FORMAT
PROGRAM CONSTANTS
TEST DATA SET
5 13 2
Alpha ++-+-++--+++-
Beta ++++--+--+++-
Gamma -+--+-++-+-++
Delta ++-+----++---
Epsilon ++++----++---
CONTENTS OF OUTPUT FILE (with all numerical options on)
phylip-3.697/doc/restml.html 0000644 0047320 0047320 00000047100 12406201173 015504 0 ustar joe felsenst_g
5 Species, 13 Sites
Name Sites
---- -----
Alpha ++-+-++--+ ++-
Beta ++++--+--+ ++-
Gamma -+--+-++-+ -++
Delta ++-+----++ ---
Epsilon ++++----++ ---
Alpha 0.000000 0.022368 0.107681 0.082634 0.095581
Beta 0.022368 0.000000 0.107681 0.082634 0.056895
Gamma 0.107681 0.107681 0.000000 0.192466 0.207319
Delta 0.082634 0.082634 0.192466 0.000000 0.015949
Epsilon 0.095581 0.056895 0.207319 0.015949 0.000000
version 3.696
Restml -- Restriction sites Maximum Likelihood program
INPUT FORMAT AND OPTIONS
10 35 4
Restriction site Maximum Likelihood method, version 3.69
Settings for this run:
U Search for best tree? Yes
A Are all sites detected? No
S Speedier but rougher analysis? Yes
G Global rearrangements? No
J Randomize input order of sequences? No. Use input order
L Site length? 6
O Outgroup root? No, use as outgroup species 1
M Analyze multiple data sets? No
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
3 Print out tree Yes
4 Write out trees onto tree file? Yes
Y to accept these or type the letter for one to change
OUTPUT FORMAT
THE ALGORITHM
PROGRAM CONSTANTS
PAST AND FUTURE OF THE PROGRAM
TEST DATA SET
5 13 2
Alpha ++-+-++--+++-
Beta ++++--+--+++-
Gamma -+--+-++-+-++
Delta ++-+----++---
Epsilon ++++----++---
CONTENTS OF OUTPUT FILE (if all numerical options are on)
phylip-3.697/doc/retree.html 0000644 0047320 0047320 00000051157 12406201173 015473 0 ustar joe felsenst_g
Restriction site Maximum Likelihood method, version 3.69
5 Species, 13 Sites, 2 Enzymes
Recognition sequences all 6 bases long
Sites absent from all species are assumed to have been omitted
Name Sites
---- -----
Alpha ++-+-++--+ ++-
Beta ++++--+--+ ++-
Gamma -+--+-++-+ -++
Delta ++-+----++ ---
Epsilon ++++----++ ---
+Beta
|
| +Epsilon
| +---3
2--1 +Delta
| |
| +-----Gamma
|
+Alpha
remember: this is an unrooted tree!
Ln Likelihood = -40.49476
Between And Length Approx. Confidence Limits
------- --- ------ ------- ---------- ------
2 Beta 0.00100 ( zero, infinity)
2 1 0.00010 ( zero, 0.04003)
1 3 0.05898 ( zero, 0.12733) **
3 Epsilon 0.00100 ( zero, infinity)
3 Delta 0.01458 ( zero, 0.04490) **
1 Gamma 0.11470 ( 0.01732, 0.22664) **
2 Alpha 0.02473 ( zero, 0.06180) **
* = significantly positive, P < 0.05
** = significantly positive, P < 0.01
version 3.696
Retree -- Interactive Tree Rearrangement
Tree Rearrangement, version 3.69
Settings for this run:
U Initial tree (arbitrary, user, specify)? User tree from tree file
N Format to write out trees (PHYLIP, Nexus, XML)? PHYLIP
0 Graphics type (IBM PC, ANSI)? ANSI
W Width of terminal screen, of plotting area? 80, 80
L Number of lines on screen? 24
Are these settings correct? (type Y or the letter for one to change)
<CLADE LENGTH="0.09362"> ... </CLADE>
<phylogenies>
<phylogeny>
<clade>
<clade length="0.87231"><name>Mouse</name></clade>
<clade length="0.49807"><name>Bovine</name></clade>
<clade length="0.39538">
<clade length="0.25930"><name>Gibbon</name></clade>
<clade length="0.10815">
<clade length="0.24166"><name>Orang</name></clade>
<clade length="0.04405">
<clade length="0.12322"><name>Gorilla</name></clade>
<clade length="0.06026">
<clade length="0.13846"><name>Chimp</name></clade>
<clade length="0.0857"><name>Human</name></clade>
</clade>
</clade>
</clade>
</clade>
</clade>
</phylogeny>
</phylogenies>
Taxonomic Markup Language
http://www.albany.edu/~gilmr/pubxml/.
and preprint at
xml.coverpages.org/gilmour-TML.pdf
published in the paper by
Ron Gilmour (2000).
Andrew Rambaut's
BEAST XML phylogeny format
See page 9 of PDF of BEAST manual at
http://evolve.zoo.ox.ac.uk/beast/
An XML format for phylogenies is
briefly described there.
treeml
http://www.nomencurator.org/InfoVis2003/download/treeml.dtd
(see also example: )
http://www.cs.umd.edu/hcil/iv03contest/datasets/treeml-sample.xml
Jean-Daniel Fekete's DTD
for a tree XML file
,>>1:Human
,>22
,>21 `>>2:Chimp
! !
,>20 `>>>>>3:Gorilla
! !
,>>>>>>>>>>19 `>>>>>>>>4:Orang
! !
,>18 `>>>>>>>>>>>5:Gibbon
! !
! ! ,>>>>>>>>6:Barbary Ma
! `>>>>>>>>>>>>>23
! ! ,>>>>>7:Crab-e. Ma
,>>>>>>>17 `>24
! ! ! ,>>8:Rhesus Mac
! ! `>25
! ! `>>9:Jpn Macaq
,>16 !
! ! `>>>>>>>>>>>>>>>>>>>>>>>>>10:Squir. Mon
! !
! ! ,>11:Tarsier
** 7 lines below screen **
NEXT? (Options: R . U W O T F D B N H J K L C + ? X Q) (? for Help)
TEST INPUT TREE FILE
phylip-3.697/doc/seqboot.html 0000644 0047320 0047320 00000067056 12406201173 015666 0 ustar joe felsenst_g
((((((((Human,Chimp),Gorilla),Orang),Gibbon),(Barbary_Ma,(Crab-e._Ma,
(Rhesus_Mac,Jpn_Macaq)))),Squir._Mon),((Tarsier,Lemur),Bovine)),Mouse);
version 3.696
Seqboot -- Bootstrap, Jackknife, or Permutation Resampling
of Molecular Sequence, Restriction Site,
Gene Frequency or Character Data
Andrew Rambaut's
BEAST XML format
http://evolve.zoo.ox.ac.uk/beast/introXML.html
and
http://evolve.zoo.ox.ac.uk/beast/reference/index.html
A format for alignments. There
is also a format for phylogenies
described there.
MSAML
http://xml.coverpages.org/msaml-desc-dec.html
Defined by Paul Gordon of
University of Calgary. See his
big list
of molecular biology
XML projects.
BSML
http://www.bsml.org/resources/default.asp
Bioinformatic Sequence
Markup Language
includes a multiple sequence
alignment XML format
Bootstrapping algorithm, version 3.69
Settings for this run:
D Sequence, Morph, Rest., Gene Freqs? Molecular sequences
J Bootstrap, Jackknife, Permute, Rewrite? Bootstrap
% Regular or altered sampling fraction? regular
B Block size for block-bootstrapping? 1 (regular bootstrap)
R How many replicates? 100
W Read weights of characters? No
C Read categories of sites? No
S Write out data sets or just weights? Data sets
I Input sequences interleaved? Yes
0 Terminal type (IBM PC, ANSI, none)? ANSI
1 Print out the data at start of run No
2 Print indications of progress of run Yes
Y to accept these or type the letter for one to change
<alignment>
<sequence type="dna">
<name>Archaeopt</name>
<data>CGATGCTTAC CGC</data>
</sequence>
<sequence type="dna">
<name>Hesperorni</name>
<data>CGTTACTCGT TGT</data>
</sequence>
<sequence type="dna">
<name>Baluchithe</name>
<data>TAATGTTAAT TGT</data>
</sequence>
<sequence type="dna">
<name>B. virgini</name>
<data>TAATGTTCGT TGT</data>
</sequence>
<sequence type="dna">
<name>Brontosaur</name>
<data>CAAAACCCAT CAT</data>
</sequence>
<sequence type="dna">
<name>B.subtilis</name>
<data>GGCAGCCAAT CAC</data>
</sequence>
</alignment>
122231111122411155
1155333333444
Saving time by combining results of separate runs
Input File
Output
Size and Speed
TEST DATA SET
5 6
Alpha AACAAC
Beta AACCCC
Gamma ACCAAC
Delta CCACCA
Epsilon CCAAAC
CONTENTS OF OUTPUT FILE
5 6
Alpha ACAAAC
Beta ACCCCC
Gamma ACAAAC
Delta CACCCA
Epsilon CAAAAC
5 6
Alpha AAAACC
Beta AACCCC
Gamma CCAACC
Delta CCCCAA
Epsilon CCAACC
5 6
Alpha ACAAAC
Beta ACCCCC
Gamma CCAAAC
Delta CACCCA
Epsilon CAAAAC
5 6
Alpha ACCAAA
Beta ACCCCC
Gamma ACCAAA
Delta CAACCC
Epsilon CAAAAA
5 6
Alpha ACAAAC
Beta ACCCCC
Gamma ACAAAC
Delta CACCCA
Epsilon CAAAAC
5 6
Alpha AAAACA
Beta AAAACC
Gamma AAACCA
Delta CCCCAC
Epsilon CCCCAA
5 6
Alpha AAACCC
Beta CCCCCC
Gamma AAACCC
Delta CCCAAA
Epsilon AAACCC
5 6
Alpha AAAACC
Beta AACCCC
Gamma AAAACC
Delta CCCCAA
Epsilon CCAACC
5 6
Alpha AAAAAC
Beta AACCCC
Gamma CCAAAC
Delta CCCCCA
Epsilon CCAAAC
5 6
Alpha AACCAC
Beta AACCCC
Gamma AACCAC
Delta CCAACA
Epsilon CCAAAC
version 3.696
Molecular Sequence Programs
INTERLEAVED AND SEQUENTIAL FORMATS
5 42
Turkey AAGCTNGGGC ATTTCAGGGT
Salmo gairAAGCCTTGGC AGTGCAGGGT
H. SapiensACCGGTTGGC CGTTCAGGGT
Chimp AAACCCTTGC CGTTACGCTT
Gorilla AAACCCTTGC CGGTACGCTT
GAGCCCGGGC AATACAGGGT AT
GAGCCGTGGC CGGGCACGGT AT
ACAGGTTGGC CGTTCAGGGT AA
AAACCGAGGC CGGGACACTC AT
AAACCATTGC CGGTACGCTT AA
5 42
Turkey AAGCTNGGGC ATTTCAGGGT
GAGCCCGGGC AATACAGGGT AT
Salmo gairAAGCCTTGGC AGTGCAGGGT
GAGCCGTGGC CGGGCACGGT AT
H. SapiensACCGGTTGGC CGTTCAGGGT
ACAGGTTGGC CGTTCAGGGT AA
Chimp AAACCCTTGC CGTTACGCTT
AAACCGAGGC CGGGACACTC AT
Gorilla AAACCCTTGC CGGTACGCTT
AAACCATTGC CGGTACGCTT AA
INPUT FOR THE DNA SEQUENCE PROGRAMS
Symbol Meaning A Adenine G Guanine C Cytosine T Thymine U Uracil Y pYrimidine (C or T) R puRine (A or G) W "Weak" (A or T) S "Strong" (C or G) K "Keto" (T or G) M "aMino" (C or A) B not A (C or G or T) D not C (A or G or T) H not G (A or C or T) V not T (A or C or G) X,N,? unknown (A or C or G or T) O deletion - deletion INPUT FOR THE PROTEIN SEQUENCE PROGRAMS
Symbol Stands for A ala B asx C cys D asp E glu F phe G gly H his I ileu J (not used) K lys L leu M met N asn O (not used) P pro Q gln R arg S ser T thr U (not used) V val W trp X unknown amino acid Y tyr Z glx * nonsense (stop) ? unknown amino acid or deletion - deletion
HIINMA*????
HIPNMGVWABT
since after the stop codon we do not definitely know that
there has been a deletion, and do not know what amino acid would
have been there. If DNA studies tell us that there is
DNA sequence in that region, then we could use "X" rather than "?". Note
that "X" means an unknown amino acid, but definitely an amino acid,
while "?" could mean either that or a deletion. Otherwise one will usually
want to use "?" after a stop codon, if one does not know what amino acid is
there. If the DNA sequence has been observed there, one probably ought to
resist putting in the amino acids that this DNA would code for, and one should
use "X" instead, because under the assumptions implicit in either the
parsimony or the distance
methods, changes to any noncoding sequence are much easier than
changes in a coding region that change the amino acid
Amino acid One-letter code ala A arg R asn N asp D asx B cys C gln Q glu E gly G glx Z his H ileu I leu L lys K met M phe F pro P ser S thr T trp W tyr Y val V deletion - nonsense (stop) * unknown amino acid X unknown (incl. deletion) ? THE OPTIONS
version 3.696
Treedist -- distances between trees
((A,C),(D,(B,E)))
has two internal branches. One induces the partition {A, C | B, D, E}
and the other induces the partition {A, C, D | B, E}. A different tree
with the same set of species,
(((A,D),C),(B,E))
has internal branches that correspond to the two partitions {A, C, D | B, E}
and {A, D | B, C, E}. Note that the other branches, all of which are
external branches, induce partitions that separate one species from all the
others. Thus there are 5 partitions like this: {C | A, B, D, E} on each
of these trees. These are always present on all trees, provided that each
tree has each species at the end of its own branch.
(((A:0.1,D:0.25):0.05,C:0.01):0.2,(B:0.3,E:0.8):0.2)
the list of partitions and their branch lengths is:
{A | B, C, D, E} 0.1 {D | A, B, C, E} 0.25 {A, D | B, C, E} 0.05 {C | A, B, D, E} 0.01 {A, D, C | B, E} 0.4 {B | A, C, D, E} 0.3 {E | A, B, C, D} 0.8 INPUT AND OPTIONS
Tree distance program, version 3.69
Settings for this run:
D Distance Type: Branch Score Distance
O Outgroup root: No, use as outgroup species 1
R Trees to be treated as Rooted: No
T Terminal type (IBM PC, ANSI, none): ANSI
1 Print indications of progress of run: Yes
2 Tree distance submenu: Distance between adjacent pairs
Are these settings correct? (type Y or the letter for one to change)
Tree Pairing Submenu:
A Distances between adjacent pairs in tree file.
P Distances between all possible pairs in tree file.
C Distances between corresponding pairs in one tree file and another.
L Distances between all pairs in one tree file and another.
Choose one: (A,P,C,L)
OUTPUT
Distances output options:
F Full matrix.
V One pair per line, verbose.
S One pair per line, sparse.
Choose one: (F,V,S)
Tree distance program, version 3.69
Symmetric differences between all pairs of trees in tree file:
1 2 3 4 5 6 7 8 9 10
\------------------------------------------------------------
1 | 0 4 2 10 10 10 10 10 10 10
2 | 4 0 2 10 8 10 8 10 8 10
3 | 2 2 0 10 10 10 10 10 10 10
4 | 10 10 10 0 2 2 4 2 4 0
5 | 10 8 10 2 0 4 2 4 2 2
6 | 10 10 10 2 4 0 2 2 4 2
7 | 10 8 10 4 2 2 0 4 2 4
8 | 10 10 10 2 4 2 4 0 2 2
9 | 10 8 10 4 2 4 2 2 0 4
10 | 10 10 10 0 2 2 4 2 4 0
11 | 2 2 0 10 10 10 10 10 10 10
12 | 10 10 10 2 4 2 4 0 2 2
11 12
\------------
1 | 2 10
2 | 2 10
3 | 0 10
4 | 10 2
5 | 10 4
6 | 10 2
7 | 10 4
8 | 10 0
9 | 10 2
10 | 10 2
11 | 0 10
12 | 10 0
Tree distance program, version 3.69
Symmetric differences between adjacent pairs of trees:
Trees 1 and 2: 4
Trees 3 and 4: 10
Trees 5 and 6: 4
Trees 7 and 8: 4
Trees 9 and 10: 4
Trees 11 and 12: 10
1 2 4
3 4 10
5 6 4
7 8 4
9 10 4
11 12 10
CREDITS AND FUTURE
TEST DATA SET 1
(A,(B,(H,(D,(J,(((G,E),(F,I)),C))))));
(A,(B,(D,((J,H),(((G,E),(F,I)),C)))));
(A,(B,(D,(H,(J,(((G,E),(F,I)),C))))));
(A,(B,(E,(G,((F,I),((J,(H,D)),C))))));
(A,(B,(E,(G,((F,I),(((J,H),D),C))))));
(A,(B,(E,((F,I),(G,((J,(H,D)),C))))));
(A,(B,(E,((F,I),(G,(((J,H),D),C))))));
(A,(B,(E,((G,(F,I)),((J,(H,D)),C)))));
(A,(B,(E,((G,(F,I)),(((J,H),D),C)))));
(A,(B,(E,(G,((F,I),((J,(H,D)),C))))));
(A,(B,(D,(H,(J,(((G,E),(F,I)),C))))));
(A,(B,(E,((G,(F,I)),((J,(H,D)),C)))));
TEST DATA SET 2
(A:0.1,(B:0.1,(H:0.1,(D:0.1,(J:0.1,(((G:0.1,E:0.1):0.1,(F:0.1,I:0.1):0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(D:0.1,((J:0.1,H:0.1):0.1,(((G:0.1,E:0.1):0.1,
(F:0.1,I:0.1):0.1):0.1,C:0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(D:0.1,(H:0.1,(J:0.1,(((G:0.1,E:0.1):0.1,(F:0.1,I:0.1):0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,(G:0.1,((F:0.1,I:0.1):0.1,((J:0.1,(H:0.1,D:0.1):0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,(G:0.1,((F:0.1,I:0.1):0.1,(((J:0.1,H:0.1):0.1,D:0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,((F:0.1,I:0.1):0.1,(G:0.1,((J:0.1,(H:0.1,D:0.1):0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,((F:0.1,I:0.1):0.1,(G:0.1,(((J:0.1,H:0.1):0.1,D:0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,((G:0.1,(F:0.1,I:0.1):0.1):0.1,((J:0.1,(H:0.1,
D:0.1):0.1):0.1,C:0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,((G:0.1,(F:0.1,I:0.1):0.1):0.1,(((J:0.1,H:0.1):0.1,
D:0.1):0.1,C:0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,(G:0.1,((F:0.1,I:0.1):0.1,((J:0.1,(H:0.1,D:0.1):0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(D:0.1,(H:0.1,(J:0.1,(((G:0.1,E:0.1):0.1,(F:0.1,I:0.1):0.1):0.1,
C:0.1):0.1):0.1):0.1):0.1):0.1);
(A:0.1,(B:0.1,(E:0.1,((G:0.1,(F:0.1,I:0.1):0.1):0.1,((J:0.1,(H:0.1,
D:0.1):0.1):0.1,C:0.1):0.1):0.1):0.1):0.1);
TEST SET 2 OUTPUT
Tree distance program, version 3.69
Branch score distances between all pairs of trees in tree file:
1 2 3 4 5 6 7
\------------------------------------------------------------------------------------
1 | 0 0.2 0.141421 0.316228 0.316228 0.316228 0.316228
2 | 0.2 0 0.141421 0.316228 0.282843 0.316228 0.282843
3 | 0.141421 0.141421 0 0.316228 0.316228 0.316228 0.316228
4 | 0.316228 0.316228 0.316228 0 0.141421 0.141421 0.2
5 | 0.316228 0.282843 0.316228 0.141421 0 0.2 0.141421
6 | 0.316228 0.316228 0.316228 0.141421 0.2 0 0.141421
7 | 0.316228 0.282843 0.316228 0.2 0.141421 0.141421 0
8 | 0.316228 0.316228 0.316228 0.141421 0.2 0.141421 0.2
9 | 0.316228 0.282843 0.316228 0.2 0.141421 0.2 0.141421
10 | 0.316228 0.316228 0.316228 0 0.141421 0.141421 0.2
11 | 0.141421 0.141421 0 0.316228 0.316228 0.316228 0.316228
12 | 0.316228 0.316228 0.316228 0.141421 0.2 0.141421 0.2
8 9 10 11 12
\------------------------------------------------------------
1 | 0.316228 0.316228 0.316228 0.141421 0.316228
2 | 0.316228 0.282843 0.316228 0.141421 0.316228
3 | 0.316228 0.316228 0.316228 0 0.316228
4 | 0.141421 0.2 0 0.316228 0.141421
5 | 0.2 0.141421 0.141421 0.316228 0.2
6 | 0.141421 0.2 0.141421 0.316228 0.141421
7 | 0.2 0.141421 0.2 0.316228 0.2
8 | 0 0.141421 0.141421 0.316228 0
9 | 0.141421 0 0.2 0.316228 0.141421
10 | 0.141421 0.2 0 0.316228 0.141421
11 | 0.316228 0.316228 0.316228 0 0.316228
12 | 0 0.141421 0.141421 0.316228 0