Introduction
Theory, Examples & Exercises

 Constrained ordination



The ecological resemblance including similarities and distances between samples, is the basic tool how to handle multivariate ecological data. Two samples sharing the same species in the same abundances have the highest similarity (and lowest distance), and the similarity decreases (and distance increases) with the differences in their species composition. All cluster and ordination methods operate with similarity or distance between samples. Even PCA and CA, even if not said explicitly, are based on Euclidean and chisquare distances, respectively.
Intuitively, one thinks about similarity among objects  the more are two objects similar in terms of their properties, the higher is their similarity. In the case of species composition data, the similarity is calculated using similarity indices, ranging from 0 (the samples do not share any species) to 1 (samples have identical species composition). Ordination techniques are usually based on distances, because they need to localize the samples in a multidimensional space; clustering methods could usually handle both similarities or distances. Distances are of two types, either dissimilarities, converted from analogous similarity indices, or specific distance measures, such as Euclidean, which doesn't have a counterpart in any similarity index. While all similarity indices can be converted into distances, not all distances could be converted into similarities (as is true e.g. for Euclidean distance).
There is a number of measures of similarities or distances (Legendre & Legendre 2012 list around 30 of them). The first decision one has to make is whether the aim is R or Qmode analysis (Rmode focuses on differences among species, Qmode on differences among samples), since some of the measures differ between both modes (e.g. Pearson's r correlation coefficient makes sense for association between species (Rmode), but not for association between samples (Qmode); in contrast, e.g. Sørensen index can be used in both Q and Rmode analysis, called Dice index in Rmode analysis). Further, if focusing on differences between samples (Qmode), the most relevant measures in ecology are asymmetric indices ignoring double zeros (more about doublezero problem below). Then, it also depends whether the data are qualitative (i.e. binary, presenceabsence) or quantitative (species abundances). In the case of distance indices, an important criterium is whether they are metric (they can be displayed in Euclidean space) or not, since this influences the choice of the index for some ordination or clustering methods.
Legendre & Legendre (2012) offers a kind of “key” how to select an appropriate measure for given data and problem (Tables 7.47.6). Generally, as a rule of thumb, BrayCurtis and Hellinger distances are better choices than Euclidean or Chisquare distances.
“Double zero” is a situation when certain species is missing in both compared community samples for which similarity/distance is calculated. Species missing simultaneously in two samples can mean the following: (1) samples are located outside of the species ecological niche, but one cannot say whether both samples are on the same side of the ecological gradient (i.e. they can be rather ecologically similar, samples A and B on Fig. 1) or they are on the opposite sides (and hence very different, samples A and C). Alternatively, (2) samples are located inside species ecological niche (samples D and E), but the species in given samples does not occur, since it didn’t get there (dispersal limitation), or it was present, but overlooked and not sampled (sampling bias). In both cases, the double zero represents missing information, which cannot offer an insight into the ecology of compared samples.
Similarity indices differ in a way how they approach the doublezero problem. Symmetrical indices treat double zero (00) in the same way as double presences, i.e. as a reason to consider samples similar. This is not usually meaningful for species composition data (as explained above), but could be meaningful e.g. for multivariate data containing chemical measurement (the fact that e.g. heavy metals are missing in both samples could really indicate similarity between both samples). Asymmetrical indices ignore double zero, and focus only on double presences when evaluating the similarity of samples; these indices are usually more meaningful for species composition data.
Fig. 2 shows an ecological example of double zero problem. Samples 1 to 3 are sorted according to the wetness of their habitat – sample 1 is the wettest and sample 3 is the driest. In samples 1 and 3, no mesic species occur, since sample 1 is too wet and sample 3 too dry  these is the double zero. The fact that the mesic species is missing does not say anything about ecological similarity or difference between both samples; simply there is no information, and it is better to ignore it. In the case of symmetrical indices of similarity, the absence of mesic species in sample 1 and sample 3 (00, double zero) will increase similarity of sample 1 and 2; in asymmetrical indices, double zeros will be ignored and only presences (11, 10, 01) will be considered.
Table 1 summarizes categories of similarity indices. Symmetric indices, i.e. those which consider double zeros as relevant, are not further treated here since they are not useful for analysis of ecological data (although they may be useful e.g. for analysis of environmental variables if there are binary). Here we will consider only asymmetric similarity indices, i.e. those ignoring double zeros. These split into two types according to the data which they are using: qualitative (binary) indices, applied on presenceabsence data, and quantitative indices, applied on raw (or transformed) species abundances. Note that some of the indices have also multisample alternatives (i.e. they could be calculated on more than two samples), which could be used for calculating beta diversity.
Qualitative (binary) asymmetrical similarity indices use information about the number of species shared by both samples, and numbers of species which are occurring in the first or the second sample only (see the schema at Table 2).
Jaccard similarity:
Sørensen similarity:
Simpson similarity:
Jaccard similarity index divides the number of species shared by both samples (fraction a) by the sum of all species occurring in both samples (a+b+c, where b and c are numbers of species occurring only in the first and only in the second sample, respectively). Sørensen similarity index considers the number of species shared among both samples as more important, so it counts it twice. Simpson similarity index is useful in a case that compared samples largely differ in species richness (i.e. one sample has considerably more species than the other). If Jaccard or Sørensen are used on such data, their values are generally very low, since the fraction of species occurring only in the rich sample will make the denominator too large and the overall value of the index too low; Simpson index, which was originally introduced for comparison of fossil data, eliminates this problem by taking only the smaller from the fractions b and c. (Note that there is yet another Simpson index, namely Simpson diversity index; each of the indices was named after different Mr. Simpson, and while Simpson similarity index is calculating similarity between pair of compositional samples, Simpson diversity index is index calculating diversity of a single community sample; you may find details in my blog post).
Percentage similarity: ,
where W is the sum of minimum abundances of various species; A and B each are sum of abundances of all species at each compared site:
Species abundances  A  B  W  

Site x_{1}  7  3  0  5  0  1  16  
Site x_{2}  2  4  7  6  0  3  22  
Minimum  2  3  0  5  0  1  11 
Quantitative similarity indices (applied on raw abundances) include percentage similarity, which is a quantitative version of Sørensen similarity index (which means that if calculated on presenceabsence data, it gives the same results are Sørensen similarity index). Note that percentage difference, calculated as 1percentage similarity, is called BrayCurtis index.
While similarity indices return the highest value in the case that both compares samples are identical (maximally similar), distance indices are largest for two samples which do not share any species (are maximally dissimilar). There are two types of distance (or dissimilarity) indices^{2)}:
An important criterium is whether the distance index is metric or not (i.e. it is semimetric or nonmetric). The term “metric” refers to the indices which can be displayed in the orthogonal Euclidean space, since they obey socalled “triangle inequality principle” (see explanation in Fig. 3). Some dissimilarity indices calculated from similarities are metric (e.g. Jaccard dissimilarity), some are not (e.g. Sørensen dissimilarity and it's quantitative version called BrayCurtis dissimilarity). In the case of Sørensen and BrayCurtis (and some others), this can be solved by calculating the dissimilarity as instead of the standard (where S is the similarity); resulting dissimilarity index is then metric. Indices which are not metric cause troubles in ordination methods relying on Euclidean space (PCoA or dbRDA) and numerical clustering algorithms which need to locate samples in the Euclidean space (such as Ward algorithm or Kmeans). For example, PCoA calculated using distances which are not metric creates axes with negative eigenvalues, and this e.g. in dbRDA may result in virtually higher variation explained by explanatory variables than would reflect the data.
BrayCurtis dissimilarity or percentage difference^{3)} is one complement of percentage similarity index described above. It is considered as suitable for community composition data, since it ignores double zeros, and it has a meaningful upper value equal to one (meaning complete mismatch between species composition of two samples, i.e. if one species in one sample is present and has some abundance, the same species in the other samples is zero, and vice versa). BrayCurtis considers absolute species abundances in the samples, not only relative species abundances. The index is not metric, but the version calculated as (where PS is percentage similarity) is metric and can be used in PCoA.
Euclidean distance:
where y_{1j} and y_{2j} are abundances of species j in sample 1 and 2, respectively.
Euclidean distance, although not suitable for ecological data, is frequently used in a multivariate analysis (mostly because it is the implicit distance for linear ordination methods like PCA, RDA and for some clustering algorithms). Euclidean distance has no upper limit and the maximum value depends on the data. The main reason why it is not suitable for compositional data is that it is a symmetrical index, i.e. it treats double zeros in the same way as double presences. Double zeros shrink the distance between two plots. The solution is to apply Euclidean distances on pretransformed species composition data (e.g. using Hellinger, Chord or chisquare transformation). An example of calculating Euclidean distance between samples with only two species is on Fig. 4.
Chord distance is Euclidean distance calculated on normalized species data. Normalization means that species vector in multidimensional space is of unit length; to normalize the species vector, one needs to divide each species abundance in a given sample by the squarerooted sum of squared abundances of all species in that sample. Chord distance is then the Euclidean distance between samples with normalized species data. An advantage of chord distance compared to Euclidean distance is that it has the upper limit (equal to ), while Euclidean distance has no upper limit.
Hellinger distance is Euclidean distance calculated on Hellinger transformed species data (and is the distance used in tbPCA and tbRDA if the species data are pretransformed by Hellinger transformation). Hellinger transformation consists of first relativizing the species abundances in the sample by standardizing them to sample total (sum of all abundances in the sample); then, each standardized value is squarerooted. This puts the species abundances on the relative scale, and squarerooting lowers the importance of the dominant species. Hellinger distance has an upper limit of and is considered as a suitable method for ecological data with many zeros.
Chisquare distance is rarely calculated itself, but is important since it is implicit for CA and CCA ordination.
When comparing two samples, Euclidean distance puts more weight on differences in species abundances than on difference in species presences. As a result, two samples not sharing any species could appear more similar (with lower Euclidean distance) than two samples which share species but the species largely differ in their abundances (see the example below).
In the species composition matrix below, samples 1 and 2 does not share any species, while samples 1 and 3 share all species but differ in abundances (e.g. species 3 has abundance 1 in sample 1 and abundance 8 in sample 3):
Species 1  Species 2  Species 3  
Sample 1  0  1  1 
Sample 2  1  0  0 
Sample 3  0  4  8 
Euclidean distance between sample 1 and 2 is lower than between sample 1 and 3, although samples 1 and 2 have no species in common, while sample 1 and 3 share all species.
The matrix of similarities or distances is squared (the same number of rows as columns), with the values on diagonal either zeros (distances) or ones (similarities), and symmetric  the upper right triangle is a mirror of values in lower left one (Fig. 5).