\documentclass[12pt,oneside]{book}
\usepackage[utf8]{inputenc}
\usepackage[a4paper, total={6.4in, 9.7in},top=1in]{geometry}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{hyperref}
\usepackage{parskip}
\usepackage{xcolor}
\usepackage[pdftex]{graphicx}
\graphicspath{{./Figures/}}
\usepackage{enumerate}
\usepackage{flafter}
\usepackage{caption}
\captionsetup{justification   = raggedright, singlelinecheck = false}
\usepackage[section,above,below]{placeins}

\usepackage{listings}
\lstset{ 
  language=R,                     % the language of the code
  basicstyle=\scriptsize\ttfamily, 	% the size of the fonts that are used for the code
  numbers=none,                   % where to put the line-numbers
  backgroundcolor=\color{white},  % choose the background color.
  showspaces=false,               % show spaces adding underscores
  showstringspaces=false,         % underline spaces within strings
  showtabs=false,                 % show tabs within strings
  frame=single,                   % adds a frame around the code
  rulecolor=\color{black},        % if not set, the frame-color may be changed on line-breaks within not-black text
  tabsize=2,                      % sets default tabsize to 2 spaces
  breaklines=true,                % sets automatic line breaking
  breakatwhitespace=false,        % sets if automatic breaks should only happen at whitespace
  keywordstyle=\color{blue!70!black},      % keyword style
  commentstyle=\color{green!50!black},   % comment style
  stringstyle=\color{orange!70!black}      % string literal style
}

\usepackage[backend=biber, style=authoryear, natbib=true, maxbibnames=6, maxcitenames=2, giveninits=true, doi=false]{biblatex}
\addbibresource{SeanPhD.bib}
\setlength{\bibparsep}{6pt}
\DeclareNameAlias{sortname}{last-first}
\renewcommand{\newunitpunct}{,\ }
\renewcommand{\intitlepunct}{}
\DefineBibliographyStrings{english}{in = {{}{}}}

\newcommand{\mytitle}{Topics in the analysis of composition data}
\hypersetup{pdfinfo={
	Title={\mytitle},
	Subject={\mytitle},
	Author={Sean van der Merwe},
	Keywords={Bayes, Dirichlet, Goodness-of-fit, regression, Posterior, Simulation} } }

\renewcommand{\chapterautorefname}{Chapter}
\renewcommand{\sectionautorefname}{Section}
\renewcommand{\subsectionautorefname}{Section}
\renewcommand{\subsubsectionautorefname}{Section}
\newlength{\figwidth} \setlength{\figwidth}{10cm} % I use this to standardise figure sizes

\usepackage{enumerate}
\usepackage{soul}
\newcommand{\com}[1]{{\color{red}#1}} % I'm going to use this command to highlight my changes, remove the red as you see fit. I'll use [] for comments inside it.
\newcommand{\new}[1]{{\color{blue}#1}}   

\newcommand{\bx}{\ensuremath{\mathbf{x}}}
\newcommand{\by}{\ensuremath{\mathbf{y}}}

\renewcommand{\baselinestretch}{1.15}

\usepackage{fancyhdr}
\fancypagestyle{C1}{\fancyhf{} \fancyhead[L]{Chapter 1 --- Introduction} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{C2}{\fancyhf{} \fancyhead[L]{Chapter 2 --- Fitting} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{C3}{\fancyhf{} \fancyhead[L]{Chapter 3 --- Discrimination} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{C4}{\fancyhf{} \fancyhead[L]{Chapter 4 --- Classification} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{C5}{\fancyhf{} \fancyhead[L]{Chapter 5 --- MGAM} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{C6}{\fancyhf{} \fancyhead[L]{Chapter 6 --- Regression} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{C7}{\fancyhf{} \fancyhead[L]{Chapter 7 --- Conclusion} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{plain}{\fancyhf{} \fancyhead{} \fancyfoot[C]{\thepage} \setlength{\headheight}{15pt} \renewcommand{\headrulewidth}{0pt}}
\fancypagestyle{preamble}{\fancyhf{} \fancyhead[L]{Preamble} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}
\fancypagestyle{contentsstuff}{\fancyhf{} \fancyhead[L]{\leftmark} \fancyhead[R]{\thepage} \fancyfoot{} \setlength{\headheight}{15pt}}

\usepackage{animate}

\begin{document}
\hfuzz=32pt
\hbadness=4800
\begin{refsection}
\pagenumbering{gobble}
\begin{center}
\includegraphics[width=1.2\figwidth]{UFSmathstatsLogo.png}

\begin{Huge}
\mytitle
\end{Huge}
\vspace{1cm}

\begin{LARGE}
Sean van der Merwe

2001007828
\end{LARGE}
\begin{Large}
\vspace{2cm}

Supervisors: Daniel de Waal, Michael von Maltitz
\end{Large}
\vspace{2cm}

Submitted in fulfilment of the requirements for the degree:\\
Ph.D. (Mathematical Statistics)\\
in the Department of Mathematical Statistics and Actuarial Science in the Faculty of Natural and Agricultural Sciences at the University of the Free State.
\vspace{1cm}

\begin{Large}
\today
\end{Large}

\end{center}

\newpage
\frontmatter
\setcounter{page}{1}
\pagestyle{preamble}
\section*{Declaration}
I, Sean van der Merwe, hereby declare that this work, submitted for the degree Ph.D. (Mathematical Statistics), at the University of the Free State, is original work and has not previously been submitted, for degree purposes or otherwise, to any other institution of higher learning. I further declare that all sources cited or quoted are indicated and acknowledged by means of a comprehensive list of references. Copyright hereby cedes to and is invested in the University of the Free State. I declare that all royalties as regards to intellectual property that was developed during the course of and/or in connection with the study at the University of the Free State, will accrue to the University. 

\includegraphics[width=60mm]{SeanvdMsignature1.png}

\newpage

\section*{Acknowledgement}

I want to thank Jesus first, for keeping my spirits up through failures and rejections, and reminding me that research is a service to the world, not for any personal gain.

I want to thank those that directly helped me with this work, for guidance and direction, even when I stubbornly refused to listen. In particular, I'm referring to Daan de Waal, Roelof Coetzer, Michael von Maltitz, and Robert Schall.

I want to thank my whole department for being such amazingly pleasant people to work with and always supporting me, even when they didn't know what I was doing or weren't supportive of it. They are all role models for me.

I want to thank my friends and family, for helping me maintain balance, and for their interest in my work.

\newpage
\section*{Abstract}
Many scientific and industrial processes produce data that is best analysed as vectors of relative values, often called compositions or proportions. Any data collected as vectors where the proportion measured in each category is of more importance than the quantity surveyed falls into this area of analysis. Such data is multivariate in nature but should not be analysed using standard multivariate analysis methods directly. Techniques specific to composition data are investigated and recommended in this work.

The Dirichlet distribution is a natural distribution for composition or proportion data. It has the advantage of a low number of parameters, making it a parsimonious choice in many cases. This work primarily explores the Dirichlet distribution from various angles. These include discriminating between populations, classifying observations, and regression of compositions on data of other types. Other models for compositions are also discussed. 

Most topics are investigated from the Bayesian perspective. This work particularly contributes to the field of Bayesian analysis using the Dirichlet distribution, expanding the theory of fitting and applying the Dirichlet type I and Dirichlet type II distributions using Bayesian techniques. Prediction and imputation is also explored.

A key contribution of this work is in developing a general approach to building advanced models for composition data, in the Bayesian framework, using the Dirichlet distribution as starting point. These models may include different types of explanatory variables for both expected values and precisions, as well as a mixture of fixed and random effects.



\vspace{4mm}

\noindent%
\textbf{Keywords:}  Composition data, proportions, Dirichlet distribution, Bayes, simulation

\vspace{4mm}

%\noindent%
%{\it Note on media:}  This document is meant to be viewed in Portable Document Format (.pdf). If you did not receive it in this format then you may obtain the latest version here: \url{https://goo.gl/yBhSUu}.

\newpage
\section*{List of Abbreviations}
\begin{tabular}[t]{|l|l|} \hline
\textbf{Abbreviation}&\textbf{Meaning}\\ \hline
D&Dirichlet type I (distribution)\\ \hline
D2&Dirichlet type II (distribution)\\ \hline
LDA&latent Dirichlet allocation (model)\\ \hline
ln&natural logarithm\\ \hline
LN&logistic-normal (distribution)\\ \hline
log&natural logarithm\\ \hline
MD&matrix Dirichlet (distribution)\\ \hline
MDI&maximal data information (prior)\\ \hline
MDirich2&matrix Dirichlet type II (distribution)\\ \hline
MEV&multivariate extreme value (distribution)\\ \hline
MGAM&multivariate gamma (distribution)\\ \hline
ML&(method of) maximum likelihood\\ \hline
SCE&sum of composition errors\\ \hline
\end{tabular}

\newpage
\pagestyle{contentsstuff}
\tableofcontents
\newpage
\listoffigures
\newpage
\listoftables
\newpage

\newpage
\pagestyle{preamble}
\section*{Problem Statement}
In many experiments or data scenarios the situation exists where measurements are relative, or only of interest to researchers when expressed relative to each other in such a way that they sum to one. The result is data that is multivariate, but not directly suitable for analyses using the popular multivariate normal analysis methods. Further, sample sizes may be small, so even if data are transformed to a free scale then the multivariate normal distribution may have too many parameters, leading to over-fitting. Excessive transformations also affect interpretability.

The Dirichlet and related distributions can be seen as a parsimonious solution, but the published literature is not yet extensive. Most notably, there is a need for Bayesian solutions involving the Dirichlet distribution.

This work seeks to address the problems of:
\begin{itemize}
\item Bayesian analysis of the Dirichlet distribution,
\item discrimination and classification with composition data,
\item and regression of composition data using the Dirichlet distribution.
\end{itemize}

\newpage
\section*{Overview}

In \autoref{ch:intro} the main topics are introduced and illustrated, relevant applications are discussed by means of examples from literature, and advances in the theory of the topics are also mentioned.

In \autoref{ch:p1} the Dirichlet type I and type II distributions are discussed in detail. New theory is derived. Objective Bayesian priors are derived. Performance of the derived Bayesian parameter estimation methods are compared to existing methods via a simulation study. Prediction and imputation using these distributions are also explained.

In \autoref{ch:p2} the discriminant function is developed for the Dirichlet and Matrix Dirichlet distributions. Uses are explained both in theory and with examples.

In \autoref{ch:p3} two new approaches to classification of Dirichlet observations are developed and discussed in detail. These are presented in the context of coal properties as inputs to a coal gasification process.

In \autoref{ch:p4} an alternative to the existing distributions for composition data is developed, namely a new class of multivariate gamma distribution. Again, classification in the context of coal gasification is used to explain the use of this approach. This alternative relates to the Dirichlet distribution in that the Dirichlet can be created as a transformation of a special case of this multivariate gamma distribution.

In \autoref{ch:p5} the use of Dirichlet distributions in a regression setting is discussed. A new approach is developed and explained. The advantages of the new approach are illustrated via simulation and example.

In \autoref{ch:conclusion} the advances made through the work above are summarised.

It is worth mentioning that many of the approaches discussed are not compared directly, as they fall in different model classes. Specifically, the Dirichlet models can be seen as the parsimonious class, as the number of parameters increases in line with the dimension of the problem; while the other models discussed (\textit{e.g.}\ multivariate gamma and logistic normal) are referred to as the rich class, since the number of parameters in those models increase in line with the square of the dimension. Models are compared to others in their own class only.

The thread connecting the topics is an expansion of the tools for working with composition data. The focus is on the Dirichlet Type I models. Being in the parsimonious class, these models require less data points for accurate parameter estimation, and they also have the benefit of not requiring transformation of composition data prior to analysis.

\newpage
\section*{Comment on notation}
Chapters 2 to 6 are based on individual papers that were each written separately, by different combinations of authors, from slightly different viewpoints. As such, they originally contained differing definitions and notation. In order to increase the readability of this document, each paper has undergone minor editing. 

In each of these chapters a link is provided to an online version of the relevant paper, in case the reader wishes to view the original. Most changes are related to a consistent notation for the Dirichlet distribution. The main reason for the differences from the original papers is discussed in the first chapter.

\newpage
\section*{Authorship and publication}
Paper 1 (\autoref{ch:p1}) is almost entirely the work of the student, under the direction of Prof.\ Daan de Waal. All derivations are done by the student. All simulation studies were designed and executed by the student. The multiple imputation approach is original to the student, adapted from the multivariate normal imputation approach. It is posted as a technical report as it discusses theory in detail to act as a reference source.

Paper 2 (\autoref{ch:p2}) is mostly the work of Prof.\ Daan de Waal, with the student providing assistance. The theory was derived by the supervisor, along with initial experiments. Results were checked and edited by the student. The student was responsible for typesetting and editing of the publication. This paper is published in the South African Statistics Journal. 

Paper 3 (\autoref{ch:p3}) contains equal contributions from Prof.\ Daan de Waal and the student, with Prof.\ Roelof Coetzer providing leadership and direction. The introduction and interpretations relating to the application were written by Prof.\ Coetzer, who posed the original problem. The first approach to solving the problem was derived by the supervisor. The second approach was derived by the student. The student was responsible for typesetting and editing of the publication, and responding to reviewer feedback. This paper is published in Chemometrics. 

Paper 4 (\autoref{ch:p4}) is mostly the work of Prof.\ Daan de Waal, with the student providing assistance and expanding the theory in places. Prof.\ Roelof Coetzer provided valuable assistance. The new distribution was developed by the supervisor. The student was responsible for the Appendix. The student was responsible for typesetting and editing of the publication, and responding to reviewer feedback. This paper is published in the South African Statistics Journal.

Paper 5 (\autoref{ch:p5}) is entirely the work of the student and is currently under review. The work has been accepted for publication in the South African Statistics Journal. 

The papers are presented in chronological order of being written; however, this order does maintain a logical flow and so the work can be read as a unit.















% >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
\mainmatter
\chapter{Introduction and Literature} \label{ch:intro}
\pagestyle{C1}
\section{Composition data}

Composition data refers to a set of multivariate observations where each individual observation vector sums to exactly one. Alternatively, it is often defined as a vector summing to a positive real number less than one, with a residual `other' component that is considered non-random making up the difference between one and the sum of the random components. Each of these definitions seem more applicable to some practical situations than others, so examples are discussed in this chapter.

When vectors are measured in whole numbers, for example in a classification process, analysis is often based on the multinomial distribution, with the total count being seen as a nuisance parameter. In cases where the total count is not relevant it seems more natural to work directly with the observed proportions. Sometimes the proportions themselves are observed directly, rather than counts and totals. This class of problem includes any situation where the quantity surveyed does not affect the expectation vector.

A good example of composition data is food composition. Most foods have their compositions analysed and reported for health and safety reasons, and being able to model changes in composition can be valuable to regulators. See the book of \textcite{greenfield} for an explanation of the creation and analysis of this type of data. In \textcite{Davis2004} the composition of food was compared between 1950 and 1999 to investigate possible drops in nutritional value over that period. Astrophysicists have considered using the composition of solar wind to determine its origin \parencite{geiss1995}. Another composition example is given by \textcite{wedepohl2011data}, where the elemental composition of glass was used to determine the glass' region of origin. Compositions are of great interest in industrial applications, where the inputs to an industrial process can affect the financial worth of the output.

\subsection{Note on conflicting definitions}

The most popular approaches to analyse composition data involve designating one component of the vector as a reference component. The other components are considered random, while the reference component is not. Analysis then proceeds relative to the reference component. These approaches were regularly applied out of necessity, due to the absence of alternatives. While there may be practical justifications to these approaches in some cases, it is argued here that applying these approaches without clear justification results in loss of practical interpretability.

In the chapters to come the theory of composition data analysis is expanded in such a way that reliance on a reference category is reduced, freeing the researcher to analyse and interpret all components directly. In accordance with this, definitions and notation have been adapted in places to reflect the idea that all components carry equal weight.

Formally, the following definition is used:

$$
\begin{aligned}
&\text{A random variable }\mathbf{X}\text{, a vector of size }P\text{ by 1, with components }X_1,\ldots,X_P\\
&\text{is called a composition random variable if }\\
&0<X_i<1\ \forall\ i=1,\ldots ,P\text{ and }\sum_{i=1}^P X_i = 1.
\end{aligned}
$$

\section{Composition data analysis methods} \label{sec:OtherMethodsCompos}

Composition data analysis is no different to most types of data analysis and most areas of statistics in that the starting point of analysis is describing the data. Practical statisticians will always advocate `eyeballing' the data before doing any modelling. This means looking at the structure, querying the collection process, looking for anomalies, summarising the data using descriptive measures, and summarising the data visually. 

However, the scale of composition data is not the same as most data types, and so transformations can be useful in the process of summarising and visualising the data. At this stage a researcher may find use in a statistical package such as `compositions' \parencite{van2008compositions} for the R environment \parencite{RCore}. The package authors explain that their package supports multiple scales and moving between them. It allows descriptive and visual analyses on all scales, and can perform principal component analysis or cluster analysis if advanced visualisation would be useful. See \textcite{rencher2002} for an explanation of these techniques.

\textcite{Aitchison1986} advocates the use of the log-ratio transform followed by classical multivariate normal distribution analysis. He calls this the logistic-normal distribution (LN). This allows advanced multivariate analyses to be carried out on composition data. Again, see \textcite{rencher2002} for an introduction to this area of statistics.

As an example, \textcite{lafferty2006} adapt their latent Dirichlet allocation (LDA) model (discussed in \autoref{sec:DirAppliIntro}) to the LN framework and call it Correlated Topic Models.

However, there are two difficulties with the LN approach. The first is interpretability. Since the transformations done are non-linear, all results should be transformed back to the original scale in a multivariate fashion if the researcher wishes to make judgements on the proportions scale. Often this is not possible, or very difficult. This difficulty persists with almost all analyses of composition data, an ever-present concern for researchers in this field.

The second difficulty is parameterisation. If the number of categories or dimensions is small then the multivariate normal approach yields great flexibility and is often useful even when the data does not fit the normal distribution perfectly. However, when the number of dimensions increases then the number of parameters inflates rapidly, and simpler models that still capture the nature of the data are required.

\section{The Dirichlet Distribution} \label{sec:DirIntroCh1}

The origin of the Dirichlet distribution is discussed in detail in the paper of \textcite{Gupta2001}. They explain that the distribution owes its name to Johann Peter Gustav Lejeune Dirichlet, who evaluated the integral that completes the distribution \parencite{Dirichlet1839}.

\subsection{Definition of the Dirichlet Distribution} \label{sec:DirDefCh1}

If $\mathbf{Y}$ (size $P$ by 1) is distributed Dirichlet$(\alpha_1,\ldots,\alpha_{P})$, denoted by $D(\boldsymbol\alpha)$, then the density is given by 
\begin{equation} \label{eq:Dintroduction}
f(\mathbf{y})= \frac{\prod_{i=1}^{P}\Gamma(\alpha_i)}{\Gamma(\alpha_0)}\left\{ \prod_{i=1}^P y_i^{\alpha_i-1}\right\}\ ,\ 0 < y_i < 1,\ \sum_{i=1}^P y_i = 1,\ \alpha_0 = \sum_{i=1}^{P} \alpha_i 
\end{equation}

The above definition is similar to the one given in \textcite{Maier2014}, who uses illustrations to help explain the distribution. Original illustrations are given here for the same purpose. In two dimensions the Dirichlet distribution is the classic beta distribution (\autoref{fig:D2illus}) which is well understood. In three dimensions (\autoref{fig:D3illus}) the flexibility of the beta is extended intuitively. In four dimensions the density becomes difficult to visualise, but a sample from the density can be illustrated as in (\autoref{fig:D4illus}). If one wishes to visualise five or more dimensions, each new dimension must be introduced in a creative way, say by changing the size, colour, or shape of the points of the sample.

\begin{figure}[hbt]
\begin{center}
\includegraphics[width=0.5\figwidth]{Dirichlet2Dg1.pdf}
\includegraphics[width=0.5\figwidth]{Dirichlet2Dg2.pdf}
\end{center}
\caption{Illustrations of the Dirichlet density with dimension two (beta density)}

\label{fig:D2illus}
\end{figure}

\begin{figure}[hbt]
\begin{center}
\includegraphics[width=0.5\figwidth]{DirichletD3g1.png}
\includegraphics[width=0.5\figwidth]{DirichletD3g2.png}
\end{center}
\caption{Illustrations of the Dirichlet density with dimension three}

\label{fig:D3illus}
\end{figure}

\begin{figure}[hbt]
\begin{center}
\animategraphics[controls,autoplay,loop,width=\figwidth ,trim=0 0 0 60]{0.5}{./Figures/DirichletD4ani}{1}{12}
\end{center}
\caption{Random sample from a Dirichlet density with dimension four}

\label{fig:D4illus}
\end{figure}


\subsection{Properties of the Dirichlet Distribution} \label{sec:DirPropsIntro}

Understanding the properties of a distribution helps the researcher to know when to apply the distribution and when not to. Before looking at applications in the next section, the main properties are briefly discussed in an intuitive way. For a more rigorous discussion, see \autoref{sec:intro}.

The Dirichlet distribution is applied to vector observations that sum to one. If one component of the vector increases then at least one other component must decrease, and thus there is a natural negative relationship between components of the vector. The negative relationship is reflected in the moments of the Dirichlet distribution, see \autoref{eq:IntroChapmeancov} below.

\begin{equation} \begin{aligned}\label{eq:IntroChapmeancov}
&E(Y_i)=\frac{\alpha_i}{\alpha_0},\ i = 1,\ldots,P\\
&Cov(Y_i,Y_j)=\left\{\begin{array}{l}
\frac{-\alpha_i \alpha_j}{\alpha_0^2 (1+\alpha_0)},\ i\ne j\\
\frac{\alpha_i (\alpha_0-\alpha_i)}{\alpha_0^2 (1+\alpha_0)},\ i=j
\end{array}\right.
\end{aligned} \end{equation}

Obtaining the mean vector and covariance matrix is a natural step in data analysis, and should the covariances be all negative then that would support the Dirichlet distribution as a candidate for direct modelling. However, in practical situations it is often the case that positive correlations occur, especially as the number of components in the vector (number of dimensions) increase. When clear positive correlations are noted then one should not apply the Dirichlet distribution directly.

There are several ways to address the issue of positive correlations between components without abandoning the Dirichlet distribution. The first is to combine components by summing them. The aggregation property of the Dirichlet distribution \citep[proved in][]{Frigyik2010} says that if two components of a Dirichlet vector are dropped and replaced with their sum then the resulting vector is again Dirichlet with the corresponding parameters replaced in the same way. Aggregating components with positive correlations can only enhance the fit Dirichlet distribution.

The second approach is to consider subsets separately where appropriate. When attempting to discriminate between Dirichlet populations it is important to note that each subset should fit the Dirichlet distribution, not necessarily the sample as a whole. As a crude example, if a researcher is interested in the effect of gender on voting patterns then that researcher can model the male voting patterns separately from the female or other gender voting patterns, and check each model fit on its own merits. See \autoref{ch:p2} for more information.

The last consideration is the case where the vectors are regressed on continuous explanatory variables. Here the requirement of negative correlations generally falls away \citep{Maier2014}. Positive correlations are assumed to be induced via joint dependence on the explanatory variable.

Another important aspect of the Dirichlet distribution is that the components of the observation vectors must be strictly between zero and one. When true zeros are observed then the Dirichlet distribution is not advised, unless the issue can be address by one of the three methods above. When values that are very small and rounded to zero are observed, then adjustments can be made to enable Dirichlet modelling. These include pulling extreme observations in from the edge of the domain \citep{Maier2014}, treating them as censored (implicit imputation) or imputing them explicitly (similar to \autoref{sec:imputation}).

Should none of the adjustments be appropriate then more complex modelling should be considered using alternative distributions (often involving transformations). For more information see \autoref{ch:p4}.

\subsection{Applications of the Dirichlet Distribution} \label{sec:DirAppliIntro}

The most common application of the Dirichlet distribution is as a conjugate prior for Bayesian analyses (\autoref{sec:whatisBayes}) involving the multinomial distribution. The result is known as the Dirichlet-multinomial distribution or Dirichlet compound multinomial distribution, which traces back to \textcite{Mosimann1962}. For a modern example related to text mining see \textcite{Madsen2005}.

The books of \textcite{Ng2011} and \textcite{balakrishnan2006} give some insight into the applications of the Dirichlet and related distributions. Another interesting example is the work by \citet{lange1995} for estimating forensic match probabilities.

The Dirichlet distribution has also been used in economics. Income inequality is a major topic of discussion in economics and it is usually illustrated via the Lorenz curve. \textcite{chotikapanich2002} show that by assuming income proportions are distributed Dirichlet they can improve modelling of such curves.

In medical science, \textcite{briggs2003} use the Dirichlet distribution to develop a fully probabilistic transition matrix for a reliable Markov model for medical interventions.

In computer science, the Latent Dirichlet Allocation model \parencite{blei2003latent} is a very popular model in machine learning, as it allows automatic characterisation of documents according to topic probabilities. The uses of this tool are extensive --- see \textcite{NIPS2010} and \textcite{Lienou2010} as starting points for exploring this interesting research area. Additionally, modern online reputation systems have the power to determine the success of a product or business, so it should be no surprise that their study and optimisation is of great value. \textcite{josang2007} discuss the use of the Dirichlet distribution in this process. Also on the theme of machine learning, \textcite{bouguila2003} develop mixtures based on the Dirichlet Distribution for pattern-recognition in image databases.

\subsection{Dirichlet regression} \label{sec:DirRegIntroCh1}

The Dirichlet distribution has been widely accepted in literature for modelling composition data, subject to the constraint that all the correlations between variables are negative \citep{Maier2014}. A wider class of distributions which allows for positive correlations defined on the same sample space is the logistic-normal (LN) \citep{Aitchison1980}. However, the LN distribution has many parameters to estimate due to the unknown covariance parameter matrix. In contrast, the Dirichlet distribution has only $p$ unknown parameters to estimate for compositions with $p$ components. Further, when the model is expanded to the regression framework then the requirement of negative correlations no longer applies \citep{Maier2014}.

The distinctive characteristic of the regression framework is that it allows for continuous measurements as explanatory variables. The goal is to simultaneously model vector observations in their entirety based on a flexible structure that allows for the analysis of relationships between the observed vectors and the explanatory variables.

As examples of applications, consider the work of \citet{boukal2015analyses} regarding the development of insects, or \citet{EspinGarcia2014} who look at genetic analysis problems, or the work of \citet{smithson2006better} where applications in psychology are discussed. 

Another area in which Dirichlet regression may be useful is in politics, where only the proportion of voters supporting a set of candidates is of interest, possibly dependent on district or demographics. In ecology, preferences of animals for specific types of prey is of interest. In any situation where people choose between a set of options, and the researcher is interested in the choice and not the number of people making it, then the Dirichlet regression model may be of use.

\section{Bayesian analysis} \label{sec:whatisBayes}

Bayesian analysis at its core refers to the idea of starting by understanding one's prior beliefs and then updating those beliefs by observing data \parencite{Efron2013}.

Besides the philosophical differences, there are many practical advantages to the Bayesian approach. It is the experience of the author that the Bayesian approach for capturing model uncertainty is often the biggest advantage as it gives more flexibility in terms of the types of questions that can be answered from a particular analysis. It also simplifies the process of creating uncertain predictions.

Probably the most widely known book on the topic of Bayesian statistics is that of \textcite{gelman2013}, which is a detailed and thorough text. It is mathematically rigorous and can help the reader gain as much insight as desired.

On the applied side, the book of \textcite{Cowles2013} takes the reader quickly from the basic concepts to the practical workings of Bayesian statistics. She explains that people naturally quantify uncertainty in terms of probabilities, but then goes on to explain that the frequentist definition of probability (the proportion of success from an infinite number of trials) is flawed in several ways. Most notably, it relies on the idea that a sequence of repeatable experiments is possible (or at least imaginable) which is often not the case.

In Bayesian statistics a probability is a person's opinion as to how likely a success is. It is unique to the person and can change over time. Sometimes researchers object to the idea of subjectivity in science, saying scientists should strive for objectivity; however, it is impossible to eliminate subjectivity and by attempting to understand and quantify it the researcher can actually increase the trustworthiness of results.

In this work the focus is on `objective' Bayesian analysis, which involves the use of non-informative priors or vague priors, because the solutions discussed are generic. In specific problems where good prior information is available we encourage the researcher to incorporate such knowledge into the analysis.

Prior knowledge is combined with observed data using Bayes' rule. \textcite{Cowles2013} states it in its simplest form as 
\begin{equation} \label{eq:BayesRule}
P(Model|Data)\propto P(Data|Model)\times P(Model)
\end{equation}
where $P(Model)$ is all the knowledge about the model (and in particular parameters) that is available before observing the data, while $P(Data|Model)$ is the likelihood.

The first powerful advantage to the above rule is that it is recursive, allowing knowledge to be updated in a natural way when further data are observed (the posterior of one sample can be used as the prior to a new sample). The second powerful advantage is that extending the definition for making predictions is straightforward:
\begin{equation} \label{eq:BayesPred}
P(New\ Data|Old\ Data)=\int_{Model} P(New\ Data|Model)\times P(Model|Old\ Data)
\end{equation}

Bayesian statistics has risen in popularity \parencite{Efron2013} due to improvements in computing hardware and software over the last few decades, which have made implementation much simpler \parencite{R2OpenBUGS}. The most common way to implement a Bayesian model is to draw random parameter samples from the $Model|Data$ using simulation techniques and then summarise those samples to draw inferences and answer questions. To evaluate the predictive distribution one can simply draw random observations given each of the randomly drawn parameter vectors and summarise those observations in turn.

\section{Outline summary}

As mentioned in the overview, the coming chapters address various topics in the analysis of composition data. The first deals with standard distribution fitting and forms a basis for the topics that follow. The second deals with discrimination between populations. The third deals with classification of observations between populations. The fourth explores a new distribution and its use. The last topic explores the general regression problem and can be seen as the main outcome of the work.

\printbibliography[title=List of References for the introductory chapter]
\end{refsection}

\end{document}
