HMMs Pattern Recognition

HMMs Pattern Recognition naming 3 of Pattern credit rating is on HMMs. It should contain a detailed report on HMMs. The topics covered should include1. An introduction to HMM and its uses.1. Problems of HMM, their explanation and relation to prior, posterior, and evidence.2. Solution to the problems of HMM and their algorithmic programs.Pattern RecognitionAssignment 3Name Muhammad Sohaib JamalAn Introduction to HMM and its UsesA Hidden Markov Model HMM is a stochastic model which has a series of observable variable X which is generated by hidden call down Y. In an indirect way HMM consist of hidden states which has output that is comprised of a set of observations. Simple Markov Model models the states are directly observables subject matter the states are directly output while in HMM the states are hidden and different form the observables or output. HMM is very reliable model for probabilistic estimation. HMM create applications in pattern recognitions such as speech recogniti on, gesture and hand writing recognition, computational Bioinformatics, etc.Suppose we are considering deuce-ace drop behinds of a move toss experiment and the person who is observing only know the results of the experiment when another person announces the result who is hidden in a closed mode from the person noting the results. The result of this coin experiment can be all set of heads and chase after e.g. THT, HHH, THH, TTT, THT etc. The person observing the results can get any sequence of heads and tails, and it is not possible to predict any special sequence that will occur. The Observation Set is completely unpredictable and random.Lets sop up that the third trail of coin toss experiment will produce more Head than the Tails. The resulting sequence will obviously prolong more turning of heads hence tails for this particular case. This is called Emission probability denoted by Bj(O).Now we suppose that the chance of flipping the third trail after the first and second trail is approximately zero. Then, the vicissitude from 1st and 2nd trail to 3rd trail will be actually very small and as an outcome yields very little number heads if the person starts flipping the coin from 2nd trail to 3rd trail. This is called Transition probability denoted by aij.Assume that each trail has some probability associated with the previous trail, then the person will start the process of flipping from that particular coin. This is cognize to be the Initial probability denoted by i.The sequence of number of heads or tails is known to be the observables and the number of trail is said to be the state of the HMM.HMM is composed ofN number of hidden states S1, S2 ., SNM number of observations O1, O2, , OMThe i (Initial state probability)Output Probability or Emission Probability B P (OM SN), where OM is observation and SN is the state.Transition probability matrix A = aij .Transition probabilities aij.mathematically the model is represented as HMM = , A, BProblems of HMM and their explanationsHMM has three basic types of problemsThe Evaluation problemSuppose we have an HMM, complete with transition probabilities aij and output probabilities bjk. We need to station out the probability that a particular sequence of observables states OT was generated by that model.The decode problemThe transition probabilities, output probabilities and set of observations OT is tending(p) and we want to determine the most probably sequence of hidden states ST that led to those observations.The Learning problemIn such problem the number of states and observation are given but we need to queue the probabilities aij and bjk. With the given set of training observations, we will determine the probabilities aij and bjk.Relation of HMM to Prior, Posterior and evidenceThe i (Initial state probability) is analogous to the Prior probability. Because the initial probability is given before the set of experiments take place. This property of initial probability is i dentical to that of prior probability.Similarly, the output probability or emission probability B P (OM SN) is analogous to the posterior probability. The posterior probability is used in forward backward algorithm.In the same manner, evidence is the probability the next state is C given that the current state is state Sj. So the evidence is analogous to the transition probability A.Solution to the problems of HMM and their algorithmsFrom the above mentioned discussion, we know that there are three different of problems in HMM. In this section we will briefly know how these problems are solvedEvaluation problem, this type of problem is solved the using Forward-Backward algorithm.Decoding problem, for such type of HMM problem we use the Viterbi algorithm or posterior decodingTraining problem, in case of this type of problem we have the Baun-Welch re-estimation algorithm to solve it.Forward-Backward algorithmThe forward and backward steps are combined by the Forward-Backward algorith m to estimate the probability of each state for a specific time t, and repeating these steps for each t can result in the sequence having the most probable probability. This algorithm doesnt guarantee that the sequence is sound sequence because it considers every individual step.The forward algorithm has the following three stepsInitialization stepIterationsSummation of overall states.Similarly, for backward algorithm we have the same steps like the forward algorithmInitialization stepIterationsSummation of overall statesViterbi algorithmViterbi algorithm is used to find the most likely hidden states, resulting in a sequence of observed events. The relationship between observations and states can be inferred from the given image.In first step Viterbi algorithm initialize the variableIn second step the process is iterated for every stepIn third step the iteration endsIn Fourth step we racecourse the best pathBaun-Welch re-estimation algorithmBaun-Welch re-estimation algorithm is used to compute the unknown parameters in hidden Markov model HMM. Baun-Welch re-estimation algorithm can be best draw using the following example.Assume we collect eggs from chicken every day. The chicken had lay eggs or not depends upon unknown factors. For simplicity assume that there are only 2 states (S1 and S2) that determine that the chicken had lay eggs. Initially we dont know about the state, transition and probability that the chicken will lay egg given specific state. To find initial probabilities, suppose all the sequences starting with S1 and find the maximum probability and then repeat the same bit for S2. Repeat these steps until the resulting probabilities converge. Mathematically it can beReferencesAndrew Ng (2013), an online course for Machine learning, Stanford University, Stanford, https//class.coursera.org/ml-004/class.Duda and Hart, Pattern Classification (2001-2002), Wiley, New York.http//en.wikipedia.orghttp//hcicv.blogspot.com/2012/07/hidden-markov-model-f or-dummies.htmlhttp//www.mathworks.com/help/stats/hidden-markov-models-hmm.htmlhttp//www.comp.leeds.ac.uk/roger/HiddenMarkovModels/html_dev/viterbi_algorithm/s3_pg3.html