Pattern Recognition System
A Pattern Recognition System is a knowledge discovery system that implements a pattern recognition algorithm to solve a pattern recognition task (to discover information patterns within a dataset).
- Context:
- It can by .
- It can range from being a Geometric Pattern Recognition System, to being a Statistical Pattern Recognition System, to being a Syntactic Pattern Recognition System.
- It can range from being a Generative Pattern Recognition System to being a Discriminative Pattern Recognition System.
- It can range from being a Pattern Recognition Classification System to being a Pattern Recognition Regression System.
- It can range from being a Unsupervised Pattern Recognization System, to being a Semi-Supervised Pattern Recognition System, to being a Supervised Pattern Recognition System.
- It can range from being a Parametric Pattern Recognition System to being a Nonparametric Pattern Recognition System.
- It can range from being a Clustering Based Pattern Recognition System to being an Ensemble Learning Based Pattern Recognition System.
- It can range from being a Sequence Labeling Pattern Recognition System to being a Multilinear Subspace Pattern Recognition System.
- It can range from being a Bayesian Network Based Pattern Recognition System to being a Markov Random Fields Based Pattern Recognition System.
- Example(s):
- Counter-Example(s):
- See: Machine Learning, Artificial Intelligence, Learning, Supervised Learning, Unsupervised Learning, Data Mining, Knowledge Discovery in Databases, Engineering, Computer Vision, Conference on Computer Vision And Pattern Recognition, Linear Discriminant Analysis, Classification (Machine Learning), Regression Analysis, Knowledge Discovery in Databases.
References
2019a
- (Pudil et al., 2006) ⇒ Pavel Pudil, Petr Somol, and Michal Haindl (unknown). "Introduction to Statistical Pattern Recognition"
- QUOTE: What is the pattern recognition process? When a human glances at a printed page and recognizes character after character, he is utilizing his past accumulated experience which has been somehow transformed into fixed decision rules. He is generally unable to explain those rules, however, their existence is out of question. The mechanism of arriving at these rules is quite obvious. It has been necessary to expose him in the learning phase to samples of respective characters and to tell him which they were. In other words to provide him with so called labeled samples from which he developed a decision rule. We can see that there are two aspects to pattern recognition - developing a decision rule and using it. The actual recognition occurs in the stage of using the rule. We have to be aware of the fact that the pattern recognition problem can be defined only if we state clearly what we wish to decide, by defining the pattern classes.
A pattern recognition problem thus begins with class definitions, which should be given a thorough consideration since in certain applications the choice of class definition may greatly influence the overall performance of recognition system. With computers becoming widely used, the field of pattern recognition ceased to be the process specific exclusively to the human brain. However, there is an essential difference between the human's recognition and mathematical (statistical) pattern recognition by means of computers. (...)
There are essentially two basic approaches to pattern recognition. In a geometric or statistical approach, a pattern is represented in terms of D features or properties and viewed as a point in D-dimensional space. The first task is to select those features such that pattern vectors of different categories will not overlap in the feature space (this ideal cannot be generally reached, so a near optimum solution is sought). Given certain labelled sample patterns from each class (training samples), the objective is to establish decision boundaries in the feature space which would separate as much as possible patterns belonging to different classes. Either the direct statistical approach can be adopted when the decision boundaries are determined by the statistical distributions of the patterns (known or estimated), or a ''non-statistical'' approach is utilized. In the latter case, first the functional form of the decision boundary is specified (linear, piecewise linear, quadratic, polynomial, etc.) and afterward the best decision boundary of the specified functional form is found. The choice of features is crucial for the quality of pattern recognition. Participation of an expert from the corresponding application area in designing the primary set of features is absolutely essential since the choice of features is data dependent(...)
Beside the statistical approach to pattern recognition there exist another approach - syntactic (or structural) pattern recognition. To make the analogy more explicit we can state that the patterns are viewed as sentences of a language, primitives are viewed as the alphabet of the language, and generating of sentences is governed by a grammar. The main advantage of the syntactic or structural approach is that, in addition to classification, it also provides a description how the given pattern is constructed from the primitives.
- QUOTE: What is the pattern recognition process? When a human glances at a printed page and recognizes character after character, he is utilizing his past accumulated experience which has been somehow transformed into fixed decision rules. He is generally unable to explain those rules, however, their existence is out of question. The mechanism of arriving at these rules is quite obvious. It has been necessary to expose him in the learning phase to samples of respective characters and to tell him which they were. In other words to provide him with so called labeled samples from which he developed a decision rule. We can see that there are two aspects to pattern recognition - developing a decision rule and using it. The actual recognition occurs in the stage of using the rule. We have to be aware of the fact that the pattern recognition problem can be defined only if we state clearly what we wish to decide, by defining the pattern classes.
2019b
- (Wikipedia, 2019) ⇒ https://en.wikipedia.org/wiki/Pattern_recognition Retrieved:2019-9-26.
- Pattern recognition is the automated recognition of patterns and regularities in data. Pattern recognition is closely related to artificial intelligence and machine learning,[1] together with applications such as data mining and knowledge discovery in databases (KDD), and is often used interchangeably with these terms. However, these are distinguished: machine learning is one approach to pattern recognition, while other approaches include hand-crafted (not learned) rules or heuristics; and pattern recognition is one approach to artificial intelligence, while other approaches include symbolic artificial intelligence. A modern definition of pattern recognition is: This article focuses on machine learning approaches to pattern recognition. Pattern recognition systems are in many cases trained from labeled "training" data (supervised learning), but when no labeled data are available other algorithms can be used to discover previously unknown patterns (unsupervised learning). Machine learning is the common term for supervised learning methods and originates from artificial intelligence, whereas KDD and data mining have a larger focus on unsupervised methods and stronger connection to business use. Pattern recognition has its origins in engineering, and the term is popular in the context of computer vision: a leading computer vision conference is named Conference on Computer Vision and Pattern Recognition. In pattern recognition, there may be a higher interest to formalize, explain and visualize the pattern, while machine learning traditionally focuses on maximizing the recognition rates. Yet, all of these domains have evolved substantially from their roots in artificial intelligence, engineering and statistics, and they've become increasingly similar by integrating developments and ideas from each other. In machine learning, pattern recognition is the assignment of a label to a given input value. In statistics, discriminant analysis was introduced for this same purpose in 1936. An example of pattern recognition is classification, which attempts to assign each input value to one of a given set of classes (for example, determine whether a given email is "spam" or "non-spam"). However, pattern recognition is a more general problem that encompasses other types of output as well. Other examples are regression, which assigns a real-valued output to each input; sequence labeling, which assigns a class to each member of a sequence of values (for example, part of speech tagging, which assigns a part of speech to each word in an input sentence); and parsing, which assigns a parse tree to an input sentence, describing the syntactic structure of the sentence.
Pattern recognition algorithms generally aim to provide a reasonable answer for all possible inputs and to perform "most likely" matching of the inputs, taking into account their statistical variation. This is opposed to pattern matching algorithms, which look for exact matches in the input with pre-existing patterns. A common example of a pattern-matching algorithm is regular expression matching, which looks for patterns of a given sort in textual data and is included in the search capabilities of many text editors and word processors. In contrast to pattern recognition, pattern matching is not generally a type of machine learning, although pattern-matching algorithms (especially with fairly general, carefully tailored patterns) can sometimes succeed in providing similar-quality output of the sort provided by pattern-recognition algorithms.
- Pattern recognition is the automated recognition of patterns and regularities in data. Pattern recognition is closely related to artificial intelligence and machine learning,[1] together with applications such as data mining and knowledge discovery in databases (KDD), and is often used interchangeably with these terms. However, these are distinguished: machine learning is one approach to pattern recognition, while other approaches include hand-crafted (not learned) rules or heuristics; and pattern recognition is one approach to artificial intelligence, while other approaches include symbolic artificial intelligence. A modern definition of pattern recognition is: This article focuses on machine learning approaches to pattern recognition. Pattern recognition systems are in many cases trained from labeled "training" data (supervised learning), but when no labeled data are available other algorithms can be used to discover previously unknown patterns (unsupervised learning). Machine learning is the common term for supervised learning methods and originates from artificial intelligence, whereas KDD and data mining have a larger focus on unsupervised methods and stronger connection to business use. Pattern recognition has its origins in engineering, and the term is popular in the context of computer vision: a leading computer vision conference is named Conference on Computer Vision and Pattern Recognition. In pattern recognition, there may be a higher interest to formalize, explain and visualize the pattern, while machine learning traditionally focuses on maximizing the recognition rates. Yet, all of these domains have evolved substantially from their roots in artificial intelligence, engineering and statistics, and they've become increasingly similar by integrating developments and ideas from each other. In machine learning, pattern recognition is the assignment of a label to a given input value. In statistics, discriminant analysis was introduced for this same purpose in 1936. An example of pattern recognition is classification, which attempts to assign each input value to one of a given set of classes (for example, determine whether a given email is "spam" or "non-spam"). However, pattern recognition is a more general problem that encompasses other types of output as well. Other examples are regression, which assigns a real-valued output to each input; sequence labeling, which assigns a class to each member of a sequence of values (for example, part of speech tagging, which assigns a part of speech to each word in an input sentence); and parsing, which assigns a parse tree to an input sentence, describing the syntactic structure of the sentence.
- ↑ Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning (PDF). Springer. "Pattern recognition has its origins in engineering, whereas machine learning grew out of computer science. However, these activities can be viewed as two facets of the same field, and together they have undergone substantial development over the past ten years."
2014
- (Jordan, 2014) ⇒ Michael I. Jordan. (2014). “I guess that I have to say something about "deep learning".
- QUOTE: I find that industry people are often looking to solve a range of other problems, often not involving "pattern recognition" problems of the kind I associate with neural networks. E.g., (1) How can I build and serve models within a certain time budget so that I get answers with a desired level of accuracy, no matter how much data I have? (2) How can I get meaningful error bars or other measures of performance on all of the queries to my database? (3) How do I merge statistical thinking with database thinking (e.g., joins) so that I can clean data effectively and merge heterogeneous data sources? (4) How do I visualize data, and in general how do a reduce my data and present my inferences so that humans can understand what's going on? (5) How can I do diagnostics so that I don't roll out a system that's flawed or figure out that an existing system is now broken? (6) How do I deal with non-stationarity? (7) How do I do some targeted experiments, merged with my huge existing datasets, so that I can assert that some variables have a causal effect?... These are a few examples of what I think is the major meta-trend, which is the merger of statistical thinking and computational thinking.
2011
- (Giacometti et al., 2011) ⇒ Arnaud Giacometti, Patrick Marcel, and Arnaud Soulet. (2011). “A Relational View of Pattern Discovery.” In: Proceedings of the 16th International Conference on Database systems for advanced applications - Volume Part I. ISBN:978-3-642-20148-6 DOI:10.1007/978-3-642-20149-3_13
- QUOTE: Pattern discovery is a significant field of Knowledge Discovery in Databases (KDD). A broad spectrum of powerful techniques for producing local patterns has been developed over the two last decades [3-5]. But, it is widely agreed that the need of theoretical fusion between database and data mining still remains a crucial issue [14, 18, 23, 24]. We would force the pattern mining methods to fit in the relational model [1] which is the main database theory. Unlike most of the proposals [6, 10, 14, 16, 20, 23, 28, 33, 34], we desire to only address the pattern mining that we distinguish from the construction of global models 17] like decision trees.
2009
- (Ding et al., 2009) ⇒ Xiaowen Ding, Bing Liu, and Lei Zhang. (2009). "Entity Discovery and Assignment for Opinion Mining Applications". In: Proceedings of ACM SIGKDD Conference (KDD-2009). doi:10.1145/1557019.1557141
- QUOTE: Entity discovery is based on pattern discovery and entity assignment is based on mining of comparative sentences. Experimental results using a large number of forum posts demonstrate the effectiveness of the technique. Our system has also been successfully tested in a commercial setting.
2006
- (Bishop, 2006) ⇒ Christopher M. Bishop. (2006). "Pattern Recognition and Machine Learning". New York, NY : Springer, 2006. DOI:10.1117/1.2819119 ISBN:0-387-31073-8, 1-493-93843-6, 978-0387-31073-2, 978-1493-93843-8.
- QUOTE: The problem of searching for patterns in data is a fundamental one and has a long and successful history. For instance, the extensive astronomical observations of Tycho Brahe in the 16th century allowed Johannes Kepler to discover the empirical laws of planetary motion, which in turn provided a springboard for the development of classical mechanics. Similarly, the discovery of regularities in atomic spectra played a key role in the development and verification of quantum physics in the early twentieth century. The field of pattern recognition is concerned with the automatic discovery of regularities in data through the use of computer algorithms and with the use of these regularities to take actions such as classifying the data into different categories.
1990
- (Fukunaga, 1990) ⇒ Keinosuke Fukunaga. (1990). “Introduction to Statistical Pattern Recognition (2nd ed.) Academic Press Professional, Inc. ISBN:0-12-269851-7
- QUOTE: This book presents and discusses the fundamental mathematical tools for statistical decision-making processes in pattern recognition. It is felt that the decision-making processes of a human being are somewhat related to the recognition of patterns; for example, the next move in a chess game is based upon the present pattern on the board, and buying or selling stocks is decided by a complex pattern of information. The goal of pattern recognition is to clarify these complicated mechanisms of decision-making processes and to automate these functions using computers. However, because of the complex nature of the problem, most pattern recognition research has been concentrated on more realistic problems, such as the recognition of Latin characters and the classification of waveforms.
1981
- (Bezdek, 1981) ⇒ James C. Bezdek. (1981). “Pattern Recognition with Fuzzy Objective Function Algorithms." Kluwer Academic Publishers ISBN: 978-1-4757-0452-5, 978-1-4757-0450-1 DOI:10.1007/978-1-4757-0450-1
- QUOTE: The term “pattern recognition” embraces such a vast and diversified literature that a definition of it always invites debate. Verhagen [1] presents a survey of definitions of pattern recognition which cites the difficulties involved in such an attempt. Nonetheless, it is pedagogically useful for us to begin by attempting to describe what pattern recognition entails. I think one can successfully defend a literal approach: quite simply, pattern recognition is a search for structure in data. (...)
Techniques of pattern recognition are applicable to data drawn from virtually any physical process. The data may be qualitative, quantitative, or both; they may be numerical, pictorial, textural, linguistic, or any combination thereof. Examples are medical records, aerial photos, market trends, library catalogs, galactic positions, fingerprints, psychological profiles, cash flows, chemical constituents, demographic features, stock options, military decisions. Techniques for data processing we call the search (...)
- QUOTE: The term “pattern recognition” embraces such a vast and diversified literature that a definition of it always invites debate. Verhagen [1] presents a survey of definitions of pattern recognition which cites the difficulties involved in such an attempt. Nonetheless, it is pedagogically useful for us to begin by attempting to describe what pattern recognition entails. I think one can successfully defend a literal approach: quite simply, pattern recognition is a search for structure in data. (...)
- ↑ C.J.D.M. Verhagen (1975) "Some General Remarks about Pattern Recognition; Its Definition; Its Relation with other Disciplines; A Literature Survey,15. Patt. Recog., Vol. 8-3 (1975), pp. 109-116.