Content

• Preface
• Classical Supervised learning
• BAYES DECISION THEORY
• Support Vector Machines(SVM)
• Linear Discriminant Analysis(LDA)
• quadratic discriminant functions (QDA)
• KNN models
• Naive Bayes models
• case study
• breast cancer
• Iris Data
• Using chemical analysis determine the origin of wines
• Appendix(code source)
• QDA algorithm
• KNN algorithm
• Naive Bayes models algorithm

Preface

Machine learning has been a hot topic in recent years. Due to its wide application, many people believe that it can do anything. This is a naive viewpoint. I myself have also been passively involved in some industry tasks that have nothing to do with machine learning.

In specific application scenarios, machine learning can solve problems that traditional mathematical methods cannot. I won't go into detail about which scenarios are suitable for machine learning and which scenarios are not, but if you are interested, we can discuss it. Actually, understanding the principles behind various algorithms and the development of classical mathematical disciplines will naturally lead to our own conclusions.

In fact, the algorithms of machine learning and many classical algorithms are similar and have a certain history, not just suddenly appeared recently. Around 2005, I myself began to pay attention to some interesting algorithms. However, I rarely spend time on mature commercial and open-source libraries. Because I am not a machine learning or statistical professional, at best I am just an enthusiast. Learning mature libraries is not meaningful to me, and understanding the principles seems to be more attractive to me.

Only by understanding the principles can we distinguish ourselves from engineers who only know how to "tune parameters".

I have written several supervised learning algorithms before, but these algorithms only target specific examples. I have never had the courage to write a universal algorithm. Actually, writing a nearly universal algorithm is not difficult mathematically, but software development involves many skills and generality, which is a good way to kill brain cells.

There are also some algorithms that I know the method, but writing universal software is very complex, so I don't bother to write it. Mainly because there is no project task drive, I simply don't have the energy to write it. Thinking of the authors of many mature machine learning libraries, they not only need advanced programming skills, but also need to work hard and be familiar with statistical and mathematical knowledge.

Here, the main implementation is classical machine learning methods, and some emerging fashionable algorithms have not yet been written. Due to time constraints, updates may be slow.