What are some resources on computational learning theory?

4

6

Pretty soon I will be finishing up Understanding Machine Learning: From Theory to Algorithms by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in JMLR's COLT.

  1. Is there a book similar to "Understanding Machine Learning: From Theory to Algorithms" that would progress my knowledge further and would go well after reading UML?

  2. Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?

(Also, taking courses in this is not really an option, so this will be for self-study).

(Note that I have also asked this question here on TCS SE, but it was recommended I also ask here.)

PMaynard

Posted 2020-04-17T16:04:40.127

Reputation: 151

Answers

7

Although I have only partially read (or not read at all) some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you.

I would also like to note that if you understand the contents of the book you are currently reading, you are probably already prepared for reading some (if not most of) the research papers you wish to read. Initially, you may find them a little bit too succinct and sometimes unclear or complex, but you need to get used to this format, so there's nothing stopping you from trying to read them and learn even more by doing this exercise.

Books

Papers

Courses (videos)

Lecture notes

Other

See also this list of resources https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc compiled by Kiran Vodrahalli.

nbro

Posted 2020-04-17T16:04:40.127

Reputation: 19 783

Thank you so much looks like a great list! I didn't know that computational learning theory was very similar to statistical learning theory. Great papers I actually read A theory of the learnable earlier this week. Lots to work through here, any other tips are appreciated as well. Additionally it would be nice to know about how far I am from being able to read and understand journals like COLT. – PMaynard – 2020-04-17T18:23:43.557

@PMaynard I will keep updating this answer with more resources. The idea is to provide a comprehensible list of useful resources. Maybe, in the future, I will also provide a description of each of these resources and for what purposes they are more useful. – nbro – 2020-04-17T18:28:28.757

@PMaynard Regarding the part of being able to read the COLT papers, I suggest you just pick 1-2 papers (e.g. from http://proceedings.mlr.press/v99/), and try to read them. This will make you understand your level. In any case, you shouldn't expect to be able to understand them in one reading iteration. It may require more iterations. The more you read, the easier and the faster it will be to capture the concepts in those papers.

– nbro – 2020-04-17T18:28:33.023

1@PMaynard Note that some papers are very specific to one topic. So, even if you have good knowledge of the basics and even certain advanced topics, you may not be able to understand those papers without having knowledge of those specific topics in those papers. This is perfectly normal. – nbro – 2020-04-17T18:32:06.673

1Thanks, once again. Someone else had the same suggestion as far as how to start reading COLT so it sounds like a great idea. Maybe also a good idea to focus in on an area I like, reading and understanding everything is probably not a very practical goal. – PMaynard – 2020-04-17T18:34:16.307

@PMaynard Yes, exactly. Once you have a good knowledge of the basics, I suggest you focus on one topic (the topic you are most interested in). Researchers don't typically tackle many topics at the same time, because, as you say, it's quite impractical and difficult to be an expert on multiple topics at the same time. – nbro – 2020-04-17T18:36:12.280

@PMaynard If you find other books, papers, etc., that you think I should add to this list, let me know and I will add them. Or feel free to propose an edit for this answer ;) – nbro – 2020-04-17T18:40:48.510