-
Building a career in ML:
- T-shaped: breadth of knowledge + significant depth in one area. - Breadth: coursework + reading. A highly organized course is an efficient way to learn, but need research papers to learn the state of the art. - Vertical: building projects, open source. Know how to actually build machine learning systems. - Importance of dirty work: downloading/cleaning dataset, plotting learning curves, e.g., looking at a lot of plots of PCA. - Lifelong learning: read a few research papers every week. Set up your life so that every week you can read/study a little bit. - Job market is correlated with your true skill, as opposed to "stamps of approval" on resume. - Biggest untapped opportunities in ML are outside the software industry in more traditional centers (e.g., agriculture, fashion). -
As a student, how does one plan to develop their career while in academia? E.g., Master's, Phd, vs. straight into industry.
- Master's or PhD on resume is helpful, but not _necessary_ (to be in the highest level technical roles). Evaluate opportunities as they come up. - Surround yourself with friends/people doing great work. When looking for a job, focus on the 10-30 persons who you will work with most closely, as well as your manager/advisor. - Don't focus on the brand of the company, unless you can get into the (potentially small) team in that company that does great work. -
Specialized vs. jack of all trades?
- Important work in ML may be cross-disciplinary, but not necessarily each individual. Interdisciplinary teams. -
10-20 research papers: basic understanding of a new area.
-
50 papers: reasonable grasp of the area.
-
100 papers: you can come up with research ideas.
-
Are there areas/techniques that could be interesting/valuable five years from now?
- Current/early practice: train a giant neural network on many tens of thousands of hours of data. - Underappreciated area: how to apply learning algorithms even on small data. If you have only 100 images then the skill of the machine learning team matters much more, and the tools are less mature. - Weakness of learning algorithms today vs. radiologists: generalization from one X-ray machine/hospital to another. Generalizing to new distributions of data. - Software engineering world: e.g., code review, scrum, agile. What are the processes for teams working on an ML project? E.g., train product managers to define requirements/specifications statistically. -
Importance of programming to ML engineers: makes an interesting analogy to literacy, where in early times everyone went to the monks in regilious institutes if they needed anything read. Due to the importance of human-machine communication, the whole world would be better off if everyone knew how to code.
-
Ways to find ML job opportunities outside major software tech industry?
- No good suggestions: search online job sites etc. - Challenge: lack of CEOs/managers who know how to build ML projects. Top question from CEOs: how do I build an ML team? How do I decide what projects to work on (i.e., what is technically feasible/valuable)? E.g., CEOs promise an aggressive roadmap for self-driving cars, and literally 20 minutes later speaks with an engineer who says what the CEO proposed is not feasible, and they are not doing it. - Suggests that managers learn about ML via Coursera (AI for everyone course -- to be released). -
How do you develop a habit of figuring out which content you're looking at is valuable, as an engineer?
- Coursework is the efficient way to pick up knowledge. - First do coursework, then, if you want to learn an area in depth: do internet searches to curate a list of related papers. - Find five initial papers in the vertical area. - Skim all five papers (10% of the effort to completely understand each paper). - Identify the important papers of those five and read them to 100%, and to find subsequent papers. - Eventually find 20-100 papers, and systematically go through them. - Apply this for each new area you want to learn about, e.g., GANs, model compression for small memory footprint. - Participate in reading groups with your friends. -
How to prepare ourselves/our children (not ML related??)?
- Cultivate lifelong learning. - Teach kids to code. - At the societal level: doesn't agree with unconditional basic income. Wants to see governments implement "conditional basic income", where people have to do something, such as study, that can increase the odds of the person being able to join the workforce in future. -
What got you interested in ML?
- Showed his father's (a doctor) paper from 1980, applying ML algorithms to medical diagnosis. Computer Aided Diagnosis of Liver Diseases, R. P. Ng.