It seems like reinforcement learning would be useful, i.e. at a high level, forming a policy for recommendations would require balancing exploration (experimenting with more risky recommendations) vs. exploitation (showing you recommendations that it knows will likely lead to clicks) and using the click-throughs, time spent watching the video, etc. as reward signals.

Does anyone know whether RL is used for recommendation in practical settings, and if so what is the current state of the art?

This is a very natural avenue and an active area of research at Google/Deep Mind. Stay tuned...

Apostol's calculus texts were my first experience with 'real' proof-based mathematics, and I also learned real analysis through his Mathematical Analysis book. Good times; his books taught me a lot.

He wrote with clarity and conciseness; when I read Apostol it always seemed like not a single word was added unnecessarily. I highly suggest picking up his books, and am sad to see that he passed away.

On the probability side, I thought Ash's "Probability & Measure Theory" was good for self-study, although some experience with real analysis is definitely a prerequisite. It can be pretty dense at times, but well worth going through.

I like the observation about the importance of concentration.

In addition to reading, I've noticed that studying mathematics is an opportunity to practice (or build) the ability to focus on frustratingly difficult material that requires a high attention to detail.

People talk a lot about programming being similar to math, programming ability being similar to mathematics ability. I've always been a shitty symbol-pusher and a reasonable programmer, so I thought this was a bunch of bullshit. Then I actually took a post-calculus math course (Intro to Analysis and Linear Algebra). The material was irrelevant but the cognitive muscles exercised were very similar.

Does this also apply to large industrial labs, e.g. MSR, IBM Research, Google research etc.? Or are there software developers who work together with researchers in those places, perhaps resulting in better quality code?

Microsoft Research has RSDEs ... research software design engineers. IBM Research also has a research software engineer position, with its own career progression track. These folks are generally fantastic developers who have been professional software engineers some point in their careers. Moreover, I have never personally seen a case where a research engineer was not put on a paper for being just a coder.

It is problematic when a team of scientists collaborate with a product group (not research software engineers but regular developers). In the limited projects I've been involved in, we ask if any product group engineer wants to collaborate on a paper we are writing. No one is typically interested and we would at least add them to acknowledgements. I found product group developers were interested in collaborating for patent submissions. I definitely don't want to generalize but take this as a data point.

Correct me if I'm wrong, but my understanding was that being an RSDE at MSR was still a dead-end position.

FWIW product engineers usually get bonuses for patents, so there's the obvious incentive of $5k in your pocket.

I made the switch from Researcher to RSDE and I'm not sure what you're talking about.

There are plenty of Principal RSDEs and some become Managers of technical leaders of teams, which sometimes even include researchers.

But of course, the track is not as clear and common as for plain Researcher or SDE (in a product team).

PS: As far as I know, the patent bonus are there for all (though I can't comment on values).

If we are talking Redmond, I've known plenty of RSDEs who have been promoted to manage their own groups. They also have the option to transfer back into a product development team, which can be very difficult for us pure researchers.

Things are changing.

Also, if you're on a patent, you get the cube (and the cash).

That makes sense, thanks

I would expect that most of the folks working at computer company labs would write their own code and be very knowledgeable in a certain domain.

Having worked as a SWE at both pure academia (as the engineer implementing for researcher-driven projects at a university) and in a similar position at a bigCO, there is certainly more "software chops" at the latter, but the general sentiment of the research types being far more competent at research than at code is not necessarily unfair. However, the "developer presence" in these groups is much higher, (and what is expected of even the pure research folks) so it's not quite as stark as outside industry. Still a different world from the pure SWE teams, however. The intesting thought I'd add to this is that the problems don't always manifest as "bad code". Things like measuring tradeoffs and compromises are not always as well thought out in the research teams. (I'd mete this statement slightly, to say that researchers have very different mental models for what tradeoffs to make in development from someone with a pure developer background, and this is not always bad, just evident.)

Agreed, I was just curious since some of the comments below seemed to imply that PhDs / PostDocs lack certain software development skills (although I personally haven't seen this). If so, this would either suggest that the code quality in computer company labs may be low, or they hire based on research potential AND ability to develop software