People that work in "ad operations" usually do that work by performing A/B tests etc. To me, it is a very unglamorous step child to a data scientist.

In this context, i think A/B tests would generally be the opposite of user tests.

Typically with an A/B test you'll test something like clicks. With a user test you'll have people in to try it out to see things that you can't see in the data. In this case, I think you could look at an A/B test and say, "Ad clicks are up! Great!" But in the user test you might say, "Oh, these clicks are accidental, so although it makes more money in the short term, it will decrease the value of the click and the value of the property.