your assumptions:

  - the "intention" of the robots.txt standard is as you state
  - the url is included in the information not allowed by that standard
  - if the url is not included it should be because of the "intention"
  - toolbars are subject to the same standards

I'm not disagreeing with you as much as just pointing out that I don't think EVERYONE agrees on these standards.