I also prefer using the terminal, but I've switched to neovim anyway. I found macvim to be difficult to configure and I just prefer having tmux terminal splits next to my vim split and I can see stuff happening as I edit the file. I also prefer iTerm2 to Terminal.app but I think it's just personal preference. I know how to use the iTerm2 settings pretty well by now to get configurations I want.
For me it's definitely mouse reporting. If my hand is already on the mouse I really like clicking tmux panes. Sometimes it's also easier to just scroll with a trackpad too.
To be fair, you can do the same for iTerm (or any application, really) by using other tools. Quicksilver, for instance, allows you to define system-wide hotkeys.
Also, I find MacVim to be a better experience than Vim in the terminal.