I tried reading the Ginsburg translation and didn't get very far. Found it to be wooden and stilted. Recently I started reading the Pevear and Volokhonsky translation and I'm having a much better time of it. It has footnotes that explain some of the references, and I'm just finding it much easier to read from a prose-style standpoint.
According to my russian wife who studied both russian and english literature, the Pevear and Volokhonsky translation is by far the best. Not only does it provide a great introduction and footnotes, it also manages to capture the spirit of the original.
I found it a highly enjoyable book and, while I'm sure I missed many references, it gave me better understanding of what it must have been like living in the USSR in those times.
I think it would be hard to get through without the discussions and some guidance.
The professor (real not in the book) was pretty good at making it more interesting.
I'm not a great reader though and read mostly non-fiction now. I go slow. I tried to read the "sound and the fury" by Faulkner once and didn't get past the first chapter.