I've been amazed how helpful people have been here to career advice questions earlier, so as I feel I have hit rock-bottom today, I give it a try myself (I am a regular poster, but anonymous this time). I guess others might be in similar situation, so discussing this might be helpful.
I have always dreamed of becoming a scientist. I always loved maths, sciences, puzzles, theory but also programming and engineering. To illustrate this, I would say that probably I look up to Sergei and Larry more for inventing PageRank, than for building up their company, although I'm interested in the workings of the economy as well, but more at an abstract level.
So getting accepted into grad school in one of the top labs in its field seemed like a dream come true. I really admire some of the professors and postdocs in the lab. Many of them have achieved ground-breaking theoretical results in the past and their groups have produced a great number of widely cited papers, surveys, useful toolboxes, etc. I was anxious to take part in similar work, however small my contribution would be.
I was even excited when I was told that I will work on a newly started, well-funded, "interdisciplinary" project, that will greatly impact people's lives. Well, I was suspicious of the buzz-wordiness initially, but how could I not trust the judgment of such smart people. After almost a year, it is fair to say that I have good view of the project and it is almost surely a giant failure. I am completely burnt out and have given up all hope of getting any meaningful work done in this project. It does not solve any real problem, there is no potential for any good science within it and no-one has the slightest idea what it is about and where it should progress. I can't say exactly what it is, but in scope it could be imagined as similar to the EU-funded Google competitor that has been widely ridiculed before.
On the surface everything still looks good, we have plenty of meetings, senior group members travel to conferences to project partners, we have an active wiki, we make demos from time-to-time, the reviewers are happy, funding is good, etc. etc. However, I find it the most soul-crushing experience to continue working while pretending that the whole thing is getting anywhere. Any attempt I made to point out ways how it could be made more practical have been ridiculed, or even trying to question some of the assumptions have met aggressive reactions from the professor, so I have given up.
Watching my collegues I have noticed different kinds of behavior. Some of them are smart and realize what is going on but they have become cynical and play along, trying to get their own stuff done on the side. Others are just too incompetent to notice something is wrong, and are happy to have a good place where they can surf the net the whole day, while pretending to be scientists. Some of my collegues are completely clueless about programming or engineering, but having good observations, comments on meetings, being generally friendly goes a long way. I have to mention that this is in a place where tuition is free, grad-school pays about 60-70% of what one could earn in the industry at the moment, so it is a safe choice financially. Not much is expected of anyone, I guess if I wouldn't turn up for a few days in a row, no-one would notice.
So what do you guys suggest I should do? I have given up getting anything useful in this project, but still research is my main dream. Is there hope of better in a different grad-school or project ? How can you tell from the outside ? Is this frustration common or is there actually a way to get honest theoretical work done in grad-school ? I consider myself a hacker as well, so working at a start-up or big co. would both be an option, but I would still most love to work in research, in the idealized way that I imagine it. (find elegant solutions to difficult problems that have wide-reaching implications). Should I ignore the environment and just try to set my own research agenda ? Is it possible to do research outside of academia ? (I'm a bit afraid of becoming a crackpot publishing papers on perpetual motion machines on arxiv :-) ? Have you been in similar situation ? How did you manage ?
Sorry for the long rant, and thanks for any suggestion...
Let me talk a little bit as a rare case: I've worked now under 7 advisors on 7 different projects and now I'm finally almost done. Most of the projects have been as you describe: "after almost a year, it is fair to say that I have good view of the project and it is almost surely a giant failure."
That's how most projects are. I'm not sure why, but I think it has to do with the fact that the projects are "planned research." The process of getting funding requires planning something that is intrinsically impossible to plan. You write a grant on hope, with the large picture in mind and then you get down to the details and things don't work out. This is normal.* Especially on the time frame of a new graduate student, it seems terrible.
To some extent, though, the system works. It does so for the same reason that some startups work: because as you look at the details you find new things that you couldn't have predicted. Those new things are your research. I'm working on a "failed project." But after 1.5 years working on it I am ready to begin writing a dissertation that I am proud of. Why? Because I found neat things along the way. That's how it works.
If you are in a good lab and surrounded by good people, I would recommend that you don't focus as much on the larger project as on learning all you can from the people around you and on understanding the details that your project will lead you to focus on. It is in helping other people, tracking down details, and playing with interesting questions that you will find the great science, not directly through the success of the larger project.
By the time I finish, I will have learned a lot. But I will have spent almost 8 years in graduate school, 2 for a masters in one field and almost 6 for a PhD in another field. If I had had my current perspective from the beginning I would have been a little more patient and I might thereby have saved myself three years. So there is a cost to hopping around even if there are benefits in terms of perspective.
Best wishes.
*I know of one exception. He explained his system: he does the research, then he gets grants for his research. This gives him time to do new research and he writes new grants based on that. He delivers because he only proposes to do what he has already more or less done!