I use to think web scraping was limited to obscure proprietary companies like connotate so I'm glad to see more of these tools becoming available for everyone to utilize and hopefully create something cool.
Intercom.io is a very cool tool indeed. We use it to extract unique product links from product overview pages of webshops for adding them to Pricepin (our own tool).
Oh well, what can you do? It's like how the Kleenex name has become genericized whereby "the popularity of the product has led to the use of its name to refer to any facial tissue, regardless of the brand. Many dictionaries, including Merriam-Webster and Oxford, now include definitions in their publications defining it as such." - https://en.wikipedia.org/wiki/Kleenex
Sounds like a nice idea but isn't Google News already doing a decent job on this? It's going to be hard to beat the speed of Twitter or Google News in my opinion but I'll be curious to see where this goes.
Not really. Google doesn't summarize facts. It aggregates articles into clusters, which is useful, but it doesn't save us much time. Google also does a rather poor job of ranking the importance of stories, and often has old news for a long time and breaking news relatively late (or later than a motivated community, like Twitter, can have it).
Google News' strength is ranking, not summarizing facts from different sources so you can avoid reading all the top 5 or 20 articles about a story.
And, philosophically, do you HN folks really want to rely on the MSM to tell you what's important? And likes, shares and troll-ridden comments on news stories?
Or do you want a democratized way to decide, collectively, what news stories and facts are important.
Don't forget the NYT, etc. screwed up on no WMD's in Iraq.
I was at a small group dinner with Judith Miller, ex-NYT, who's nice but she messed up.
"The most important input for your pricing is quite simple: are enough people buying it at its current price? If not, you might have a pricing problem, but not necessarily. If enough people are buying it, you face the interesting question: can you charge more?"
Wished he'd defined what's "enough" at least for him.
I think that's the point: He can't, and if he did, it wouldn't be relevant to anyone else. Figuring out what entails "enough" is exactly the exercise that makes the question useful.
Ugh, so depressing... go to college, rack up enormous amounts of debt, and hope to find a job so that you can start paying off the monthly minimums for the seemingly the rest of your life.
In certain fields the "go into debt to pay for an education which will help you get a good job" strategy works well. Unfortunately, governments and educational institutions have taken "college graduates earn X% more and have a Y% lower unemployment rate" statistics and pretended that they apply to all college graduates, regardless of their field.
Only a fool would think that a degree in underwater basket weaving would provide the same employment prospects as a degree in any of the STEM fields, but fools are precisely the target market for institutions providing such courses.
It hasn't been too long since I was starting college. For freshman (in the US at least), colleges encourage students to try lots of classes and take their time picking a major. Many schools don't require students to declare a track until they are in their sophomore year -- halfway through a four-year program! Many can be completed in 2-3 years (even "hard" majors, like physics), and the schools design the degree programs with this buffer so students can wait to make a decision. Of course, the schools don't mind recieving four years of tuition for an education that could be completed in three.
But it's easy to blame the schools. In my experience, students don't really make career choices (like picking a major) by weighing the economic costs and benefits. Our current culture in college encourages students to take on massive amounts of debt regardless of their field. For many liberal arts degrees, this is probably a foolish financial decision. But for many college freshman, the apparent difficulty of a subject is considered thoroughly, while the job prospects of a career path are a secondary consideration.
for many college freshman, the apparent difficulty of a subject is considered thoroughly, while the job prospects of a career path are a secondary consideration.
Right. My point was that this is partly because of the educational culture which promotes "getting a degree will make you earn more" rather than "getting a degree in subject X, Y, or Z will make you earn more". Students who enter college intending to study science but can't hack it would probably be better off dropping out; instead, most end up in liberal arts and are surprised when their degree turns out to be useless because all the advertising they ever saw promoted "having a degree" as the only relevant flag.
But you can be smart about this by hedging your bets and still achieving the same effect. A degree in basket weaving and a degree in Engineering will both serve to expand your abilities by teaching you critical-thinking skills, teaching you how to communicate more effectively, and, in short, providing you with a framework for "learning how to learn". But one will make it far easier to get a job than the other.
My rule of thumb has always been this: Write down all of the things that you're interested in and would enjoy spending four years or so learning about. Then, from that list, pick the ones that are also employable to some extent. At the same time, don't simply pick a degree solely because of employment prospects, since markets are cyclical, even in classical STEM fields.
Except Chemistry, biology, and biochemistry (massive R&D cuts by the pharmaceutical industry has flooded the market with experienced scientists).
Except CivEng (oops, not much new construction).
STEM degrees are often considered harder by employers, so for many jobs that don't strictly require them, they're used as signalling mechanisms in exactly the way that having a degree at all used to be a signal. That means there's a problem with encouraging more people to get such degrees because you flood the market with STEM degrees and repeat the process of devaluing the signal.
What does that leave? CS degrees are currently very much in demand, but will that remain the case for the next 10-20 years? Will it even be true next year? There were a lot of people who had employment trouble after the first bubble burst, I would hate to see that happen again. Not to mention that this is hardly a solution that will work on a large scale. The industry does not require a million new software engineers every year.
Would anyone have predicted ten years ago that going to law school would be a disastrous financial and career decision for many thousands of people? That was supposed to be one of those "safe" direct-to-career educations. Hasn't worked out that way.
The average debt of college graduates for the class of 2010 is $25,000, and is estimated to be $27,000 for the class of 2011 [1]. Compared to housing, this is not a lot of debt. Many people these days will even buy a car on credit, which can be a comparable amount of debt, of course depending on the car.
I know education is a common theme here on HN, and I always read articles like this. I still think education (esp. in US and UK) is ripe for entrepreneurial disruption, and I enjoy seeing what investors are doing in this space. The system is clearly broken, because incentives are out of proportion to costs.
Alas, entrepreneurial disruption in this field is made near impossible by the need for governmental oversight of all things related to education.
Until "official" degrees stop being an unspoken requirement when considering job applicants, startups such as Khan Academy will remain useful, but ultimately powerless to truly disrupt education.
I wouldn't be surprised if the system operated in the system's best interest, and not the student's. I only have anecdotal evidence, but from my own experience, the whole system was designed to keep you in the system for as long as possible.
This may have been exacerbated by stupid government targets to get, say, at least 50% of all pupils into university. And in at least some of those cases, you're only going because you're brainwashed into it.
Yep. Incentives always get out of whack when the government gets involved. It's what's happened to health care, retirement funding, mortgage financing, on and on and on.
When incentives aren't aligned with human nature, inefficiencies amplify and eventually break whichever system the government stepped in to fix.
Facebook is the best thing that's ever happened to me. The people from my past that I really don't want to converse with are likely tied up in their simple little Facebook world by now. Let them wonder why I'm not on Facebook. Problem solved!
Just tweeted this to the author and thought I cross-post it here as well: Wonderful piece, I just had those same thoughts in the midst of all this facebook IPO frenzy... stay hungry, humble, and healthy! (Hmm, I can call it the three H's :))
