Shameless plug here - The author of the article does build time-series databases for a living, and more specifically the Datadog monitoring platform - which will gladly collect your millions of metrics, graph them and alert on them, along with all the events you care to keep :-)
We've been through a number of data stores ourselves, starting with Postgres back in 2010 - then on to Redis + Cassandra before we built our own. But that's a story for another post...
We're hiring engineers that specialize in one of the following:
+ Data visualization in the browser
+ Go and distributed systems
+ System programming and low-level performance optimization
+ Data Engineering on the hadoop/spark/python stack
We're one of the very few NYC startups to be engineering-driven and working on hard, large-scale data analysis and visualization problems. We're also growing very fast :-)
Our product is built by engineers for engineers which means you can and will have a lot of impact on it.
> it has the grace you'd expect of a 60's housing project in the eastern bloc
Maybe because I'm now writing this comment from inside a 70's housing project in the (former) eastern bloc, but lately (meaning the last 2-3 years) I've seen a re-appraisal of the eastern bloc esthetics.
"Brutalism - for people who like living in unfinished construction sites covered in seeping water damage"
I cheer a little on the inside whenever I see a brutalist building being torn down. There's so very few buildings of the type that are worth anything, and so many of the type that barely last a generation before needing repairs that amount to new construction cost.
I also live a couple blocks from it and the only upside is that it'll eventually block the rusty looking Barclays Center from view. I'd gladly take the old Freddy's back.
Does rust automatically turn people off because of cultural connotations? It's self-finishing, already appears established, and doesn't require maintenance.
I like the material, particularly when it's juxtaposed with nature, but its use on the Barclays Center doesn't do it for me. There's nothing around the structure that relieves it.
I work at Datadog - we're only using ElasticSearch for full-text structured events, not time-series, which represent 10,000 - 100,000 times more data in volume.
We had to build our own Time-Series streaming / storage / query so we could handle millions of points per second and years of retention.
FWIW we've been using the mortar platform to run large pig jobs without a fuss at http://datadog.com and we've been very happy with it. Glad to see them contribute their recommender code too.
We don't use the recommendation engine but the underlying platform, which makes it really simple to write and run pig jobs. Though the majority of our business deals with real-time data processing, the ability to crunch numbers in batch without dev or ops overhead is attractive and well worth the price to us.
We're looking for engineers passionate about either
* Data visualization using D3
* Large-scale real-time data processing in Go, Python and C
* UX/UI Design and development
* Writing technical pieces you'd want to read
We're a growing, well-funded startup. And we're hiring engineers for all positions from back-end to ops to data visualization.
Think tens of billions of data points a day to stream, store, visualize and analyze. We use Python, Numpy, Cython, D3, JS, Cassandra, Postgres, Redis, Elasticsearch, and more...
We've been through a number of data stores ourselves, starting with Postgres back in 2010 - then on to Redis + Cassandra before we built our own. But that's a story for another post...
http://datadog.com