Last time I check, multi-process byte-range file locking was a mess. I'm surprised by your statement about BerkeleyDB, because as far as I can tell it simply cannot be safely implemented.
That's a fair criticism, and well taken, but keep in mind I'm learning the language as I go, and most of the commits are just rewriting things because they were suboptimal, not idiomatic, or hard to read. At this stage, I would consider adding comments wasteful.
I also have no intention of making this project live as long or have as many users as Redis does.
If you want good examples of Rust code, you should probably look into other repositories. Today I was checking out how Rust's HashMap works and I found it quite readable:
I don't know Rust, and the only thing opaque to me in that line is the `'a` syntax. I suppose it is related to memory safety.
In plain English:
`search_entry_hashed` is a parametric function. It works in a type safe way on any kind of K and V. It takes as arguments a reference to a mutable hash table, a hash function and a key, and returns an `Entry` object whose precise type depends on the type of the parameters.
You could hardly express that more succinctly. You can't understand it by skimming it, but it is a complex definition.
Your guess about 'a is correct. 'a is here used to signify that the return value can only be safely used while the value table exists and is not being concurrently modified. This implies that the value is not copied on return, but only a reference into the table is returned.
I'm surprised that RawTable isn't parametrized on the type of hash function. Something like this (possibly wrong way to express it, but you get the idea)
"'lifetime" gives no useful semantic information there. All 'somethings are lifetimes; it would be like naming a type parameter "Type," or a variable "variable." If you don't have a specific shard context that would give the name meaning, `'a` is as good as any. I do agree that something like Key or Value would probably be better than K or V.
Yes, in general, it is a good idea to give more descriptive names but in the context of a hashtable, K and V are good enough. I seriously doubt that there is anyone reading the source code of a hashtable and cannot figure out in milliseconds what K and V mean (how fast is the human brain?:).
Depends if you know Rust. The language can do things with types and memory that (say) something like Python or Ruby don't do, which means Rust needs extra syntax to mark those things. Most of the <> or ' are to do with types or memory safety.
I had no intention of making the end result useful, but I run into interesting problems.
First, I wanted to make it as pure Rust as possible. I tried to avoid UNIX specific code, and since there is no library with Windows support for asynchronous IO in Rust, I was pushed into spawning thread and blocking waiting for client data. I quickly noticed that the benchmark was way below Redis (around 60% of ops/sec with 50 clients). But then someone point out to me[1] that I was running tests in a machine with two cores, and this actually may be better for machines with multiple cores[2]. I have yet to try it out and benchmark the results.
So far Rust API was disappointing for network operations. For example, `TcpStream.read()`[3] and `TcpListener.incoming()`[4] do not have a timeout. Maybe because its development is driven for Servo and not for servers.
I have thought about doubling down on multithreading and instead of a global database lock as rsedis is using now, having one per key (or some other arbitrary partition), and having concurrent operations, which is hard to do safely in C. But I have not gotten there yet.
> set_timeout has been removed for now (as well as other
> timeout-related functions). It is likely that this may
> come back soon as a binding to setsockopt to the
> SO_RCVTIMEO and SO_SNDTIMEO options. This RFC does not
> currently proposed adding them just yet, however.
And on UDP:
> All timeout support is removed. This may come back in
> the form of setsockopt (as with TCP streams) or with
> a more general implementation of select.
I'm on shaky wifi and my phone, so I can't find a citation for this, but I also believe it was removed due to 1) Rust not having any stable representation of time and 2) needing to shim certain behaviors on some platforms, which we decided wouldn't happen in the first round of stable interfaces.
That said, the lack here certainly hurts, and we did manage to stabilize Duration, paving the way for timeouts to return.
Thanks for the clarification. I wanted to subscribe to rust's internals debates and proposals, but I was not sure how to find them. Should I be looking at https://github.com/rust-lang/rfcs or is there anywhere else?
1. We keep open issues on that repo to track ideas.
2. At some point, someone may decide to formally propose an idea. They may or may not post a "pre-RFC" to internals.rust-lang.org to get early feedback.
3. An actual RFC will get filed at that repo as a PR.
4. The relevant sub team will check it out and comment, and at some point, consensus will be reached either way.
5. The RFC will go to 'last call' for a week, making sure all voices have been heard.
6. Assuming nothing in last call blocks moving forward, the RFC will be accepted.
7. The RFC will be implemented.
So, in short, yes, subscribing to that repo will notify you of the relevant discussions.
IRC is used for quick casual conversation. The internals forum is used for discussion, "pre-RFC", and the like. The rfcs repo is use for actually discussing formal RFCs, as well as wishlist language and library bugs. The rust repo is for the actual implementation of rustc and the standard library itself.
Your comment is interesting to me because although i've only looked at Rust from a distance, its seemingly lack of features related to server side code ( threading , non blocking io and networking api) have made me wait before i actually start coding in that language.
It depends on what you're doing. See https://github.com/jonhoo/volley, which uses blocking IO and 1:1 threads, and we're still damn fast.
There are also libraries like mio that can give you no blocking IO. Rust's standard library is deliberately minimal, exploring the ecosystem a bit is helpful, though admittedly not as easy as it could be. Working on it!
Right. People should realize that there is no latency difference between blocking and non-blocking IO -- in fact there is a small difference in blocking IO's favor. The difference is in throughput when there is a very large number of concurrent requests, namely more than about 10K and certainly more than 2K. If you're serving under 2K concurrent requests, blocking IO is probably a better choice than nonblocking. Of course, the number of concurrent requests you need to serve depends on the time you spend serving each request; Little's Law and all that.
Also notice that the main developer only has two months of experience with Rust, so it is probably not as stable and well tested as Redis. As stated in the README, the main goal is to learn Rust.
This article feels like a follow up on "On Ruby"[1] and related to "It’s OK for your open source library to be a bit shitty"[2]. The common theme I find in all three is the quality and health of a dependency. I agree with OP that stars and latest commit are not clear indicators of quality.
I think more useful indicators are tests, documentations and pull requests.
* Tests. These are things you can actually run to see if the library actually works. In most cases, you can read them and understand how it is supposed to be used. On some platforms even if the tests passed in the past, they may not pass now, for example using different version of the language or platform. The coverage of the tests would be nice, but it is hard to measure at first sight, therefore using the ratio of tests LOC to library LOC might be an indicator.
* Documentation. The documentation itself does not change how the library behaves but it is a clear indicator whether the author expected someone else to be able to use it or not, and shows a sort of responsibility. Most of weekend hacks would not have one.
* Pull Requests. If there are old and open pull requests, that seem useful, it is a bad sign for the maintainer of the project.
The technical quality of the project has been so far debatable[1][2].
The Partido de la Red results seem to reveal a problem that is obvious at first sight: the underrepresentation of those who need representation the most[3].
I have doubts about whether direct democracy is a good idea, at all[4]. Summarizing, dividing the burden of decision making is inefficient for the population at large, and it does not guarantee better results.
People behind this project admit that this does not fully solve the problem it is attempting to solve[5].
Even if I could ignore the previous statement (which of course I can't), I wonder if the advantages of moving from an indirect democracy to a direct democracy are enough to even try to solve the problems. I think the cost of switching is high, and I cannot see any benefit.
In USA the main problem with democracy is the lack of interest of population in government elections. Does micromanaging decisions make this problem better or worse? The question is valid. It could be better if the feeling is that no matter who wins nobody will represent the voter; it could be worse if people do not care about the decisions that have to be make. There are other possible options, of course, but I don't really know if this solves problems or makes it worse.
I wonder how selection bias would affect decision making. People will mostly vote on subjects they feel passionate about, and ignore the others. Which will probably lead to what programmers know as bikeshedding[6].
Now I would like to highlight the positive of thinking about this problems and attempting a solution. Also following up Congress's debates and actions is something that should happen more often, for example Congressional Dish[7] attempts to do so.
It's not "direct democracy". It's just democracy. If you trace back the term used by the Greeks, they called both the technology and the concept agora. Etymologically, agora means "speaking in public", "thinking with others" and in many verses it's used as antonym of war.
Democracy is always a work in progress. Otherwise it would be a totalitarian concept.
So what we are trying to understand is simply democracy but for the digital age. Using computers and networks, and the power of software. Open source, collaborative free software.
> It's not "direct democracy". It's just democracy.
Uh? I think we are in a Representative Democracy[1] and what DemocracyOS proposes is a Direct Democracy[2]. It is not "just democracy" since both Representative and Direct democracies are forms of democracy.
we had votes in every single neighborhood of the city. Our correlation was better with progressive parties (ARI) than conservative (PRO) ones even. Actually in Congress, The Workers Party (PO) brought more users to DemocracyOS than ANY other party.
I tried to do the math myself about the correlation, but it seems like my statistics teachers would be disappointed[1]. What R^2 did you get compared to ARI and what with PRO?
At first sight, it does not seem significant.
My claim was about poor and rich people, and you used progressive and conservatives parties as a proxy indicator, which is not great. In USA, for example, California and New York are two of the richest states and they are progressive, while the poorest tend to be conservative.
In the "comunas" 4, 8 and 9 (Lugano, Mataderos, Soldati, Pompeya, Barracas) the party "UNION PRO" did better than "UNEN" (a.k.a.: "ARI"), and in the 12, 13, 14 (Urquiza, Belgrano, Palermo) their percentages were really close.
If anything, your statement is completely unrelated to my question.
> Actually in Congress, The Workers Party (PO) brought more users to DemocracyOS than ANY other party.
Correlation does not imply causality. One possible explanation is that both DemocracyOS and PO have a stronger impact on young people.
Rant? Do you really consider I was ranting at any point?
I think I've exhibit thoughts, tried to look for mathematical proof, quoted people who are smarter than me, wonder about known problems when making decisions, and you consider that a rant?
You have no addressed a single concern I have shared. You don't have to, but your replies are infuriating because you are missing the point every time.
You are just repeating marketing phrases like "every single kid with a huge smartphone". Well, what does that prove? Does it prove that internet has as much penetration in lower income population than it does in high income one? I think not. And if you keep repeating that you are just insulting the young people who live in a slum and do not have smartphone, because you are blaming them for not having one.
> the gap is more generational than Socio-economical
That gap is not related to democracyOS but to technology. You will have a stronger impact on a young generation not because "you are thinking in long term" but because they'll understand it faster.
This comment was mostly a rant. And from now on I'll refuse to answer any comment you write until you want to provide something useful to talk about, or even back your statements.
Sí he estado, no desde que se popularizaron los smartphones, y mientras hablas de experiencias personales no deja de ser estadísticamente insignificante.
Y no, me calenté con tu comentario, antes estaba argumentando, a diferencia de vos que no tuviste un solo arugmento.
lol; yes, I'm hiding when I'm saying (a) you are using threading wrong (b) my personal experience is statically insignificant (c) I'm trying to have a debate while you have not presented a single argument.
> Stats? 72% of the Homes of Buenos Aires have Internet access. Of those, 88% with broadband access.
> Oh, I forgot: 93% of < 30 year olds access social media at least once a week in Buenos Aires.
Is that Buenos Aires City, or Buenos Aires City and Surrounds, or Buenos Aires Province, or Buenos Aires City and Province?
72% is the average, how is it distribute? we are talking about income inequality here, not about internet access.
Even in the most optimistic case where it is evenly distributed, what are you trying to prove? My criticism was that Partido de la Red got a lot better results on wealthier areas, and I was wondering how do you justify.
Saying that poor people also have internet actually makes your case worse, it means they actively decided that your project does not represent them.
A better argument would have been that access is getting there but they don't have it yet.
I replied from a Mobile App. Stats are for Buenos Aires city, that's where we run. Im proving that you never been into a slum.
You've said "personal experience is statically insignificant"
I agree, that's why I replied 93% of under 30 year olds in Buenos Aires access social media at least once a week.
I mentioned my personal experience because it was obvious how many of these kids access the internet. You see that 93% stat everywhere if you actually cared and waled the streets of the city.
These attacks where the most common ones we had during the campaign. So let me give you an update: the largest organisation that implemented DemocracyOS is a political party from Kenya. Over there 25% of the GDP is already transacted through mobile devices.
I traveled a lot around the world, specially Latin America. And I did notice that civic adoption of technology is way stronger in developing nations that are struggling than in developed ones. Venezuela is an impressive case if you ever get to see what's going on ever there.
So no, I don't bullshit. I've been working on this for many years now.
You proof is irrelevant, and incorrect. I lived the past two years in California. Were does stats also correct two years ago? Or your "never" actually means "recently"?
> These attacks where the most common ones we had during the campaign.
Attacks? Really? I see that Partido de la Red is following Frente para la Victoria speech where you are either 100% with them, or you are the enemy. Am I getting paid be the Clarin Group now as well? I made questions.
> if you actually cared and waled the streets of the city
Ugh, that is an attach, and I don't really like your tone. You are implying that you only care if you are physically there, which is incorrect on every possible level.
You are systematically ignoring the questions I have asked, posting cherry-picked facts that are at most mildly related to the conversation I was trying to have with you.
My original question was: "How do you justify that most of the votes came from wealthy neighborhoods of the city?"
Your answer was repeating that people, even poor people, use internet. Well, cool, that's awesome. Now what do you think about my question for a change?
I see this mainly as a database that might work well in mobile, giving persistence for free with an API similar to the NS{Dictionary,Array,String,Data,Number} classes, and without using as much memory.
