Couple of my friends that work for IBM told me that it still needs a lot of work because researchers that take blind tests can almost always tell apart the cold, inhuman responses that are devoid of any human decency from the responses generated by the computer.
Love the idea and execution, but this is less an attorney and more a legal librarian. Attorneys spend most of their time on the facts of the case.
In law school, most lawyers are taught to present each argument in a standard format called IRAC: Issue, Rule, Analysis, Conclusion. The issue is like the question presented on the front page here: "can courts pierce the corporate veil where a corporation has misappropriated funds?" ROSS spits out the relevant rule, and any information it can find on how to interpret that rule in specific situations. However, this is only the beginning of a lawyer's job. The largest part is taking that rule and actually applying it to the facts of the specific case. ROSS doesn't seem to be able to do this, yet. Finally, and least importantly, the lawyer comes up with a conclusion (or a series of probability-weighted conclusions). ROSS doesn't seem to be able to do this either. In law school and in court, it's not enough to cite the rule of law. It's much more important to apply the reasoning of the rule to every specific fact in the specific case.
It would be very interesting to see this intelligence applied not just to legal research, as here, but to e-discovery, which is the other time intensive task given to associates and contract attorneys. That would, I think, be the next step into turning this into a general purpose attorney that _would_ be able to handle the analysis and perhaps conclusory stage of a legal issue.
If anyone who worked on ROSS is here, where did you purchase your corpus from? West or lexis? How are you doing shepardizing and pruning of bad law?
Right now, there are a lot of billable hours for what Ross can do in seconds. I don't think it's unfair to say that this is replacing a substantial portion of _some_ lawyer's jobs.
I'd like to see a concrete description of the difference between what ROSS gives you versus what, say, LexisNexis does.
If its just giving you a simple answers and not actually producing the kind of research results a research attorney using traditional research tools would -- where the simple answer would be part of the heading, but sources and analysis would be part of the report -- its not going to be useful except as a novelty. In law, its rarely as important to get a simple answer as to have an answer that you can support as most correct and explain why other potential answers are less correct for the specific circumstances.
And, from the vague marketing hype, it doesn't seem like what is really needed in law is what ROSS is being sold as doing.
Great comment. What ROSS does in addition to what you've described is ask questions against the context of a case. The case facts (inputed by users) will be factored into the queries so as to produce the most relevant information (rules, precedents and other connections across a vast body law) that would take an associate (especially a junior) many hours to unlock. And absolutely, ROSS is not the end. The quality of the execution lies in a legal researcher working well with a cognitive system that provides around-the-clock legal intelligence. And e-discovery is definitely on the radar for the future.
Yes, e-discovery is definitely the missing piece here. At the end of the day, lawyers need to be empowered by technology to do their job better, and ultimately make the legal system better.
E-discovery has already gone through an impressive cost deflation with the application of more sophisticated scanning, OCR, and search algorithms that recently (in the past decade) came onto the market. I've seen more than a few law firms that developed an over-dependency upon discovery-related fees in their business model either drastically downsize, abruptly change, or in one case shutter their doors entirely (after a painful circling the drain).
If deep learning technology like Watson deflates further the remaining e-discovery market as well as the initial case research billables, it will be interesting to watch how law firms adjust their business models. For those that can stand out with consistently innovative, novel and creative legal services delivery (like nearly always coming up with newly-accepted interpretations of case law) that depends upon people, I could easily see rates for those attorneys go dramatically up, partly to backfill the revenue gap that is created in the wake of automation of these aspects of legal services business models, partly due to an exacerbation of the bifurcation effect automation seems to have upon labor markets.
I'm impressed, and I'd love to see it in action. But I'm also worried that this is going in the wrong direction.
If the law is getting too complex for humans to handle, the solution is not to create supercomputers that help us. The solution should be to simplify the law.
If the answers are comprehensive and accurate, this could aid the simplification of law by revealing conflicts and inconsistencies.
Reforming law to be machine readable might eventually provide a litmus test to prevent the passage of bad laws. It could also provide a way to flag prior law that is invalidated by a new law.
The more you simplify the law, the more complex the application of the law becomes.
Murder occurs when a person, of sound memory and discretion, unlawfully kills a person, with malice aforethought.
Seems simple the there are hundreds of cases that tell you what exactly the limits of these elements are. Is being drunk sound mind? What constitutes malice aforethought.
The law is complicated because reality is complicated.
The civil law system keeps it a bit less complicated by not using precedent from old cases. But that means the application in your case is harder to predict. It is your judge making the gut call instead of a history of judges.
> How do we know eliminating some of the complexities of current laws won't re-introduce some of the problems those complexities were trying to solve?
The other side of that coin is: are the bad outcomes produced by the current codebase bad enough that we will replace them and accept the risk of re-introducing the other bugs?
Alternatively, the entire codebase is in English (+/-) so one could document the bugs that changes are attempting to address.
In theory that would be great, and it's been attempted multiple times throughout history - the law has pretty much always been too complex.
The problem is that much (although admittedly not all) of the complexity in the law is a result of attempts to clarify and provide consistency to the nuance and obscure edge cases that occur in real life. Removing that nuance from codified law results in it being even more down to a judge's individual interpretation than it is today, meaning individuals would be even less clear on where they stand than they do now.
If a computer could accurately interpret the law, and that computer was freely available to everyone, I think we'd all be in a far better position.
The law has always been complicated. I'm pretty sure the only people who would be able to "simplify the law" would be lawyers, and I can't really imagine they would think it would be in their best interests to do that.
> The law has always been complicated. I'm pretty sure the only people who would be able to "simplify the law" would be lawyers, and I can't really imagine they would think it would be in their best interests to do that.
Lawyers are often behind movements to simplify the law. The problem is that efforts to simplify the law often lead to (what some substantial group feels are) substantially negative impacts given the complexity of the reality the law is meant to deal with.
Simplifying often sounds better in the abstract than in concrete terms.
Law written by lawyers (e.g. Model Codes, Restatements, Procedural Rules) are usually far shorter and more clear than law written by legislatures. The political process introduces a lot of special cases and legislatures are hesitant to leave discretion to the judiciary. Both for obvious reasons.
Also, it's a reflection of reality and human values, both of which which happen to be very complex, nuanced, and have plenty of edge cases for any rule we might try to dream up.
I think it's a reflection of an uneducated society that cannot apply common sense or the Golden Rule. Everything has to be specified in extreme detail.
> it's a reflection of an uneducated society that cannot apply common sense
The problem of common sense is that everybody has his own, you can't build rules on sand and that's what "common sense" is.
For one person it's common sense to snip bits of genital organs from infants, for the next it's common sense to pray 5 times a day, for yet an other one it's common sense to flog your children to bleeding, outright own other human beings or threaten those who don't agree with you with maiming or death. You will find people telling you it's common sense to have live grenades on your coffee tables, plant anti-personnel mines in your garden, answer a door-knock by shooting through the door or walk around with enough weaponry to take over a small country.
Relying on "common sense" is relying on hyperlocal groupthink, it's not even workable in tribal societies, it's barely workable within a small family.
Common sense is sufficient for most things. The problem with law is that what gets to court are the weird edge cases that aren't covered by common sense.
Often by the time something reaches the court, there is no possible outcome that provides justice for all, and the court instead has to determine which of the parties to screw the least.
Some things have no unequivocal decision metric. e.g. "We know it's not always fair but that's the way we have decided it goes, so everyone knows where they stand".
I think this is demonstrated by Coase's "The Problem of Social Cost" [1]
> Because in the real world there are costs of bargaining and information gathering, legal rules are justified to the extent of their ability to allocate rights to the most efficient right-bearer.
There have been some important efforts to make the law more accessible in the last, say, 50 years. E.g.:
* Jury instruction have been rewritten using the minimum number of terms that jurors won't already know.
* Legal decisions are written to be more readable to contemporary laypeople. (At least they seem that way to me -- it's a little hard to tell given the change in writing styles generally.)
* Uniform legal codes have refactored the law to make it more consistent state-to-state, and to remove complexity when for example two causes of action could be reduced to one.
* Property law (which probably moves more slowly than others) has moved away from centuries-old "magic phrases" that could break inexpertly-written deeds.
But all of this is basically syntax. It makes the system easier to learn, but doesn't reduce the fundamental complexity of the model.
And you can't reduce that complexity much, if you want a system that is "fair" and "just" at scale. "Fair" meaning that it treats like cases alike, and different cases differently. "Just" meaning that it mostly comes to what most people consider the right outcome in individual cases.
The scale here is hundreds of millions of people over hundreds of years, any two (or more) of whom can interact in a way they want the law to settle. Treating all of those cases "fairly" means at least one of: already having a rule that roughly guides the outcome for a given dispute (statutes and regulations), or taking into account the decisions other people came to in similar cases in the past (caselaw). Treating the cases "justly" means those rules and analogies have to be fine-grained -- if one of the parties points out a special circumstance in their case, and most people would feel that it should change the outcome, then it needs to be taken into account for the system to be just.
So law is complex because the potential interactions between hundreds of millions of unique individuals are complex.
You might want to reduce complexity by reducing the reach of the law, but I don't think you can reduce it that far and still have a recognizable society. For example, many libertarians would prefer a system with greatly reduced criminal law and commercial regulation -- but where the government still enforces contract and property law in a way that is predictable and fair. The system for fairly, justly, reliably, predictably enforcing the contracts entered by hundreds of millions of people over hundreds of years is going to be complex -- the same kind of complexity we have now.
Obviously the situation could be better or worse -- the stuff I listed at the top are small examples of how it's gotten better. Making the system (say) half as complex would be great. But it wouldn't fundamentally change the problem space, or make tools like Watson unhelpful for solving it.
Things like case law become more complicated due to the march of time. More cases get decided, more case law is created. Another is the question of legislative intent. The edits to law as it is being drafted can be used to argue a case. These edits increase with time.
This sort of device is only applicable for very specific areas of "law". It may work for generalized law questions, retrieval of known legal principals, or even for electronic discovery (data mining) but once you get beyond law school there is a human element. I would like to see a machine come up with a novel means of describing a realworld legal situation.
Ask him to determine whether John Stewart's use of media from other networks (ie fox news) constitutes fair use for purposes of copyright law. That very basic question, answerable by any second year law student, requires image recognition, cultural understanding, even a sense of humor. It cannot be answered from legal databases alone.
Or here is one I get all the time: What constitutes "reasonable security" for a law firm handling client information? How about for a hospital? Or a Bank? No two lawyers will ever agree on those standards. They cannot be gleaned from case law and change every day in response to new threats/technology/needs. I doubt the machine would have much to add.
Or, does a CDN violate the principals of net neutrality? Lol. Have at it R2D2.
Its not meant to replace humans. Its meant to reduce the amount of tedious research work they need to do.
If you reduce the vast tedium of researching stuff in Westlaw/LexisNexis, you can spend more time on your research memo. You can do more actual analysis work. Thus firms can reduce the number of associates they need on staff, because the associates won't be wasting time on wrangling research results.
Yes, you still need a smart human to pose the right questions to Watson.
It probably won't even replace the vast tedium of researching West/Lexis for a while, but it might replace the "attorneys" that West/Lexis employ to summarize and tag articles w/ metadata. There is a ton of humanhours put into West/Lexis databases. They aren't just keyword searches.
Perhaps, if you are the 1% of lawyers who are associates at big law firms doing appellate-level research. They basically the same job as law students do in LRW class, pulling hundreds of quasi-relevant caselaw from a massive database. That side of things has, in recent years, been outsourced overseas.
Yes, but this is a system that gives you unlimited queries for even cheaper than if you were to outsource research. And since it is a cognitive computing system, it gets better over time.
This century-long decent into lesiure is what's really killing us. I wish I could remember back when cars were unaffordable, computers were unattainable, and hobby supplies were expensive specialist shops. I bet those were the days!
I'm optimistic that it's a good development, the more automation displaces high-salary occupations, the more likely society will change to address fundamental economic shifts associated with singularity, or post-scarcity, or whatever you want to label it. Doubly true since it's an frequent past occupation of politicians.
Does anyone know of in-depth articles that explain Watson internals (or even general such as what family of algorithms they're using) ?
A friend of mine got to visit their R&D department a few years ago, and told me they basically explained nothing at all and remained on the marketing level. But there's got to be something somewhere, like previous research articles by the team's members, right ?
Does anybody know if this is in the "idea" stage right now or if it is a working product?
If I understand correctly, this is a project by a team of University of Toronto undergraduate students for a competition to receive 100,000 $ seed funding by IBM and continued access to Watson. It seems to me that the stated goals are incredibly ambitious // hard to reach with these resources?
Honest question -- This is amazing! But why limit the search only to legal profession, why not apply Watson to all general questions and compete with Google?
I'd love an answer to a question without a list of results.
Well, in a nutshell, the way Watson Ecosystem works, you are granted access to an instance of Watson in which you can build a corpus of knowledge only in a particular domain.
Well, solving a specific domain is easier than solving for all domains. Begin small. It's not the same type of technology as Google. It most likely contain a lot of specific configuration for understanding the semantics and expressions of law.
Is there an amazing demo of this tech that i missed ? because if not-this is just another marketing site for watson/ibm, nothing more. we've seen plenty of those done by IBM- without much result yet(for example, where is watson the chef app they promised), 4 years after jeopardy.
"fill up this survey, and you might win a change to test watson". Doesn't really inspire confidence. and maybe there's a good reason for that - like it's not ready for prime time, or for heavy testing by technologists ?
And that sound like what review says here[1]:
"it’s worth reiterating that, at this early stage, this software really does not stand on its own. That’s not to say it’s not useful (it is!) but it has a ways to go before it’s ready for prime time."
And since this is field nobody has tried to build apps for(because of limited commercial value),there's nothing to compare it to, which makes it harder to evaluate the tech.
Again, we're looking for working technologies - not articles, movies or books.Its much easier to bullshit your way through a book than a working software - and comparing that to the state of the art and other commercial offerings.
Unequal access to our legal system is the shadow problem of our age. It touches so many aspects of our society and we barely seem to be aware of the issue.
Hopefully this technology and others in the same vein will bring costs down and speed up the process so more people can have access to what they should have already by right.
This brought to mind a passage from Stanislaw Lem's "The Cyberiad", where one of the main characters cooks up a one-use lawyer, much like a throw away shell script:
And Trurl went home, threw six heaping teaspoons of transistors into a big pot, added again as many condensers and resistors, poured electrolyte over it, stirred well and covered tightly with a lid, then went to bed, and in three days the mixture had organized itself into a first-rate lawyer. Trurl didn't even need to remove it from the pot, since it was only to serve this once, so he set the pot on the table and asked:
"What are you?"
"I'm a consulting attorney and specialist in jurisprudence," the pot gurgled, for there was a bit too much electrolyte in it.
If you've never read The Cyberiad, you really should.
This might be a way to help overburdened court appointed defense attorneys better represent their clients (as well as speed up the overburdened judicial system).
I'd like to think of this as comparable to CAD for architects, except attorneys have never had any serious domain-focused assistive software, beyond word processing. Maybe some have, but it's not a well-known category that I've been able to see.
There are some packages for very specific sub-fields, things like Collection Master and such but I'm not aware of anything generic enough to cover the field of 'law'.
There is a lot out there, for everything from case management to practice management to discovery to legal research to legal composition to litigation support ...
I hope I can see the day when the justice system is run completely by benevolent AI. Legislative too, for that matter, and at least partially the executive functions of government as well, while we're at it, including law enforcement.
The matrix was a movie about being connected to and living in a simulation without knowing it. The judges could very well be real peoples' consciousnesses. Not really a good comparison.
Why is this only offered to lawyers and legal professionals? I have had countless scenarios where I just want to look up something regarding the laws surrounding a topic but get lost looking for accurate legal information... This seems like a perfect solution for that use case, no?
Watson's good, but it's not that advanced. It's an 'expert system', in that it's a system for experts, not a system that is an expert.
This is a smarter, domain-specific search engine. You still need to be able to ask the right questions and then understand interpret the results it provides.
I still don't disagree that for certain 'low-stakes' issues it could be a useful tool for lay persons, but they'd need to understand the limitations. Perhaps they're worried that giving such access might tarnish the Watson brand -- if a medical doctor has tried Ross and found it wanting then when they roll out 'House' she might dismiss it as junk based on that experience.
No. It's more like StackOverflow. You wouldn't send a non-programmer there for basic questions, it's for people who work in the field looking for specific answers.
I am so glad other people are thinking about this. I think the potential use of ML and AI for helping with legal matters is absolutely huge, and I think it is one of those things that would add incredibly to a large number of people's lives. I've been financially intimidated by the threat of frivolous lawsuits far too often over the last five years, and I know the problem isn't unique to me. Even if you are undeniably right, you can easily drop $100k proving it. It's the same problem that is at the root of patent trolls. This is a fantastic idea, and it's good to see the Watson team working on such fantastic stuff. I honestly hope I get a chance to help them with their efforts at some point.
I think the most obvious way would be to offer this as a subscription service to the lawyers of your platform. But, remember we're talking about IBM, this will cost you.
As far as I know, litigators do spend a fair amount of time researching cases as it there job to assemble a case that is capable of being presented in court - although most litigation gets settled before going anywhere near a court.
However, in most law firms I know litigators are in the minority - and I don't think the likes of corporate lawyers spend much time researching cases?
Why should the final price tag remain the same? Free market economics says that if we introduce efficiencies, the price comes down, and/or fewer people produce the same output. In general (exception is music, as far as I can tell) that's the case. Other labor markets have suffered horrible, drastic shake-outs, why not Law?
Now that this "Attorney" can do legal research, I guess the next steps are to get it to marshal the relevant facts and evidence, analyze documents and information, interview witnesses, negotiate with the other side, and present a case to the jury.
> and if I want Ross to help me sue IBM, does Ross act independently? :)
I know you are joking, but your comment is a good response to the other commenter who wished for a future when law is administered by a "benevolent AI". Who writes the code for the AI?
They already are. Law dropped in 2008 recession and hasn't recovered. Law school applicants are down 1/3 or something like that, and it's causing much stir [1].
