People want to see 100% success in all things. But that isn't very economical. Compare the SLS to Starship.
SLS has had a 100% success rate. One launch, one success. But it costs $2B per launch (and climbing) and has taken 12 years to get here. Plus, it's stealing pieces of the Space Shuttle, which was developed in the 70s over a long period of time.
Starship will launch a half a dozen "failures" before they achieve success, but they will have a bigger, better launch system that's fully reusable and costs orders of magnitude less per launch.
>> People want to see 100% success in all things. But that isn't very economical.
That sounds logical, but isn't really a thing in aerospace/space. Complicated high-energy systems have thousands of failure points. So to have any chance of success each failure point needs to be engineered below, by way of example, a 0.0001% chance of failure. That costs lots of money. But say one decides to accept more risk for less cost. Ok. So you switch from 0.0001 to 0.001 failure rates. You risk is now 10-fold higher at each failure point, but with thousands of failure points adding up you are now essentially doomed. And you haven't saved anything. The cost of 0.001 components isn't fundamentally different than the 0.0001 components were. SpaceX can save money though different business practices, by trimming people/money/contracts/compliance and such, but if you look at their rockets they are not fundamentally any less-perfect than anyone else's. They cannot afford to be. This is why rocket failures, like aircraft failures, are taken so seriously. There is an extremely fine line between "works ever time" and "never worked twice" with very little money to be saved between the two.
Across many areas, risk-v-cost math never really happens. It is either go or no go. Take CPU production. Intel spends billions at each of hundreds of fabrication step to push down miniscule error rates because any of a million errors can destroy a chip. There is no money to be saved by allowing any one process to become less than as perfect as it can possibly be. A detected slip from 0.0001 to 0.001 at any step would result in an entire fab being shut down in order to diagnose the problem. The marginal savings of a less-than-perfect process isn't worth the exponential increase in the risk of total system failure.
I'm not proposing something, I'm describing it. This is what SpaceX is doing, and it's quite successful.
And you can look all the way back to their little hopper version of the Falcon 9 and see that this strategy has been the key to them undercutting the launch market significantly.
My prediction: SpaceX will have a 5th 100% successful launch of Starship before the SLS has a 5th successful launch. They'll just have ten not-100%-success launches before then.
Completely agree. I've worked in "old space" and the fundamental problem is that they can't afford to experiment. SpaceX has the option to physically test an idea that's holding them up before investing in fully bringing it up to production quality, only to have to redo all that work next iteration. That's why they can make new things and do it cheaper and faster.
Ironically, "old old space" = Soviet space was so damn good at innovating it probably would have made SpaceX look old-fashioned. Really I don't think there's a fundamental reason why we can't have two SpaceX's (Spaces X?) so why is there no other?
Haha I had a feeling someone would bring this up. To me, old space isn't farther back in time in the golden age of space, but rather what the space industry eventually calcified into. New space is like a Renaissance.
There are other new space companies, but they're just not as good.
Except SpaceX’s failure rates are similar with every other successful launch system. Rather than looking at failure as a constant rate you need to consider these numbers change with every flight. Initially major design flaws are identified and workers become more skilled at a process, eventually new errors creep in etc etc.
They have greatly benefited from being able to use modern tools and seen where other systems failed. Many rocket companies have failed when trying to go fast and break things because it isn’t an easy shortcut. Instead SpaceX has used the normal approach used by other successful organizations and simply executed it well.
> Except SpaceX’s failure rates are similar with every other successful launch system.
Falcon 9 has the record for most consecutive successful orbital launches. Their last failure was AMOS-6 in September of 2016. Since then they've had 189 successful launches in a row.[1] In that same time Soyuz has had 113 launches with 3 failures. Soyuz's longest success streak was 100 launches from 1983 to 1986.[2] The US's Delta II had 100 consecutive successes from 1997 to 2018, though it has since been retired. A total of 155 Delta IIs were launched with 2 failures.
Falcon 9's current successful landing streak of 110 missions exceeds the competition's best launch streak. By any metric one can measure, SpaceX has the most reliable rocket.
You can always slice and dice data to make one side look better.
The actual number of successes vs partial successes vs launch failures vs fatalities are the best data we have. Throwing away any of that data because it makes you look worse isn’t a good idea.
Similarly we needs to understand that there’s a huge difference between risk and what actually happens. People get lucky in Vegas every day, what matters to most of us is the accuracy of the estimate of underlying odds not just the exact outcomes out to seven decimal places.
> actual number of successes vs partial successes vs launch failures vs fatalities are the best data we have
Current vehicles are vastly different from the originals. What we’re trying to do is predict the probability of the next launch failing. Equally weighting far historicals and recents is bogus statistics.
> What we’re trying to do is predict the probability of the next launch failing.
I thought we were comparing methods. Unless the next payload is yours then the odds of the next launch failing is meaningless to most of us, but we can learn something from the methods used.
But sure, if you have a bet in Vegas or something then feel free try and calculate things as closely as possible. Just understand that several of Soyuz failures didn’t kill the crew so there’s other metrics people might care about.
What does this mean? The question most of us care about is which method resulted in a more reliable rocket. And SpaceX’s track record shines uniquely in that respect. The frequency, moreover, makes the results robust. Legacy rockets like Ariane will never reach that confidence because the likelihood of fluke successes won’t have been minimised when the rocket is retired.
As to why their methodology is important this isn’t the Falcon 9 this is a new launch system which is likely going to have multiple failures before it’s own streak can begin.
So sure, we can reasonably assume that Starship will get to a state of reliability similar to current Falcon rocket, eventually. We can’t assume the first few commercial Starship launches are going to even approach that level of reliability. And in fact the best point of comparison may be the early days of Falcon 9.
Speaking of methodology, it's incorrect to relate a development test result to reliability or risk. Source is my personal experience doing reliability calculations for a NASA rocket component and working with the statisticians incorporate my numbers into their risk model.
You have it mixed up. I've worked with the stats at NASA. Mission success and failure counts. Test quantity and quality count, test freedom counts, how they learn from test counts, but the test result does not count. This isn't a mission.
This was a partially successful test nothing more and nothing less. I get people really really think SpaceX had done an excellent job and I don’t disagree but people who are comparing the end result of a long process Aka the current state of falcon 9 with a new system like Starship are going to be disappointed.
Starship is extremely likely to fail repeatedly before achieving anything close to the same streak as the Falcon 9 has. That’s not an issue with SpaceX that’s an inherent aspect of doing something really difficult.
I don't think you realize this, but when you said that we can't exclude the test failure from the risk/reliability assessment, that's exactly what you're saying. I didn't realize until just now that you're actually defending the test failure as being acceptable.
This isn't a production flight so why are you treating it at one? What if I told you that you can't compile your code until you deploy it for the first time and you don't get to change it much after that? I'll leave you to contemplate that thought experiment yourself.
I am not treating this as anything but a test flight. Several government test flights have similarly achieved core objectives while failing to achieve every objective.
I guess I'm not sure what point you're making. If you consider a test flight failure to be part of the overall failure rate, then you're treating it the same as a "real" flight. The government is constrained to less experimentation on every level from a daily basis up through test flights. Overall they do less useful engineering and more unnecessary work.
> Except SpaceX’s failure rates are similar with every other successful launch system
Really? I make is 189 successful F9 launches since the last issue in 2018 (there's a couple of landing failures, but given that everyone else apart from the space shuttle has a 100% loss...)
If you look at the "finished product" of block 5 that makes 162 launches and zero failures.
That's reliability far beyond any other launch system, including the space shuttle and Ariane 5 which are the only ones to come close in numbers of launches. Ariane 5 is certainly a reliable system as far as spaceflight goes, but it flies 3 times a year, Falcon 9 flies 3 times a month
Hardly, landing on launch pad had a lower success rate and requires significant fuel so many otherwise perfectly reusable boosters were sacrificed for a higher launch payload.
They got great publicity from it, but landing vertically is a major compromise.
Sure, ignoring past failures can always make someone on a winning streak look invincible. But calculating the underlying odds to hit even a 200 long winning streak with the observed failure rates on other systems wouldn’t be particularly unlikely.
These systems all are quite good, and they have tended to get better over time.
My point would be that while SpaceX does save lots of money, it doesn't do so by producing cheap or less reliable rockets. It would never accept a failure rate any different than anyone else because, in aerospace, that isn't really a thing. SpaceX can certainly blast forwards with different business practices and different tolerance for developmental risks, but the final product will not be fundamentally different than anyone else: near-perfect machines resulting in near-perfect performance. There will never be a "cheap" version of a commercial rocket with an accepted less-than-perfect failure rate. Spaceflight is an all-or-nothing game.
> My point would be that while SpaceX does save lots of money, it doesn't do so by producing cheap or less reliable rockets.
It does though. During the development process. SpaceX will do five launches with lower reliability vehicles before the big launch providers will even do a single launch, and those five will have be cheaper in total. Of course, they aren't putting essential cargo (or god forbid people) on these higher risk test flights. By the time they're doing that they have developed certainty in the design.
A number of Tesla related non-deaths can be attributed to Autopilot safety features working as promised. Tesla will argue that the number is higher, based on number of crashes per mile statistics. I'm not sure if that's true, but assuming for everyone who died due to an Autopilot failure, someone else survived a human error crash that didn't happen, would that be a good thing? What about if it was, for example, 10 people saved for every 1 killed?
And statistically many more people would have died without AP. But you're correct in that Tesla is using the same playbook on FSD as SpaceX, launch HW (and SW) early and iterate often, and I'd bet they'll save way more lives trying to get to autonomy like SpaceX rather than like NASA (the Waymo approach).
> But you're correct in that Tesla is using the same playbook on FSD as SpaceX
Except Tesla, unlike SpaceX, is willing to put passengers in its test vehicles. The SpaceX approach would be to let a bunch of FSD Teslas crash into things and each other before giving them payloads.
Putting someone in an experimental rocket is quite different from being essentially a safety driver required to pay attention and take over at any time. If you have an accident on FSD is is probably (though not always) your fault for not paying attention.
I “unwillingly” have to share the road with people who murder 40,000 people a year with their vehicles. Thankfully we are developing the technology to get these reckless maniacs out of the driver’s seat.
> And statistically many more people would have died without AP.
Citation needed. Not Tesla's "stats" that if the people compiling them completed anything more than high school statistics are intensely misleading.
Comparing a subset of miles driven on the simplest and easiest roads (because the systems can't be used and are turned off) and comparing to accident stats across ALL roads is disingenuous to the extreme, and Tesla continues to tout it.
Short of pulling over, humans don't have the opportunity to say "let's disengage, because it's a bit challenging", and then not have to worry about "counting" any accidents from there forward.
Agreed. It’s like if I unit tested 0.01 percent of my code but ran the unit test 10 million times, with no failures, and claimed it was therefore “statistically” better than code that had been 100% manually tested.
SpaceX does not have a difference in intended reliability, or a difference in design reliability. (At least for the rocket as a whole. One could argue that 33 engines allows lower engine reliability through redundancy)
What they DO have is a significant difference in prototype reliability for live launch. This is clear when you look at their launch history.
Or one could say that SpaceX has basically the same tolerance for failure as the traditional rocket companies had back in the 50s and 60s when they too were first learning to build rockets. That pre-launch "tolerance" is basically zero, with every post-launch failure being investigated as a mistake to be corrected rather than an acceptable cost of doing business.
I don't think that is accurate. There is a difference between 9%, 99%, and 99.999% confidence of success going into a launch.
You can almost always delay builds and launces to run more simulations, tests, and studies and increase confidence.
A simple example is SpaceX could have chosen to wait until they had a booster test with 100% engine ignition before moving on a full launch. Instead they choose move forward anyways without more stationary booster testing.
> There will never be a "cheap" version of a commercial rocket with an accepted less-than-perfect failure rate. Spaceflight is an all-or-nothing game.
Why? If you're launching people I see why you want near-perfect, but if you're launching something with a low replacement cost (ex: getting fuel to orbit to support other projects) it seems to me that as volumes get large enough eventually "use lower quality and accept a slightly higher probability of failure" starts to be cost effective.
Because there is no way of building a cheaper rocket with a less reliability. Take aircraft. Does anyone deliberately build cargo aircraft with less reliability than passenger aircraft? Does anyone build a smaller airliner with less reliability than the big airliners because fewer lives are at risk in the smaller aircraft? No. All aircraft are designed and built to amazingly high standards because, in such as complex high-energy environment, there is no money to be saved by building less-than-perfect machines.
Most cargo planes are expected to run out of civilian airports where a failure could result in debris launching themselves into populated vehicles or buildings. In contrast a crop duster launching off a dirt runway and expected to go no more faster than highway speeds actually can be built fairly loose, and often are. For example, this duster here [1] won a award in the 70s for innovations like a "pressurized cockpit" and "air conditioning".
Listen, we don’t need to speculate. Read the History of Falcon9 development.
Your comments in this thread all go against the development approach of that rocket— Falcon9 is a stable platform now as it is used in production. In the development phase there were tons of explosions. This all happened in the past.
No. All aircraft are designed and built to amazingly high standards because, in such as complex high-energy environment, there is no money to be saved by building less-than-perfect machines.
But this is totally false. There are entirely different standards applied if you want to design an aircraft for commercial airline passenger transport vs for general aviation. There are entirely different requirements for instrumentation reliability if you're building a day-VFR aircraft vs one that is allowed to fly in instrument conditions. And there are entirely different requirements for aircraft that you sell to the public vs ones that you build yourself.
The aviation side is full of examples of exactly the sliding scale you're saying doesn't exist.
> there is no way of building a cheaper rocket with a less reliability
Of course there is. The famous example is radiation hardening. SpaceX opted for redundancy instead. Not only cheaper, but more modern, too.
> in such as complex high-energy environment, there is no money to be saved by building less-than-perfect machines
SpaceX has launched zero humans. (EDIT: Totally wrong!) It aims to, so target reliability is high. (I would argue their track record in production is a product of their willingness to push the envelope in tests.) But there is a large market for cheap, if unreliable, launches. Because there is an emerging market of cheap satellite makers.
I'm with you on spaceflight but people definitely have built small cargo carrying UAVs to a lower standard than passenger aircraft. Such a thing is conceivable anyway.
Ya but smaller UAVs aren't operating in the same energy environment. They are small enough that them randomly dropping on people's houses doesn't matter much. Aerospace is about things large/fast/high enough that all failures put lives at risk.
Okay, so you're not saying it has to be expensive, but almost the opposite? Perfect and slapdash have similar production cost, so properly designed rockets will all be just about perfect?
> My point would be that while SpaceX does save lots of money, it doesn't do so by producing cheap or less reliable rockets
Indeed, if we assume that the each launch is an IID binomial coin flip (which isn’t really the right way to evaluate right-censored data), and observe (by reading Wikipedia) that SpaceX has had at least 450 successful Falcon 9 launches since the last in-flight failure, then they have at least five nines of reliability:
0.999995^450 ~= 449/450.
Which appears to be an industry-leading stat.
For context (excluding the Columbia re-entry failure), the space shuttle only had 4 nines:
I'm going to challenge those numbers: Using a posterior probability density function for a binomial distribution, the lower bound in Falcon 9 reliability is .9934 with 95% confidence (assuming 450/450 successful trials). The reliability of F9 could be much lower than five 9s and still reasonably give you 450/450 successful trials. There's only a 0.44% chance that F9 reliability is at least five 9's given the data.
Sounds exactly the same. There are no cheap aircraft. Everything that flies is subject to innumerable laws and regulations to ensure that it is built to exceedingly high standards. There is no such thing as a discount aircraft with lesser reliability. Some are cheaper than others but none are deliberately less-reliable, not in any fundamental way. Even ultralights have to abide many regulations.
To me, this comment implies that SpaceX is better/smarter than NASA, Northrop, Boeing, et. al. That may be true, but it's worth remembering that the goals are very different. Because of congressional oversight, SLS is largely a jobs/pork program. Spaceflight is incidental.
> To me, this comment implies that SpaceX is better/smarter than NASA, Northrop, Boeing, et. al.
I don’t think SpaceX has smarter engineers. I don’t think they even have smarter managers, since many of their executives used to work for NASA and/or traditional aerospace firms (e.g. President+COO Gwynne Shotwell started out in aerospace at a private non-profit research centre doing contract engineering work for the NASA Space Shuttle and the US military space program)
One big difference is Elon Musk at the founding of SpaceX told his executives to take big risks (as in “if this fails we go bankrupt”). I think Tory Bruno at ULA is a great CEO, but no way is Boeing or Lockheed-Martin ever saying to him “we want you to take such big risks that we might go bankrupt if they fail”. He, and all the people under him, are only allowed to take small-to-medium sized risks. But that puts a definite limit on what they can achieve compared to SpaceX whose executives and engineers have the freedom to make much riskier decisions
>My prediction: SpaceX will have a 5th 100% successful launch of Starship before the SLS has a 5th successful launch. They'll just have ten not-100%-success launches before then.
I'm not the best way to frame this is "first one to a single successful launch". Reliability matters when you're dealing with high-risk scenarios, so a better measure is "probability of a given launch being successful".
That only matters after you’ve gone operational though. The difference here isn’t risk appetite for operational launches, it’s risk appetite for test launches. SpaceX expects to do many, many more test launches than SLS, in fact counting Starship upper stage launches they’re already well ahead.
I'm not asking as a 'gotcha' question, I'm acknowledging that the sample size matters. A lot of these statistics are bandied about without really elaborating on the full context of the way reliability is actually measured in industry.
Without checking for the most up-to-date numbers, I believe the F9 is slightly better than Soyuz on raw numbers (successful flights / total attempted). But Soyuz probably has around 6-7x more flights, meaning we have much more confidence that the Soyuz numbers accurately reflect reality.
Soyuz has only had 140 flights, and had some non lethal mission failures in recent decades. It’s a manned mission vehicle only though. Overall I don’t think you can really say one is more reliable than the other, they’re both very reliable. I’m just saying the suggestion that SpaceX approach is inherently less safe is very much contrary to the evidence.
Having said that I do think propulsive landing for crewed vehicles is pretty scary. F9 firsts stage landings have been pretty reliable for a while, and they now have several boosters with 10 or more flights, so we’ll see. It’s not like capsule landings are 100% safe either.
There has been over 140 crewed launches. For context, you seem to be counting both test/demo, crewed, and uncrewed F9 launches. Again, I don't know the exact number off the top of my head, but there's probably ~10 crewed F9 launches, so it's an order magnitude difference using the same metric. It's gets better though, when comparing total launches.
This is true -- there is a lot of small number statistics in estimating the reliability of launch vehicles. And there are a lot of small risks you never see until you accumulate a lot of lunches. But that question is completely incidental to the design methodology.
A new vehicle will always be unproven. But one that's flown 5 times in testing (4 of them unsuccessfully) before its first flight will have had more flight time on most of its systems than something like the SLS where you take 10 years to think about what could go wrong and then do one test launch and call it good.
As long as the organization has and maintain a culture that is aggressively seeking out problems large and small, and proactively fixing aprox. all of them, you will end up with a program with a high rate of operational success.
It's the X-origin vs Slope issue - steeper slope always wins.
The problem is maintaining that aggressive problem-seeking culture after long periods of success
This is part of the culture distinction. On one hand SpaceX is attacking problems, but IMO they often don't go towards attacking the root causal understanding. That means there's a possibility of unknown latent risk.
As an example, they had issues with failures related to their COPVs rupturing. On the one hand, they addressed the problem by redesigning their system. On the other, they never really investigated fully why the COPVs were failing in the first place. Instead, NASA decided to fund that investigation on their own. One possible consequence is that their redesign didn't fully address the risk because they never fully investigated the root cause.
But the apples to oranges here is >1 launches (SpaceX, some successful and some unsuccessful) vs 1 successful launch (traditional modern aerospace).
From an outcome perspective, it's hard to ever see the lower launch rate dominating from any perspective.
You don't have economies of manufacturing scale, because your assembly rate is so low it doesn't make sense not to treat each as exquisite.
You don't have rapid iteration on manufacturing improvements, because the tyranny of safety checks on manufacturing time balloons {time from fix to flight}, after a proposed fix is identified.
And most importantly, you leave yourselves extremely vulnerable to unknown-unknowns, that you can't imagine in the design phase.
For example, if NASA had been launching the shuttle more rapidly, with the bulk of those being uncrewed launches, they probably would have picked an uncrewed launch to test expanding the temperature bounds at Cape Canaveral, and Challenger would have exploded without a crew.
As was, NASA's shuttle launches were so rare that there wasn't acceptable launch rate and weren't low-impact launch opportunities to do so. So they tested it on a crewed mission with disastrous results.
Point being: they backed themselves into a low-volume/high-risk corner of their own strategic design
SpaceX's most brilliant achievement was using Starlink to artificially boost launch demand and give them a minimally-profitable/break-even place to sink higher-risk launches.
> Point being: they backed themselves into a low-volume/high-risk corner of their own strategic design
Because they knew they had to "get it right" the first time because a bunch of buffoons(congress) would consider any crash or explosion as a failure and pull funds immediately.
>Because they knew they had to "get it right" the first time because a bunch of buffoons(congress) would consider any crash or explosion as a failure and pull funds immediately.
I agree with many of your points, but it comes across as slightly biased because you don't acknowledge a single downside to any of them.
>You don't have rapid iteration on manufacturing improvements, because the tyranny of safety checks on manufacturing time balloons
Rapid iterations has obvious benefits. But there are also downsides because it makes it harder to arrive at a stable, reliable design, it introduces vendor issues, etc. Tesla is also known for iterating faster compared to their major competitors but it has resulted in logistics and reliability issues.
IMO SpaceX's most brilliant achievement was leveraging govt contracts to work out the kinks of their designs, which could then be leveraged at a lower risk for Starlink. In effect, they let the public take the burden of the risk (because the govt is really the only entity capable of shouldering that size of a risk for an unproven quantity) and then transitioned to a private means of revenue in Starlink. (I'm not saying that as a slight btw, I think it's mutually beneficial).
Frankly comparing Tesla and SpaceX is getting to be a tiresome argument. They're owned by one "eccentric" billionaire, but he's not an engineer, they don't share staff, facilities or manufacturing outside of "hey this alloy is pretty good".
SpaceX's strategy for the Falcon 9 worked and it's one of the most reliable rockets in the world, flying the most often.
>Frankly comparing Tesla and SpaceX is getting to be a tiresome argument.
You're focused on the wrong takeaway. You're making this about a person, I'm talking about a process. I used Tesla because it's easy to see how one culture translates to the other. Insert any company that uses rapid iteration in place of Tesla if you prefer.
The point is that there are certain circumstances where rapid iteration is useful and others less so. When reliability matters, rapid iteration may be working against you. (It's a continuum, of course, so the real question is where is that tipping point)
>SpaceX's strategy for the Falcon 9 worked
The point I'm driving at is there is a distinction to be made when finding out why certain iterations didn't work vs. just changing the design without fully understanding the failure mechanisms of the first. One leads to a greater understanding than the other. It's a difference in an engineers mindset and a scientists mindset.
No. I am not saying it’s the “industry” as much as the context of risk. That's why it doesn't matter if the analogy is Tesla or some other safety-critical manufacturer. To be clearer: how many F9 launches have carried humans?
Now go look at the history of Shuttle for the equivalent number of launches at that risk level. Would you claim they are equivalent in terms of human-rated safety?
If not, it’s only because you have the benefit of knowing the long-tail probabilities with the Shuttle.
>It's the same industry. Building the same type of product.
By extension of your logic, Starship should then already have the same launch reliability as F9. So either this is an example of a low-probability event, or your logic is flawed.
There were 135 space shuttle launches, of which 2 failed. There have been 162 launches of the F9 block 5 with 0 failures. Why do you think we have more knowledge of the long-tail probability of the Shuttle than F9?
True, few of those F9 missions were crewed, but that's the point. There's no difference between a crewed and a noncrewed F9 launch vehicle, so there's no reason to think the presence of humans would change the risk. So, you get to accumulate most of that reliability data without putting people at risk doing so.
>Why do you think we have more knowledge of the long-tail probability of the Shuttle than F9?
Because the nature of the two programs was fundamentally different. The Shuttle was a product contract, while CCP is a service contract. On the former, the govt has much more control, and will detail more rigorous acceptance criteria. This generally gives a much higher pedigree on quality control. On the latter, they take a much more hands off approach and have limited insight.
As an analogy, imagine you are making a big bet on acquiring a software company. One company gives you their source code, shows you all their most recent static analysis, unit test results, allows you to interview their programmers etc. The other company allows you none of that, but gives you a chance to play around on their website to see for yourself. Both systems seem to work when you try the end product, but which do you have higher confidence in?
At the end of the day, "reliability" is just a measure of how much confidence we have that a product will do what we ask of it, when we ask.
Are you asserting that we gain better insight into the reliability of a system by thinking about it deeply rather than by observing it perform its function? Because I don't believe that for a minute.
I'm not saying you can get by without thinking, but it's difficult for humans to estimate the reliability of a complicated system. Reality, though, has no problem doing it.
Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.
No, I'm not saying by "thinking about it" (although that has its place). Everything I listed is a form of testing. But there's a distinction between iterative testing at a lower level, and end-to-end testing. Again, both have their place.
Take the example of the F9 strut failure. They could have tested the material outside of the final test configuration and saved themselves a lot of trouble. They chose to forego that testing, and instead 'tested ' it as part of part of their launch configuration. (I put it in quotes because it's not clear to me that this was a conscious testing decision).
There’s also a difference between “we’re not completely sure of the fundamental principles, but our testing indicates it works” and “our testing indicates it works and we have a solid understanding why”. The latter allows you to know the limits of your application much more readily. The risk in the former is that you don’t know what you don’t know, so you can never be wholly sure if you’re good or lucky. And luck can be fickle. And this is also where rapid iteration can lead to issues: the more you change, the less sure you can be about whether your results are attributable to luck.
>Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.
They have many engineers who want more access and are effectively told to back off because it's not their place in this type of contract. So I'm not saying they have no access, I'm saying they have very limited access by comparison. It would have been better if I made the analogy that they get the results to a small select number of tests, but not all.
Considering his vast wealth of knowledge and expertise in the field he might as well be called an engineer. In fact probably more so than many real engineers.
That doesn't really apply here. The dynamics of stage separation on Starship are orders of magnitude more complex than anything used on any previous rocket. If you want to look at what failure looks like for a super large rocket compare it to the failed Soviet N1. This flight demonstrated better performance in a single test flight than every N1 test flight over years. The N1 suffered catastrophic failure after catastrophic failure. Starship was pushed way past its design tolerance and showed its design is technically sound. Failure is unavoidable, the Saturn V almost failed its first test flight way earlier than Starship did here.
I feel like the N1 is unfairly demonized. They didn't have the benefit of modern closed loop computer flight controls that we have now. Detailed fluid dynamics simulations. Modern manufacturing techniques and production accuracy.
You're talking about failure rates per rocket, but we're talking about the overall development program. Falcon 9 certainly has very low failure rates on its individual components now (below your suggested 0.0001 number). But the way they got there was via a higher risk iterative development process, same as what we're seeing with Starship. The Falcon is the most reliable rocket on the planet at this point, so clearly it's working!
Your intel example doesn't fully hold water. Chip design has taken a binning approach for decades at this point. The silicon is designed for failure in such a way that yield is maximized assuming there will be a variance in quality. Launch and iterate happens for silicon just like anything else. During the initial product launch, there is a lot of waste, but as processes are improved, microcode updated, rework procedures defined, and value engineering efforts completed, yields go up and costs go down.
It depends on what you count as a chip. Modern "chips" are actually a great many things all one one slab of silicon. Failure then knock out single components, resulting in the final chip going into one bin or another. And each of those components can be as complex as an entire "chip" from a few years ago. That is really creating reliability through numbers. Within every component any slight error will still brick that component. The binning process is really a edge case where the fab is playing at the margins. Any slight increase in error rate would quickly see every chip going into a "bad" bin, with the entire processing becoming uneconomical. Intel, and everyone else building chips, strives for perfection with a zero tolerance mantra.
But binning is largely binning against performance curves: "we doped this side of the wafer just a little too much and it's a little slow, but this one from the other side is just right" - while checking for faults (full chip scan and other manufacturing tests) result in dies being thrown out.
> you switch from 0.0001 to 0.001 failure rates. You risk is now 10-fold higher at each failure point, but with thousands of failure points adding up you are now essentially doomed
Our ex ante ability to estimate these probabilities is lacking. Especially when they’re coupled. And redundancy, instead of tighter tolerances, is often cheaper than the classic approach. These are proven lessons from SpaceX.
SpaceX is doing the classic statistic thing[0], making spacecraft stronger where they explode until they don't. It's more like a hyperparameter search and less like QA for individual parts.
SpaceX is just doing what the Soviets did. The Soviets preferred to launch and learn just like SpaceX does today. Soviets also ran on a much smaller budget than NASA and preferred simplicity over complexity and constant tweaking.
The Soviets proved that methodology works and SpaceX continues it to this day. I have a feeling most that follow SpaceX think they are trying a new revolutionary approach here.
The N1 didn't really fit the testing driven development approach, they couldn't static fire the flight engines, they ended up relying on test firing extra engines from the same batch and assuming all in a batch were the same.
I'm no rocket expert, but the Wikipedia article you linked seems to have an idea about that:
>Adverse characteristics of the large cluster of thirty engines and its complex fuel and oxidizer feeder systems were not revealed earlier in development because static test firings had not been conducted.[9]
Time and money. Powers at be decided to stop funding the project. If SpaceX goes bankrupt next year, they could fail too and it won't have anything to do with the technology.
Strategically, they were caught with their pants down when the US demonstrated they were serious about going to the Moon. The soviet space program was great at sending cosmonauts to LEO, but they had no ability, and no desire to spend the time and effort to develop the ability, to go to the Moon.
They planned and expensed the N1 as if it was another LEO Soyuz variant.
The N1 was always doomed to fail because the USSR spies had been found and were given misleading information and plans for the US space program. The US intelligence community snuck bugs into the N1 design by sneaking them into the designs their spies thought they were stealing.
Only if the failure points are arranged in series - A AND B AND C must happen to have a successful outcome.
If the failure points are arranged in parallel - X OR Y OR Z must happen to have a successful outcome, with multiple redundant paths to success, your total failure rate is the chance that ALL of X, Y, and Z fail. This is a much lower number than when they are in series.
To use concrete math - say that Starship has 33 raptor engines with a failure rate of 0.01%, 3 grid fins with a failure rate of 0.05%, and a fuel tank with a failure rate of 0.001%. If it's engineered so that all 33 raptor engines, all 3 grid fins, and the fuel tank all need to work for a successful launch, the success rate of the whole system = 0.9999^33 * 0.9995^3 * 0.99999 = 0.9952 = ~0.5% chance of failure. If it's engineered so that it can get to orbit on 28 out of the 33 raptor engines, 2 out of the 3 grid fins, and there is a double-hull to the fuel tank with a failure rate of 0.005%, then the chance of failure for each subsystem is 0.0001^5 = 10^-20, 0.0005^2 = 2.5 * 10^-7, and 0.00001 * 0.00005 = 5 * 10^-10, and when you multiply out those subsystem failure rates you get 1 - (1 - 10^-20) * (1 - 2.510^-7) * (1 - 510^-10) = 0.9999997495 = ~0.000025% chance of failure.
Moreover, lets look what happens if you take the multiply-redundant design above and then increase the chance of failure of each component 100x. Raptor engines are now 99% reliable, grid fins are now 95% reliable, and fuel tanks are now 99.9% reliable. The overall failure rate for each subsystem becomes 0.01^5 = 10^-10, 0.05^2 = 2.5 * 10^-3 and 0.001 * 0.005 = 0.000005. When you multiply out those subsystem failure rates you get 1 - (1 - 10^-10) * (1 - 2.510^-3) * (1 - 510^-6) = 0.00250498759 = ~0.2% chance of failure. The multiply-redundant system, even with component failure rates 100x higher, still has better reliability than the perfectly-engineered system where every component must perform exactly to spec.
This principle is used all the time in practical engineering. It's why Google builds server farms out of thousands of commodity PCs, hooked up in primary/replica clusters with replication and transparent failover. It's why ships have watertight compartments and double-hulls. It's why passenger jets have multiple engines, multiple hydraulic control systems, and multiple flight computers. Any engineer worth their salt is going to avoid SPOFs and assume that components will fail, then build redundancies into the design so that a partial failure does not endanger overall mission success.
>So to have any chance of success each failure point needs to be engineered below, by way of example, a 0.0001% chance of failure. That costs lots of money. But say one decides to accept more risk for less cost. Ok. So you switch from 0.0001 to 0.001 failure rates. You risk is now 10-fold higher at each failure point,
It seems like a strawman to suggest that you would let failure probability increase by a factor of ten as the first step.
>but with thousands of failure points adding up you are now essentially doomed.
And it was an easy one to tear down.
>And you haven't saved anything. The cost of 0.001 components isn't fundamentally different than the 0.0001 components were.
There is no reason to believe that at all. Invert the question: if a system is 99.999% reliable, does that mean it should be free to make it more reliable? And why?
>It is either go or no go. Take CPU production. Intel spends billions at each of hundreds of fabrication step to push down miniscule error rates because any of a million errors can destroy a chip. There is no money to be saved by allowing any one process to become less than as perfect as it can possibly be.
This is completely wrong. Chip factories do produce chips with defects; those chips are sold for cheaper, since they still work, but not as fast. IIRC most of TSMC's modern processes are designed to sell the good chips at a high price and the bad ones cheaper.
Evaluating the reliability of every component is extremely expensive. If you had perfect knowledge of the reliability of every component, then sure. It would be cheaper to build a perfect rocket first, then launch it. The cost saving comes because it’s cheaper to launch the rocket and see where it fails than it is to exhaustively evaluate the reliability of every component.
But you still need to do it for something like a rocket that is supposed to carry people in the future. In to long term, it's better to evaluate reliability of every component at the beginning, than do it after if fails.
You actually don’t. With a high launch cadence you can make statistical inferences about the reliability of the system as a whole. This isn’t the 80s where you launch one disposable rocket every 6 months. Starship is fully reusable and will likely fly multiple times a week carrying non human payloads.
Have you heard of instances where airplanes failed due to something as basic as a nut or a screw not meeting the required specifications? It's difficult to trust the reliability of a rocket, which is largely based on statistics, when you're putting hundreds of people on it. How can we be certain that NASA/FAA and other organizations will permit such a risk?
The crash of Partnair Flight 394 in 1989 resulted from the installation of counterfeit aircraft parts. Counterfeit bolts, attaching the vertical stabilizer of a Convair CV-580 to the fuselage, wore down excessively, allowing the tail to vibrate to the extent that it eventually broke off.
This happens to aircraft, and the only way to ensure it doesn't happen to rockets is to have extreme control of your supply chain.
You can’t verify the reliability of an aircraft by flying it 1000 times because you’d be putting the life of the pilots at risk. Autonomous rockets have no such constraint. You could launch starship an unlimited number of times without putting a single person at risk.
It's not even about putting the life of the pilots at risk - it's about cost.
It's starting to look like the lack of flame diverter was a huge mistake that will cost a lot of money and time, how do you think this will affect the whole project? What if after they fix the launchpad, it gets destroyed again, simply because some other preventable failure? How many more of these "tests" can they have without going bankrupt?
Pouring new concrete is cheap, they’ve already done it multiple times from damage due to test fires. Each time they reformulate the concrete and it needs less repair.
After they get to orbit they can start putting payloads on board, and the tests will pay for themselves. I’d say the risk of bankruptcy is very low.
Edit:
Just saw photos of the crater under stage zero. Looks like they do need a flame diverter lol.
Once they realized they needed a flame diverter, I think they traded off the cost and delay of putting one in vs the cost of just filling the hole each time.
It's totally feasible for them to keep testing and keep refilling that hole while in parallel they build a perfect launch pad with a flame diverter somewhere else. That way they don't have a gap in testing cadence.
Maybe you're trying to make the point that for very complicated systems with many failure points, the reliability of a single component is less impactful than redundancy and there is some truth to that but I must point out that risk vs cost calculations are definitely happening in both industries you mention.
Triple redundancy is not a thing in general aviation while, for some systems, it is in commercial. That's a risk vs cost calculation.
Semiconductor manufacturers do risk vs cost calculations through the entire development and manufacturing process. Source: I've worked in semiconductor, doing those calculations.
What you've missed here is that failure points can only be guessed or simulated prior to testing. That analysis is often more expensive than the cost of a test and test article.
So SLS does a LOT of analysis and manages to find and rectify failure points prior to flight. They pay a lot in analysis and inflexible design to do this.
SpaceX does some analysis, but then flies to confirm the analysis early. That way they identify actual failure points and can use sensors to see how close they got to failure.
For instance today I'm sure they learnt a lot from each failed engine, parts on the booster that stopped working, detailed telemetry on ship behavior on flight and sensors for the non-separation of booster from ship.
SpaceX builds to the same high tolerances as other rocket manufacturers, but they don't try to avoid testing through overly rigorous analysis. They also don't gold-plate their manufacturing, instead making tradeoffs to allow cheaper volume production with recovery instead.
> risk is now 10-fold higher at each failure point, but with thousands of failure points adding up you are now essentially doomed. And you haven't saved anything. The cost of 0.001 components isn't fundamentally different than the 0.0001 components were.
exactly. optimizing for an acceptable risk for space launches, esp space travel, seems still to be leading to $2B launches. at which point, what's the difference?
but like, SpaceX's whole thing has been blowing up test rockets over and over as they experiment, iterating quickly. once they've nailed the design down, the result is pretty reliable, but look at how many booster recovery fails they had before they got successful. your logic works more for defect rates on parts than on the design itself.
Before the anti-Musk cats get too wet, Elon Musk has said, and I quote, "If we get far enough away from launch pad before something goes wrong, then I think I would consider that to be a success. Just don't blow up the pad.": https://edition.cnn.com/2023/04/16/world/starship-spacex-lau...
Right. The criteria for this test was "Failure = CATO (catastrophic at takeoff)". It took off, it didn't CATO, it even hit max Q, therefore success. That's not "partial success". That's 100% success, you won the game show, but you didn't get all the points in the sudden death bonus round.
[A] ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pound of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot — albeit a perfect one — to get an “A”. Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes — the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.
The counterpoint to it is the probably apocryphal Soviet nail factory (but with more verifiable examples nearby):
Once upon a time, there was a factory in the Soviet Union that made nails. Unfortunately, Moscow set quotas on their nail production, and they began working to meet the quotas as described, rather than doing anything useful. When they set quotas by quantity, they churned out hundreds of thousands of tiny, useless nails. When Moscow realized this was not useful and set a quota by weight instead, they started building big, heavy railroad spike-type nails that weighed a pound each.
I was just poking that the parable ignores that the iterative side also requires an additional element - a target state to converge to over time, whatever that may be - repetition isn’t sufficient.
Yes, but let's not take the analogy too far. For people who work in safety-critical design (including software), it takes a different mindset than what is often prevalent in silicon valley. When the stakes are high, you don't really want to "move fast and break things".
You can actually see this in SpaceX. In development when stakes are relatively low (e.g., no payload or passengers), the risk threshold is high. But they start taking a more measured approach when that risk starts to ratchet up. The danger being, advocates of one approach don't always know when/how to transition to the other.
I previously worked in medical image processing/transcoding and you are correct, but most people probably don't know precisely how or why.
The knee jerk mindset that most people have in safety-critical design, is being ultra conservative.
In the fields of medical and space (and likely others), you have an asymmetrical risk-reward profile. Think about it this way, if an engineer takes a risk on refactoring some software logic, and it speeds the system up 3%, what is their reward? A raise? A promotion? Fat chance. If that refactor instead breaks 3% of the time; engines blow up, people die, customers yell at them, they get fired, perhaps in some situations they even get sued.
The engineers then converge to a local maxima by the means of: "If it ain't broke, don't fix it", and various other ultra conservative leanings. This mindset also will often get selected for in hiring, and rewarded.
Now take the limit of this as time goes to infinity, you have bloated, legacy software that is full of spaghetti code and can't take new features easily (if at all). In the case of NASA's space shuttle program, it was extremely expensive, and the cost wasn't falling significantly over time either.
One might view ultra conservatism as the problem, but the real issue is the asymmetrical risk-reward profile. Solving that takes a head-on approach with great leadership, deploying capital, state of the art testing/QA, great deployment pipelines, and more. Shield people from the risk, and intentionally reward people when they push the envelope.
Imagine if you had a software testing process and product specification that was 99.9999% effective (and no, yours is not even close to that), you could then move at a silicon valley "fail fast" pace and advance the technology and architecture rapidly.
> You can actually see this in SpaceX. In development when stakes are relatively low (e.g., no payload or passengers), the risk threshold is high. But they start taking a more measured approach when that risk starts to ratchet up. The danger being, advocates of one approach don't always know when/how to transition to the other.
This corroborates "fail fast". They achieve reliability and safety by launching far more than traditional space companies and seeing failure in those launches. They prove it out before adding the risk of human lives or expensive cargo. Meanwhile traditional companies will develop their rocket for 15 years till it's "perfect". SpaceX figured out that achieving perfection is best done through actual attempts, not laboratory experiments.
Nothing contradicts that. SpaceX does do this contextually. Once they nailed down the Falcon 9 rocket, the boosters and the Merlin engine, they mostly stopped messing with it and focused on operational excellence. But to get to that point they failed fast, because it's the most effective way to get reliability.
See the last comment of my OP. The risk I'm poking at is a cultural one, where "failing fast" becomes an acceptable mode of operating, regardless of the context.
Let's look back at Star Hopper. SpaceX literally hired a company that builds water towers to build a prototype tank for Starship - and flew it! They were primarily trying to figure out how to build it, move it, etc. Obviously the risk tolerance was high. That's really the difference between them and say Boeing. SpaceX starts with higher risk tolerance just to figure out the lay of the land, but they start reducing that tolerance as development progresses. Boeing aims for perfection out of the gate (apparently).
Alternative take: SpaceX is so new they don't know what they don't know.
Take their example of a failed F9 strut, where the material supplied by a vendor didn't come close to meeting the necessary specs. A mature aersopace company would have processes in place to check the material for these specs before use. SpaceX has since levied these new process checks, but prior to that failure lots people may have pointed to them as being more efficient because of their 'streamlined' process.
A mature aersopace company would have processes in place to check the material for these specs before use.
Would a "mature aerospace company" also know to not use O-rings outside the temperature range specified by its engineers? Or know to test whether foam traveling at high velocity would penetrate the TPS?
Look, this is hard stuff. It's very easy to tell when you screw up, but very difficult to tell how close you are to screwing up. You're deluding yourself if you think some entities are immune to screwing up just because they've been around.
Whataboutism aside, I’m not delusional; I’m quite upfront that these types of biases exist at every organization that is staffed by human beings.
The difference is I don’t allow “gosh, space is hard!” as a rationale for thinking one organization is immune to those shortcomings. So instead of taking a look and asking something like, “Hmmm. Every other organization seems to have a supplier vetting process for safety critical stuff, I wonder if we should too?” We can instead just pretend we’re smart and different and be forced to learn already solved problems the hard way. The supplier thing is very standard quality control process stuff that transcends industry. Knowing if foam can penetrate tile or o-rings operate out of spec are not, precisely because they were non-standard conditions. That’s not to say that the decisions weren’t flawed, but I don’t think it’s as good of an analogy as you may think. Besides, the investigations largely pointed to broken cultures so I don’t know if that’s the type of company you want SpaceX associated with.
What’s the saying? “A fool learns from his own mistakes. A wise man learns from somebody else’s”
That's fair, my point was just that you made it sound like "mature aerospace companies" were some special beings that didn't make mistakes.
It's good to learn from other people's mistakes, but you also can't let everything people have done before go unchallenged or no progress would ever be made from rockets that cost $2B per launch.
Yeah, I realize now that probably wasn't worded as well as it could have been. To say it differently, I would expect well-run companies (whether 'mature' or not) to have the processes in place to better control the well-known problems. When it comes to those 'unknown unknowns' sometimes you can't learn except by trial-and-error.
Or, you can take a risk-informed approach, and understand what risks are prevalent (e.g., the risk of a bad vendor) and put the appropriate checks in place to mitigate that risk. "Learning" doesn't always mean taking the highest risk option and just rolling the dice.
Indeed, and one way to verify you meet the spec is to test it, which they did and found it to be deficient. Having done so, they decided to improve their process. This is the definition of learning from your mistakes
This is an really optimistic outlook. One way to test the rocket is also to see if it fails when humans are aboard. But it may not be the best way to balance risk and what you learn.
It would have been much more economical to test a coupon of the material upon receipt, like what is considered standard practice throughout aerospace companies. Or, like you said, you can blow up a rocket and launch pad instead. Same result, different risk profiles.
I don't think anyone would argue it would have been better had they known better and done things right the first.
The question is how you transition from the state of not knowing to knowing.
If you have poor processes and a lack of knoledge, how do you get better?
In this specific instance, the root cause analysis and remediation are vastly more complicated that presented in this thread. It is not like SpaceX wasnt doing testing on incoming materials at all or ignorant of the concept.
You’re right. Like most failures of this type, it’s rarely simple and these forums are super conducive to long-form discussion. They did checks, but they were inadequate.
Regarding knowing if you have a poor process or not, it depends on the uniqueness of your problem. For proteins that are relatively common, like material checks, you can shorten your learning process by looking at other organizations that have been through it for decades. For more exotic non-standard problems, you might have to learn the hard way.
I think the distinction I'm making is that the "form" of learning you take should be proportional to the risk and that all forms are not equal in value.
I agree with everything in your comment. One thing I've wondered about re space exploration though is how we reconcile what I think is pretty much universal acknowledgement that we have to do everything possible to avoid loss of life with the inherent danger of space travel, and the "drag" that a zero-tolerance safety focus can have on culture.
Put another way, what would global exploration have looked like if sailors refused to accept the risks of early ocean crossings?
That is a real issue. I think part of it involves creating a culture where it's acceptable to make failures as long as those failures were the result of a sound decision process.
What you don't want is a culture that is either a) afraid to take any risk because they are afraid of career consequences or b) willing to roll the dice with bad decision processes due to biases and bad incentives.
Example for a): bureaucrats who are unwilling to push the envelope because a bad outcome would effectively end their career
Example for b): making high-risk decisions due cost/schedule pressure, like competition for a contract
Early modern sailors were uneducated manual labourers with few economic prospects in a world where simply living as a lower class individual was more dangerous than nearly any job that exists in the developed world today. Sailing often paid better than jobs on land which made up for the risk, and it offered the potential of massive reward to the high class leadership of the vessel.
There is no consummate economic incentive for being an astronaut. The incentive is the experience and making some impact on science, and while that motivates many people the probability of attracting the best and the brightest goes down as the probability of exploding goes up.
I'd say there's unlikely to be enough of an economic incentive to justify riskier manned space travel until the earth becomes a whole lot less habitable.
If what you're saying is true, how come astronaut programs have many more applicants than they can take? People have other goals in life beyond economic incentives.
People volunteer for the military, and I don't think anyone's under any illusion this is risk free. The important thing is not lying to people about the risk involved.
>People have other goals in life beyond economic incentives.
They acknowledged this point, though:
>The incentive is the experience and making some impact on science
I think you might be interpreting their last point differently than me. I took it to mean that the "Mars or bust" is an inspiring, but impractical, narrative.
I don't know what you're talking about. The Space Shuttle blew up twice, killed too many people out of sheer incompetence. And it flew again both times. Both times because NASA needed the vehicle to deliver on its promises. The Space Shuttle should never have flown after Columbia was lost. But it did, because NASA decided to live with the risks.
My expectation is that as soon as someone gets something going beyond Earth orbit and beyond the need to obtain launch licenses from the FAA, the zero-tolerance safety culture will be reduced. The Martians will build nuclear power plants.
I think it's a matter of failing in a safe environment. Clearly that's what space x is doing. They fail spectularly, but they do so within the confines of a safe environment.
For the real launch they've already mapped out the failure modes and are able to prevent them when it really matters.
I've worked in aerospace. The organizations (both public and private) would all claim to fail "within the confines of a safe environment".
One small BS detector is when there's some unplanned/unmitigated test outcome that gets characterized as a "test anomaly" rather than being transparent about the details.
>they've already mapped out the failure modes and are able to prevent them when it really matters
This remains to be seen. The shuttle also had all their failure modes mapped out. As did CST-100. Yet massive failures still occurred.
Well in the case of the Shuttle both accidents that killed people were due to previously identified failure modes. At the end of the day, risk will always be a number greater than 0%. At some point someone is going to have to make a judgement call that the risk is low enough to proceed, and sometimes that call is going to be wrong.
The failure modes may have been known but the effects were not. A FMEA needs both to work.
Regarding the foam, they had a difficult time even recreating it after the fact. It was apparently only on a lark that they decided to turn the gun up to 11 and, viola, now the foam had the physical properties capable of damaging the tile catastrophically. So, yes, they knew the mechanism of foam shedding but did not realize the effect properly.
With the o-rings, they similarly just didn’t have the test data for this conditions. They incorrectly extrapolated on the test data they did have.
The technological developments in the 60s/70s in aerospace never cease to amaze me. The SR-71, X-15, Saturn 5 rocket all developed when computers were less powerful than the NES yet they achieved things that no one else has been able to accomplish since (although I am sure there these have been surpassed via classified Skunk Works).
> they achieved things that no one else has been able to accomplish since
These sorts of statements are infuriating. You're dropping all of the context of past achievements (budget, political will, design goals, need, etc) and then castigating modern engineers.
The SR-71, X-15, and Saturn V achieved their design goals. There's no modern need to go back and re-achieve those goals.
The SR-71 is an amazing plane. When it was originally built there was a need for a reconnaissance plane that could fly high enough and fast enough to evade Soviet high altitude air defense systems. In the intervening years reconnaissance satellites have obviated that need. Outside of a few other tasks there's not much need for a Mach 3+ manned aircraft. Even for the tasks where one is useful the SR-71 still exists. That's not to say a replacement couldn't be built today. In fact many believe one exists in the form of the rumored Aurora.
It's just ridiculous to make assertions about "no one" being able to accomplish those projects today. If NASA or the DOD put out an RFP and had a budget for a modern X-15 it could definitely be built.
The test was a massive success. So happy to see starship clear the towers. The largest rocket ever built, larger than the Saturn V. Are you kidding me?
Elon was setting expectations a couple days ago when he said "Let's just not explode on the launch pad" or somesuch. In that sense, mission definitely accomplished. "Massive" success? Not sure I would go that far, but success of a massive project? Absolutely. It would have been nice to see the mission hit all it's objectives, but I'm sure they got quite a bit of data and operational experience.
What would be a failure is if the next one has the same problem.
Until now, every kg we launch has been horrendously expensive. So all things in orbit and sent to other planets have lots of expensive design features to reduce weight or to reduce size in the fairing.
StarShip is so huge and the reuse will be so cheap that it's going to be 100x or more cheaper. We could send heavy equipment into space and on to other planets that just didn't make sense before.
Once Starship can land 100 tons on the moon, the question isn't "what can we fit" but "how do we fill all this capacity usefully?!" So the science objectives we can achieve grow enormously.
The viability of the Starlink constellation depends on getting large numbers of satellites on orbit quickly. And the newest generation of Starlink birds are too large to fit in the Falcon 9 fairing.
If you are planning on several y and z anyways, absolutely. On the most recent SmarterEveryDay video on encasing a Prince Rupert's drop in glass, sculptor Cal Breed talks about the moment when a process fails. He could stop there and restart, saving some time, but instead all the pressure is off, and he "makes as many mistakes as possible" for the rest of the build.
Quote is towards the end, but the whole vid is worth a watch.
Adjacent to this discussion is the "All Up Testing" concept from the Saturn era. The conservative testing strategy was to test each component individually, then put them together and test them as a system. All up eschewed this conservative approach, testing everything that was ready. It's only marginally germane to this discussion, but a great historical note since other commenters are comparing the Starship with the Saturn V.
Maybe if you were only planning on doing something once. They've already nearly completed the next rocket. The whole point of this launch was a test. There was nothing on it. The next iteration will also be a test, and likely the next few after that. They'll even fly their own satellites up on test vehicles until they've worked out more of the kinks. Then before you know it, it'll be as reliable as falcon 9 and launching 100 times a year.
Look at all the Martian rovers. That is a notoriously hard environment to operate in, so success has often been defined as "the rover functions for at least 30 days". And then some rovers ended up working for over a decade. That doesn't mean that every rover that didn't last an entire decade was a failure though! It just means that, regardless of whether you hit your main goal, if you still have something working left over at that point, of course you keep using it.
The main goal here was to clear the pad and get some atmospheric experience with the entire stack. Goal met. But of course they had contingency plans to get as much more experience with it as possible, if things continued to function nominally (as they have with some rovers, or indeed some previous first SpaceX test flights, like the Falcon Heavy).
As you point out, "success" depends entirely on your goals for the task.
SpaceX said their goal was to get off the launch pad going into this, which indeed may have been setting a low bar for success. However, there is no other alternative definition.
If you are tasked with recording X, and anything else beyond is bonus, doing X is success.
If the rocket had exploded on the pad, that would’ve been an unsuccessful test. The launch pad and tower are way more expensive than the rockets and take much more time to replace. They call it “stage 0.”
Btw this isn’t moving the goalposts. Clearing the launch tower was always the success criterion; the rest is gravy.
yeah I think many people saw the launch as the launch of the finished Starship+booster when what was really happening was a test of maybe v0.0.2 if not pre-alpha. It's not done yet, there's a long ways to go before the expectation becomes 100% success.
Interesting thought: Starship just expended more engines on a *test* (39?) than I believe the entire SLS program has budgeted through the end of the 2020's (16 from shuttle + 18 new = 34?).
edit: Never mind, it's slightly less – see the comment below correcting me. The SLS program has 40, not 34, RS-25's.
Raptor2 also is being upgraded with things like electric gimbaling vs hydraulics. I think that was a hydraulic unit at the bottom of super heavy that exploded a few sec after lift off. So... electric gimbaling it is then. ;)
i'm a big fan of spacex and poked fun at SLS with all their delays and stuck valves etc. Well now it's the SLS fans turn to do a little ribbing. I still think starship will win out in the end but I'll tip my hat to SLS.. occasionally.
Well said. On the other hand, the SLS — having a launch escape system and not requiring sophisticated acrobatics in order to reach land safely — is totally the ship I'd rather be on.
SpaceX approach when it comes to safety is to make sure everything that can go wrong goes wrong during testing and lesser value launches, so that when it is time to put people onboard, the issues will have been fixed.
NASA is about making doing right the first time though careful design.
For SpaceX approach to work, they need to test a lot, with a lot of explosions, so they need to make a lot of rockets, they also need repeatability. That's why they are using assembly lines and mass production techniques.
Which approach is the best, I don't know. The traditionally designed Delta IV never failed, but on the other hand, we lost two Space Shuttles and their crews. As for the Falcon 9, it had a couple of failed missions, but it has proven very reliable over time.
I disagree. SLS is hard-limited to a small total number of launches. Starship is going to get into the hundreds soon enough. By that point Starship will legitimately be safer and more reliable than SLS will ever be able to accomplish, so I'd pick it.
I suspect SpaceX will have to add something resembling an escape system before the thing gets "man-rated". That system could also be used to protect the passengers in case of a failed landing. It will add weight and thus lower the carrying capacity but they seem to have enough margin to allow for such an addition. They already have the header tanks in the top of Starship, adding a number of escape engines and some explosive bolts to separate the nose cone from the rest of the ship should be doable. Add some parachutes to make the thing land at a survivable speed and you're done - beer coaster calculation style that is.
> Unusually, the ejection system was tested with live bears and chimpanzees; it was qualified for use during 1963 and a bear became the first living being to survive a supersonic ejection.
I was thinking more in terms of the B-1 crew escape capsule [1] consisting of the whole cockpit equipped with rocket engines and parachutes to push it away from the plane and allowing it to land in one piece. The same could be done with the top of Starship, the part above the fuel tanks.
They want to carry 100 people. An escape system for those many people is the same scale of a commercial airliner escape system, which is designed to be operated on a stationary plane, on land or water. Astronauts will be well trained to use it but I think that there must be a lot of openings to let 100 people get out quickly in mid air. Maybe they'll agree that Starship is its own escape system.
Before the "Ejection escape systems" there are a few paragraphs about "inflight crew escape system"
> The vehicle was put in a stable glide on autopilot, the hatch was blown, and the crew slid out a pole to clear the orbiter's left wing. They would then parachute to earth or the sea
But 100 people is a different matter IMHO.
Ejections seats and capsule were not pursued, the Wikipedia page explains the reasons.
The big problem with the Space Shuttle was that the orbiter was located next to the fuel tank and between the solid boosters instead of on top of it. This made it impossible to perform the normal "accelerate away from the big boom" manoeuvre which normal rocket escape systems use. On Starship the passengers will be situated above the explosives instead of next to them/between them. As to whether it is possible to add escape engines of sufficient power to pull away the nose cone, push it up into a parabolic trajectory of sufficient height to give parachutes the chance to deploy I don't know but at least it could be done in theory where the design of the Space Shuttle and its "close cousin" Buran made this impossible.
We also went by "Human Interference Task Force" and "Angus MacHammer and the Ukrainian Glowplugs," renown in North Texas for our musical mediocrity. We were once introduced as "DJ Control Rat and MC 1000 Inch Buddha," which was interesting 'cause we were completely unrelated to MC 1000 Foot Jesus and played bluegrass.
Fun, little known fact: The Shuttle program's only successful post-launch abort was performed by Challenger in STS-51-F (not to be confused with Challenger's STS-51-L, which ended... suboptimally)
If you're counting the capsule itself, Starship qualifies too.
If you're talking about the launch escape tower, that's for during launch. It's gone by the time you reach orbit, leaving you in the same scenario of Lunar Starship in the event of a failure; in space, but no way to get down.
Maybe someone can explain to me why SpaceX lands their boosters?
I would think weight of fuel required to land booster + legs + steering fins would be much greater than a parachute or two and a water landing. That weight savings would be a cost savings as well as more payload that could be lifted (also a cost savings).
Would the booster be destroyed landing in the ocean by parachute?
Salt water is bad … evil stuff … they already try to avoid siting on the pad as much as possible to reduce salt spray corrosion from the sea air… even the barge trip back to land is less than ideal… salt water is the enemy of any complicated metal objects… so they are doing everything possible to avoid hitting the salt water and to keep the rocket as far away from it as practicable.
> On the other hand, the SLS — having a launch escape system and not requiring sophisticated acrobatics
> in order to reach land safely — is totally the ship I'd rather be on.
Have you ever flown on a Boeing 747, without a launch escape system?
That said, I do agree about the flip maneuver. I am very interested to see how that evolves.
SLS will never reach land, you'll be splashing down in the middle of the ocean, stuck inside a tin can getting battered by waves until NASA can scramble their limited resources to get you.
The booster presumably like you will have been thrown into the ocean. Very wasteful system.
This is pretty hyperbolic. In the shuttle era, these contingencies were thought of and planned for. The amount of preparation NASA would do before shuttle launches was incredible, including flying medical and rescue teams to the chosen launch abort sites.
"Preparations of TAL sites took four to five days and began one week before launch, with the majority of personnel from NASA, the Department of Defense and contractors arriving 48 hours before launch. Additionally, two C-130 aircraft from the space flight support office from the adjacent Patrick Space Force Base (then known as Patrick Air Force Base) would deliver eight crew members, nine pararescuers, two flight surgeons, a nurse and medical technician, and 2,500 pounds (1,100 kg) of medical equipment to Zaragoza, Istres, or both. One or more C-21S or C-12S aircraft would also be deployed to provide weather reconnaissance in the event of an abort with a TALCOM, or astronaut flight controller aboard for communications with the shuttle pilot and commander."
Do you think that is fundamentally different with, say, Boeing or SpaceX?
If you look through all the failures and close calls in aerospace they are often rooted in human psychological errors. The pressures that lead to them may change with different organizations, but they don't go away.
This almost sounds like the start of a joke.... 'so an engineer, a politician, and an accountant walk into a bar.' One's the head of SpaceX, one's the head of NASA, ones the head of Boeing. So yeah, I do think there's a fundamental difference there.
So how would you characterize that difference, both in terms of strengths and weaknesses? I have a few thoughts but would be curious to hear yours first.
While this is an interesting question I'm not going to give an especially interesting answer. I see things as you might imagine. And while it might seem unfair I'd also appeal to reality. It's now been more than half a century since a human left low earth orbit. NASA and Boeing (et al) had all this time to succeed. They failed, and there's no real excuse for their failures besides themselves, and their own motivations.
Keeping it brief SpaceX/engineer is genuinely trying to get people to Mars, largely driven by ideological reasons with extensive technical creativity/competence backing them up. Accountant/Boeing wants to make more money. Outsource our software development to guys in India bidding $9/hour? Awesome! That's another 0.037% profit, what could go wrong!? Something doesn't work? Who cares!? We're on a cost+ contract baby, what you call "failure to deliver", I call delivering value to my shareholders!
And then there's the politician. In this particular case, he's not only a life long politician but also 80 years old on top. The only 'bright side' is that, due to his political influence, he's gone to space before. On the other hand Charles Bolden was a genuine astronaut and absolutely everything one would think they would want from a NASA head, yet he was a miserable failure. It may simply be that political style leadership (even when not a politician) isn't really conducive to meaningful progress in modern times.
>They failed, and there's no real excuse for their failures besides themselves, and their own motivations.
I'd argue the incentive wasn't there until CCP. That was the fundamental difference in the last 20 years. Without CCP, I don't think SpaceX would be successful, either. But I will say they've done much more than the Boeing at executing on that incentive.
I do think you may be overly cynical in your characterization, though. It wasn't too many years ago that Boeing was listed as the most desirable company to work for by college students. The reason isn't that they thought it was because they couldn't wait to gouge the public coffers, it was because aerospace has always been considered a sexy engineering discipline. You'll almost never find a civil engineering firm on those lists because "roads and commodes" just aren't considered cool.
Back to the question, I'll weigh in with my perspective. They are all responding to incentives, albeit different ones. But we have to acknowledge the downsides of each. SpaceX is awesome, but they aren't without their own psychological pressures and biases. I've brought it up elsewhere in this thread, but they have wanted to rapidly iterate rather than fundamentally understand some of their design issues. I suspect this is partly cultural (where operational tempo matters more than scientific rigor...i.e., "we don't need to know why it works, as long as it works") and some of it is business (i.e., they have specific contractual deadlines to consider). Those are also some of the issues that lead to mishaps dating back to Apollo and Shuttle.
Considering you seem to think the legacy downsides are due to the business/shareholder side, do you see the same issues encroaching on SpaceX if they go public?
Absolutely, I do believe that SpaceX going public will largely be the death of that company. And, depending on when this happens, it could even herald the second death of space in America. The one thing that's good here is that Elon has stated that he will not be taking SpaceX public until transit between Earth and Mars is well established.
The point I'd make is that leadership really matters. Boeing, at its peak, almost certainly had orders of magnitude more talent than SpaceX did in its early years - in no small part because of what you mentioned. And they absolutely had many orders of magnitude more money and access to funding. But their leadership was just absolutely abysmal, and consequently the potential of that talent was left completely untapped.
But on the other hand, like you mentioned there are incentive problems. Even though Boeing failed to tap into their potential, their stock price has been constantly and steadily going up for decades. Even their planes literally falling out of the skies was but a brief stumble, the damage there largely repaired owing to the start of a profitable new war. So in this regard I doubt their leadership is particularly disappointed with their results. They achieved what they set out to do after all. And that's pretty disappointing.
>The point I'd make is that leadership really matters.
I wholeheartedly agree. You're probably aware, but there have been a number of good write-ups detailing an overall erosion of engineering leadership at Boeing [1].
>Elon has stated that he will not be taking SpaceX public until transit between Earth and Mars is well established.
I think maybe the difference between you and me is that I take many of Musk's promises with a boulder of salt. If I was a gambler, I'd bet that we see some wordsmithing about what the definition of a "well established Mars transit" when it comes time for an IPO.
A decade ago, the word was that SpaceX would go public "when we are reliably launching F9." Then Tesla went public and Elon decided he didn't want anything more to do with public companies. If anything, I think that statement should only be interpreted as it absolutely not happening before that condition is fulfilled, not that it will happen once it is.
... they have wanted to rapidly iterate rather than fundamentally understand some of their design issues.
You say this with seeming knowledge of the thoroughness of SpaceX's failure investigations. Care to elaborate how you can say this with such authority?
In large engineering organizations, it is not uncommon for highly specialized engineers to not be aware of the status of the overall project. Plus, you hear A LOT of different stories, rumors and expectations as projects progress. I'm SURE they started with "Oh Yeah! This Baby is Going into ORBIT!" but as time went on they were like "Well.. we don't have time to build these 18 components to spec, so we loosened the tolerance so we think it'll launch but might explode 10 seconds later." And the nameless drone in sector 47 only heard the first projection.
I would be very surprised if Shotwell didn't have an all-hands to set the whole team's expectations beforehand.
SpaceX is well known to have a very "flat" management structure. They expect all their scientists and engineers to learn, understand, and contribute across-the-board.
I've have my own version of this more times than I can count.
For me, I run a trading firm and when I started it was "if we can survive the day without losing a ton of money, its a win." I was happy if the positions even made it to the exchange the way we wanted, haha.
Which is rightly, awesome... They just launched an 11 million pound firecracker....
It would be AWESOME if elon included actual fireworks which would go off in the event of a RUD to add some fabulousness to such events.
In fact, all the devices that are stripped out for these test launches should be replaced with their equivalent weights in fireworks. Detonate them when possible during a RUD to allow for more spectacle!
This is a fantastic idea, I'm sure the FAA will be thrilled when SpaceX asks them for permission to turn their rocket into a disneyland night-time spectacular orbital missile!
All Americian launches have a Flight Termination System (FTS) which is a bomb attached to ensure the off-course vehicle explodes before returning to earth. There are personnel in the control room who can activate the FTS when something goes wrong and there is a danger to people on the ground.
That is what detonated the StarShip today.
You'll find it was notably absent in Soviet rocket tests, for instance with the N1 which veered off course, returned to the ground intact and exploded there instead.
NASA did so many studies where they tried to put together Space Shuttle parts to make ‘low cost’ rockets (saving development expenses) but whatever they came up with was terribly expensive since those parts were expensive and that didn’t matter if it was a rocket with a cluster of 4 SSMEs or a rocket that strapped on 3 of the solid rocker booster.
You’d think NASA would have read their own studies and given up rather than proceed on the predictable Artemis boondoggle.
The way I'm hearing it was not a failure but an unbelievable smashing success. No one expected the rocket to get to the launchpad, much less off of it. The second stage was not expected to be reached at all, it's cool that they even tried. Greatest test launch of all time. From what I've read this morning if the test flight had worked all the way it would be like man on moon levels of good.
Eh, if SpaceX had genuinely not expected the rocket to get off the launch pad, they would not have launched it.
I expect their math was something like:
95% chance of enough engines lighting to launch
90% chance of clearing launch pad
60% chance of making 60 seconds of flight
30% chance of 2nd stage separation
15% chance of re-entry
>> Plus, it's stealing pieces of the Space Shuttle, which was developed in the 70s over a long period of time.
That is not always a bad thing. I mean you typed this on a computer running an ISA that started in the 70's or 80's. Maybe 90's if you are on PowerPC ;)
They're still designing it, and learning to build it (building the "machine that builds the machine"). They've already scrapped booster 4 / ship 20 without even a test flight.
Today's rocket was a byproduct of what they're actually working on, so it's hard to pin a price on it. I'd guess a few million dollars in engines, and a few hundred thousand in steel? This is all stuff they would've needed to find a way to dispose of anyway; the next booster in line (B9) already has hundreds of improvements. Among these are improvements to thrust vector control and engine shielding, both of which were involved in the issues with today's flight.
You touch upon an interesting advantage that private space engineering has that, depending on one's attitude on the topic, may or may not be fair.
NASA has to do everything in the public eye and their process reflects this. Private enterprise is allowed to hide information, which implies they are allowed to massage information.
(... this probably says unfortunate things about the nature of governance by public sentiment that it's best not to think about over-much if one is super-fond of democracies as engines of progress...)
True. On the other hand, we do know that SpaceX has taken about $10 billion in funding since its inception.
With that, they developed Falcon 1, Merlin engines, Falcon 9, Falcon Heavy, first full-flow staged combustion engine in the methalox Raptor engine, Starship, Starship Heavy, Dragon, Crew Dragon, Re-usability, Starlink satellite manufacturing and 4,000 satellites launched to orbit, and quite a bit more.
The SLS has spent 23.8 billion in nominal dollars so far. SpaceX has received about 9.8 Billion total investment, which puts an upper limit on operating losses.
$9.8 billion legally-obligated-disclosable total investment. The disclosure requirements for NASA are stricter than for private industry (and there may also be an apples-to-oranges question on auditing and accounting: if NASA grants a company $500 million to do R&D, and that company turns around and pays its staff and fabrication costs with that money, is that a billion of expenditure on the SLS project or $500 million?).
I agree there are some Apples to oranges comparisons issues. For example, you would want to add pure R&D grants to the SpaceX total.
I don't follow your NASA example though. We know the NASA/government spend amount for the SLS (23.8 billion). We don't need to speculate on if there is double counting of government spend and contractor spend.
The problem is that the public and Congresspeople don't tend to see it that way. They aren't generally of the opinion that spending a couple billion to blow something up on the pad or shortly after launch is an acceptable resource expenditure (especially not at the Congressional level).
Rocket specialists know this is part for the course, but most of the public isn't rocket specialists.
I believe you underestimate the modal American congress-person and rate-payer.
In congress, maybe about 25% of them can't be bothered to remember what state they're from. They don't really matter as they'll pretty much vote however the last person to talk to them wants them to vote. This is why congressional legislative directors try to schedule appointments immediately before their congress-person votes.
About 10% of congress-people are SUPER sharp and will understand this whole "you have to break eggs to make an omelet" concept. 50% more could be educated. The remainder will act randomly depending on what the internal polling says.
SSL and Boeing and the old school guys know a critical part of their job is to lobby congress-persons and staffers. They're not paying them off, they're just making campaign contributions to ensure they get access to pitch their side of the story.
As for the modal American voter? They don't care about space. They care about whether they're getting a raise next year, the mortgage is paid and inflation doesn't price them out of a good meal every now and again. When they get economic security, THEN they start caring about other things like who goes in what bathroom, why they can't buy TANG or light-bulbs at the grocery store anymore or whether they're getting value for money in their national space program.
My gut feeling after doing polling for a few years is the numbers are about the same: 25% of American voters can't tie their shoe-laces, 10% will understand you sometimes fail when you try to do something innovative and the remainder will need some convincing.
The good news is (effectively) no-one in the US looks at national budgets. Heck, most congressional staffers never read the whole thing, just the bits they're interested in. Many (most?) voters (and congress-persons) look to membership organizations for direction. If someone is a member of the Planetary Society, and they happen to be chatting w/ their elected representative, you can be pretty sure they'll mention how important the space program is. If someone is a member of Drunk Middle-Aged Regressive Science Haters of America, you can probably guess what they think about anything with the pong of science about it. Fortunately, this latter class of Americans usually doesn't know who their congress-person is or that they have a congressional representative.
So... to make an already long post short... I don't think you have to convince EVERYBODY, just the people that matter. The message that "it's okay to fail from time to time as long as you're making substantial forward progress" is something at least half of the people who affect US budget priorities can get their brainstems around.
[And as an aside... having worked with US congress-people in the past, I can report they're frequently much smarter than you give them credit for. And you don't get to be a Legislative Director in a congressional staff without being reasonably sharp. But they do worry about how large donors will respond to their votes. We're entering a phase of debt-ceiling debates. The GOP controls congress at the moment and we'll see a lot of wheeling and/or dealing. It's quite telling to see what each party thinks is important. Biden seems to have invested a small portion of personal reputation in the SLS, probably because of it's history over the Obama administration. Despite it's actual benefits, various GOP members may use that to rail against it (SLS) as a proxy for railing against Biden. (I'm trying to avoid being partisan here, some Dems railed against the Constellation program during the Bush years, so I'm not saying it's ONLY something the GOP does. It's just the typical railing against the other party to try and fire up your base so you don't have to explain why you didn't follow-through on your campaign promises.) But I ramble...]
> NASA has to do everything in the public eye and their process reflects this. Private enterprise is allowed to hide information, which implies they are allowed to massage information.
That would be a good point if we were talking about NASA vs Blue Origin. But SpaceX is arguably building more in the open than NASA.
> How would you argue that? I mean, please step me though this argument.
The person I replied to was making the case that NASA can't take risks because they develop in the open whereas private companies can develop in hiding, so they can take more risks.
But a quick glance on reddit, nasaspaceflight, and youtube tells me that SpaceX is very much developing in the open, and to a greater extent than NASA.
Which means that there is some other reason why SpaceX feels more free to take risks than NASA.
In contrast, Blue Origin is famous for not disclosing almost any information to the public. For a long time Tory Bruno, the CEO of ULA, was the one providing the most public information on Blue Origin's BE-4 engines.
If you cared to, you could download the entire US federal budget, NASA's budget, NASA's contracts with SSL, Boeing, SpaceX, etc. and all the ancillary data regarding milestones and results. You could, if you wanted, download more information that you might ever want about Blue Origin's contracts w/ the federal government. And SpaceX's contracts with the federal government.
Not everything is there, but enough to get a very decent picture of what's going on. Sometimes relevant information takes YEARS to be published.
But it's not easy to find. I've had to directly email people sometimes. Sometimes stuff is classified for no good reason other than someone thought at one point that a particular program was dual-use (commercial/military). Sometimes you DO have to file FOIA agreements.
I'm completely not coming down on anyone for not spending their time doing this. It takes a fair amount of time to piece things together from spreadsheets and contract addenda.
But it is possible.
What SpaceX does is they make it easy to see what they're doing. They upload videos to YouTube and Shotwell speaks at events and conferences from time to time. But you are getting their side of the story. Every now and again Casey Dreier over at the Planetary Society will dig up some previously difficult to find nugget of information about how various programs are being funded and exactly what they're being funded for. But Dreier's job is to focus more on planetary exploration missions rather than aerospace development, so I don't think he's focusing on SpaceX, SLS and Blue Origin.
You can only massage information so far though. The costs do actually matter and you can't turn a profit if all of your launches are horrendously expensive (above what you're charging for them), and you just keep it a secret.
What's the value add on your comment? Of course costs matter for IBM and Amazon! They're publicly traded companies, and they need to make a profit or else!
Well... the US tax-payer spent $2.9B for a starship or two. Or to be more accurate, there's a contract between NASA and SpaceX to deliver a Lunar Lander Starship variant (NextSTEP-2, Appendix H, Option A.) Last year it was upgraded to Option B which (I think) calls for a beefed-up Starship that can perform multiple lunar missions.
I'm sure they're not just dumping 2.9B in Elon's bank account (I mean... if they did he would just buy back all the Tesla stock) but there are a series of milestones that need to be met for the government to release the next chunk of funds. I don't know if NASA published the exact milestones, but maybe "not exploding on the launch-pad" was enough to release the next block of funds.
By comparison, Falcon Heavy costs about $62M per launch, but Gwynne Shotwell predicted they can shave 40% off that price tag when they start re-using heavies.
But Elon has supposedly said the TDC (total development cost) of the Starship is projected to be 2-10 billion. [I found this at the Daily Mail website, so take it with as large a grain of salt as you wish.]
If it costs $10B to build one and $10M per launch and you get 10 launches per vehicle, that's $1.1B per launch if you amortize the development costs. But if it takes $2B to build one and $2M per launch and you get 10 launches, then it's only $220M per launch. But if you build five operational starships that each have 10 launches each, things start to get downright affordable.
So if the question is how much has SpaceX spent on the Starship to date? I don't know that's public info. If the question is how much will they spend on development, the answer is $2B-$10B (though that's from the Daily Mail.) If the question is how much will each launch cost (including amortizing development costs)? It could be as cheap as $42M. Or depending on how much of the $2.9B from NASA they're able to apply to previous development costs... who knows!? They could make $1.6M per launch as long as the GSA doesn't audit them too closely.
There's a myth in US government purchasing that competition drives costs down, and that's probably true for commodities. But Starship class super-heavy-launch vehicles aren't commodities. There's a TREMENDOUS amount of cost (both opportunity cost and dollars) associated with fiscal oversight of large projects. The hope has always been that when Blue Origin built something to compete, it would bring total costs down. I am skeptical.
They literally have another booster/starship ready to go. The project cost may be 2-10B but the individual launch vehicles will cost less than 200M$ each.
I'm not sure why people need to be reminded of this. This isn't a reply to anyone making complaining about the lack of success. It comes off as fanboyism.
They have two more ships built and two under construction. I think the iteration time could be driven by the (1) condition of the launch pad (2) the cause of S24 destruction.
100% success matters when you need to hit a launch window for interplanetary transit. or when you have spacecraft that are integrating sensors from dozens of commercial and academic partners. 100% success matters when rockets are actually launching payloads. what is the evidence that this fail fast method is superior to the already tried and tested method? everyone just assumes that. the differences between spx starship and SLS go far beyond just that method anyway so you cannot possibly compare
Your comment was dead when I got to it - I disagree with your words but vouched because I think being skeptical is valuable (terms and conditions may apply, etc).
I can't personally attest to this but another commenter mentioned that this was what the soviets did as part of a space race with a smaller budget.
You should note that success rate is crucial once you are in production and ready to go - judging overall success against that bar while developing and testing is nonsense.
Finally - I think everyone learns faster by trying more things and making more mistakes? I wouldnt blame you for wanting to see that studied - kind of a neat question after all - but it seems like you are assuming one method or another was picked because it was superior as opposed to a reality of material constraints.
I feel like if NASA had budget for blowing up more rockets to prove/disprove points that they would be doing so. Actually now that I say it out loud I can't imagine anyone choosing 'not rocket'.
Zero, given that the launch corridor doesn't pass over land, and all rockets are equipped with a Flight Termination System that blows up the rocket if it deviates from that corridor.
SLS has had a 100% success rate. One launch, one success. But it costs $2B per launch (and climbing) and has taken 12 years to get here. Plus, it's stealing pieces of the Space Shuttle, which was developed in the 70s over a long period of time.
Starship will launch a half a dozen "failures" before they achieve success, but they will have a bigger, better launch system that's fully reusable and costs orders of magnitude less per launch.
"Fail fast" applies to more than just software.