There's denial and there's vehement to the point complete denial from multiple different companies.
It's either a giant conspiracy by the FBI and multiple mega-corporations to blatantly lie, on public record, about a matter that if happened would most likely come up in the future again.
Furthermore if apple and amazon were both notified for comments, there is good reason to suspect that the FBI would hear of the article and try to censor such an article for national security reasons, especially so if they made apple and amazon lie about it.
Or ... Bloomberg didn't do their due diligence and were too eager to be duped by agents who wanted to push an agenda to move manufacturing away from china or something similar.
>there is good reason to suspect that the FBI would hear of the article and try to censor such an article for national security reasons
The rest of your post aside, the FBI cannot do that. It wouldn't matter if it was secret. It's not even a close call, it's been explicitly and repeatedly slammed down by courts even in extreme cases like classified information being illegitimately leaked, for example with the Pentagon Papers (SCOTUS ruling [1] against prior restraint). It just came up again a few months ago when a federal judge tried to use prior restraint and depublishing against the LA Times over their publication of information about a confidential informant and bargain that was accidentally published in full on PACER. A rung bell cannot be unrung.
Now, if the FBI could find a leaker who had signed an agreement with the Federal government they could go after them in person. If a newspaper broke the law to obtain a story then that separate violation could independently be prosecutable (in public). But none of that means the publicly released information can then be taken back. And even if some random blogger might be intimidated illegally and not find the resources to fight back, that wouldn't be an issue for a major publication.
I don't take issue with your skepticism in general but it's not helpful to ascribe special powers to government that it doesn't actually have either.
> It's either a giant conspiracy by the FBI and multiple mega-corporations to blatantly lie, on public record, about a matter
Like, say, the matter of secret surveillance on the mass scale? I mean, the track record here is not exactly pristine.
> Bloomberg didn't do their due diligence and were too eager
That is a distinct possibility too. But I think we are now beyond the point where we could say "major tech companies would never lie together with the US security apparatus on a matter of public importance". They would, if they think it's worth it.
> But I think we are now beyond the point where we could say "major tech companies would never lie together with the US security apparatus on a matter of public importance". They would, if they think it's worth it.
This is just conspiracy theory thinking. You offer no evidence for this incredible assertion. Some companies have previously collaborated with the government, generally without explicitly lying, but we cannot jump to the conclusion that all companies would voluntarily lie in a coverup conspiracy -- which, by the way, opens them up to investor lawsuits and risks destroying their branding, for no good reason. We also do know that the government cannot legally compel companies to lie, only to remain silent.
It's not "theory", it's publicly known facts that companies participating in mass surveillance denied it, and US official lied under oath to Congress (and weren't punished for it) to conceal it. There's no "theory" here.
> You offer no evidence for this incredible assertion.
The evidence to the above is publicly available and has been discussed to death. If you somehow managed to miss all of it, start with https://en.wikipedia.org/wiki/Mass_surveillance_in_the_Unite... and go on the links from there, it will take you some time.
> Some companies have previously collaborated with the government, generally without explicitly lying,
Yes, saying "we do not conduct this particular kind of surveillance ordered by this particular person" while knowing they conduct a slightly different kind of surveillance, ordered by different set of persons - is not explicitly lying. Just like saying "we don't have surveillance technology installed by FBI" if it's installed by NSA instead. There are many ways of lying without "explicitly lying".
> but we cannot jump to the conclusion that all companies would voluntarily lie in a coverup conspiracy
We can not and we do not. We do not know whether any specific company would lie - we just know this option is now very much on the table.
> which, by the way, opens them up to investor lawsuits and risks destroying their branding, for no good reason.
Being on good terms with somebody as powerful as US federal government is a very, very good reason. And I don't see anybody's branding being destroyed so far by the revelation of mass surveillance. We know about https://en.wikipedia.org/wiki/Room_641A and https://en.wikipedia.org/wiki/Hemisphere_Project - has AT&T brand been destroyed? Not in the least. And the government granted them immunity from lawsuits related to this.
Your whole post is filled with unsubstantiated conspiracy theories. The only program that named these companies was PRISM, which ingests data from targeted electronic wiretaps conducted by the FBI. None of the companies lied about that.
Correct me if I'm wrong but if you are referring to in PRISM the denials were quite vague. Both facebook and google said they never heard of a program called Prism but never denied being part of one. And facebook denied "direct access".
Even when those responses came out there was confusion about what they meant in the statements. There doesn't seem to be any confusion here.
It's easy to pretend to be confused about a denial. Most people here do it so they can have something to post about.
The PRISM denials were not vague and were perfectly true. PRISM was a system for handling subpoenas, and nobody denies getting subpoenas. If you ask e.g. Facebook they also won't deny proactively reporting child abuse to the FBI, which is another real US legal requirement.
I think the matter of distinction of your second point isn't that they would or could, it's that we have evidence that in the past, they have already done this.
It is possible that because of the nature of counterintelligence investigations, the individual companies agreed to gag orders while law enforcement does its work.
Also, these companies want to continue to do business in China, and likely do not want to be on the record accusing the government of a massive criminal conspiracy.
https://www.apple.com/newsroom/2018/10/what-businessweek-got...
Finally, in response to questions we have received from other news organizations since Businessweek published its story, we are not under any kind of gag order or other confidentiality obligations.
Statements are very specific so it's easy to craft them in such a way that it basically respects the gag order and does not actually contain any untrue statement. Even " Apple has never found malicious chips, “hardware manipulations” or vulnerabilities purposely planted in any server."
Would this statement hold true if it was found not by Apple?
"Apple never had any contact with the FBI or any other agency about such an incident."
If Incident was being handled by 3d party private entity which was in contact with some gov entity this would hold true.
"We are not aware of any investigation by the FBI, nor are our contacts in law enforcement." If it was contacted by entity other than FBI this would hold true.
Also national security related investigations are highly segmented in corporations -- in the company the size of apple for instance there may be a handful of people with knowledge that the investigation is ongoing or even exists. Its not like the PR department (or even the full board) may be aware of its existence.
They throw out some massive numbers though. You can kinda discount 6 "apparatus" sources (but usually for stuff like this it's just one or two, so it's already unusual), but they also mention several people at Apple, Amazon and the FBI, plus someone else in the "discouragement" meeting. That's a lot of agents.
If it's made up, it's more realistic to believe the publication is complicit in the fabrication.
Could these companies to anything _but_ outright deny it?
Silence would be perceived as confirmation, and claiming to not know would be terrifying to their customers. Confirming it would risk their entire supply chain.
Sure, maybe it's not true... but the only move here is to deny it even if it were true. The corporations involved have a massive amount to lose here.
The FBI is not the only jurisdiction these companies have to follow, if they want to continue to operate internationally. Couldn't such silencing orders come from China or something?
I also feel that the hack described is only borderline technically feasible.
They describe a microcontroller the size of a decoupling capacitor that is installed between the main CPU and main memory (as far as I can tell from the vague description).
I assume this would have to be done without layout changes. On a part of the board that is quite sensitive to layout changes. It just doesn’t seem likely that you’d do a hack like this. You’d need a micro controller or ASIC that was running as fast as main memory. You’d need to make it cope with different kernels... and edit memory such that remote servers could reliablely be contacted.
Why not just swap out some other part? Like the IPMI controller? Or the Ethernet controller? Something that has access to main memory, that would hide the functionality even better, and that would give the attacker more space to work with?
Except... a key word in the article makes the hack perfectly believable and feasible: "baseboard management controller".
If the chip is inserted on the serial data line between the SPI flash memory and the BMC CPU, then, as an ex-InfoSec engineer, the whole thing sounds very plausible and even easy IMHO...
You have to expect that any article about intricate tech details written for a general audience will get parts of their descriptions wrong. Like you I was raising my eyebrows when I started reading. But when they mentioned the BMC, I believed. The author did not make a vague mistake when mentioning this very specific technical term.
In fact, the BMC is the perfect target for such a hardware hack: low-speed SPI flash memory interface easy to man-in-the-middle, BMC more privileged than the OS (can virtualize storage, keyboard, etc), BMC code independent of the OS (infect both Linux and Windows at once), BMC code changes so rarely that a backdoor making assumptions about the code layout and content would still work after many years of updates, etc.
Edit: I checked Supermicro servers from the 2015 era. Most used the AST2400 BMC. It boots from SPI flash so this backdoor chip only has to intercept and modify bytes on the data out (DO) line to inject malicious code.
This was my thinking as well. I was curious how such a device would do what they are claiming. And BMC is the perfect candidate to make this whole bit work. Reading that phrase definitely made the gears turn and the whole thing started to piece together a clearer picture.
AST2400 has option for two spi memories, one overrides the other by default. They simply put a microscopic spi flash in place of the second "recovery" flash.
I heard before the rumors of Chinese server mobos "talking" some gibberish on ICMP, so that must be it.
You could be right. But stuffing megabytes of a full copy of the BMC flash image in such a chip may be overkill. You could just as easily man-in-the-middle bytes sent by the legit flash over the SPI DO line...
It does, and all the technical details mostly match up. From the plausible attack vector (targetting the BMC with an implant) to CG animations of a compromised motherboard showing the implant at the exact spot between the BMC and its firmware chip (which makes perfect sense if it's some kind of SPI interposer).
Yes, they need to provide more details on how its supposed to even work. We can come up with all kinds of theories but proof is needed that this device is even a chip. I want layer photographs of that thing.
Look at the stock of supermicro, since 2015 (the year of revelation) it's been going down for them - reason might be because none of the companies bought new hardware from them.
> Look at the stock of supermicro, since 2015 (the year of revelation) it's been going down for them - reason might be because none of the companies bought new hardware from them.
> "The delay primarily relates to the magnitude of work that the company must still perform in order to review the company's accounting judgements, estimates and records for transactions that occurred during fiscal year 2015 through 2017, as well as the assessments and conclusions on the effectiveness of its internal control over financial reporting."
It's interesting that the irregularities started in 2015.
Tinfoil hat time. Based on my assertions elsewhere I suspect this is posturing plausible deniability for a more local actor being responsible for implants that may or may not be discovered already.
The vigorous denials from Apple and Amazon are suspiciously against the grain in these situations.
If you compare to MSFT etc whenever there is a "global cyber incident" the story is the same and correlates with the governments etc which is business as usual.
Apple and Amazon probably have a small set of TS/SCI cleared employees who dealt with this mess. It’s likely 99.99% of the employees at those firms had no idea what was going on. The switching out of thousands of compromised servers was probably made to look like routine maintenance or upgrades and the whole affair was kept secret. That is, until some high level government employees intentionally leaked it to the media, probably under direction of the White House to garner support for a more aggressive stance on China - the trade war in particular. Read between the lines.
That's a lot of unsubstantiated assumptions. It's also explicitly denied by both Amazon:
>It’s untrue that AWS knew about a supply chain compromise
and Apple:
>we have conducted rigorous internal investigations based on their inquiries and each time we have found absolutely no evidence to support any of them.
There's no in "between the lines" or ambiguous wording there, they flat out deny it. Unless this small set of "TS/SCI cleared employees" worked completely on their own without reporting to anybody else in the company this means that they are lying in these statements.
It's possible that they do just that but it's a bit strange to me that they wouldn't find an easier way to deflect the issue without using such a strong and explicit language. Something vague like "we've been working with the authorities and have no reason to believe that any sensitive information has been leaked etc..." would be easier to spin if it turns out that somebody can prove that these attacks took place.
Surely if the scale of the attack was as large as reported by Bloomberg in their article it should be possible to find one of these backdoored boards in the wild? Or at least have testimonies by employees in these company that could testify that batches of motherboards were suddenly replaced for no obvious reasons?
And if trade war is the reason why deny it now? What do they have to gain from that, they're the victims in this story as far as I can tell.
> this means that they are lying in these statements.
Cyberwar with China is the current equivalent of the old Cold War scuffles with the USSR in third countries. How many lies were told back then, to cover up the worst mishaps? And then they were "uncovered" when it was safer or more convenient to do so.
I can totally believe that there are tons of actual conspiracies regarding the tumultuous relationships between China and the USA but in what configuration does this particular one make sense? As far as I can tell the only people who come out badly out of this are the Chinese intelligence agencies and the complicit Chinese manufacturers. Why would the Apple and Amazon go out of their way to protect them? I can imagine that the US government could coerce them into doing that but to what end?
If these companies are caught red handed lying on behalf of the Chinese or US governments it would set an absolutely terrible precedent, I don't see why they would risk it when they have so little to gain.
> As far as I can tell the only people who come out badly out of this are the Chinese intelligence agencies
The richest and most powerful American companies, some of whom are critical to national security and/or make their operational security as a critical selling point (the CIA among others), would be found not to be in fundamental control of the essential infrastructure powering their core business. It would trigger expensive large-scale review of every server in the world, and investors would run for the hills. This would be massive, and have heavy repercussions in the markets. FAANG are not protecting the Chinese, they are protecting their own finances - and the authorities will let them get away with it because the alternative is unpalatable.
If you keep it as a denied rumour, there is an official excuse for people to just get on with business as usual - nobody wants to deal with a market crash, not even most traders, and after all newspapers say many things, not all of them true. Maybe Supermicro is compromised, and maybe if you really care about hardware security you should buy elsewhere <wink-wink>; but it's not official, so most people can just pretend nothing is happening and go about their day, until a solution can be found.
> I don't see why they would risk it when they have so little to gain.
Just wanted to thank you for posting this, as I [self-censor].
I'd like try to boil things down a little further for people:
1) Most things in the modern world require high degrees of trust. Once a significant portion of people begin to question the system, it fails.
2) The main goal of most organizations/governments, in general sense, who find themselves in a favorable position is to keep the game going.
3) Though it's impossible to control for all variables, 'we' believe we can manage most common ones. The uncommon (foreseen and unforeseen) often arise in times of crisis (panic behavior) and often are unwieldy.
So... the veracity of the story is generally less important than is managing reactions to it.
4) IANAL, but I worked w/them for many years and have crafted many statements. Communication is an art form open to interpretation.
5) I've long suspected that such "tampering" was standard practice, for any global power. Why? See point 2.
I encourage anyone skeptical about any portion of the story, to don his/her 'megalomania'-cap for a bit. Then everything, at least conceptually, should make sense.
Those employees wouldn't be authorized to talk about the information to anyone not cleared. Which includes the PR department or anyone tasked with crafting a statement.
That's not how TS/SCI works. You've been watching too many movies. Source: was TS/SCI cleared for 15 years. Information being classified just means you can't divulge the information, not that you can't divulge the existence of the info.
That's a bit misleading. There are code word programs and classified investigations where just revealing the existence of the info would be a serious violation.
In this case, if a cleared employee is asked: is this information true, is there such an investigation? Then simply by saying they can't comment on the question, they reveal the info to be true.
I think most people with high-level clearances would play it safe in such situations and just deny any knowledge of the situation.
I can't comment is exactly what they can and do say very frequently. Apple could have not commented on any investigation or not responded to a request for comment. This would have been completely normal. "That's not something I can talk about" is a very common phrase.
Absolutely correct, but extraordinary claims require proof. I'm certainly willing to be proven wrong, and of course I could be, but there is no indication that this is anything other than Bloomberg being misled at this point. Location of clearly exploited hardware, acknowledgement by anyone involved, or analysis of traffic from one of these boards would all corroborate the story.
>>just means you can't divulge the information, not that you can't divulge the existence of the info.
The existence of it is important. I can't tell the content of x letter sent from China's Amb to the mother ship but I can tell you that we intercepted the letter. How was it intercepted? That in itself means a lot.
Where did you get that from? This is well known to not be true in the US, the government can't even stop you from publishing nuclear secrets or the Pentagon Papers. See New York Times v. United States (1971) and United States of America v. Progressive, Inc.
The language is not very explicit. They allow the reader to interpret thier statement through an implied explicit denial. Great liars if not just plainly unclear.
This whole thing wreaks. Is it fake news? Are the statements from Amazon and Apple expertly crafted to fool people who aren't very good at reading comprehension (most people)?
There is a lot of information in the Bloomberg story, which I mean in the information-theoretic sense, and not in the sense that it must be "true" information for any definition of "true". A lot of very specific claims. Those claims did not come from the reporter writing the story, which is about 99.9999% likely to be incapable of making up such a plausible story with such details.
So one must think about "Where did this information come from?"
It is at least a plausible theory that the story is largely true (though as I posted in another thread, I'd bet money it's not all entirely true), and that the denials are either made by people who are unaware of the truth, or are being made deliberately. (Actually, the people who literally prepared those statements almost certainly believe the truth of the statements. One of the most plausible ways for a group of people to lie is for all communication to come from individuals who genuinely believe the lies; no body language tells or any other such leakage about it not being true.)
It is also plausible that the denials are in fact true, in which case one is left with the very interesting question of "Where did all that information come from and why is it wrong?"
As a couple of people have also said, there's also the option that the story is largely true, and the denials are true if you parse and read them like a lawyer, but meant to mislead anybody who doesn't. I can't say I've examined them for that, but it's definitely a possibility to consider.
I'd actually suggest "propaganda" isn't a great explanation; propaganda does not generally depend on making lots of specific, refutable claims, and certainly not followed up by refutations immediately. It is usually designed to speak directly to people's emotions and fan pre-existing flames in ways specifically designed to not be refutable. If this really was government propaganda (note how that is more specific than my previous unqualified "propaganda", because anybody can propagandize, not just governments), I would expect the American companies would be strong-armed into agreeing with the story for the propaganda's purposes, or that the story would never run at all if they couldn't be sure the companies weren't going to back them.
An alternate theory that might fit all the facts is industrial espionage. Let's say the story is completely untrue, the denials completely true. What is the result of this story? Supermicro in particular stands to lose some business. Perhaps someone who benefits from that planted this story.
Another alternative is stock market shenanigans. As I write this, the delayed feeds aren't showing it yet, but Supermicro stock (SMCI) just took a 31% bath overnight. Who benefits? Short sellers, put option traders.
I have no idea myself. "Unknown unknown" is still a pretty large chunk of my personal probability estimates.
Supermicro was already delisted prior to this article and had about $370k of US dollar trading volume yesterday in the OTC markets. Don't think people will make much money shorting it as this point, since it is already so illiquid. Highly doubt that is the explanation for the article.
EDIT: I read the volume amount incorrectly - it's more like 6m of volume which is not too bad. Someone could probably make money shorting it but again since its delisted already it would be difficult/risky to do so.
I wonder if their delisting is related to this hack, or if it's a sign that they have poor internal controls and the company is just poorly controlled every where from accounting to engineering.
> We did not uncover any unusual vulnerabilities in the servers we purchased from Super Micro when we updated the firmware and software according to our standard procedures.
Does that mean they did uncover some "usual" vulnerabilities?
Typically yes. This is base hardening. The usual vulnerabilities can be just default configuration options or unwanted firmware/software features.
Similarly no security team is going to say "There are no vulnerabilities in the servers we purchased." It's just not true, they're always there and expected.
There's enough "usual" vulnerabilities that the practice of fixing them has a name: hardening. For supermicro motherboards specifically, "open services with really shitty default credentials" is apparently one of them — definitely one that is common in a lot of other contexts.
In this context, "unusual" vulnerabilities would be evidence of a deliberate attack, rather than just common security mistakes.
> Or at least have testimonies by employees in these company
The original article directly addressed this: "The companies’ denials are countered by six current and former senior national security officials, who—in conversations that began during the Obama administration and continued under the Trump administration—detailed the discovery of the chips and the government’s investigation. One of those officials and two people inside AWS provided extensive information on how the attack played out at Elemental and Amazon; the official and one of the insiders also described Amazon’s cooperation with the government investigation. In addition to the three Apple insiders, four of the six U.S. officials confirmed that Apple was a victim. In all, 17 people confirmed the manipulation of Supermicro’s hardware and other elements of the attacks. The sources were granted anonymity because of the sensitive, and in some cases classified, nature of the information."
It is entirely likely that the companies affected were directed by the IC agencies working on this not to discuss or reveal their knowledge of the hack. Often in intelligence operations it is important and useful to not alert your adversary that you are aware of their intrusions until you are fully ready to take action against them, or have fully removed the danger.
I don't see any reason to take the companies' categorical denials as evidence that this did not happen or that they were not targeted. Those statements are what one would expect in a national security incident and investigation of this magnitude, with such serious implications.
Provides only minimal substance here, but I'll mention I have worked in a company where there were times I knew information I wasn't even allowed to share with the CEO of the company. He knew who our customer was, but he was not allowed to know the substance of the work we were doing due to the non-disclosure agreements.
What I'm trying to get at is, it is possible for an agreement to be created for someone internally to know some "important" information in a company, and for it to not be known by others, even those higher than the employee.
100%. Work at a company that provides equipment to the government. There where people that have a clearance and know why and how thing where being used, but the exec staff up to the CEO just knew who the customer was and no details beyond. It’s prtty normal.
> Either Apple and Amazon are lying or Bloomberg is wrong
Apple and Amazon are compelled to lie. It's a classified investigation and likely only cleared employees are aware of it, and they have to deny its existence or stand to lose their clearance (worth $$$).
How can companies be forced to issue such press statements? They may be forced to give incomplete answers when asked, but these statements go far beyond answering questions.
The people issuing press statements are almost certainly out of the loop. And of course, no one with knowledge of the situation is going to step forward, given they have a strong incentive to stay silent.
I mean, think of your own company's PR team. If you've worked for a large company, you've likely had interaction with a PR team that was pretty ignorant about the inner workings of the company. Now, add to that the fact that this is a classified investigation being run by FBI counterintelligence, and interfacing only with cleared employees (none of whom work for the PR team).
You are a PR guy. A respected and big news agency puts out an article saying your company is involved in some secret stuff - stuff that you wouldn't know about.
Do you release a press release denying everything without asking the guys who are supposed to know about it? Would those guys say to you it's all fabricated and urge you to deny it or will they evade answering it (or even more likely they'll go up the chain)?
Right, PR folks may not understand how all the stuff works in their company, but they sure as shit know which topics require senior approvals. That is basic self-preservation for them.
In corporations you don't address the federal government without your legal team in the conversation, and your legal team would also be in the approval path for any public statement about such an allegation. It's extremely unlikely that a public company would comment on something like this without connecting the dots internally first.
Yesterday in America, an SMS message went out nationwide from FEMA. It was labeled a 'presidential alert' so many people assumed it was Trump that sent it. This went viral on social media and people began photoshooting funny messages that Trump or other theoretical leaders might send.
Normally I'd assume this sort of leak would only be the result of orders from above. But there was recently the credible allegation that Trump backed off the ZTE sanctions after China threw a few million dollars at the Trump Organization in a real estate deal[1]. Whether that was a successful case of bribery or not I can see a highly placed employee looking at that and deciding they can't trust their superiors to do the right thing in this matter.
"Apple has never found malicious chips, “hardware manipulations” or vulnerabilities purposely planted in any server. Apple never had any contact with the FBI or any other agency about such an incident. We are not aware of any investigation by the FBI, nor are our contacts in law enforcement."
Bloomberg's article and Apple's statement can't both be right.
>"Three senior insiders at Apple say that in the summer of 2015, it, too, found malicious chips on Supermicro motherboards. Apple severed ties with Supermicro the following year, for what it described as unrelated reasons."
> Bloomberg's article and Apple's statement can't both be right.
Possible that Apple employees with security clearance are the only ones with this knowledge i.e. it's fully possible that even Tim Cook doesn't know about this
Even if they were bound by a non-disclosure order, this response goes beyond what's necessary to refute the story and conveys material information which would be used against Apple by shareholders if it is later found to be factually inaccurate.
A lot of people are unaware of how anonymous sources in a serious news organization work. Here, it means that the multiple high-level intelligence officials described in the article are known to and vetted by Bloomberg. They've looked at their resumes and bona fides, and confirmed their backgrounds. They're just not revealing their names to us.
So which is more likely: that multiple intelligence officials are making this up, or that Apple/Amazon/Supermicro feel obligated to lie because this is an ongoing classified counterintel investigation?
The language of the Apple refute is so strong, to the point of directly attacking Bloomberg and calling them irresponsible. So yes, in this case there are at least a few lies being peddled by the Bloomberg intelligence contacts
Best line from an otherwise serious and and important piece of reporting:
"Two of Elemental’s biggest early clients were the Mormon church, which used the technology to beam sermons to congregations around the world, and the adult film industry, which did not."
Conspiracy theorists aside, the main new thing that came out of the Snowden revelations was that Google’s physical security for data-center-to-data-center traffic was compromised by the NSA, which Google never denied, and responded by hardening server to server traffic.
It almost makes you wonder if there is a process for ensuring companies comply with secret investigations and are forced to act publicly and privately as if they have never happened.
Regardless about how you feel about the hack, outsourcing the vast majority of our technology to another country just doesn't seem like the smartest idea. Why would we put our most trusted technology into someone else's hands—just because it's going to save a few bucks? Wouldn't it be worth it to just do these things ourselves?
You should talk about "our technology" only if you are a big stock owner or, maybe, a naive employee. They are not called "global corporations" because of their patriotism.
It's not called "global capital" because it cares where to reproduce.
I'm sure that companies will start to re-shore production on to friendlier countries or at least require sub-contracting restrictions. From the article it sounds like a sub-sub-sub-contractor (SuperMicro->Main Chinese Contractor->Compromised Chinese Contractor) was the weak link.
This is interesting in how vehemently all the companies are denying everything. I am pretty clueless about how the feds work so I'll ask: is it possible they would be violating secrecy laws or leaking classified info if they acknowledge this really happened? Could they already be under NDAs or whatever the equivalent is in the national security world?
Or is it simply a matter of their shareholders having lofty expectations about tapping the biggest market in the world (China) and saying anything that angers China is the worst thing you could possibly do from a PR perspective?
I don't think there needs to be a conspiracy for vehement denials to make sense. Security is a hugely important reputational good for both Apple & AWS.
That people who made promises not to divulge information continue doing so on a regular basis is the disturbing part for me. If you make an agreement then stick with it. If you no longer can then quit.
The AVGO/QCOM LBO didn't happen ostensibly because of the role 5G plays in national security, not trust issues with the design. However, I'll grant that there could be other covert reasons (nationalism, these very trust issues, etc).
I think the article is implying that the implanted chips might not have been detected due to their low profile design. Given that the bad manufacturers were subcontractors it's likely only a fraction of all the boards manufactured were compromised. It's either that or someone at Supermicro was in on it.
The article alleges (b), and give indications of why it wouldn't necessarily be picked up:
> In one case, the malicious chips were thin enough that they’d been embedded between the layers of fiberglass onto which the other components were attached, according to one person who saw pictures of the chips. That generation of chips was smaller than a sharpened pencil tip, the person says. (Amazon denies that AWS knew of servers found in China containing malicious chips.)
That is possible but unlikely. Firstly you'd have to do this at the PCB fab phase as this can't be retrospectively applied, so you'd have to have infiltrated two times as many companies and add complex manufacturing exceptions to their workflow. Secondly you'd have to get it through AXI process which is used on these larger boards (Automated X-ray Inspection) which is used by the larger companies for manufacturing AND design validation. So you'd have to infiltrate another company.
Also stuff like this tends to show up on boundary scans. It's not that easy to cock around with signal integrity on these sorts of boards and get away with it.
I find the whole thing infeasible from a cost and logic perspective. The SMbus firmware and Aspeed sub-vendor are so much easier to hit and don't leave any corpses around to find after the fact.
I've always wondered if that passive device in the great crest at the US embassy in Moscow had equivalents which got hooked up to consumer devices with high voltage parts (to make people reluctant to play inside)
Remember the furore when Zenith was the last domestic manufacturer of TVs in the USA? We've come a long way since then..
It's not that dangerous to open electronics ever since they stopped using electron guns. But, consumer devices don't need to secretly spy on you when consumers buy them literally for the purpose of being spied on.
I have not seen anything that indicates how installing this chip would do anything at all without also modifying the trace design and fabrication of the PCB itself.
Also does anyone have information about the "baseboard management controller" mentioned? I would like to understand the complexity required to MiTM a ROM or FLASH memory read by such a controller before concluding the feasibility and number of players in manufacturing chain required for it to work.
The BMC is an ARM microcontroller that has complete access to everything, exactly like the Intel Management Engine, but the server vendors prefer a separate chip for the same job. What is described in the article is very easy to do by inserting a microcontroller on the SPI link that connects the BMC with the flash memory containing the BMC programs, which are copied from there to a RAM at boot.
However, such a microcontroller would need 8 pins for at least 7 signals: ground, power, SPI clock and 4 SPI data, to and from BMC and flash memory.
Nonetheless, 8-pin packages can be very small, e.g. 0.8 mm by 1.35 mm, i.e. only slightly larger than 1 square millimeter.
So from a technical point of view, all is easily feasible.
The problem is that it would require compromised people in several places at the subcontractors, because the design files for the PCB must be replaced wherever the PCB is made and in another place, at the PCB assembly, the pick & place document must be replaced and an extra reel with the backdoor component must be mounted on the equipment and that reel must come from somewhere else than from the normal suppliers of the assembly line without raising suspicions.
It can be done, but many accomplices are required. Because most of the time the backdoor component will pass the SPI data signals transparently, it will not be detected at any electrical testing and the usual optical inspections are unlikely to detect such a small change.
I am using many Supermicro motherboards, so I am wondering if this story is true. If it were true, it would not be much of a surprise, because they did not do something really novel but they just matched what USA also did, e.g. in the Cisco case.
But also if a modification of the SPI bus can result in exploitation then the manufacturer will just patch the memory in place. Cheaper, easier, less detectable. So... the manufacturer is not the source of attack.
One thing I don't see much about in the article is the supposed chip which allegedly compromised Super Micro servers. The cover image of the piece shows a surface mount component with three solder pads balanced on a finger tip. Looks like a very simple SMD part to me.
On top of that Apple, Amazon and Super Micro are flat out denying this - I suspect Bloomberg messed up here.
Can someone point me to the exact laws or cases with precedent that would make Apple and Amazon's statements illegal if the Bloomberg article were true? I'm not doubting it, but I can't find much about this online. Pump and dump schemes, ponzi schemes, and insider trading are illegal, and those aspects of securities fraud are well documented online, but this more general "lying to the public" that these companies may or may not be doing has proved trickier to find precedence for. I am not a lawyer, and my only resource here is google, but I think there is an assumption at play that this form of lying is illegal, and I'm honestly not sure that it is.
I could be wayy of base here, but if its not illegal, than it would be pretty obvious what is going on.
The cost of manufacture is a very small part of the cost of assembling computers. For the iPhone, for instance, it represents less than $10 out of the $240 or so total cost. Thus shifting production to other another locale with higher costs would not significantly increase the price of the product, and in any case labor costs are going up in China as well.
Thus you wonder why more production isn't being shifted from China to, say, Thailand, Indonesia or India. Steve Jobs once said the industrial capacity to do the work simply isn't available outside of China, in terms of skilled people and supply chains, and that may be a big reason why.
I’m not sure if you’ve noticed, but IT isn’t really taken very serious outside of IT. I mean, they want money from it, but you don’t get money by telling the world that your primary tech companies are compromised.
Though judging by the replies, I think Bloomberg needs some really solid sources, so there is probably nothing to it.
The US government argued for years that chips needed to be made inside the US - which they were for certain military applications. Then, President Clinton allowed huge technology transfers from the US to China, and the US was no longer making its own chips for military applications.
Because at least half of the people wouldn't believe them and consider this another conspiracy theory. Even when Bloomberg pushed this there are still so many deniers in this thread.
Hell, how many conspiracy theory's have come out as true in since 2001.
Remember the "The telephone companies are routing all our calls and internet data to the NSA" conspiracy.
A bunch of people said that was fake, and there was no way this could happen.
Then more evidence came out, and the same people said there was no way that could happen, its too big of conspiracy and it would have leaked way before then.
Then the government gave the telco's retroactive immunity for spying on the public.
"In the three years since the briefing in McLean, no commercially viable way to detect attacks like the one on Supermicro’s motherboards has emerged—or has looked likely to emerge."
I know a commercially viable way to detect hardware attacks.
Standardized hardware designs, such as the "x86 standard," ARM IP licenses, and more recently, RISC-V, decentralizes manufacturing and drives the cost to commodity levels. I'm specifically proposing that the U.S. Government appropriate the patents on whichever hardware design and declare them a National Security asset, and then guarantee royalty-free licenses to any company that wants to use them.
When it's no longer a big profit center, China is no longer as interested in owning a monopoly on it.
And presto: there's no longer a monopoly on the hardware. Thus it's no longer a guarantee that your hardware is being bent to the will of a single nation-state. Hardware attacks can be detected as variations between the hardware made by one nation vs. the hardware made by another nation.
The downside is that Apple can't have the same profit margins that come from closed, proprietary hardware.
The upside is that manufacturing and process innovation (such as Intel used to do) becomes extremely desirable. It becomes so valuable that we saw Intel reluctant to offshore their best processes.
There: economic solution and political points, to boot!
Apparently one of the big factors in Supermicro success is that it has over 900 different motherboard designs, and hundreds of hardware specialists which can customize them further to client wishes.
Apparently one of the big factors in Supermicro success is that it has over 900 different motherboard designs, and hundreds of hardware specialists which can customize them further to client wishes.
How is that a sequitur arguing about "a few open-hardware motherboard designs" and the projections of them being "commercially succesful"?
This response piece was published one hour after the original report. Bloomberg obviously had both ready. It would have responded already if intend to.
For what it's worth, this exact article is linked directly from within the original article, and is addressed.
> Read: Statements from Amazon, Apple, Supermicro and Beijing
> The companies’ denials are countered by six current and former senior national security officials, who—in conversations that began during the Obama administration and continued under the Trump administration—detailed the discovery of the chips and the government’s investigation.
>"Somewhere in the Linux operating system, which runs in many servers, is code that authorizes a user by verifying a typed password against a stored encrypted one. An implanted chip can alter part of that code so the server won’t check for a password—and presto! A secure machine is open to any and all users."
I realize the intended audience for this article is not a technical crowd but can someone walk me through in practical terms how such a chip might subvert the /bin/login binary?
> Another common feature is virtual USB disk media, which can be used to infiltrate or exfiltrate files or to provide new boot media. The combination of these capabilities and remote power cycling would allow an attacker to seize control of most common server configurations. For instance, they could restart the system and boot from a virtual live CD, then directly copy or modify data on the host’s storage devices
If the IPMI has write access to disc and/or main memory, you can do it more directly - drop a new /bin/login on the disc, or patch it in memory (similar to the LoJax attack: https://news.ycombinator.com/item?id=18090651)
It's interesting that the numerous sources all decided to confine in Bloomberg. How did they all know to go to the same paper? And if some of them went to other papers then we should hear about it tomorrow, I guess.
> Two of Elemental’s biggest early clients were the Mormon church, which used the technology to beam sermons to congregations around the world, and the adult film industry, which did not.
It's either a giant conspiracy by the FBI and multiple mega-corporations to blatantly lie, on public record, about a matter that if happened would most likely come up in the future again. Furthermore if apple and amazon were both notified for comments, there is good reason to suspect that the FBI would hear of the article and try to censor such an article for national security reasons, especially so if they made apple and amazon lie about it.
Or ... Bloomberg didn't do their due diligence and were too eager to be duped by agents who wanted to push an agenda to move manufacturing away from china or something similar.