On the note of Jupyter notebooks and version control - there was a talk at this year's Pycon Ireland about using a built in cleaner for notebooks when committing the JSON (discard the cell results), and then dropping the whole lot into a CI system utilising remote execution (and Bazel or similar) to run and cache the outputs. Was a talk from CodeThink. No video up yet though. Scenario was reproducible notebooks for processing data from a system under test.
> On the note of Jupyter notebooks and version control - there was a talk at this year's Pycon Ireland about using a built in cleaner for notebooks when committing the JSON (discard the cell results)
Yup, I use a long "jq" command [0] as a Git clean filter for my Jupyter notebooks, and it works really well. I use a similar program [1] for Mathematica notebooks, and it also works really well.
scoot, you may find the current mini-series by the podcast Unexplainable to be interesting. It's on sound, and one episode is about tinnitus and research into it.
"FTP is dead" - shared web hosting would like a word. Quite a few web hosts still talk about using FTP to upload websites to the hosting server. Yes, these days you can upload SSH keys and possibly use SFTP, but the docs still talk about tools like FileZilla and basic FTP.
They mention that the "FTP" service includes SFTP, which is file transfer over SSH (not actually related to classic FTP), which is perfectly secure and supported by most FTP clients like Filezilla.
The premium "SSH connection" you mentioned seems to refer to shell access via SSH, which is a separate thing.
They also support FTP without the SSH transport, and it's not FTPS either. Various IP cameras still support FTP as a way to write files out periodically; I use this to provide a "stream" from a camera (8 seconds per frame because reasons) to the world. Actual streaming via RTSP is also available, but I could never get a stable stream to a video host (like YT or Twitch) from the camera (partially because of a poor quality network connection that can't be upgraded easily). So, FTP + credentials -> walled off directory that's not under the web root -> PHP script in web root -> web browser.
Transport encryption should be a huge priority for everyone. It's completely unacceptable to continue using unencrypted protocols over the public internet.
Especially for the use case of transferring files to and from the backend of a web host. Not using it in that scenario is freely handing over control over your backend to everything in between you and the host, putting everyone at risk in the process.
How would you know if the transfers were interfered with? Do you take checksums of the files you upload and then check that the files apparently uploaded are the same?
Also, how do you know that there isn't someone performing a MITM (man in the middle) attack? FTP has no mechanism that I know of to verify that you're connecting to the server that you think you are.
It may well be that you're not a sizeable target and that no-one is interested in hacking your site, but that's just luck and not an endorsement of unencrypted FTP.
How would you know that your neighbours aren't secretly spying together on you and interfering with your life in ways you don't notice?
We have to put a limit to paranoia. If things work correctly for decades and there are no signs of foul play after endless real world usage, it's safe to say nobody is hacking our FTP.
It's different if you're a bank or the KGB or the CIA.
> It may well be that you're not a sizeable target and that no-one is interested in hacking your site, but that's just luck and not an endorsement of unencrypted FTP.
Needing an armored car or protection from neighbours is specifically to guard against proximity based exploits and those are very unlikely threats to most people. FTP interception can be easily performed from anywhere in the world with a little bit of DNS poisoning and then perform a MITM attack (or even just alter the data in transit from a malicious wifi hotspot).
It costs approximately zero to use encryption and protect against the FTP exploits, so why continue to use FTP? There's literally no advantage and several possible disadvantages. Just relying on not being hacked before seems a foolish stance to me.
If it's so easily done, then most FTP websites would be hacked every week. But hundreds of millions of people have FTP websites and never get hacked in decades.
I challenge you to select any FTP website of your choosing and make a tiny change to prove that you've hacked it and let me know here.
There's little reason to expect to be run over when you're on a bike, jut like there's little reason to expect your website to be hacked because you use FTP. If you're a normal person.
We have to be proportional when we do risk assessment. Just because it's part of modern programmer faith to be against FTP, doesn't mean it's sensible. Most hackers are just repeating what others have told them, and a lie becomes common sense.
If FTP is considered unsafe, then riding any non-armored vehicle should also be unacceptable.
Not true. Your hosting provider already has physical access to the computer you're connecting to.
Whether or not the connection you're using is encrypted doesn't really matter because the ISP and hosting provider are legally obligated to prevent unauthorized access.
(It's different if you're the NSA or some other state-level actor, but you're not.)
ISPs very frequently do not give a shit about the law. There are so many instances of major ISPs intercepting and modifying traffic, injecting ads, redirecting people to gambling websites, etc. It's not some freak incident involving the NSA targeting you, it happens all the time. All it takes is one bribe.
And what happens if your ISP is compromised without their knowledge? What happens when it's a consumer device such as a router? Don't forget that nearly every TP-Link router has an active malware infection.
It's not just one ISP that you have to trust, it's every single intermediate piece of equipment.
Intercepting traffic is a trivial & common form of compromise, and the problem multiplies by how many different parties you are handing your data to. It is wildly irresponsible to not attempt to protect against this.
"You <-> ISP <-> Bank webpage" is an entirely different security threat model than "You <-> Server you rent from an ISP".
Also, unsanctioned wiretapping is an entirely different criminal offense than stealing leaked credentials.
You can't make blanket statements like that without understanding ISP peering agreements and how data is stored and where.
Let's not pretend like slapping cryptography over L3 is the entirety of being secure. Often (most of the time?) cryptography doesn't even matter much for security.
P.S. Security (prevent stealing sensitive data) and verification (making sure nothing extra is added during transfer) are different problems.
> "You <-> ISP <-> Bank webpage" is an entirely different security threat model than "You <-> Server you rent from an ISP".
...In what world do people rent servers from consumer ISPs? This used to exist in the 1990s, but is nonexistent now.
If this still exists, it's email-only and has already been outsourced elsewhere. No consumer ISP currently in existence is running these sorts of services on their own hardware.
> Also, unsanctioned wiretapping is an entirely different criminal offense than stealing leaked credentials.
I want to be very clear: There are countries that effectively do not have laws that would ever be adequately enforced on ISPs, either because of corruption, a lack of resources in the courts systems, or both. The use of bribery to compel ISPs into intercepting and recording internet traffic is already rampant at scale. You can't rely on the law to protect you when the internet goes across borders.
> Let's not pretend like slapping cryptography over L3 is the entirety of being secure. Often (most of the time?) cryptography doesn't even matter much for security.
Not sure what your point is. Yes, transport security is not the solution to every problem. But it is by far the lowest hanging fruit, the threat modelling is incredibly clear and obvious. There is a reason transport encryption has become universal across every use case imaginable - it's the literal first step to not getting completely pwned before you've even done anything.
> P.S. Security (prevent stealing sensitive data) and verification (making sure nothing extra is added during transfer) are different problems.
And? On the transport level, they have the same solution: TLS. Confidentiality and integrity work hand-in-hand. It's very rare you will need one without the other.
Unencrypted FTP does not give you either of these, and in fact by being limited to password authentication, it helps turn every passive data collection attack into a persistent remote control attack.
> It's completely unacceptable to continue using unencrypted protocols over the public internet.
That is nonsense. The reality is that most data simply is not sensitive, and there is no valid reason to encrypt it. I wouldn't use insecure FTP because credentials, but there's no good reason to encrypt your blog or something.
You're missing the opposite issue - people might not care about your data, but you might well care if their data (e.g. porn sites) was uploaded to your blog.
It's not so much about the data, but protecting your credentials for the server.
I'd argue that most people like knowing that what they receive is what the original server sent(and vice versa) but maybe you enjoy ads enough to prefer having your ISP put more of it on the websites you use?
Jokes aside https is as much about privacy as is is about reducing the chance you receive data that has been tampered. You shouldn't only not use FTP because credentials but also because embedded malware you didn't put there yourself.
This is the usual horseshit people say about this topic when they don't understand it. It's not just about encryption, but authentication (tamper-resistance). Your blog might not contain sensitive information, but if the entire website is intercepted and becomes malware, you're in trouble.
The bad news with FTP in particular is that only one request has to be intercepted and recorded to have persistent compromise, because the credentials are just a username and password transmitted in clear.
Shared hosting is dying, but not yet dead; FTP is dying with it - it's really the last big use case for FTP now that software distribution and academia have moved away from FTP. As shared hosting continues to decline in popularity, FTP is going along with it.
Shared hosting is in decline in much the same way as it was in 2015. Aka everyone involved is still making money hand over fist despite continued reports of its death right around the corner.
The number of shared hosting providers has drastically declined since the 2000s. I would posit that things like squarespace/hosted wordpress took the lion share, with the advent of $5-10 VPS filling the remaining niches.
The remaining hosting companies certainly still make a lot of money, a shared hosting business is basically on autopilot once set up (I used to own one, hence why I still track the market) and they can be overcommitted like crazy.
> The number of shared hosting providers has drastically declined since the 2000s
Yeah, there’s definitely been some wild consolidation. I’ve actually been involved in quite a few acquisitions myself over the last decade in one form or another.
> (I used to own one, hence why I still track the market)
I’m still in the industry, though in a very different segment now. I do still keep a small handful of legacy customers, folks I’ve known for years, on shared setups, but it’s more of a “you scratch my back, I’ll scratch yours” kind of thing now. It’s not really a profit play, more a mix of nostalgia and habit.
Probably worth noting also that declining number of providers does not equal a declining number of customers. I know every company I engaged with ~15-years ago has been acquired at least once.
No, not at all the case. There has been continued consolidation of the shared hosting space, plus consumer interest in "a website" has declined sharply now that small businesses just feel that they need an instagram to get started. Combine that with site builders eating at shared hosting's market share, and it's not looking good for the future of the "old school" shared hosting industry that you are thinking of.
Seems short sighted, a lot of older people and privacy conscious people of all ages avoid social media. But I guess if they are sustaining a business on only Instagram, good for them.
> There has been continued consolidation of the shared hosting space
That’s been happening, at least from my own memory, since at least the mid-2000s.
> plus consumer interest in "a website" has declined sharply now that small businesses just feel that they need an instagram to get started.
Ah yes, the 2020s version of “just start a Facebook page.” The more things change, the more they stay the same I suppose.
> Combine that with site builders eating at shared hosting's market share
I remember hearing that for the first time in I wanna say...2006? It sure did cause a panic for at least a little while.
> and it's not looking good for the future of the "old school" shared hosting industry that you are thinking of.
Yes, I've heard this one more times than I can count too.
The funny thing is, I’ve been hearing this same “shared hosting is dying” narrative for nearly two decades now. Yet, in that time, I’ve seen multiple companies launch, thrive, and sell for multi-million dollar exits.
But sure, this time it’s definitely the death knell. Meanwhile, I assure you, the bigger players in the space are still making money hand over fist.
I was in the space from the late 90's, acquired ~30 brands and was the largest private consolidator of shared hosting, and sold to a Fortune 500 in 2015. Sounds like you had a similar experience as mine. There's no way you can deny that the glory days of shared hosting are over - while there is still a little money to be made by setting up a VPS with cPanel, and money to be made if you are WebPros or Newfold, the market is contracting and has been for years due to the factors I listed. The Cheval list used to be the hottest marketplace on the planet and now is just a shell of it's former self, unfortunately.
I think everyone is underestimating how much B2B file exchange happens over SFTP/FTPS. I'm in healthcare and my system moves thousands of files up and down from over 100 unique hosts daily.
Also worth noting that FTPS (FTP over TLS) exists and obviates the fuss around SSH TOFU and key management etc. Especially given we're in the era of free certificates via Let's Encrypt, this is a great option.
The main downside is people will sometimes assume you mean SFTP (not having heard of FTPS or realising they are different), and then get upset when it doesn't work as they expect. However good tooling will support both e.g. Filezilla.
Anecdotally, this rings true for me. Father walked out on mother essentially, when I was in my early 40s. Rocked my world thoroughly, hadn't seen it coming. Several years of therapy to come to terms with it.
Also anecdotally, I experienced the opposite. After three decades of constant fighting, the divorce was almost cathartic. Their marriage was more traumatic for me than their divorce.
Interestingly, their relationship became much better after and they are ok as friends now.
Research suggests there is a big difference between how children experience divorce between high-functioning and low-functioning families - children from high-functioning families often experience parental divorce as traumatic, children from low-functioning families often experience it as a relief from ongoing traumatisation
Even within the same family, both divorce and dysfunction can be experienced very differently due to differences in each child’s individual psychology and also family dynamics (sometimes one child is made to bear the brunt of the dysfunction much more than the others-the “black sheep” versus the “golden child”)
I can second this - my parents were good friends after their divorce, as all of the issues of contention were related to sharing a house and relationship.
I would assume ones's parents divorce would be much easier to come to terms with for someone in their 40s compared to 20s, teens or kids (that doesn't mean it wouldn't suck).
my parents split and got divorced when i was in my 20s, and both of them had remarried within about 5 years. it didnt really bother me very much at all, i was already living very far away from home, and its good to see theyre both happier than they were before the divorce. the biggest change to me is i have to visit two houses when i go home for a holiday instead of just 1 house
Nothing against Fedora and the rpm-based platforms but I prefer the debian-derived distros. My preference is due to Debian feeling like a community project rather than being driven by corporate interests. Ubuntu was doing for a while but that started changing a few years ago.
> Heck, I use Fedora Server as my homelab OS to run Incus. Works For Me.
In your case I guess it makes sense since you have to run Fedora at work, but I was under the impression that the support for Incus (i.e. official packaging etc) was better on Debian.
It might be better on Debian, but it's absolutely fine on Fedora too - in that I've hit zero snags. Maybe there are problems somewhere, but I've not encountered them yet (with a mix of all three supported ways to run things).
Sunsets with no land in sight. Sunsets framing the land. Sunrises with no land in sight. Sunrises framing the land. Thunder and lightning rolling up one side of an island in the distance, putting on a show.
Dark skies and the stars.
The peace and tranquility of quiet places with just nature and you. Until the sod over the hill turns on their generator.
Fresh fish.
The feel of sea spray, wind in the ears, the connection to your boat, knowing if things are right just by the feel and sound.
And that's just from doing non-ocean-crossing sailing.
Not the person you're asking, but I sail a 7 ton, 10 metre yacht. I can change course by 90 degrees in under 10 seconds if I need to (probably far less, depends how hard of a crash tack I do).
people do need to sleep if they're soloing offshore, so the warning would need to be at least a minute to be able to wake up, then there's putting on the safety equipment to go out on deck etc. So make that multiple minutes of warning for it to be useful..
Im not aware (which is not to say it doesn't happen) of that many incidents with sailing vessels and whales. However, you've prompted me to look, and it's more than I thought.
reply