I wonder why no one ever talks about architectures in the middle between those two - modular monoliths.
The point in time where you're splitting your codebase up in modules (or maybe are a proponent of hexagonal architecture and have designed it that way from the beginning), leading to being able to put functionality behind feature flags. That way, you can still run it either as a single instance monolith, or a set of horizontally scaled instances with a few particular feature flags enabled (e.g. multiple web API instances) and maybe some others as vertically scaled monoliths (e.g. scheduled report instance).
In my eyes, the good part is that you can work with one codebase and do refactoring easily across all of it, have better scalability than just a monolith without all of the ops complexity from the outset, while also not having to worry as much about shared code, or perhaps approach the issue gently, by being able to extract code packages at first.
The only serious negatives is that this approach is still more limited than microservices, for example, compilation times in static languages would suffer and depending on how big your project is, there will just be a bit of overhead everywhere, and not every framework supports that approach easily.
Because there is no newly invented architecture called "modular monolith" - monolith was always supposed to be MODULAR from the start.
Micro services were not an answer to monolith being bad. Something somewhere went really wrong with people's understanding and there is bunch of totally wrong ideas.
That is also maybe because a lot of people did not knew they were supposed to make modules in their code and loads of monoliths ended up being spaghetti code just like now lots of micro service architectures end up with everything depending on everything.
The interesting thing about microservices is not that it lets you split up your code on module boundaries. Obviously you can (and should!) do that inside any codebase.
The thing about microservices is that it breaks up your data and deployment on module boundaries.
Monoliths are monoliths not because they lack separation of concerns in code (something which lacks that is not a ‘monolith’, it is what’s called a ‘big ball of mud’)
Monoliths are monoliths because they have
- one set of shared dependencies
- one shared database
- one shared build pipeline
- one shared deployment process
- one shared test suite
- one shared entrypoint
As organizations and applications get larger these start to become liabilities.
Microservices are part of one solution to that (not a whole solution; not the only one).
Monoliths don’t actually look like that at scale. For example you can easily have multiple different data stores for different reasons, including multiple different kinds of databases. Here’s this tiny little internal relational database used internally, and there’s the giant tape library that’s archiving all this scientific data we actually care about. Here’s the hard real time system, and over there’s the billing data etc etc.
The value of a monolith is it looks like a single thing from outside that does something comprehensible, internally it still needs to actually work.
> But all those data sources are connected to from the same runtime, right?
Yes, this is an accurate assessment from what I've seen.
> And to run it locally you need access to dev versions of all of them.
In my experience, no. If the runtime never needs to access it because you're only doing development related to datastore A, it shouldn't fall over just because you haven't configured datastore B. Lots of easy ways to either skip this in the runtime or have a mocked interface.
> And when there’s a security vulnerability in your comment system your tape library gets wiped.
This one really depends but I think can be an accurate criticism of many systems. It's most true, I think, when you're at an in-between scale where you're big enough to be a target but haven't yet gotten big enough to afford more dedicated security testing at an application code level.
> But all those data sources are connected to from the same runtime, right?
Not always directly, often a modern wrapper is setup around a legacy system that was never designed for network access. This can easily mean two different build systems etc, but people argue about what is and isn’t a monolith at that point.
Nobody counts the database or OS as separate systems in these breakdowns so IMO the terms are somewhat flexible. Plenty of stories go “In the beginning someone built a spreadsheet … shell script … and the great beast was hidden behind a service. Woe be unto thee who dare dare disturb his slumber.”
This actually feels like a good example of the modularity that i talked about and feature flags. Of course, in some projects, it's not what one would call a new architecture (like in my blog post), but rather just careful usage of feature flags.
> But all those data sources are connected to from the same runtime, right?
Surely you could have multiple instances of your monolithic app:
If the actual code doesn't violate the 12 Factor App principles, there should be no problems with these runtimes working in parallel: https://12factor.net/ (e.g. storing data in memory vs in something external like Redis, or using the file system for storage vs something like S3)
> And to run it locally you need access to dev versions of all of them.
With the above, that's no longer necessary. Even in the more traditional monolithic profiles without explicit feature flags at work, i still have different run profiles.
Do i want to connect to a live data source and work with some of the test data on the shared dev server? I can probably do that. Do i want to just mock the functionality instead and use some customizable data generation logic for testing? Maybe a local database instance that's running in a container so i don't have to deal with the VPN slowness? Or maybe switch between a local service that i have running locally and another one on the dev server, to see whether they differ in any way?
All of that is easily possible nowadays.
> And when there’s a security vulnerability in your comment system your tape library gets wiped.
Unless the code for the comment system isn't loaded, because the functionality isn't enabled.
This last bit is where i think everything falls apart. Too many frameworks out there are okay with "magic" - taking away control over how your code and its dependencies are initialized, oftentimes doing so dynamically with overcomplicated logic (such as DI in the Spring framework in Java), vs the startup of your application's threads being a matter of a long list of features and their corresponding feature flag/configuration checks in your programming language of choice.
Personally, i feel that in that particular regard, we'd benefit more from a lack of reflection, DSLs, configuration in XML/YAML etc., at least when you're trying to replace writing code in your actual programming language with those, as opposed to using any of them as simple key-value stores for your code to process.
You're talking about something very odd here... a monorepo, with a monolithic build output, but that... transforms into any of a number of different services at runtime based on configuration?
Is this meant to be simpler than straight separate codebase microservices?
This is actually quite a nice sweet spot on the mono/micro spectrum. Most bigger software shops I've worked at had this architecture, though it isn't always formally specified. Different servers run different subsets of monolith code and talk to specific data stores.
The benefits are numerous, though the big obvious problem does need a lot of consideration: with a growing codebase and engineering staff, it's easy to introduce calls into code/data stores from unexpected places, causing various issues.
I'd argue that so long as you pay attention to that problem as a habit/have strong norms around "think about what your code talks to, even indirectly", you can scale for a very long time with this architecture. It's not too hard to develop tooling to provide visibility into whats-called-where and test for/audit/track changes when new callers are added. If you invest in that tooling, you can enforce internal boundaries quite handily, while sidestepping a ton of the organizational and technical problems that come with microsevices.
Of course, if you start from the other end of the mono/micro spectrum and have a strong culture of e.g. "understand the service mesh really well and integrate with it as fully as possible" you can do really well with a microservice-oriented environment. So I guess this boils down to "invest in tooling and cultivate a culture of paying attention to your architectural norms and you will tend towards good engineering" ... who knew?
> You're talking about something very odd here... a monorepo, with a monolithic build output, but that... transforms into any of a number of different services at runtime based on configuration?
Shudder...a previous team's two primary services were configured in exactly this way (since before I arrived). Trust me, it isn't (and wasn't) a good idea. I had more important battles to fight than splitting them out (and that alone should tell you something of the situation they were in!).
Its really not odd at all...this is how compilers work...we have been doing it forever.
Microservices were a half baked solution to a non-problem, partly driven by corporate stupidity and charlotan 'security' experts - I'm sure big companies make it work at enough scale, but everything in a microservice architecture was achievable with configuration and hot-patching. Incidentally, you don't get rid of either with a MCS architecture, you just have more of it with more moving parts...absolute sphegetti mess nightmare.
It’s not that odd. Databases, print servers, or web servers for example do something similar with multiple copies of the same software running on a network with different settings. Using a single build for almost identical services running on classified and unclassified networks is what jumps to mind.
It can be. If you have two large services that need 99+% of the same code and their built by the same team it can be easier to maintain them as a single project.
A better example is something like a chain restaurant running their point of sale software at every location so they can keep operating when the internet is out. At the same time they want all that data on the same corporate network for analysis, record keeping, taxes etc.
> You're talking about something very odd here... a monorepo, with a monolithic build output, but that... transforms into any of a number of different services at runtime based on configuration?
I'd say that it's more uncommon than it is odd. The best example of this working out wonderfully is GitLab's Omnibus distribution - essentially one common package (e.g. in a container context) that has all of the functionality that you might want included inside of it, which is managed by feature flags: https://docs.gitlab.com/omnibus/
Now, i wouldn't go as far as to bundle the actual DB with the apps that i develop (outside of databases for being able to test the instance more easily, like what SonarQube does, so you don't need an external DB to try out their product locally etc.), but in my experience having everything have consistent versions and testing that all of them work together makes for a really easy solution to administer.
Want to use the built in GitLab CI functionality for app builds? Just toggle it on! Are you using Jenkins or something else? No worries, leave it off.
Want to use the built in package registry for storing build artefacts? It's just another toggle! Are you using Nexus or something else? Once again, just leave it off.
Want SSL/TLS? There's a feature flag for that. Prefer to use external reverse proxy? Sure, go ahead.
Want monitoring with Prometheus? Just another feature flag. Low on resources and would prefer not to? It has got your back.
Now, one can argue about where to draw the line between pieces of software that make up your entire infrastructure vs the bits of functionality that should just belong within your app, but in my eyes the same approach can also work really nicely for modules in a largely monolithic codebase.
> Is this meant to be simpler than straight separate codebase microservices?
Quite a lot, actually!
If you want to do microservices properly, you'll need them to communicate with one another and therefore have internal APIs and clearly defined service boundaries, as well as plenty of code to deal with the risks posed by an unreliable network (e.g. any networked system). Not only that, but you'll also need solutions to make sense of it all - from service meshes, to distributed tracing. Also, you'll probably want to apply lots of DDD and before long changes in the business concepts will mean having to refactor code across multiple services. Oh, and testing will be difficult in practice, if you want to do reliable integration testing, as will local development be (do you launch everything locally? do you have the run configurations for that versioned? do you have resource limits set up properly? or do you just connect to shared dev environments, that might cause difficulties in logging, debugging and consistency with what you have locally?).
Microservices are good for solving a particular set of problems (e.g. multiple development teams, one per domain/service, or needing lots of scalability), but adding them to a project too early is sure to slow it down and possibly make it be unsuccessful if you don't have the pre-existing expertise and tools that they require. Many don't.
In contrast, consider the monolithic example above:
- you have one codebase with shared code (e.g. your domain objects) not being a problem
- if you want, you still can use multiple data stores or external integrations
- calling into another module can be as easy as a direct procedure call in it
- refactoring and testing both are now far more reliable and easy to do
- ops becomes easier, since you can just run a single instance with all of the modules loaded, or split it up later as needed
I'd argue that up to a certain point, this sort of architecture actually scales better than either of the alternatives, in comparison to the regular monoliths it's just a bit slower to develop in that it requires you to think about boundaries between the packages/modules in your code, which i've seen not be done too often, leading to the "big ball of mud" type of architecture. So i guess in a way that can also be a feature of sorts?
I'd like to challenge one part of your comment - that microservices break up data on module boundaries. Yes, they encapsulate the data. However, the issue that causes spaghettification (whether internal to some mega monolith, across modules, or between microservices), is the semantic coupling related to needing to understand data models. Dependency hell arises when we need to share an agreed understanding about something across boundaries. When that agreed understanding has to change - microservices won't necessarily make your life easier.
This is not a screed against microservices. Just calling out that within a "domain of understanding", semantic coupling is pretty a fact of life.
that's not at all accurate of any of the monoliths I've worked on. This in particular describes exactly zero of them:
- one shared database
Usually there's one data access interface, but behind that interface there are multiple databases. This characterization doesn't even cover the most common of upgrades to data storage in monoliths: adding a caching layer to an existing database layer.
.NET Remoting, from 2002, was expressly designed to allow objects to be created either locally or on a different machine altogether.
I’m sure Java also had something very similar.
Monolith frameworks we’re always designed to be distributed.
The reason distributed code was not popular was because the hardware at the time did not justify it.
Further, treating hardware as cattle and not pets was not easy or possible because of the lack of a variety of technologies such as better devops tools, better and faster compilers, containerization, etc.
I would actually disagree - to me you can have "decent separation of concerns in your code" but still have only built the app to support a single entry point. "Modular monolith" to me is a system that is built with the view of being able to support multiple entry points, which is a bit more complex than just "separating concerns"
If your concerns are well separated in a monolith (in practice this means being able to call a given piece of functionality with high confidence that it will only talk to the data stores/external resources that you expect it will), adding new entry points is very easy.
Now, it's not trivial--going from, say, a do-everything webserver host to a separation of route-family-specific web servers, background job servers, report processing hosts, and cron job runners does require work no matter how you slice it--but it's a more or less a mechanical or "plumbing" problem if you start from a monolithic codebase that is already well behaved. Modularity is one significant part of said good behavior.
My theory is that microservices became vogue along with dynamically typed languages. Lack of static types means that code becomes unmanageable at a much lower level of complexity. So the complexity was "moved to the network", which looks like a clear win if you never look outside a single component.
I've wondered if it's not a ploy by cloud vendors and the ecosystem around them to increase peoples' cloud bills. Not only do you end up using many times more CPU but you end up transferring a lot of data between availability zones, and many clouds bill for that.
A microservice architecture also tends to lock you into requiring things like Kubernetes, further increasing lock-in to the managed cloud paradigm if not to individual clouds.
> I've wondered if it's not a ploy by cloud vendors and the ecosystem around them to increase peoples' cloud bills. Not only do you end up using many times more CPU but you end up transferring a lot of data between availability zones, and many clouds bill for that.
Disagree. I'd argue that microservices are inherently more cost effective to scale. By breaking up your services you can deploy them in arbitrary ways, essentially bin packing N microservices onto K instances.
When your data volume is light you reduce K and repack your N services.
Because your services are broken apart they're easier to move around and you have more fine grained scaling.
> further increasing lock-in to the managed cloud paradigm if not to individual clouds.
Also disagree. We use Nomad and it's not hard to imagine how we would move to another cloud.
More granular scaling. Scaling up instances of 8, 16, 32GB or even larger instances is much more expensive than 1,2,4GB instances. In addition, monoliths tend to load slower since there's more code being loaded (so you can't scale up in sub minute times)
Obviously there's lazy loading, caching, and other things to speed up application boot but loading more code is still slower
The archetypical "microservice" ecosystem I am aware of is Google's production environment. It was, at that point, primarily written in C++ and Java, neither very famous for being dynamically typed.
But, it was a microservice architecture built primarily on RPCs and not very much on message buses. And RPCs that, basically, are statically typed (with code generation for client libs, and code generation for server-side stubbing, as it were). The open-source equivalent is gRPC.
Where "going microservice" is a potential saving is when different parts of your system have different scaling characteristics. Maybe your login system ends up scaling as O(log n), but one data-munging part of the system scales as O(n log n) and another as just O(n). And one annoying (but important) part scales as O(n * 2). With a monolith, you get LBs in place and you have to scale you monolith out as the part that has the worst scaling characteristic.
But, in an ideal microservice world (where you have an RPC-based mechanism taht can be load-balanced, rather than a shared message bus that is harder to much harder to scale), you simply dial up the scaling factor of each microservice on their own.
Amazon was also doing microservices very early and it was a monolithic C++ application originally (obidos).
Microservices was relly more about locality and the ability to keep data in a memory cache on a thin service. Rather than having catalog data competing with the rest of the monolithic webserver app on the front end webservers, requests went over the network, to a load balancer, they were hashed so that the same request from any of the webservers hit the same catalog server, then that catalog server usually had the right data for the response to be served out of memory.
Most of the catalog data was served from BDB files which had all the non-changing catalog data pushed out to the catalog server (initially this data had been pushed to the webserver). For updates all the catalog servers had real-time updates streamed to them and they wrote to a BDB file which was a log of new updates.
That meant that most of the time the catalog data was served out of a redis-like cache in memory (which due to the load balancer hashing on the request could use the aggregated size of the RAM on the catalog service). Rarely would requests need to hit the disk. And requests never needed to hit SQL and talk to the catalog databases.
In the monolithic world all those catalog requests are generated uniformly across all the webservers so there's no opportunity for locality, each webserver needs to have all the top 100000 items in cache, and that is competing with the whole rest of the application (and that's even after going to the world where its all predigested BDB files with an update service so that you're not talking SQL to databases).
It depends. If your monolith requires, say, 16 GB to keep running, but under the hood is about 20 pieces, each happily using less than 1 GB, tripling only a few of these means you may be able to get away with (say) 25 GB. Whereas tripling your monolith means 48 GB.
You can obviously (FSVO "obvious") scale simply by deploying your monolith multiple times. But, the resource usage MAY be substantially higher.
But, it is a trade-off. In many cases, a monolith is simpler. And in at least some cases, a monolith may be better.
If you go with static languages you are pretty much stuck with Microsoft or Oracle, both which are ahole companies who cannot be trusted. There is no sufficiently common 3rd static option, at least not tied to Big Tech.
Not really. You could do the same by having different organisational units being responsible for different libraries. And the final monolith being minimum glue code combined with N libraries. Basically the same way your code depend on libraries from other vendors/OSS maintainers.
The problem is that orgs are not set in stone. Teams get merged and split in reorgs, buyouts and mergers happen, suddenly your microservices designed around "cleanly defined boundaries" no longer make any sense. Sure you can write completely new microservices but that is a distraction from delivering value to the end customer.
So? The solution is that services travel with teams. I've been through more than one reorg. That's what always happens. The point is to have clear ownership and prevent conflicts from too many people working on the same code.
> monolith was always supposed to be MODULAR from the start
Well, that certainly is sensible, but I wasn't aware that someone had to invent the monolith and define how far it should go.
Alas, my impression is that the term "monolith" doesn't really refer to a pattern or format someone is deliberately aiming for in most cases, but instead refers to one big execution of a lot of code that is doing far more than it should have the responsibility to handle or is reasonable for one repository to manage.
I wish these sorts of battles would just go away, though, because it's not like micro services are actually bad, or even monoliths depending on the situation. They're just different sides of the same coin. A monolith results from not a lot of care for the future of the code and how it's going to be scaled or reused, and micro services are often written because of too much premature optimization.
Most things should be a "modular monolith". In fact I think most things should start out as modular monoliths inside monorepos, and then anything that needs to be split out into its own separate library or microservice can be made so later on.
No one had to invent the monolith or define how far it should go; it was the default.
Microservices came about because companies kept falling into the same trap - that because the code base was shared, and because organizational pressures mean features > tech debt, always, there was always pressure to write spaghetti code rather than to refactor and properly encapsulate. That doesn't mean it couldn't be done, it just meant it was always a matter of time before the business needs meant spaghetti.
Microservices, on the other hand, promises enforced separation, which sounds like a good idea once you've been bitten by the pressures of the former. You -can't- fall into spaghetti. What it fails to account for, of course, is the increased operational overhead of deploying all those services and keeping them playing nicely with each other. That's not to say there aren't some actual benefits to them, too (language agnostic, faults can sometimes be isolated), but the purported benefits tend to be exaggerated, especially compared to "a monolith if we just had proper business controls to prevent engineers from feeling like they had to write kluges to deliver features in time".
The individual code bases of a microservice might not involve spaghetti code, but the interconnections certain can. I'm looking at a diagram of the service I work on, with seven components (written in three languages), five databases, 25 internal connections, two external interfaces, and three connections to outside databases, all cross-wired (via 6 connections) with a similar setup geographically elsewhere in case we need to cut over. And that's the simplified diagram, not showing the number of individual instances of each component running.
There is clear separation of concerns among all the components, but it's the interconnections between them that take a while to pick up on.
Fair; I should have been more explicit - your code can't fall into spaghetti (since the services are so small). Of course, you're just moving that complexity into the infrastructure, where, yeah, you still have the same complexity and the same pressures.
> Because there is no newly invented architecture called "modular monolith" - monolith was always supposed to be MODULAR from the start.
Isn't "non-modular monolith" just spaghetti code? The way I understand it, "modular monolith" is just "an executable using libraries". Or is it supposed to mean something different?
The way I see it, spaghetti code is actually a very wide spectrum, with amorphic goto-based code on one end, and otherwise well structured code, but with too much reliance on global singletons on the other (much more palatable) end. While by definition spaghetti code is not modular, modularity entails more. I would define modularity as an explicit architectural decision to encapsulate some parts of the codebase such that the interfaces between them change an order of magnitude less frequently than what's within the modules.
In my world, the solution depends on the requirement. I can't take all the criticism of each as if they are competition to each other. Also, multiple monolithic (can't stand that word as well) can be applied to distribute resources and data, and to reduce dependencies.
Compute, storage, and other services have gotten to the point where they are unlimited, they were originally designed for what was considered monolithic applications. Shared tenancy is not good in the age of multiple dependencies and in terms of security needs for mission-critical applications and data.
Cloud host providers got too eager in seeking to create new lines of business and pushed micro-service architectures far too early to maturity, and now we're just beginning to see it's faults, many which can't be fixed without major changes that will likely make them pretty much useless, or alternatively just similar to monolithic architectures anyway.
Profit and monopolistic goals shouldn't drive this type of IT innovation, solving critical problems should. We shouldn't just throw away all that we've engineered over the past decade and reinvent the wheel... Heck, many liars are still running FINTECH on COBOL.
> * got too eager in seeking to create new lines of business and pushed * far too early to maturity, and now we're just beginning to see it's faults, many which can't be fixed without major changes that will likely make them pretty much useless
> Because there is no newly invented architecture called "modular monolith" - monolith was always supposed to be MODULAR from the start.
Unless you're in a large established org, modulatity is likely a counter-goal. Most startups are looking to iterate quickly to find pre-PMF. Unless you write absolutely terrible code, my experience is you're more likely to have the product cycle off a set functionality before you truly need modularity. From there, you either (1) survive long enough to make an active decision to change how you work (2) you die and architecture doesn't matter.
"Modular monolith" is a nice framing for teams who are at that transition point.
Agree. IMO modularity/coupling is the main issue. My issue w/ the microservice architecture is that it solves the modularity problem almost as a side effect of itself but introduces a whole host of new ones that people do not anticipate.
Yes, if you, at the outset, say we will separate things into separate services, you will get separated services. However, you do NOT need to take on the extra complexity that comes with communication between services, remote dependency management, and additional infrastructure to reduce coupling.
I explicitly claim to use a "citadel" archicture [0] when talking about breaking off services for very similar reasons. Having a single microservice split out from a monolith is a totally valid application of microservices, but I've had better results in conversation when I market it properly.
I've found this to go so far as to have "monolith" understood to mean "single instance of a system with state held in memory".
This is one of the things I love about Elixir and OTP+The Beam that underpin it all. It's really great that you can just slowly (as is sensible) move your system over to message passing across many VMs (and machines) before you need to move to a more bespoke service oriented architecture.
With Umbrella apps this can already be setup from the start and you can break off bits of your app into new distinct message-able components (micro services you could say) as you like while still being able to share code/modules as needed.
The other thing I'd say is you can introspect how these "services" are running in realtime with IEX and it feels like magic being able to run functions in your application like some sort of realtime interactive debugger. One of the biggest issues I had with micro-services is figuring out what the hell is going wrong with them (in dev or prod) without the layers of extra work you need to add in terms of granular logging, service mesh and introspection etc.
Creating a lot of actors and messaging for the business logic of your application is considered an anti-pattern in Elixir, and a typical novice mistake. Applications in Elixir are structured using functions that are in modules that call other functions in other modules. Yes you can use OTP applications to isolate dependencies but none of this is done with the intent to more easily break up your app into a bunch of micro-services.
Which is a distinct feature made for breaking up the logic of your applications into smaller, domain bounded libraries. Umbrella apps are for the most part like regular libraries, just internal to the project which let's you do neat things like share config and deployment across them.
They don't require interacting with OTP functionality unless you make them that way and I think the OP was crossing some wires there.
No, you do not even need OTP functionality on the child project, that's my point. Not everything uses OTP.
Edit: We may be talking past each other, from Sasa Juric:
"It’s worth noting that poison is a simple OTP app that doesn’t start any process. Such applications are called library applications." Which is what I'm thinking of. He also says "Finally, it’s worth mentioning that any project you build will mix will generate an OTP app."
I was mainly talking about you don't have to use anything like GenServer or other OTP functionality with the split out app so they're more like the library application but that is still in fact an OTP application even if you're not using any of the downstream OTP features.
Each separate child project is still an OTP application, even if you do not use "OTP" features in them. OTP app is just the term for that artifact, similar to assembly or archive in other languages but it is not only terminology, each one will have `start` called when the VM starts, even if they don't spawn any servers.
I've argued for a long time that microservices should grow out of a monolith like an amoeba. The monolith grows until there is a clear section that can be carved off. Often the first section is security/auth, but from there it's going to be application specific. A modulith could be just another step in the carve up process.
But, there is no right answer here. Application domain, team size, team experience, etc... all matter and mean a solution for one team may not work for another and vice versa.
In my experience with enterprise software one of the things that cause most trouble is premature modularization (sibling to the famous premature optimization).
Just like I can't understand how people can come up with the right tests before the code in TDD, I can't understand how people can come up with the right microservices before they start developing the solution.
Perhaps a better way to think of "writing the tests first" in TDD approaches (or, more generally, test-first development, which is a term that has gone out of favor) is that you write test cases in that testing framework to express your intent and then from there can start to exercise them to ensure correctness. It's not a crime to change them later if you realize you have to tweak a return type. But writing them up front necessitates a greater depth of thinking than just jumping in to start hacking.
Not doing this runs the risk of testing what you did, not testing that it's right. You can get lucky, and do it right the first time. Or you can make a bunch of tests that test to make sure nothing changed, with no eye as to whether it's what anybody actually wanted in the first place.
Sitting down with somebody for whom TDD is natural is educational and makes it very hard to argue against it, particularly for library code.
The issue with most TDD or even BDD I have seen is that it is usually worthless...
You end up testing that numbers came out of a function or that some thing was called, but it doesn't actually solve the real issue which is to get your use cases correct. Instead it encourages you to break down the problem into a bunch of unrelated bits, it doesn't actually check that your approach is correct or what the customer wants, just that you wrote some bits of code which do things, whether those things are the right things...
As it is often practiced it is usually a failsafe for people who struggle to write code at all.
Acceptance tests are too happy pathy, integration tests are rarely done or deemed 'unnecessary', so the only place left to 'think about the problem' becomes actually writing/designing the code. And so tests tend to come last, because you already decided what works, and they check nonsense.
For truly difficult code, such as a mathematical algorithm which is difficult to break down, unit tests make sense, the majority of "when X is true do A, when X is false do B" of unit tests are utter garbage.
Well when you hire 40 engineers and 4-5 dev managers, each manager wants an application they can call their own, so they divvy up the potential product into what they feel are reasonable bits. Hence: micro services. It’s the same reason React was created: to more easily Conways-Law (used as a verb) the codebase
On the TDD argument, you should know what a return for a function f should be when given an argument b. Ideally, you have a set of arguments B to which b belongs, and a set of results C to which the return belongs. Your tests codifies mapping examples from B to C so that you can discover an f that produces the mapping. Take another step and generate random valid inputs and you can have property-based testing. Add a sufficient type system, and a lot of your properties can be covered by the type system itself, and input generators can be derived from the types automatically. So your tests can be very strong evidence, but not proof, that your function under test works as expected. There's always a risk that you modeled the input or output incorrectly, especially if you are not the intended user of the function. But that's why you need user validation prior to final deployment to production.
Likewise, with a microservice architecture, you have requirements that define a set of data C that must be available for an application that provides get/post/put/delete/events in a set B to your service over a transport protocol. You need to provide access to this data via the same transport protocol, transform the input protocol to a specified output protocol.
You also have operational concerns, like logging that takes messages in a set C and stores them. And monitoring, and authorization, etc. These are present in every request/response cycle.
So, you now split the application into request/response services across the routing boundary-> 1 route = 1 backing model. That service can call other apis as needed. And that's it. It's not hard. It's not even module-level split depolyment. It's function-level deployment in most serverless architectures that is recommended because it offers the most isolation, while combining services makes deployment easier, that's mostly a case of splitting deployment across several deployment templates that are all alike and can be managed as sets by deployment technologies like Cloudformation and Terraform [1].
You can also think of boundaries like this: services in any SOA are just like software modules in any program - they should obey open/closed and have strong cohesion [2] to belong in a singular deployment service.
Then you measure and monitor. If two services always scale together, and mutually call each other, it's likely that they are actually one module and you won't effect cohesion by deploying them as a single service to replace the two existing ones. easing ops overhead.
Not deploying and running as a monolith doesn't mean not putting the code to be run into the same scm/multiproject build as a monorepo for easy shared cross-service message schema refactoring, dependency management, and version ingredient. That comes with its own set of problems -- service projects within the repo that do not change or use new portions of the comm message message schema shouldn't redeploy with new shared artifact dependencies; basically everything should still deploy incrementally and independently, scaling it is hard (see Alphabet/Google's or Twitter's monorepo management practices, for example); but there seems to be an extra scale rank beyond Enterprise size that applies to, it's very unlikely you are in that category, and if you are you'll know it immediately.
We like to market microservice architecture as an engineering concern. But it's really about ops costs in the end. Lambdas for services that aren't constantly active tend to cost less than containers/vps/compute instances.
> from there it's going to be application specific
It is actually sensible to keep all the business logic layer(s) in the monolith while it is possible. Easier to grasp the domain for new team members, easier to bound context.
Have you seen this done well more than a few times? Honest question because it’s something I think everyone agrees with as being a good idea but it never gets actually done. It’s definitely not an industry practice that big monoliths eventually get split up in modules and quality increases. It’s something you have to fight for and actively pursue.
I’ve been on projects with very good developers, who were very bought into the idea of separating modules in the monolith, yet for one reason or another it never got done to an extent where you could see the quality improving. This leads me to believe this technique just isn’t practical for most teams, for reasons that are more psychological than technical.
It’s just something about how people approach a single codebase that leads to that, it’s harder to think about contracts and communication when everything is in the same language.
But if you force them to communicate over the wire to something in a different stack then all of a sudden the contracts become clear and boundaries well defined.
It is not a binary flip between monolith and modular monolith, it is on a gradual scale, and I saw teams moving toward modularity with a various degree of success. They may not even use the term of distributed monolith to name their approach. Sometimes, they do it to keep the monolith maintainable, sometimes as the first steps towards microservices - defining boundaries, simplifying dependencies certainly help against the antipattern of distributed monolith.
Ideally, the decision to build modular monolith should be made and implemented from the very start of the project. Some frameworks like Django help with keeping separation.
I found that fitness functions help with policies and contracts. You run then in your CI/CD and they raise alarm when they detect contract / access violation across the boundaries.
I've been quite happy with multi module projects over the years. I.e. where your modules are a directed graph of dependencies. Whatever modular structure you have right now is never perfect, but refactoring them can be straightforward, which does require active pursuit of the ideal, yes. You start such a project with an educated guess about the module structure, then the team goes heads down developing stuff, then you come up for air and refactor the modules. If you don't do that part then things get increasingly crufty. I think a best practice in these environments is to not worship DRY too much, and instead encourage people to do non-scalable things in the leaf libraries, then later refactor re-usable stuff and move it to the branch libraries.
It helps very much to be in a language and build environment that has first class modularity support. I.e. good build tooling around them and good IDE support. And at the language level, good privacy constraints at the package level, so the deeper libraries aren't exposing too much by accident.
What modules patterns have I seen work over the years? Generally, having business logic and data access apis lower in the tree, and branching out for different things that need to be deployed differently, either because they will deploy to different platforms (say, you're running business logic in a web server vs a backend job), or because they are deployed on different schedules by different teams (services). A nice thing about the architecture is that when you have a bunch of services, your decision as to what code runs on what services becomes more natural and flexible, e.g. you might want to move computationally expensive code to another service that runs on specialized hardware.
But you need to refactor and budget time to that refactoring. Which I think is true in any architecture--it's just often more do-able to do in one pass with the multi module approach.
In my experience, i've seen the modular code approach more often than separate deployment approach, quite possibly because the latter is still a bit harder to do when compared to just having 1 instance (or people are just lazy and don't want to take the risk of breaking things that were working previously for future gains), but sooner or later the question of scalability does come up, at least in successful projects.
Let me tell you, as someone who has delivered a critical code fix for business continuity after midnight a few times, slapping N instances of an app runtime in a data center somewhere is way easier than having to struggle with optimizations and introduce more complexity in the form of caches, or write out Hibernate queries as really long and hard to debug SQL because people previously didn't care enough about performance testing or simply didn't have a feasible way to simulate the loads that the system could run into, all while knowing that if your monolith also contains scheduled processes, none of your optimizations will even matter, because those badly optimized processes will eat up all of the resources and crash the app anyways.
In short, the architecture that you choose will also help you mitigate certain risks. Which ones you should pay attention to, however, depends on the specifics of your system and any compliance requirements etc. Personally, as a developer, fault tolerance is up there among the things that impact the quality of my life the most, and it's pretty hard to do it well in a monolith.
In my eyes the problem with contracts is also worthy of discussion, though my view is a bit different - there will always be people who will mess things up, regardless of whether you expect them to use modules someone else wrote and contribute to a codebase while following some set of standards or expectations, or whether you expect them to use some web API in a sane manner. I've seen systems that refuse to acknowledge that they've been given a 404 for a request numerous times (in a business process where the data cannot reappear) and just keep making the same request ad infinitum, whenever the scheduled process on their side needs to run.
So, having a web API contract can make managing responsibility etc. easier, however if no one has their eye on the overall architecture and how things are supposed to fit together (and if you don't have instrumentation in place to actually tell you whether things do fit together in the way you expect), then you're in for a world of hurt.
To that end, when people need to work with distributed systems of any sort, i urge them to consider introducing APM tools as well, such as Apache Skywalking: https://skywalking.apache.org/ (sub-par interface, but simple to set up, supports a decent variety of technologies and can be self hosted on prem)
Or, you know, at least have log shipping in place, like Graylog: https://www.graylog.org/ (simpler to setup than Elastic Stack, pretty okay as far as the functionality goes, also can be self hosted on prem)
There is a more serious downside that you don’t mention: splitting things into modules takes time and involves making decisions you likely don’t know the answer to. When starting a new product, the most important thing is to get something up and running as quickly as possible so that people can try it and give you feedback. Based on the feedback you receive, you may realize that you need to build something quite different than what you have. I’ve seen plenty of successful products with shoddy engineering, and I’ve seen plenty of well engineered products fail. Success of a product is not correlated with how well it’s engineered. Speed is often the most important factor.
Selling effectively is the most important factor, not speed. Speed is second as a factor (and obviously very important). That's actually what you're describing when you say success of a product is not correlated to how well it's engineered. It's correlated to how well you can sell what you have to the audience/customers you need. That's why some start-ups can even get jumpstarted without having a functional product via pre-product sign-ups and sales. Getting to selling as reasonably quickly as you can, in other words.
Go when you have something to sell. That's what the MVP is about.
Which also isn't the same as me saying that speed doesn't matter - it matters less than how well you sell. It's better to sell at a 10/10 skill level, and have your speed be 8/10, than vice versa (and that will rarely not be the case). Those are bound-together qualities as it pertains to success, so if you sell at 10/10 and your speed is 1/10, you're at a high risk of failure. Give on speed before you give on selling and don't give too much on either.
> There is a more serious downside that you don’t mention: splitting things into modules takes time and involves making decisions you likely don’t know the answer to.
Partially agreed. Domain driven design can help with answering some of those questions, as can drilling down into what the actual requirements are, otherwise you're perhaps reaching for your code editor before even having an idea of what you're supposed to build.
As for the inevitable changing requirements, most of the refactoring tools nowadays are also pretty reasonable, so adding an interface, or getting rid of an interface, creating a new package, or even in-lining a bunch of code isn't too hard. You just need to know how to use your tools and set aside time for managing technical debt, which, if not done, will cause more problems down the road in other ways.
> Speed is often the most important factor.
If you're an entrepreneur or even just a business person who cares just about shipping the feature, sure. If you're an engineer who expects their system to work correctly and do so for the years to come, and, more importantly, remain easy to modify, scale and reason about, then no, speed is not the most important factor.
Some business frameworks like COBIT talk about the alignment between the tech and business, but in my experience their priorities will often be at odds. Thus, both sides will need to give up bits of what they're after and compromise.
If you lean too heavily into the pace of development direction, you'll write unmaintainable garbage which may or may not be your problem if you dip and go work for another company, but it will definitely be someone else's problem. Thus, i think that software engineering could use a bit more of actual engineering it.
Not necessarily 50 page requirement docs that don't conform to reality and that no one cares about or reads, but actually occasionally slowing down and thinking about the codebases that they're working on. Right now, i've been working on one of the codebases in a project on and off for about 4 years - it's not even a system component, but rather just some business software that's important to the clients. In my personal experience, focusing just on speed wouldn't have been sustainable past 1 year, since the codebase is now already hundreds of thousands of lines long.
For a contrast, consider how your OS would work if it were developed just while focusing on the speed of development.
> Speed [of delivery] is often the most important factor.
Depends on the org and the app type. If banks "moved fast and broke things" using millions of dollars, they'd be shut down or sued into oblivion. If it's merely showing dancing cats to teeny-boppers, sure, move fast and break things because there's nothing of worth being broken.
I agree with your position, I'm a big fan of the modular monolith approach. I took a look at your post. This is one thing that jumped out to me:
> Because the people who design programming languages have decided that implementing logic to deal with distributed systems at the language construct level... isn't worth it
I'm not sure if this is just a dead end or something really interesting. The only language I really know that [does this is Erlang](https://www.erlang.org/doc/reference_manual/distributed.html), though it's done at the VM / library level and not technically at the language level (meaning no special syntax for it). What goes into a language is tricky, because languages tend to hide many operational characteristics.
Threads are a good example of that, not many languages have a ton of syntax related to threads. Often it's just a library. Or, even if there is syntax, it's only related to a subset of threading functionality (i.e. Java's `synchronized`).
So there might not be much devotion of language to architectural concerns because that is changing so much over time. No one was talking about microservices in the 90s. Plus, the ideal case is a compiler that's smart enough to abstract that stuff from you.
Recent languages do have syntax related to -concurrency- though.
Languages pre multi-core probably provided a VM/library, as you say, for threading, and then said "generally you should minimize threads/concurrency" (which for mainstream languages were the same thing).
Languages since then have embraced concurrency at the syntax level, even if not embracing threading. Node has event listeners and callbacks (and added async/await as a syntactical nicety), go has goroutines (with special syntax to indicate it), etc.
It's interesting that while languages have sought to find ways to better express concurrency since it became necessary to really use the chips found in the underlying hardware, they largely haven't sought to provide ways to better express distribution, leaving that largely to the user (and which has necessitated the creation of abstracted orchestration layers like K8s). Erlang's fairly unique in having distribution being something that can be treated transparently at the language level.
Mind you, that also has to do with its actor based concurrency mechanism; the reason sending a message to a remote process can be treated the same as sending a message to a local process is because the guarantees are the same (i.e., "you may or may not get a response back; if you expect one you should probably still have a timeout"). Other languages that started with stronger local guarantees can't add transparent remote calls, because those remote calls would have additional failure cases you'd need to account for (i.e., Java RMI is supposed to feel like calling a function, but it feels completely different than calling a local function. Golang channels are synchronous and blocking rather than asynchronous and non-blocking, etc. In each case you have a bunch of new failure conditions to think about and address; in Erlang you design with those in mind from the beginning)
Patterns and automation supporting modularization hasn't received the attention that patterns and automation around services has over the past 10 years.
In practice, modularization raises uncomfortable questions about ownership which means many critical modules become somewhat abandoned and easily turn into Frankensteins. Can you really change the spec of that module without impacting the unknown use cases it supports? Tooling is not in a position to help you answer that question without high discipline across the team, and we all know what happens if we raise the question on Slack: crickets.
Because services offer clear ownership boundaries and effective tooling across SDLC, even though the overheads of maintenance are higher versus modules, the questions are easier and teams can move forward with their work with fewer stakeholders involved.
Modules have higher requirements for self discipline the services. Precisely because the boundaries are so much easier to cross.
And also because it is harder to guard module from changes made by other teams. Both technically and politically, the service is more likely to be done by a single team who understands it. Module is more likely to be modified by many people from multiple teams who are just guessing what it does.
I think anyone who speaks about monolith today assumes you do use modules appropriately.
I mean for "starting with a monolith and splitting out when necessary" you kind need property structures code (i.e code which uses modules and isn't too tightly coupler between them).
Through in my experience you should avoid modularizing code with the thought of maybe splitting it into parts later on. That's kinda defeats the point of starting with a monolith to some degree. Instead modularize it with the thought of making it easy to refactor and use domain logic as main criterium for deciding what goes where when possible ( instead of technical details).
To be fair, modular monoliths weren't talked about much before microservices. I'm thinking about the heyday of Rails, where there were bitter debates over whether you should even move critical business logic out of Rails. I got the impression that most devs didn't even care. (That was sort of the point where I saw that I was a bit of an odd duck as far as developers go if I cared about this stuff, and pivoted my career.) I really enjoy contexts where I can make modular monoliths, however. I'm thinking of mobile/desktop apps here, mostly.
Software design is something of a lost art currently. This is partially due to the current zeitgeist believing that anything that automated tools cannot create/enforce can't be that important in the first place. Of course, there's a whole swath of concerns that cannot be addressed with static or dynamic analysis, or even different languages.
Microservices is a response to the fact that a monolith without any real design moves quickly but can easily converge on spaghetti due to the fact that everything is within reach. It enables teams to create an island of code that talks to other islands only via narrowly defined channels. Additionally, they support larger dev teams naturally due to their isolationist take, and reinforce the belief that "more developers = always better" in the process. In other words, they mesh perfectly with the business context that a lot of software is being built in.
Factor in the fact that $COOL_COMPANY uses them, and you have a winner.
Modular design is decades old though. Rails devs argue about it because it's against the rails way, and that makes things harder for them. But I've been modularizing my Java webapps since I graduated from college, 18 years ago.
But also, microservices weren't pioneered by rails devs. They were pioneered by huge companies, and they definitely have a role to play there, as you point out.
What I think nobody talks about is: (1) the legitimate reason for breaking up services into multiple address spaces, and (2) that using different versions of the runtime, different build systems, and different tools for logging, ORM, etc. in different microservices is slavery, not freedom.
(1) is that some parts of a system have radically different performance requirements than other parts of the system. For instance 98% of a web backend might be perfectly fine written in Ruby or PHP but 2% of it really wants everything in RAM with packed data structures and is better off done in Java, Go or Rust.
(2) The run of the mill engineering manager seems to get absolutely ecstatic when they find microservices means they can run JDK 7 in one VM, run JDK 8 in another VM, run JDK 13 in another VM. Even more so when they realize they are 'free' to use a different build system in different areas of the code, when they are 'free' to use Log4J in one place, use Slf4J someplace etc, use Guava 13 here, Guava 17 there, etc.
The rank and file person who has to actually do the work is going to be driven batty by all the important-but-not-fashionable things being different each and every time they do some 'simple' task such as compiling the software and deploying it.
If you standardize all of the little things across a set of microservices you probably get better development velocity than with a monolith because developers can build (e.g. "make", "mvn install") smaller services more quickly.
If on the other hand the devs need to learn a new way to do everything for each microservice, they are going to pay back everything they gained and then some with having to figure out different practices used in different areas.
(Throw docker into the mix, where you might need to wrangle 2G of files to deploy 2k worth of changes in development 100 times to fix a ticket you can really wreck your productivity, yet people really account for "where does the time go" when they are building and rebuilding their software over and over and over and over again.)
In my humble opinion microservices are "hot" because in theory you can scale a lot with them if you are able to do cloud provisioning.
Microservices needs DevOps+Orchestration Service.
A good example of microservices architecture is how K8s is designed: I think it is an overkill for most average needs, so think twice before entering in microservice trip tunnel.
Microservices solve pretty much one problem: you have a larger organization (> 10 devs, certainly > 100) and as a result the coordination overhead between those devs and their respective managers and stakeholders is significantly limiting overall forward progress. This will manifest in various concrete ways such as "microservices allow independent component release and deployment cycles" and "microservices allow fine grain scaling", and "microservices allow components written in different languages", but really it's all Conway.
This is a pretty critical point. The drum I tend to beat is that the positive read of microservices is that they make your code reflect your org chart.
(If they don't do this, and that usually resolves into developers each owning a bunch of microservices that are related concepts but distinct and are context-switching all the live-long day, you've created the reverse of a big ball of mud: you've created a tar pit. Companies used to brag to me when they had 250 microservices with 100 developers, and I don't think any of those companies are going concerns.)
i always think of microservices as a product an organization offers to itself. If you don't have the team including managers and even marketing to run an internal product then you probably shouldn't be doing the microservice thing.
Well in my eyes equating microservices with Kubernetes is a problem in of itself. I actually wrote about Docker Swarm as a simpler and more usable alternative to it (for smaller/simpler deployments), though some other folks also enjoy Hashicorp Nomad which is also nice (another article on my blog, won't link here not to be spammy myself).
If you evaluate your circumstances and find that microservices could be good for you, then there are certainly options to do them more easily. In my eyes some of the ideas that have popped up, like 12 Factor Apps https://12factor.net/ can be immensely useful for both microservices and even monoliths.
So i guess it's all very situational and a lot of the effort is finding out what's suitable for your particular circumstances. For example, i made the page over at https://apturicovid.lv/#en When the app was released and the page was getting hundreds of thousands of views due to all of the news coverage, scaling out to something like 8 instances was a really simple and adequate fix to not break under the load.
We've been following this modular monolith approach as well (for 3 years now), bounded contexts and all, but our architectural end goal is still "mostly microservices". Maybe it's specific to PHP and the frameworks we use (what the modulith is written in) but the startup time of the monolith is just unacceptable for our processing rates (for every request the framework has to be reinitialized again, with all the 9000 services via DI and what not). Microservices tend to be much more lightweight (you don't even need DI frameworks), and monolith also encourages synchronous calls (calling a bounded context's API in memory is so simple) which has been detrimental to our performance and stability, because microservices, at least in our architecture, encourage event-based communication which is more scalable, and allows clean retries on failure etc. But again, your mileage may vary, maybe it's specific to our tools.
In my experience, long build times mostly come from not caring about build times. Not from large codebases. A surprising number of developers don't think build times (including running all the tests) are important. They'll just go "ok, so now it takes N minutes to build" rather than thinking "hold on, this shouldn't take more than a few seconds - what's going on here?"
Tried it twice, never pulled its weight. It introduces abstraction layers everywhere, as a premature optimisation, even though they might never be needed, and its a bad fit for more verbose and statically typed languages due to all the ceremony thats required. Anyone made similar experiences?
I've mixed feelings about it. As advertised it can be somewhat useful for business rules-rich apps, yes, but most of the times the application side of the project will just grow exponentially faster than your core - especially if you don't keep an eye on commits, as some of your coworkers may choose the easy way out and push business logic on the application side instead of refactoring... It's a clean architecture from a business logic standpoint, indeed, but it doesn't do a lot to keep the outside of the hexagone well organized (beside the port/adapter pattern), and that side of the app can grow into a mess really fast if your team lack experience.
Not as many abstraction layers than in a classic J2EE app, though. It's not that bad.
I use it all the time. It is my default architecture choice. It works great in my opinion. I wonder why you think there is a lot of ceremony needed? The core idea doesn’t require much. The Hexagonal Architecture idea can be implemented in many ways. I recommend Googling “Elm Architecture” to see an example. It isn’t described as being a Hexagonal Architecture but I will argue that it is one of the best ways to implement it I have seen. It has zero ceremony (a few functions define the whole architecture).
I agree. Started with a simple React for frontend and NestJS for backend. Now I am running microservices for distributing third-party widgets, my search engine, and the analytics.
Works well and it actually simplifies things a lot, each service has its repository, pipeline and permissions, developers don't need to understand the whole application to code.
You also don't have to start with Kubernetes to make microservices work, many tools can act as in-betweens. I am using app engine from gcloud, yes it's a lot of abstraction over kubernetes and it is overpriced, but I don't care. It works perfectly for these use cases and even if overpriced, it stays a low absolute value.
The caveat is that you really need to start off with a "stateless mindset".
I even wonder why the word "monolith" got such a bad connotation that it is now used synonymously to "big ball of mud".
I mean, monoliths in the original sense (Washington Monument, that 2001 - A Space Odyssee moon thing, ...) are all but messy.
It's usually not obvious where to put the seams ahead of time so you can cut them when you need to split into microservices.
Plus keeping the API boundaries clean costs time and resources, and it's tempting to violate them just to launch this one feature. This extra discipline doesn't have any payoff in the short term, and it has unknown payoff in the long term because you're not sure you drew the boundaries ahead of time anyway.
So I think in practice what happens is you create a monolith and just eat the cost of untangling it when the team gets too big or whatever.
> I wonder why no one ever talks about architectures in the middle between those two
Because that's the default. It doesn't make intuitive sense to integrate every separate service you develop internally into one huge bulky thing, nor to split up every little feature into even more small services that need a ton of management mechanisms and glue to even be functional. Only after advocates for both extremes sprung up does it make sense to invent, as you did, a new word for the thing in the middle. It's just the sensible thing to do in most situations.
Agree. But start with modules immediately. SOLID applies to modules as well as classes. [And if we're talking about Rust, then substitute "crate" wherever I write "module"]. Basically be constantly refactoring your code into classes (or equiv.) and the classes into modules.
It's super easy to join multiple modules into a single application: the linker, JVM, whatever does it automatically. It's insanely hard to break a monolithic application into modules.
The problem with modular monoliths is the relative difficulty of managing dependencies in the code. It’s easy to hack the modular interfaces or expose internals, which can make it hard to maintain and painful to pull out into another service down the road.
“But I would never hack an interface like that!” you say. Oh, you sweet summer child.
IME it's the drive for microservices that encourage the modular monolith. That is to say, the monolith is typically loosely modular (but still with many rule-breakages) until the push for microservices starts _then_ the big refactor to modularity begins.
Splitting code into modules has the same downsides as splitting it into microservices. You can still end up making the wrong splits and needing to back track on things you once thought were modular but no longer are.
The logistics of microservices are rarely the hard part. It's the long term maintenance. Everyone who's ever maintained a "core" library knows the same pain, at some point you just end up making sacrifices just to get things to work.
> Splitting code into modules has the same downsides as splitting it into microservices.
Not to be pedantic, but it has some of the same downsides. Microservices have other major downsides in that they bring in all the fallacies of network computing. Even if you manage to stabilize these in the end they just waste so much time in development and debugging.
The point in time where you're splitting your codebase up in modules (or maybe are a proponent of hexagonal architecture and have designed it that way from the beginning), leading to being able to put functionality behind feature flags. That way, you can still run it either as a single instance monolith, or a set of horizontally scaled instances with a few particular feature flags enabled (e.g. multiple web API instances) and maybe some others as vertically scaled monoliths (e.g. scheduled report instance).
I wrote more about that approach on my blog, as one of the first articles, "Moduliths: because we need to scale, but we also cannot afford microservices": https://blog.kronis.dev/articles/modulith-because-we-need-to...
In my eyes, the good part is that you can work with one codebase and do refactoring easily across all of it, have better scalability than just a monolith without all of the ops complexity from the outset, while also not having to worry as much about shared code, or perhaps approach the issue gently, by being able to extract code packages at first.
The only serious negatives is that this approach is still more limited than microservices, for example, compilation times in static languages would suffer and depending on how big your project is, there will just be a bit of overhead everywhere, and not every framework supports that approach easily.