How Adopting a Modular Monolithic Architecture Enables Our OSS

In the previous articles in this series, Appsmith's journey from SaaS to OSS, I discussed my opinions on the ongoing monoliths vs. microservices debate, the benefits of each architecture for developers, and how all of this has been factored into decisions regarding how Appsmith (an internal tool building platform) is built — culminating in our adoption of a modular monolithic architecture.

While our decision to implement a modular monolith was heavily influenced by the advantages that would be realized when we open-sourced, we didn’t anticipate just how strong a driver the architecture would be for the community that has become a key part of the Appsmith project.

In this next part of the series, I will look at how the modular monolithic architecture improved how we interact with our open-source community so that other open-source projects can borrow from our experiences.

Our stewardship role in leading an open-source project

Just as there are positive and negative factors that must be considered when choosing an application architecture, there are trade-offs to consider — especially the potential effect on your development process — before you open your code to the world.

Traditional vs. open-source development

In a traditional, proprietary software development process, developers can dedicate their full efforts towards the project because they are being paid a full-time salary. They are also more centralized and can work from a uniform plan from the beginning, especially when it comes to major components of the software like the architecture and infrastructure.

However, proprietary software comes with the risk of the product not gaining traction. It often has a limited reach if potential users are unsure whether it fits their needs, are put off by the upfront costs involved, or are concerned about the lifespan of the product. Developing proprietary software can cost a lot of money, with the risk that it doesn’t even get off the ground.

Contrast this against the open-source development model, in which the startup costs are much lower. There is still a lot of time and effort that developers have to put in, but by encouraging an open-source community to form around a project, it’s far easier to develop and validate ideas and ensure that you have an audience with less upfront financial risk.

Having an active community also provides a wider diversity of ideas and use cases. Community suggestions and feedback will reflect end users’ actual desires far better than any internal research will, since the ones developing the software are the end users themselves.

Finally, the quality of the product is also improved under an open-source model. Any developer can dig into the source code and implement what they need themselves and then share it with everyone else, which is something that cannot be done with traditional projects.

Decentralization of the project leads to wider adoption due to lower barriers to use, leading to more users and, in turn, more testing of your product in different environments and for different use cases.

However, the benefits of open-source come with a price. Decision making is also decentralized and, if not managed properly, can lead to the project being pulled in different directions, to its detriment. This is especially true with deep, architectural changes that could have significant consequences downstream. Volunteer contributors may also not be fully focused on the agreed-upon mission statement, but only on addressing their own requirements. For big complicated projects, this can significantly limit the contributions that individual developers can make.

An alternative approach: a hybrid between traditional development and open-source

Given these tradeoffs, we found ourselves in a similar situation as with our decision between monoliths and microservices — caught between two paradigms and wanting the benefits of both. And, as we found with our compromise between monoliths and microservices, we could leverage the advantages of both centralized and decentralized development in order to drive innovation in the underlying technology.

We have full-time internal development teams who dedicate all their efforts towards standardizing the deeper parts of the architecture, while we also benefit from a wider community of open-source contributors who bring new ideas to test and build on Appsmith’s core work. Wider adoption and testing from the open-source community allows us to stay extremely aligned with what developers want without getting stuck in our own biases.

In this dynamic, we see ourselves as the stewards of the Appsmith project. It’s our job to provide a stable architecture and an efficient development process. Community members can then contribute their suggestions and focus on building highly-customizable value-adds that don’t require major alterations to the core of the software. This has struck the right balance between the benefits and drawbacks of each development model on its own.

How the principles of modularity foster OSS contribution

The key benefit that we saw in regard to open-source contributions when moving from a purely monolithic architecture to a modular monolith was better organization – of both the codebase and the teams working on it. Code isolation, abstraction, ownership, development consistency, and simplicity all improved the developer experience.

While it’s clear how all of these principles can help any software team produce better software faster, several of these can be particularly beneficial for working with the OSS community. Determining boundaries and finding the right level of abstraction were very important for helping contributors understand the codebase (at least well enough to contribute), and embracing consistency and simplicity were key to further encouraging contributions.

We wanted to give OSS contributors the ability to alter and improve any aspects of the code that they wanted, without requiring them to have an extremely deep understanding of the architecture. Otherwise, developers would probably shy away from contributing due to the sheer amount of studying the codebase they’d have to do before becoming productive. It was important for us to cut down this barrier to entry.

Creating strict boundaries between different portions of the code also made the Appsmith codebase more intuitive for our internal teams and those contributors working on core components. Developers can be confident that each section of the code is responsible for exactly one thing without any overlap. It allows them to make changes, confident that they won’t break anything downstream in unexpected ways.

During modularizing, and while we were refactoring in preparation for open-sourcing, a consistent style began to emerge for how the different modules were created. The “Appsmith way of doing things” became clearer and easier to communicate to contributors — enabling developers to jump into a part of the codebase that they’d never seen before because they saw many familiar patterns from elsewhere in the codebase. Modularization encouraged more contribution, with clear ongoing benefits to the project.

Modularity also extends to how we address documentation. We are working to create the simplest developer experience possible and documentation is a big part of this — we’ve broken down the codebase into complicated architectural components that should be changed very infrequently and less-complicated feature-oriented components that can be changed frequently with no dangers. Instead of trying to document our codebase uniformly, we can prioritize the high-traffic feature-oriented areas so that developers have all the tools they need to contribute where they’re needed most.

Appsmith’s open-source experience

During Appsmith’s first year as a company, we were closed-source. We found that we could iterate much faster by just focusing on developing a minimum viable product, rather than diverting our focus to building an open-source community as well. And, to be honest, we felt a little vulnerable about putting our code out there for everyone to judge.

A year later, once we had a more stable foundation underneath us, we decided that pivoting in the open-source direction was now the right move. By that point, we had the major architectural components solidified and we knew that the best way to drive forward innovation in internal tool-building technology was to go ahead and open up our code and start building a community.

The immediate positive effects

For one, community feedback encouraged us to improve the organization and consistency of our codebase and procedures across the board. Because we chose the hybrid modular monolithic approach rather than a microservices-oriented one, contributors had access to the entire Appsmith codebase. This allowed them to compare and contrast different sections to identify and fix inconsistencies much more easily. This has made our code much more consistent, which is why I promote this practice to other small teams who want to improve the readability and reliability of their code.

Opening up the project to a much wider developer base also forced us to think through some of our development processes in more depth and streamline them as much as possible. Creating targeted documentation for certain sections of the code that could benefit from more development was a major key to this. This had the effect of simplifying onboarding for Appsmith team members as well, which we didn’t anticipate but did make sense in hindsight.

The complications we faced

One of the biggest mistakes that we made early on — and something that we are still trying to get right — is finding the best level of abstraction. There is a constant battle between minimizing complexity and maximizing control. This is something that even large teams need to be aware of, as it impacts developer frustration, and could therefore dramatically slow or halt adoption of your project in the community.

Overall, the open-source community has been very kind, but also honest about where we can improve. They tell us where our code is too convoluted and also where our processes and documentation can be improved. Making this leap is critical to take a project to the next level, but I would encourage anyone looking to open-source their project to have the right mentality before doing so: humility. You need to be able to take constructive criticism and view your project as a work in progress that is constantly being polished rather than something that is being shot down when it is criticized by others.

The solutions we found

We are currently encouraging open-source contributors to focus on the three to four areas of the codebase that can benefit most from customization. We’re then prioritizing documentation of those areas. I would encourage everyone with an open-source project to do the same: think about the areas of their codebase that could most benefit from more development and prioritize documentation of those.

Focused documentation helps to prevent contributors from being overwhelmed by a massive codebase and shying away from contributing. Contributors need to feel like they can jump right in and start making tangible improvements without needing to understand every nook and cranny of your architecture and without constantly having their pull requests rejected.

All of this has only been possible because we already put in the hard work to move toward a modular monolith that is well-organized with separate functionality clearly isolated into different modules within the codebase. We plan to double down on this approach throughout 2023 and beyond by adding a marketplace of modules and more customization for Appsmith installations, in order to further foster OSS development within our project.

How this architecture affects everyone involved

As stewards of the Appsmith project, we think that the path we have chosen — combining the strengths of monoliths and microservices and centralized and decentralized development — will continue driving innovation in the industry. We believe that, through this model, we can get higher-quality tools into developers’ hands faster than we otherwise could.

Throughout the process of re-organizing our project, we’ve had to balance the interests of our end users, our open-source community, and Appsmith as a company. We needed a simple deployment process for our end users self-hosting Appsmith. We needed an organized codebase and efficient development process for our community contributors. And we needed a strong internal team that could solidify our architecture and infrastructure to ensure the long-term viability of the project. The modular monolith was the key to successfully balancing all of these interests and we believe that it will allow us to continue building great tools in the future.

We hope that this series has given you a better idea of where we stand on the monoliths vs. microservices debate, how we’ve implemented this in Appsmith’s architecture, and how our architecture allows us to better serve our contributors and users in the community. If this is a project that you’d like to be a part of, be sure to try out the free, cloud version of Appsmith!