New Intelligent Drag and Drop Experience for Appsmith’s User Interface Builder
Table of contents
Developers can quickly build any custom business software with pre-built UI widgets that connect to any data source on Appsmith. It’s a reliable and fast method to develop internal tools quickly. We created Appsmith to help developers save valuable time building complex applications for internal uses within their organizations. For this to work for everyone, we believe that the core parts of Appsmith need to run smoothly and should be continuously improved. Appsmith’s UI is built using React, Redux, Web Workers, Immer, among other things.
One of the key issues that users faced with Appsmith was that when they would drag the widgets onto the canvas, they would only get dragged in if there was enough space on the drop area. This was not a pleasant experience; it would involve dragging the widget onto some other free area on the canvas, re-designing the desired drop area, and then dragging it up. We realized that this was a significant flaw slowing down the UI building process. So we immediately fixed it.
In this blog, we’ve interviewed Appsmith engineers Ashok M and Rahul Ramesha to learn more about the process and challenges involved in solving this problem.
What was the issue with resizing and dragging widgets?
Simply put, the problem that we had was that whenever we would drag a new widget or any existing widget into a position where it would collide with any other widget, that kind of movement was often restricted. We did not have an option to auto-resize the widget dragged into a particular space. There was also no possibility for the existing widgets on the page to automatically move around to make space for a new widget. For anyone trying to create an application, this can be a frustrating experience because when users design things, they don’t necessarily do it in order. There are many instances where they might remember to add something later. We wanted the experience of making an app on Appsmith should be smooth and delightful.
Take a look at the screenshots below to see the previous experience:
When you try to resize a widget, and there’s a widget already in the path, a user would not be able to resize the new widget without explicitly moving the existing widgets out of the way.
In this image above, you can see that the ‘Container’ widget cannot be resized into the size shown in the image below without moving the ‘Checkbox’ widget.
These problems often arise when there is a real estate shortage on the canvas, when the widget's size falls short by a small size change or when the movement of other widgets is restricted within a particular space. For example, when placing a button between two existing buttons, the widget being dragged is one column size larger than the available space on the canvas. Generally speaking, users don't usually know how much resizing needs to be done to the existing widgets to make space for a new widget or dragging other previously dragged-in widgets.
What was user feedback around this issue?
Users often asked us to allow dropping widgets on top of each other (which some of the UI building products and most of the canvas building products provide) to deal with UI block collision checks. These checks ensure that no two widgets are overlapping and are fully/partially out of the main canvas. Going in this direction would have meant building layers and dealing with layers.
For some context, layers are actually Z-Index layers, which could have allowed for dropping widgets one on top of the other by adding a higher Z-index value. An example that comes to mind is Adobe Photoshop; Tooljet, Miro, Figma also allow layers in a way. For Appsmith, this kind of a solution isn’t ideal because one can often forget that there are widgets behind a widget in a lower Z-index layer, and adding more layers would mean more time for the dom to render and paint.
After a few internal discussions around this, we found that this would not be a scalable solution, and it would also make resizing, selecting, focusing widgets very difficult. We also want to develop the experience of building UI on Appsmith to be more intelligent.
Can you elaborate on this vision of enhancing UI building experience and the solution you created?
When we brainstormed this issue, we knew that the solution had to be scalable. It also had to be intelligent enough to auto-adjust according to the screen resolutions of different devices. We wanted to develop Reflow as a solution to this problem. Reflow is a process of technically deciding which widgets to move and resize in real-time to allow space for the dragging/resizing widget. Widget resizing allows the user to resize a widget while holding another widget to make space. This only works when the widget is cramped against a boundary on the canvas.
How did you go about developing the solution? What were some other approaches you had considered, and what were their limitations?
Conceptualizing and building this feature took less than time than expected. However, we spent time thinking about the right solution. We did this by trying out POCs of different solutions. We built three POCs to realize that reflowing while dragging would be an essential part of our solution. We then also had to consider the two behaviors of Reflow: Natural and Relative.
Natural:
While resizing a static widget, when colliding with a widget in a particular direction, the widgets reflow after cascading collision without maintaining any relative spacing
While dragging a static widget, the widgets reflow similar with cascading collisions. Even here, the dragging widget can be made to fit into any space.
Relative:
While resizing a static widget, when colliding with a widget in a particular direction, all the widgets in the path of collision of the colliding widget will be moved while maintaining relative spacing till the edge of the canvas. At the edge of the canvas, it reduces the relative spacing on further resizing a static widget.
While dragging a static widget, when colliding with a widget in a particular direction, all the widgets move as per the reflow algorithm, similar to resizing reflow. The direction of collision is critical while reflowing with dragging. The static widget itself can move other widgets that can help fit in between any space on the canvas.
We developed two more POCs to get feedback on which reflow was more user-friendly and likable. We understood that ‘Natural’ was more predictable, but both behaviors had their own merits. Finally, we built “Drag and Drop Experience” to resize widgets at the corners to allow space for the dragging widget, which seemed essential.
Can you explain your the new algorithm for the experience?
At its core, the algorithm’s behavior is to push all the widgets the dragged widget is colliding with. Let us explain what happens under the hood in more detail; consider the widget dragged on the canvas to be a ‘static’ widget. When this type of widget is dragged onto the canvas, we compare its coordinates with all other widgets on the canvas to check for overlapping collisions. The overlapping widgets are further put through the same process recursively. This helps create a tree structure of widgets, wherein a parent node will have overlapping widgets as children nodes and become parents for their overlapping widgets. With the help of this tree structure, the direction of the static widget, displacement of the static widget and canvas boundaries, X and Y movement values of each widget are calculated. When moved along the X and Y axis from their original position, these widgets will create the illusion of pushing the colliding widgets.
Here’s a link to the code where this algorithm is implemented.
This is the core logic of our algorithm, but there’s a lot more to this. For example, we are tweaking the direction of movement in corner cases, keeping track of multiple directions of widgets, smooth canvas exits, and entries, among a few more.
Can this algorithm be applied in other scenarios or projects?
So we will extend this project to the cut/copy/paste feature where you can paste a widget anywhere on the canvas, and the rest of the widgets will move away to make space for the copied widget. We will also be including it in the dynamic height project, where widgets like Table, List etc. can grow in height and push other widgets to the bottom. Another extension for this algorithm would be to push widgets around based on device resolutions, ie, develop position responsiveness of widgets.
Can you talk about the performance of your fix? What happens when there are hundreds of widgets on the canvas?
We tested it out with 100 widgets, and there was no problem with performance, but performance is expected to degrade with more and more widgets. We tested this out with our high-performance laptops by slowing down the CPU by 6x using Chrome’s CPUthrottle; there were minor lags but nothing that is unusable.
What is the roadmap of this particular feature? Are there any further enhancements and improvements that you’re planning to make?
We think that this is just the beginning! We’ve got some significant enhancements planned.
Multiple widget reflow (Major Enhancement):
Reflow widgets even when multiple widgets are moved together.
Locked widgets (Major Enhancement):
So container jumps(moving a widget from the main canvas into a container or vice versa) will be tricky and irritate some users because people might not want to move widgets from the carefully designed positions. So we will lock a widget not to allow it to resize or move from its position.
Dynamic resize limit (Minor Enhancement):
There is a resize limit for our widgets: 4 rows x 2 columns, and the same for all widgets. We can’t go below these dimensions. It doesn't make sense for widgets like a divider or sometimes button and checkbox, so we might try to get the minimum dimensions in real-time based on the widget it affects.
What was the most challenging part of building this feature?
Building this feature was quite challenging because there aren’t many readily available examples on the internet; and building this also meant enabling others to understand what was in our minds. We wrote close to 8000 lines of code. Still, we’ve pushed only 4500 lines into the repo because we have had to build two behaviors to understand the solution among internal stakeholders better.
We learned that there is no right way to build new experiences. Different solutions helped solve issues in different scenarios. We could always come up with a scenario that would cause the existing solution to fail. In the end, we had to choose a solution that catered to most of the scenarios and not all.
In the beginning, we built a solution to do one thing: to push colliding widgets in a direction and then add code to tackle one problem at a time. As the solution started to feel more and more refined, other problems surfaced. While trying to tackle a complex problem, identifying the core logic of the solution and adding to it one step at a time is critical in solving it.
Ashok M is a Frontend Engineer at Appsmith.
Rahul Ramesha is a Frontend Engineer at Appsmith.
We hope that you enjoyed reading this blog.
Appsmith is built in public, and we love sharing our knowledge with everyone. If you have a special request to know more about behind-the-scenes for specific features, write to me vishnupriya@appsmith.com.
