New Intelligent Drag and Drop Experience for Appsmith’s User Interface Builder
7
March
2022
Engineering

New Intelligent Drag and Drop Experience for Appsmith’s User Interface Builder

New Intelligent Drag and Drop Experience for Appsmith’s User Interface Builder
Vishnupriya Bhandaram
0
 minutes ↗
No items found.
Engineering

Developers can quickly build any custom business software with pre-built UI widgets that connect to any data source on Appsmith. It’s a reliable and fast method to develop internal tools quickly. We created Appsmith to help developers save valuable time building complex applications for internal uses within their organizations. For this to work for everyone, we believe that the core parts of Appsmith need to run smoothly and should be continuously improved.  Appsmith’s UI is built using React, Redux, Web Workers, Immer, among other things. 

One of the key issues that users faced with Appsmith was that when they would drag the widgets onto the canvas, they would only get dragged in if there was enough space on the drop area. This was not a pleasant experience; it would involve dragging the widget onto some other free area on the canvas, re-designing the desired drop area, and then dragging it up. We realized that this was a significant flaw slowing down the UI building process. So we immediately fixed it.  

Click here to view the issue


In this blog, we’ve interviewed Appsmith engineers Ashok M and Rahul Ramesha to learn more about the process and challenges involved in solving this problem. 

What was the issue with resizing and dragging widgets? 

Simply put, the problem that we had was that whenever we would drag a new widget or any existing widget into a position where it would collide with any other widget, that kind of movement was often restricted. We did not have an option to auto-resize the widget dragged into a particular space. There was also no possibility for the existing widgets on the page to automatically move around to make space for a new widget. For anyone trying to create an application, this can be a frustrating experience because when users design things, they don’t necessarily do it in order. There are many instances where they might remember to add something later. We wanted the experience of making an app on Appsmith should be smooth and delightful. 

Take a look at the screenshots below to see the previous experience:  


When you try to resize a widget, and there’s a widget already in the path, a user would not be able to resize the new widget without explicitly moving the existing widgets out of the way. 



In this image above, you can see that the ‘Container’ widget cannot be resized into the size shown in the image below without moving the ‘Checkbox’ widget.


These problems often arise when there is a real estate shortage on the canvas, when the widget's size falls short by a small size change or when the movement of other widgets is restricted within a particular space. For example, when placing a button between two existing buttons, the widget being dragged is one column size larger than the available space on the canvas. Generally speaking, users don't usually know how much resizing needs to be done to the existing widgets to make space for a new widget or dragging other previously dragged-in widgets.

What was user feedback around this issue? 

Users often asked us to allow dropping widgets on top of each other (which some of the UI building products and most of the canvas building products provide) to deal with UI block collision checks. These checks ensure that no two widgets are overlapping and are fully/partially out of the main canvas. Going in this direction would have meant building layers and dealing with layers. 

For some context, layers are actually Z-Index layers, which could have allowed for dropping widgets one on top of the other by adding a higher Z-index value. An example that comes to mind is Adobe Photoshop; Tooljet, Miro, Figma also allow layers in a way. For Appsmith, this kind of a solution isn’t ideal because one can often forget that there are widgets behind a widget in a lower Z-index layer, and adding more layers would mean more time for the dom to render and paint. 

After a few internal discussions around this, we found that this would not be a scalable solution, and it would also make resizing, selecting, focusing widgets very difficult. We also want to develop the experience of building UI on Appsmith to be more intelligent. 


Can you elaborate on this vision of enhancing UI building experience and the solution you created? 

When we brainstormed this issue, we knew that the solution had to be scalable. It also had to be intelligent enough to auto-adjust according to the screen resolutions of different devices. We wanted to develop Reflow as a solution to this problem. Reflow is a process of technically deciding which widgets to move and resize in real-time to allow space for the dragging/resizing widget. Widget resizing allows the user to resize a widget while holding another widget to make space. This only works when the widget is cramped against a boundary on the canvas.

How did you go about developing the solution? What were some other approaches you had considered, and what were their limitations? 

Conceptualizing and building this feature took less than time than expected. However, we spent time thinking about the right solution. We did this by trying out POCs of different solutions. We built three POCs to realize that reflowing while dragging would be an essential part of our solution. We then also had to consider the two behaviors of Reflow: Natural and Relative. 

Natural: 

  • While resizing a static widget, when colliding with a widget in a particular direction, the widgets reflow after cascading collision without maintaining any relative spacing
  • While dragging a static widget, the widgets reflow similar with cascading collisions. Even here, the dragging widget can be made to fit into any space.

Relative: 

  • While resizing a static widget, when colliding with a widget in a particular direction, all the widgets in the path of collision of the colliding widget will be moved while maintaining relative spacing till the edge of the canvas. At the edge of the canvas, it reduces the relative spacing on further resizing a static widget.
  • While dragging a static widget, when colliding with a widget in a particular direction, all the widgets move as per the reflow algorithm, similar to resizing reflow. The direction of collision is critical while reflowing with dragging. The static widget itself can move other widgets that can help fit in between any space on the canvas.

We developed two more POCs to get feedback on which reflow was more user-friendly and likable. We understood that ‘Natural’ was more predictable, but both behaviors had their own merits. Finally, we built “Drag and Drop Experience” to resize widgets at the corners to allow space for the dragging widget, which seemed essential. 


Can you explain your the new algorithm for the experience? 

At its core, the algorithm’s behavior is to push all the widgets the dragged widget is colliding with. Let us explain what happens under the hood in more detail; consider the widget dragged on the canvas to be a ‘static’ widget. When this type of widget is dragged onto the canvas, we compare its coordinates with all other widgets on the canvas to check for overlapping collisions. The overlapping widgets are further put through the same process recursively. This helps create a tree structure of widgets, wherein a parent node will have overlapping widgets as children nodes and become parents for their overlapping widgets. With the help of this tree structure, the direction of the static widget, displacement of the static widget and canvas boundaries, X and Y movement values of each widget are calculated. When moved along the X and Y axis from their original position, these widgets will create the illusion of pushing the colliding widgets.

Here’s a link to the code where this algorithm is implemented

This is the core logic of our algorithm, but there’s a lot more to this. For example, we are tweaking the direction of movement in corner cases, keeping track of multiple directions of widgets, smooth canvas exits, and entries, among a few more.


Can this algorithm be applied in other scenarios or projects?

So we will extend this project to the cut/copy/paste feature where you can paste a widget anywhere on the canvas, and the rest of the widgets will move away to make space for the copied widget. We will also be including it in the dynamic height project, where widgets like Table, List etc. can grow in height and push other widgets to the bottom. Another extension for this algorithm would be to push widgets around based on device resolutions, ie, develop position responsiveness of widgets.


Can you talk about the performance of your fix? What happens when there are hundreds of widgets on the canvas?

We tested it out with 100 widgets, and there was no problem with performance, but performance is expected to degrade with more and more widgets. We tested this out with our high-performance laptops by slowing down the CPU by 6x using Chrome’s CPUthrottle; there were minor lags but nothing that is unusable. 


What is the roadmap of this particular feature? Are there any further enhancements and improvements that you’re planning to make? 

We think that this is just the beginning! We’ve got some significant enhancements planned. 

  • Multiple widget reflow (Major Enhancement):

Reflow widgets even when multiple widgets are moved together.

  • Locked widgets (Major Enhancement):

So container jumps(moving a widget from the main canvas into a container or vice versa) will be tricky and irritate some users because people might not want to move widgets from the carefully designed positions. So we will lock a widget not to allow it to resize or move from its position.

  • Dynamic resize limit (Minor Enhancement): 

There is a resize limit for our widgets: 4 rows x 2 columns, and the same for all widgets. We can’t go below these dimensions. It doesn't make sense for widgets like a divider or sometimes button and checkbox, so we might try to get the minimum dimensions in real-time based on the widget it affects.

What was the most challenging part of building this feature? 

Building this feature was quite challenging because there aren’t many readily available examples on the internet; and building this also meant enabling others to understand what was in our minds. We wrote close to 8000 lines of code. Still, we’ve pushed only 4500 lines into the repo because we have had to build two behaviors to understand the solution among internal stakeholders better. 

We learned that there is no right way to build new experiences. Different solutions helped solve issues in different scenarios. We could always come up with a scenario that would cause the existing solution to fail. In the end, we had to choose a solution that catered to most of the scenarios and not all.

In the beginning, we built a solution to do one thing: to push colliding widgets in a direction and then add code to tackle one problem at a time. As the solution started to feel more and more refined, other problems surfaced. While trying to tackle a complex problem, identifying the core logic of the solution and adding to it one step at a time is critical in solving it.



Ashok M is a Frontend Engineer at Appsmith. 

Rahul Ramesha is a Frontend Engineer at Appsmith. 


We hope that you enjoyed reading this blog.

Appsmith is built in public, and we love sharing our knowledge with everyone. If you have a special request to know more about behind-the-scenes for specific features, write to me vishnupriya@appsmith.com


How 40 Lines of Code Improved Our React App’s Performance by 70%
14
February
2022
Engineering

How 40 Lines of Code Improved Our React App’s Performance by 70%

How 40 Lines of Code Improved Our React App’s Performance by 70%
Vishnupriya Bhandaram
0
 minutes ↗
#
engineering
#
engineering diaries
#
web-development
#
reactjs
#
analytics
Engineering

On Appsmith, developers can quickly build any custom business software with pre-built UI widgets that connect to any data source. These widgets can be controlled with JavaScript. We created this framework to help developers save on valuable time building complex applications for internal uses within their organizations. For this to work for everyone, we believe that the core parts of Appsmith need to run smoothly and should be continuously improved.  Appsmith’s UI is built using React, Redux, Web Workers, Immer among other things. 


In this blog, Sathish Gandham, a frontend engineer focusing on UI performance at Appsmith, will talk about how we improved the editing experience on Appsmith.

What is the Editing Experience on Appsmith? 

The editing experience on Appsmith involves writing bits of code to customize the functionality of the widgets and writing special commands and actions into the application. It is a crucial function in the Appsmith framework. 

Lag and Delay 

Building an application on Appsmith involves dragging and dropping widgets onto the canvas and writing custom code on the editor; however, we noticed that while editing text/code in the canvas, the editor would often freeze, resulting in a less than optimal user experience. When building an application, there should be no delay or lag. This was a huge issue that needed our immediate attention from an engineering perspective. For a natural typing experience, users want the keypress latency to be under 100ms, though 50ms would be ideal. 

To solve this problem, we needed to understand what happens when a user types. For this, we used: 

React profiler: This measures how often components in the application render and the “cost” of rendering. The profiler helps in identifying parts of an application that are slow. In our case, this allowed us to understand what components were rendered as we typed. 

Chrome Performance tools: This helped us quantify the problem, measure our progress, find the code taking longer to execute, and find unnecessary repaints. 

Please note that the React profiler is part of the React Developer tools browser add-on which is available for Chrome and Firefox.

From the React profiler, we see three pairs of long commits; each of these corresponds to the following property pane, and UI widget renders. Ideally, the property pane should render much faster than the canvas since there is nothing changing in the property pane except the input we are editing. Even the canvas should be rendering the widgets currently in use and not the rest. We realized that this was not the case and needed to be fixed. 

We profiled the property pane issue in isolation to identify what it takes to render it. For this, we used the performance tab in Chrome DevTools to see what happens when the property pane opens. This gives us some helpful information. 

  • ComponentDidMount of code editor is taking a lot of time 
  • Styles are also taking a long time to render
If you see the property pane commit in the screenshot above, you will notice that evaluatedValuePopup also takes significant time.


Here’s how we listed the tasks that lay ahead of us: 

  1. Identify as to why all the widgets were rendering when they don’t have to and fix it
  2. Optimize the code editor [Not apparent from the React profiles]
  3. Identify why all the controls in the property pane are rendering and fix it
  4. Optimize the global styles
  5. Optimize the evaluatedvalue pop-up

In this blog, I will talk about how we went about the first task. Before I get to that, here are a few tips for profiling: 

  • Try to split your problem into smaller pieces and profile them. With this, you won’t crowd your profile, and you can find the performance issues with ease. 

Example 1: To improve the editing experience, we just profiled a single keypress. 

Example 2: To profile a drag and drop action, we can split that into drag start, move, and drop.

  • Leave the application idle for 5 seconds after starting the profile and before stopping it. It will make it very easy to identify the work that has been done. [See A & C From profile above]
  • To measure the overall performance improvements, instead of measuring each optimization individually, it’s better to consider focussing on the overall scripting and time taken to render during an action. You can get this number from the chrome performance tab. [B & D from profile above]
  • In the React profiler, don’t just focus on the large commits. Go through each commit at least once, and see what’s getting rendered in that commit. The chances are that one or two small components are accounting for all those renders.

Here’s a short guide on reading the React profile: 

  • A: List of commits during our profile
  • B: The commit we are looking at
  • C: Details about the selected component (WidgetsEditor). Our widgets editor rendered three times during the profile at 6.1s, 8.6s, and 14.1s. 102ms, 328ms,83.1ms is the duration each commit took; it is not the total time the selected component took to render.
  • D: Zoomed in view on the component we selected and its children.

Here are the notes on the profile based on which we worked on improving the editing experience. You can download the attached profile and import it in your React profiler to follow along or just refer to the image above.

Please note that the React profiler is available only when you open a react app in dev mode in Chrome/Firefox, if you don’t have a local React development set up, you can use the standalone React developer tools to read the profile. 

Here are instructions on how to install it and start it: 

Install

# Yarn

yarn global add react-devtools

# NPM

npm install -g react-devtools

Run

react-devtools

Follow this link to read the detailed notes from the profile we did to improve the editing experience on Appsmith. 

I’ve put some notes here for your reference: 

1. Evaluated value opening. Not related to editing.

2. Widgets editor, not sure why.

3. Editor focused. We should be able to avoid the rest of the property pane from rendering.

4. Small changes to property pane, its header, lightning menu, and action creator. Nothing changes for them, so they should not be re-rendering. Memoization can help here.

5. Same as above. 

6. We get the evaluated value back. Entire widgets editor is re-rendered (Deduced this from one of the two updates to table), we can optimise this

- If each widget subscribes to its data independently, we should be able to avoid the unnecessary renders of the widgets by

- Doing a deep check at the widget level

- update the store with only values that changed. 

7. PropertyPane is rendered with the updated value. EvaluatedValue taking most of the time.

8. From 8 to 17, these are commits like 4 & 5 above. 

9. 18 & 19 are widgets editor and property pane. I don’t see why these are required. I will look into it. 



Widgets Render When Not Needed


One of the most loved features of Appsmith is reactive updates. With reactive updates, you can see the widget change and show data. With traditional programming, you would have to reload the page in order to see the update in the widget. This is achieved by updating the data tree as and when you change something on the canvas and using the updated data tree to re-render the app. Due to the amount of data we have and the number of calculations we need to do, it took a long time and blocked the main thread.

To solve this problem, we moved the evaluations to a web worker freeing the main thread. A brilliant move to solve the problem at hand, but this created a new issue. The problem here was due to object reference changing. Since the data tree is coming from the worker, we would always get a new reference for every item in the tree even though only some of them changed. This reference change was making all the widgets re-render unnecessarily.


A few approaches we tried to solve this problem were:

  1. Get what keys changed from the worker (worker has this information) and update only those values in the reducer. This did not work because the list of keys was not complete. 
  2. Compute the diffs between the current data tree and the one received from the worker and update only what changed. Though this prevented the renders, we did not improve the overall scripting time we measured earlier. The reason is, computing the diffs itself took a lot of time, which would happen twice for each change.


Web Worker to the Rescue 


We moved the task of computing the diffs to the worker and used the deep-diff library to compute the diffs and let immer take care of immutability.

This helped us in two ways:

  1. Offloaded the expensive task of computing the diffs on the main thread.
  2. Reduced the size of the data we transfer between worker and the main thread (this was never a bottleneck for us, though).


This change alone brought down the keypress latency by half.


Instead of replacing the entire data tree from the worker, we get the only changes (updates) and apply them to the current state. applyChanges is a utility method from deep-diff. Immer takes care of the immutability.


If there’s anything to be said about performance improvement, it’s this, don’t take performance for granted and profile your code on a regular basis. Even a few lines of change or configuration can lead to a great performance gain. 



I hope you found this blog helpful. If you’d like to get in touch with Satish, ping him on Discord or visit his website.

Hacktoberfest 2021: Let the Contributions Begin!
27
September
2021
Announcement

Hacktoberfest 2021: Let the Contributions Begin!

Hacktoberfest 2021: Let the Contributions Begin!
Vishnupriya Bhandaram
0
 minutes ↗
#
hacktoberfest
#
announcement
#
developer
#
engineering
Announcement

Time indeed does fly because it feels just like yesterday when we hosted our first ever Hacktoberfest last year! And what a year it has been.

We started Appsmith because we wanted to create a developer-first product to make an engineer’s life easy. From being a small, little-known project on Github, we now have 5K stars. This wouldn’t have been possible without our community; apart from giving us incredible feedback, they also contributed immensely to our codebase, from contributing new integrations to helping solve bugs. We now have over 100 contributors, and we want to expand our community (that means you!), empowering them to contribute to a project that will always remain developer first.

And so, we’re back again this year for an exciting month-long virtual festival supporting open-source projects. We love it because it’s celebratory and community-oriented! At Appsmith, we genuinely believe open-source can take the way we build things to new heights. And our commitment to the community will never waver.

In that spirit, we are kicking off the Hacktoberfest 2021. In the run-up to the actual code-fest, there are going to be many talks! We will be bringing together folks from other awesome projects to talk about Hacktoberfest and Open Source. Be sure to check these out. You will definitely get to learn more about how to get involved in open-source projects. And yes! We’re ready to swag it out, too, for every valid PR! And contributors who submit more than 4 PRs and invite others to contribute to our repository will get a special prize. You are welcome to contribute to the issues on Appsmith’s repository on Github. Be sure to check contribution guidelines, review, and merge policies. Please note that not all pull requests will count towards Hacktoberfest 2021.

Contributing to open-source projects is an excellent way for users to become contributors. And it’s both a fun and rewarding experience. Sure, it can be scary to begin, and many think that only code that implements significant features matters. Well, I am here to tell you that it’s not true. We believe that no contribution is too small. Bug fixes, documentation updates, submitting articles are all fantastic ways of participating.

Hacktoberfest is open to all: Students, developers, anyone really. Background no bar. To this end, this year, we’ve included several newbie-friendly issues in the challenge to give a warm welcome to first-timers.

If you’ve still got some doubts about participating or need some queries answered, head over to our Hacktoberfest channel on Discord, and we will be happy to help you out!  To know more about how you can participate in this event, take a look at our Contributor Guidelines and Code of Conduct.

Happy Hacking! 🐱‍💻

Launching the Appsmith Writers Program!
22
September
2021
Announcement

Launching the Appsmith Writers Program!

Launching the Appsmith Writers Program!
Vishnupriya Bhandaram
0
 minutes ↗
#
writers program
#
announcement
Announcement
Editor's Note: We have paused the form for new applications for the writer's program. If you wish to contribute, please send an email to writersprogram@appsmith.com along with a brief outline of the topic you wish to write on, links to two writing samples, and mention your areas of interest/expertise. Please note that we receive a high volume of emails, do allow us some time to get back to you. Be assured that we will do our best to get back to you in a timely manner.

Are you an engineer who enjoys creating content and sharing your knowledge with the world? We'd love to talk to you!

We've been creating a lot of content at Appsmith over the last few months for our growing audience of developers to educate them and share our knowledge with them. You can check it out on our Youtube or our blog. As an open-source project, we don't just want to share our codebase with our community but also our collective knowledge.

What is the Appsmith Writers Program?

At Appsmith, we want to work with engineers who love writing or creating videos. Whether you're a veteran of the trade or a young student just starting out, we'd love to hear from you.

And of course, pay you for the work you do.

Topics that excite us are internal tools, low code, open-source, databases, application performance, engineering best practices, Javascript. And needless to say, newer ways to use Appsmith.

All that we care about is whether you can write in-depth and in detail. We're looking for long-form pieces which help educate our community (1500 words and up).

We don't care about your style and grammar. We've got a great editorial team to help with that.

Is it paid?

Yes! We pay anywhere between $200-$400 for content created (video, templates, blog posts, tutorials).

What kind of content should I send in?**

Below are the types of content we're most excited about:

Blog Post / Templates

If you want to write an engineering post that could be relevant for Appsmith's audience of backend developers but not directly about Appsmith (for example, this), we'd be very excited to discuss those too.

If your proposal is accepted, we’ll feature it on our website with your name and share it with our rapidly growing open-source community.

Video

These videos can fall into different categories:

  • A Tutorial where you show how you built something using Appsmith.
  • A How-to guide explaining some of Appsmith's features.

If you want to have an ongoing engagement with Appsmith, where you want to create Youtube videos on broader engineering topics (not directly related to Appsmith) on the Appsmith Youtube channel, please do reach out as well; we can figure something out.

Great, how do I get started?

The best way is to fill our form to share an outline, and we can take it from there. (Note: We're going to only consider previously unpublished content)

If you're keen on exploring this opportunity but don't have a clear idea of what you'd like to write, we're happy to help! Just mention this in the form, and we will get back to you ASAP!

Engineering Diaries: How We Created a Google Sheets Integration for our Product
15
September
2021
Engineering

Engineering Diaries: How We Created a Google Sheets Integration for our Product

Engineering Diaries: How We Created a Google Sheets Integration for our Product
Vishnupriya Bhandaram
0
 minutes ↗
#
product
#
integrations
#
engineering diaries
Engineering

Everything starts with a spreadsheet. Well, almost everything. Spreadsheets are the backbone of all business operations, whether budgeting, people management, expense management, organizing lists, etc. Spreadsheets often become the first choice for fledgling businesses, mainly owing to their versatility and flexibility. Little goes a long way with spreadsheets.

For a startup, smart utilization of available funds is critical, and Google Sheets often comes in handy to plan projects, analyze risks, report metrics, generate quotes and predict financial outcomes. Start-ups even use spreadsheets to keep track of client lists, investor lists, and more. So what’s the problem? Things can get messy once you work on it collaboratively or establish strict flows around maintaining a spreadsheet database. Data can get corrupted, and not knowing the latest version, too many changes by too many people who have access to the database and no admin control.

This is where the power of low-code can be melded with the power of spreadsheets. Turning an excel sheet into a web application is a great way to contain errors due to poor data management, allow for granular control, admin and user access, and these applications scale along with your business.

Today, it’s pretty easy to make an application from Google Sheets in record time. There are many low code and no-code tools out there that can help you do this, including Appsmith. In this blog, we will talk a bit about the Google Sheets integration on Appsmith, how we built the integration and all the things you can do with it.

Behind the Scenes

One of our colleagues, Nidhi Nair, worked on making this integration a possibility. Nidhi is a platform engineer at Appsmith and she joined Appsmith a little less than a year ago, and she enjoys the creative liberty to explore her ideas at Appsmith.

“It was possible to use Google Sheets on Appsmith even without our integration. Users could do this using the REST API plugin. However, it’s not the most convenient, and I found it to be unwieldy and something that every user couldn’t configure intuitively or easily,” says Nidhi.

The Google Sheets integration was created to simplify the interaction for end users. “We identified a set of actions that users would want to use Google Sheets for and optimized the way they interact with data in their sheets,” adds Nidhi. This meant not having to deal with cells and columns, but just arrays. “We defined the scope of the integration to be similar to that of a database. A single sheet was understood to be a table that we wanted to manipulate with the integration. We identified the relevant actions for this,” she says. Implementing this integration meant doing considerable research around how users interact with sheets. For us, reducing the friction for users was a key priority. “We introduced something called RowObjects in the integrations that makes sure that users don’t necessarily have to tinker with the data themselves,” says Nidhi.

Key Challenges

“We wanted to be able to support the DB integration style interaction and also allow users that want to work on it as an excel sheet to be able to continue to do so. This was a relatively easy solution because of how flexible our logic for these integrations is,” says Nidhi, adding that a user could say ‘get me rows 1-10’ and for the next page, ‘get me rows 11-20.’ But they can also do something like: ‘Get me cells D3:J8, and on the next page, get me D11:J16’ (or whatever other logic they would like to use). While this may sound trivial, having the liberty to navigate across the sheet at will means that they can organize their data separately from how it is consumed in Appsmith.

The biggest challenge in creating this integration was to use Appsmith credentials as a provider for all instances to make it easy. With this, users don't have to set up configuration on Google; Appsmith has already done that for them. Setting up the configuration on Google comes with painful scrutiny, and it’s not for everyone, especially for people who do not deal with tech. “Appsmith’s one-click approval makes it easier,” says Nidhi.

However, this has a downside; Google Sheets on self-hosted instances cannot be used unless they connect to Appsmith’s cloud API.

The engineering team is also working on and hopes to solve shortly for storing authentication on a per-user basis. This will allow users to access the part of the sheets they have access to and limit access to those they don't.

To read more about the roadmap for features, follow this link.

How to Use Google Sheets Integration on Appsmith

With Appsmith's inbuilt Google Sheet Integration Plugin, you can use any Google Sheet as a data source or a backend to build robust applications.

Set-up Google Sheets Plugin

  1. Create a new account on Appsmith (it’s free!), if you are already an existing login to your Appsmith account.
  2. Create a new application by clicking on the Create New button under the Appsmith dashboard.
  3. We’ll now see a new Appsmith app with an empty canvas and a sidebar with Widgets, APIs, and DB Queries.
  4. Click on the + icon next to the APIs section and choose the Google Sheets option. Next, click on the New Datasource button, set the scope to Read and Write, and click Authorise.
  5. This will ask us to log in from our Google Account, choose the account we want to access Google Sheets, and log in. After successful authorization, this will redirect back to your Appsmith account.
  6. Now, you’ll find your Google Sheets Datasource under your APIs, and you can create the necessary queries by choosing this data source.

Excellent; now that you’ve completed the set-up, follow the instructions in our docs and get started on your app!

Learn How To Make An App With Google Sheets

Are you interested in learning more about our engineering processes? Follow us on Twitter and Linkedin to stay tuned!

Write to me, vishnupriya@appsmith.com, and I’d love to get to know what you’re building with Appsmith!

Build a GitHub Dashboard and Track Any Open Source Project
10
August
2021

Build a GitHub Dashboard and Track Any Open Source Project

Build a GitHub Dashboard and Track Any Open Source Project
Vishnupriya Bhandaram
0
 minutes ↗
#
github
#
open-source
#
app-development
#
ui
#
javascript

Did you know that Appsmith is a fully remote distributed organization? We have colleagues in 5 countries! This makes total sense for us as an open-source platform with contributors from over 100 countries (take a look at our activity on Github). It’s also why we think of ourselves as learners; we read, talk to people, and love feedback. And in that spirit, every new team member makes an app on Appsmith in their first week upon joining us! And there is only one rule, that there are no rules! These apps range from functional to fun to downright silly!

This week, we’re featuring our colleague, Favour Ohanekwu’s app. Favour is a front-end engineer from Ibadan in Nigeria. Over the years, he has been actively involved in building web applications across several industries ranging from health, agriculture, and IoT. You can follow his work here.

First steps

Favour toyed around with a few ideas before landing on a final app idea. Some of these app ideas were:

  • An application that allows employees to review their team members and view reviews about them from their fellow team members.
  • An application that shows the current availability status of each employee at any time of the day. On this application, employees will be able to set their status (Active, Busy, Away) and be able to see the current working status of other team members.
  • A dashboard that reports the company's GitHub repositories and compares them in terms of several metrics.

Favour chose to work on the third idea to build a Github Dashboard to view and track several projects, using Github’s REST API.

Click here to see the app!

We’ll outline the steps involved in making this app, and you’ll see how easy it is to build this application in few simple steps using Appsmith!

Getting Started

This short tutorial uses the cloud version of Appsmith. However, you can always export and import Appsmith apps to different environments (cloud, self-hosted, local). The recommended way to use Appsmith is to use Docker; follow the documentation here for detailed instructions if you want to build locally.

  1. Create a new account or sign in with an existing account, redirecting it to our dashboard.
  2. Click on Create New under your organization, and this will create a new Appsmith application.
  3. Rename the application to GitHub Organisation Dashboard (or any name you’d like) by simply double-clicking on the existing one.
  4. On the left, find the entity explorer; this is where you can manage all the widgets and data sources of the entire application.

Now you can build a simple UI with Appsmith widgets to build the dashboard and display all metrics crucial to your project!

Fetching Data from Github Organisation

Now that you’ve configured the app, you will need to configure a data source that will fetch all the repos from a Github organization. To do this, follow the steps below:

  • Click on the + icon next to the Widgets, find the input widget, and drag and drop it on canvas. The idea is to enter the organization name and dynamically fetch all the repos created under that organization.
  • Next, click on the + icon next to Data Sources, and create a new API as a data source. Rename the API as get_organization_repos and paste the following in the URL:
https://api.github.com/orgs/{{Input1.text}}/repos
  • Here as you can see, we’re dynamically passing the organization name to the GitHub API from the input widget.
  • Go back to the canvas, and drag and drop a new Select widget; now paste the following snippet to pollute the options with repos in the select widget from the API.
    {{
    get_organization_repos.data.map((repo)=>     ({
        "label":repo.name,
        "value": repo.name
    }))
    }}
  • Drag and drop a new button widget and set the onClick action to Execute a Query and select the get_organization_repos
  • With this, whenever you enter a new organization name, the select widget automatically populates it with all the repos from that organization.

Additionally, you can drag text widgets and add some context regarding the same for the dashboard. Here’s how it looks:

Screenshot 2021-08-10 at 11.10.52 PM.png

Fhere

Fetching Data and Events of a Particular Repository

Displaying Core Metrics

Now that the repositories are listed on the application, you can fetch the data of a single repository and display all the crucial information such as network count, forks, subscribers, and a few more.

  • Click on the + icon next to the Data Sources and add a new API and set the URL to the following:

https://api.github.com/repos/{{Input1.text}}/{{home_select_repo.selectedOptionValue
  • This URL will fetch the repo details; we’ll name it as get_organization_repo_details
  • Similarly, add one more API data source and name it as get_repo_events with the following URL:
https://api.github.com/repos/{{Input1.text}}/{{home_select_repo.selectedOptionValue}}/events
  • For both of these data sources, we send the repo name from the Select widget, which is named as home_select_repo, and the selectedOptionValue property will help access the selected option from it. You need to create a UI for displaying all the core repositories based on the selected option. For this, drag and drop a few text widgets onto the canvas, and set the Value property to the following:
TextWidget1 Value: {{get_organization_repo_details.data.network_count}}
TextWidget2 Value: {{get_organization_repo_details.data.forks_count}}
TextWidget3 Value: {{Get_organization_repo_details.data.subscribers_count || "-"}}
TextWidget4 Value: {{Get_organization_repo_details.data.open_issues_count }}
TextWidget5 Value: {{Get_organization_repo_details.data.watchers_count}}

Additionally, you can add more text widgets and a container widget to organize this more beautifully.

Here’s how Favour did it:

Screenshot 2021-08-10 at 11.11.47 PM.png

Displaying Repository Events

In this section, you can display a table widget to show all the events for a repository. In this way, you can monitor who is creating pull requests, pushing new changes, etc.

Follow the steps below to do this:

  • Drag and drop a new table widget onto the canvas.
  • Open the property pane of the table widget, and set the following in the Table Data property.
{{
        get_repo_events.data.map(event => ({
            "Date": moment(event.created_at).format("LLL"),
            "Type": event.type.match(/[A-Z][a-z]+/g).join(" "),
            "Actor": event.actor.login,
            "Avatar": event.actor.avatar_url,
            "View": event.actor.url
        }))
   }}

With this code snippet, you can display all the data from the get_repo_events API and bind it on the table widget. Following is how the output looks like:

Screenshot 2021-08-10 at 11.13.25 PM.png

Displaying Statistics

Last, you can add a couple of charts that will help you visualize the number of repository forks and the number of watchers for the repository by following the below steps:

  • Drag and drop a chart widget onto the canvas.
  • Set the title to Number of Repository Forks
  • Choose the Pie Chart under the Chart Type property.
  • Now create a new series and set the title as Repository Forks and add the following under `Series Data
{{
Get_organization_repos.data.map(repo=>({
  "x": repo.name,
  "y": repo.watchers
}))
}}
  • Similarly, add more chart widgets, set the chart type to Bar Chart, and set the Series Data to the following:
{{
Get_organization_repos.data.map(repo=>({
  "x": repo.name,
  "y": repo.watchers
}))

Following is a screenshot:

Screenshot 2021-08-10 at 11.14.50 PM.png

Now, wasn’t that easy to do with Appsmith?

Have you made something using Appsmith? Write to me (vishnupriya@appsmith.com), and I would love to feature you on our blog!

If you’re interested in building an app on Appsmith, sign up today. We have docs, and tutorials and live help, and a vibrant discord community to help you along the way. So go ahead, put your ideas out there!

Do you want to join Appsmith? We’ve got a bunch of openings; take a look here and apply!

Don’t forget to join our community and sign up for live events.

Build Tools for Your Fitness Start-up
27
August
2021
Tutorial

Build Tools for Your Fitness Start-up

Build Tools for Your Fitness Start-up
Vishnupriya Bhandaram
0
 minutes ↗
#
open-source
#
developer
#
development
#
developer-tools
Tutorial

This week, we’re back to feature apps made by Appsmith’s newest team members! Each new team member has to make an app on Appsmith as part of the hazing welcoming ritual! 😬 We’re a fully remote distributed organization with colleagues in more than five countries, and this is a fun and educational way of onboarding new members onto the team. And of course, it makes sense for people working on making Appsmith better to use it, understand it, and know the framework’s shortcomings. These apps made during the onboarding process can range from functional to fun or straight-up silly!

This week, we’re featuring our colleague Ashit Rath’s app. Ashit is a Sr. Frontend Engineer from Bhubaneswar in India. He has experience in building scalable web applications, and he loves to travel. You can follow his work here.

Ashit decided to make a simple food nutrition app to list items, and it would show you the nutrition details. Ashit’s recipe analyzer is an excellent example of the range of applications you can make on Appsmith.

During the app-making process, Ashit found the following things great about Appsmith:

  • Composing UI widgets was very intuitive
  • Adding API data source was super easy
  • Good performance overall

In the next part of the blog, Ashit has listed the steps to build the recipe analyzer.

Getting Started

This short tutorial uses the cloud version of Appsmith. However, you can always export and import Appsmith apps to different environments (cloud, self-hosted, local). The recommended way to use Appsmith is via Docker; follow the documentation here for detailed instructions if you want to build locally.

  • Create a new account or sign in with an existing account, redirecting it to our dashboard.
  • Click on Create New under your organization, and this will create a new Appsmith application.
  • Rename the application to GitHub Organisation Dashboard (or any name you’d like) by simply double-clicking on the existing one.
  • On the left, find the entity explorer; this is where you can manage all the widgets and data sources of the entire application.

There are three parts to building this Recipe Analyzer app:

  1. API for nutritional data
  2. Connecting the form to get the data from API
  3. Crunching numbers to show the nutrition

API for nutritional data

‍ The API that we have used here comes from Edamam. They have a robust API for any kind of food/recipe/ingredient level search.

Sign up for their Nutrition Analysis API as a Developer account to try out without a subscription. The Developer account might give out less information than it should, but it should be fine for our use case.

Once signed up, we need to generate the API key from here.

  1. Click the "Create New Application",
  2. Select "Nutrition Analysis API"
  3. Fill out the form and hit "Create Application"
  4. Once the application has been created; you would be greeted with the application API details page. Keep "Application Keys" and "Application ID" in place. We would need it in the next step.

Connecting the form to get the data from API

‍ We have the API key from the previous step, so now we will use that to make API calls to Edamam for our searches.

Now head over to Appsmith to create a data source.

  1. Create a new Datasource by clicking the + button on the sidebar.
  2. Click "Create new" for a new data source
  3. Click "Create new API"
  4. Change the API Name to nutrition_api, it would be "Api1" by default.
  5. Click on the "GET", a dropdown should open up and select "POST".
  6. Add the following to the URL bar; replace with the "Application ID" and with the "Application Keys" that we got from the previous step

https://api.edamam.com/api/nutrition-details?app_id=<application id>&app_key=<application key>

This should create our data source for fetching the nutrition data.

Let's create a new input and connect it to the data source to fetch.

  1. Add a new text widget and name it as FoodInput
  2. Resize the text widget according to need and add the following to the Placeholder property
1 cup rice

1/2 cup dal

100gm chicken
  1. Now add a new button widget; on clicking this, we need to trigger an API request (to the API we added in the previous section).
  2. Open the button widget properties and scroll down to Actions section; there would be an onClick property. Click the JS and a blank text box should open up
  3. Add the following lines to the text box opened in the previous step
{{

(function () {

        if (FoodInput.text.trim()) {

            const foodInputArray = FoodInput.text.split("\\n")

            storeValue('foodInputArray', foodInputArray)

            const onSuccess = () => {}

            const onFailure = (response) => {

                showAlert('Invalid quantity or name.', 'warning')

            }

            nutrition_api.run(onSuccess, onFailure)

        } else {

            showAlert('Please enter something to analyze', 'warning')

        }

})()

}}

This code basically takes the FoodInput text, modifies into proper API request format (array of string), stores the value in global storage using (setValue) and triggers the nutrition_api (nutrition_api.run)

That's it; we have connected our API and our form to send requests and get our awesome nutrition data!.

Crunching numbers to show nutrition values

Now we create the part where we display the data from Edamam and show it in a neat format.

All of the data points are calculated in a similar fashion, so we will demonstrate the only one just to get the idea behind it.

Let's consider Total Fat, the value for it can be derived by having the following code in the text property.

{{(function() {

    if (!nutrition_api.data.totalNutrients?.FAT?.quantity) return "-";



    return parseFloat(nutrition_api.data.totalNutrients.FAT.quantity).toFixed(2) + nutrition_api.data.totalNutrients.FAT.unit

})()}}

This first checks if totalNutrients.FAT.quantity exists or not; if not, then we display "-.” If it exists, then parse the value and append the unit to it

So "10.12520" becomes 10.12 gm (gm comes from nutrition_api.data.totalNutrients.FAT.unit)

Similarly, the Total Fat Daily Percentage can be displayed using the same logic.

{{(function() {

    if (!Api1.data?.totalDaily?.FAT?.quantity) return "-";



    return parseInt(Api1.data.totalDaily.FAT.quantity) + "%"

})()}}

We build the whole UI by adding similar code but changing the key from FAT to whatever macro/micronutrient is required to be shown.

Wasn’t that simple?

This recipe analyzer can be used as part of the many tools a fitness company/studio can give its members. We took the recipe analyzer a few steps further and envisioned the various other things to help a fledgling fitness studio tech up.

See the screenshots below to get a better idea:

Screenshot 2021-08-25 at 1.37.50 PM.png
You can make a fully automated system to keep track of progress for clients, with our modal widget, you can automate payment collections etc. Just connect your data source to get started!
Screenshot 2021-08-25 at 1.38.52 PM.png
Connect your database from Airtable or Google Sheets and start building!

For a detailed tutorial on how to build dashboards and admin panels, follow this link.


Have you made something using Appsmith? Write to me (vishnupriya@appsmith.com), and I would love to feature you on our blog! If you’re interested in building an app on Appsmith, sign up today . We have docs, tutorials, and live help on our vibrant Discord community to help you along the way. So go ahead, put your ideas out there!

How We Made Connecting Data to Widgets Much Easier
6
August
2021
Engineering

How We Made Connecting Data to Widgets Much Easier

How We Made Connecting Data to Widgets Much Easier
Vishnupriya Bhandaram
0
 minutes ↗
#
open-source
#
ui
#
ux
#
navigation
Engineering

What do we want our users to do on Appsmith? The obvious answer is that we want them to build fantastic internal apps for their organizations super fast. But what we're doing solves a more significant and perhaps intangible problem: avoiding repetitive and tedious grunt work. Developers don't want to do this, and we want to enable them to get to solutions faster. For this, Appsmith needs to be smarter and better too. The first step is to have a smooth onboarding experience — it's good for our users and great for us!

However, we noticed that users couldn't easily find critical elements essential to understanding how Appsmith works.

image.png

Our data connectors were hidden, and there were no obvious means of accessing them. There were a few more glaring pain points; for example, we felt that our widgets with pre-filled data, discouraged users from playing around with the platform. And even if our users were landing on the queries section, the flow was confusing and switching between data queries and widgets was non-intuitive. The product was not hitting the intended direction; to address all this confusion and more, we changed the navigation experience.

Simply put, our overarching goal with this update was to get people to connect their data to the UI.

Widget ➡️ Datasources ➡️ Querying Data ➡️ Connecting Widget With this flow in mind, we made several changes to the navigation experience.

Users can now:

  • Connect data sources faster
  • Find the right widget for available data
  • See relationships between UI and Data clearly

In this blog, we will talk about Appsmith's new navigation experience and how our design and platform pods went about the production process. Hopefully, you'll also get an inside look into the collaborative engineering and design practices here at Appsmith!

Merging APIs and DB Queries under Datasources

When a user is in an exploratory stage, what do they do first? Build UI or connect data sources? For us, it was a little bit of a chicken and an egg situation. We wanted to limit the fields to not overwhelm users with too many options and manage the flow into smaller edible nuggets.

Having APIs as a separate entity was counterintuitive. Merging it under data sources makes it easier for discovery and helps the user understand that all data comes from a data source — whether it's an API or a database query.

Along with this, we've also added subtle nudges and prompts such as this:

image.png

This reinforces the importance of connecting data sources to get apps working. ‍

The Right Widget for Your Data ‍

If you're confused about finding the best way to represent available data (a chart or a table?), the Appsmith platform can now predict the best widget based on your data.

image.png

This feature helps users narrow down options based on the type of data they have, speeding up the process of building an app. Earlier, users had to add the data source, write the query, go back to the canvas, scout for the right widget, (deep breath), drop it, and then bind it. With the new and improved flow, users can add data sources, write queries, select recommended widgets. Appsmith does everything else!‍

Clarity on Entity Relationships‍

Once you're inside a widget, it's essential to see how your data is linked to each other. We realized that when there are multiple data sources and queries on your application, sometimes it might be hard to navigate between data sources. So, we’ve added a way to be able to see entity relationships more clearly. With the upgraded new navigation experience, all the defined queries are listed on the widgets under incoming entities; you can directly choose (what) without writing JS bindings. When the widget performs any actions, for example, when a button is clicked, or a row is selected, these queries can now be seen under the outgoing entities section.

image.png

Make things more obvious

If there's one thing we've come to believe in after spending almost two months on this update, it's that keeping things simple tends to go a long way. This reflects even in the copy

Bind Data — Connect Data

APIs and DB Queries — Datasources

We introduced slash commands as a quicker and simpler way to connect widgets to a data source. Now you can trigger commands by typing "/" and use it anywhere you're writing Javascript (within moustache bindings) on Appsmith.

image.png

Slash commands also give a heads up for developers to start from somewhere. They become the way to initiate writing custom code and our auto-complete feature speeds up the process!

Our efforts with this new navigation experience were to make things simple and enable our users to find and understand features easily. As an open-source organization, we will always be a work in progress powered by our incredible community. We will strive to keep improving Appsmith!

‍If this new navigation experience helped you do let us know! As always, we would love to learn more about you and what you're building using Appsmith; please feel free to email me (vishnupriya@appsmith.com).

Become a Betasmith
Have a say in shaping Appsmith's future!‍

Join our community‍ Contribute or generally hang out with us!‍

Sign up for Events‍ Stay up to date on new features, live demos, etc.

Build a Custom Application using Slack API
10
August
2021
Tutorial

Build a Custom Application using Slack API

Build a Custom Application using Slack API
Vishnupriya Bhandaram
0
 minutes ↗
#
open-source
#
applications
#
slack
#
app-development
#
javascript
Tutorial

I recently joined Appsmith, and I am one of the few non-technical employees here! And as part of the onboarding process, all new team members have to build an app on Appsmith! It doesn’t matter if you’ve studied journalism and anthropology, have no experience in coding or JavaScript, no excuses! The brief was that it could be fun or functional, and anything under the Sun! Anxiety and panic were having a field day while I thought about what to build and where to start and finally decided on making something fun yet functional for Appsmith.

We have a vibrant channel on Slack called the #reading-club, where channel members can share and discuss books they have read. I expect (and hope) that this channel will have a high volume of messages, and the members are likely to grow. Even though there are only 26 members in the channel right now, one has to scroll up to get to earlier recommendations! It can be pretty frustrating to do all that when you’re not checking the channel every day, and it can be confusing to go through different recommendations by different members. The channel is a great way to share your thoughts, but there is no way of organizing those by name. To this end, I created the Appsmith Reading Club App, and here’s what I hoped it would do:

  • A way for non-members to also keep track of book recommendations
  • Be a one-stop-shop to see what Appsmith colleagues are reading without the clutter of the group chat

To build the app, here’s what I did:

Building a Rad UI

As a 'visual person,' and I feel most comfortable mapping out the elements and figuring out a flow. It helps me get a clear idea about what each component looks like and sets the scope.

I wanted one part of the app to be a list of names of the channel members, a button that could filter out their messages and display them in chronological order. Here are the widgets I used to build my UI. It took me under 10 minutes to put this together.

  • A table widget
  • A list widget
  • Text widgets
  • Image widgets

It's the next steps which are unfamiliar for a non-technical user like me: connecting a data source, writing code, and ensuring my app works! For this part, I enlisted my colleague, Vihar's help.

‍Here's a short tutorial!

‍Connecting Slack API‍

To pull information from a channel on Slack you need to work with the Slack API.

You have to then create an app and then generate OAuth tokens; to be able to do this, you need admin access to your organization’s Slack.

‍Here’s what our oauth configuration looks like:

oauth_config:
 scopes:
    user:
      - channels:history
      - channels:read
      - files:read
      - identify
      - users.profile:read
      - users:read
      - identity.basic

Next up, you have to link this configuration to your Appsmith application. This short tutorial uses the cloud version of Appsmith. However, you can always export and import Appsmith apps to different environments (cloud, self-hosted, local). The recommended way to use Appsmith is to use Docker; follow the documentation here for detailed instructions if you want to build locally.

In the next step, you will connect to the Slack API that will return all the users from the entire organisation. Since this is a secured API, we need to use the bearer token from the slack developer application to use it.

Follow the steps listed below:

  1. Click on the + icon next to the Data Sources.
  2. Select Authenticated API under the APIs section.
  3. Name the Query as completeList
  4. Use the following endpoint to fetch all the users from the slack organization: https://slack.com/api/users.list.
  5. Set the Authentication type to bearer token and copy the key here from the slack app. Hit RUN on the top right to execute the query.
image.png

Now in the response pane, you should be able to see all the users from the Slack organization. However, since our goal is only to fetch the users from a particular slack channel; in my case, this was #reading-club. For this, you have to write one more API that fetches details of a specific slack channel.

To do this, follow these steps:

  1. Click on the + icon next to the Data Sources.
  2. Select Authenticated API under the APIs section.
  3. Name the Query as filteredIds
  4. Use the following endpoint to fetch all the users from the slack organization: https://slack.com/api/conversations.members?channel=C01H8CPUVUK&limit=100.
  5. Set the Authentication type to bearer token and copy the key here from the slack app.
  6. Hit RUN on the top right to execute the query.

Here, as you can see, we’ve sent an additional parameter, which is channel-id. You can find this by clicking the details of the slack channel.

image.png

Now, to fetch all the messages from the slack channel, create a new query called getMessages. Follow the same steps, and replace the endpoint with the following:

https://slack.com/api/conversations.history?channel=C01H8CPUVUK&limit=1000

image.png

Here, you can limit the number of messages by sending an additional limit parameter. Now that we have everything, you can start building the UI for the app.

First step is to show all the members of the slack channel using a Table widget; when clicked, it will show all the book suggestions and other messages on a different List widget.

Follow these steps:

  1. Click on the + icon next to the widgets, find the Table widget, and drag and drop it onto the canvas.
  2. Open the table’s property pane by clicking on the cog icon on the table.
  3. Now, under the Table Data property, paste the following JS code snippet:
{{ 
function() {
    let all = completeList.data
    let filtered = filteredIds.data
    const result = all.members.filter(({
        id
    }) => filtered.members.includes(id));
    return result
}()
}}

In this snippet, we first fetched profiles of all the members in the slack organization and filtered them by user_ids with the selected slack channel. You can use a map function to return all the details of the users.

Next, we configure columns in the table by clicking on the cog icon for each column. For example, to render images from the API, we’ll need to access the profile key. Hence, we set the column type to image and set the column computed value to the following:

{{profile.image_original }}
image.png
image.png

With all the connections in place, the application will be fully functional. To take a look at my app, click here

‍It took me approximately 30-40 minutes to make this! My next steps would be to refine this app and add a few more elements, for example, displaying the date and time the messages were sent, images, and replies, filtering out system-generated messages, etc. While my experiments with Appsmith and this world of low code tools may have started with my soft spot for books, I quickly thought of other use-cases that can be implemented via the same app, such as automating other critical conversations/task-management on Slack channels or any other messaging applications. These would make my day-to-day job much easier! :)

It’s a pity that the world of internal apps is not given due attention simply because it’s not always customer-facing. Shifting our perspective to consider all users as customers can be a significant first step in building great internal apps. And low code is the future — internal applications can be tailor-made in a fraction of the time and cost while helping businesses focus on what matters most to their growth. ‍

I love hearing from our users and contributors. If you’re interested in building something on Appsmith or are currently building something, I'd love to pick your brain, please email me: vishnupriya@appsmith.com