26
January
2021
Engineering

Evaluating JS in the Browser for a Low Code Product

0
 minutes

Appsmith is an open-source low code platform for developers to build internal apps and workflows.

In Appsmith, our developer users define business logic by writing any Javascript code in between {{ }} dynamic bindings almost anywhere in the app. They can use this while creating SQL queries, APIs, or triggering actions. This functionality lets you control how your app behaves with the least amount of configuration. Underneath the hood, the platform will evaluate all this code in an optimized manner to make sure the app remains performant yet responsive.

Let us take an example of binding a query response to a table widget.

It all starts with the binding brackets {{ }} . When the platform sees these brackets and some code in it, in a widget or action configuration, it will flag the field as a dynamic field so that our evaluator can pick it up later. In our example let us bind usersQuery to usersTable

Screenshot_2020-12-05_at_2.46.08_PM.png

Since we have added this binding in our tableData field, we will flag this field and store it in our widget config

// usersTable config
{
  "usersTable": {
        ...
        "tableData": "{{
            usersQuery.data
                .map(row => ({
                    name: row.name,
                    email: row.email
                }))
            }}",
        "dynaminBindingPathList": [
            {"key": "tableData"}
            ...
        ]
    }
}

In the background, our evaluation listener, always keeps a lookout for such events that would need an evaluation. For our example, this is a scenario that definitely needs an evaluation, so it kicks off our evaluator.

We pass on our current list of app data constructed in what we call as DataTree to the evaluator thread and patiently wait to hear back from it ⏱

// DataTree
{
    "usersQuery": {
        "config": {...},
        "data": [...]
    },
    "usersTable": {
        "tableData": "{{
            usersQuery.data
                .map(row => ({
                    name: row.name,
                    email: row.email
                }))
            }}",
        "dynaminBindingPathList": [{"key": "tableData"}]
    }
}

For performance reasons, we run our evaluation process in a separate background thread with the help of web workers. This ensures that evaluation cycles running longer than 16ms do not hang up the main thread giving the app bandwidth to always respond to user events.

Inside the thread, the event listener gets a wake-up call and gets to work.

  • Get differences: First it will calculate differences in the DataTree from the last time. This will ensure we only process changes and not the whole tree. In our example, we would see the usersTable.tableData has changed and usersTable.dynamicBindingPathList has a new entry. It takes each difference, filters any un-important changes, and processes the rest.
  • Get evaluation order with dependency map: It also maintains a DependencyMap between various entity properties. The evaluator will notice if any bindings have changed and recreate the sort order accordingly.For our example, we will infer that usersTable.tableData now depends on usersQuery.data. This means that the query response should always be evaluated before we can evaluate the table data and that whenever we see a change in the query response, we need to re-evaluate the table data as well
// DependencyMap
  {
      ...
      "usersTable.tableData": ["usersQuery.data"]
  }
  // Evaluation order
  [
      "usersQuery.data",
      "usersTable.tableData"
  ]
  • Evaluate: After creating an optimized evaluation order, we will evaluate the update the tree, in that said order. Evaluation happens via a closed eval function with the whole DataTree acting as its global scope. This is why we can directly reference any object in our DataTree in our code.
// Evaluator

  const code = `
    usersQuery.data.map(row => ({
      name: row.name,
      email: row.email
    }))
  `;
	const scriptToEvaluate = `
    function closedFunction () {
      const result = ${code};
      return result
    }
    closedFunction()
  `;
	const result = eval(scriptToEvaluate);
  • Validate and parse: We always want to make sure the values returned after evaluation to be in the right data type that the widget expects. The ensures the widget always gets predictable data even if your code has returned some errors. This is also needed for any function down the line in the evaluation order, if it refers to this field, will always get a reasonable data type to work with.

And that completes it. At the end of this, we will have a fully evaluated DataTree that we can then send back to the main thread and start listening for any new event to do this whole process again.

// Evaluated DataTree
{
    "usersQuery": {
        "data": [...] 
    }
    "usersTable": {
        "tableData": [...]
    }
}

Our main thread gets an event saying the evaluation is complete, with the new evaluated DataTree which it stores in the app redux state. From here, the widgets pick up their data and render it.

Screenshot_2020-12-05_at_2.46.22_PM.png

Summarizing our philosophy

  • Pull vs Push: While building a low code app builder for varied developers, we thought hard about how the written code works with the rest of the platform. We wanted configuration to be easy to start yet powerful when it needed to be. For this reason, we went with a Pull based architecture rather than Push. What this means is that in most places, you won't have to think about how the data will get to a field. You write code that pulls everything from the global DataTree and sets it to the field where you write it. This way the moment the underlying data changes, it get propagated to all the fields dependant on it and you as a developer do not have to orchestrate ui changes.
  • One-way data flow: Since we are built on top React.js and Redux, we strongly embrace the one-way data flow model. What this means is that you cannot set a table's data directly to that field from some other part of the app. If you do need to update the table, you will have to trigger the query to run, which will then cause the table to re-render with the new data. This helps the code you write easy to reason about and bugs easy to find. It also encapsulates each widget's and action's logic in itself for good separation of concern.
Evaluating JS in the Browser for a Low Code Product

Appsmith is an open-source low code platform for developers to build internal apps and workflows.

In Appsmith, our developer users define business logic by writing any Javascript code in between {{ }} dynamic bindings almost anywhere in the app. They can use this while creating SQL queries, APIs, or triggering actions. This functionality lets you control how your app behaves with the least amount of configuration. Underneath the hood, the platform will evaluate all this code in an optimized manner to make sure the app remains performant yet responsive.

Let us take an example of binding a query response to a table widget.

It all starts with the binding brackets {{ }} . When the platform sees these brackets and some code in it, in a widget or action configuration, it will flag the field as a dynamic field so that our evaluator can pick it up later. In our example let us bind usersQuery to usersTable

Screenshot_2020-12-05_at_2.46.08_PM.png

Since we have added this binding in our tableData field, we will flag this field and store it in our widget config

// usersTable config
{
  "usersTable": {
        ...
        "tableData": "{{
            usersQuery.data
                .map(row => ({
                    name: row.name,
                    email: row.email
                }))
            }}",
        "dynaminBindingPathList": [
            {"key": "tableData"}
            ...
        ]
    }
}

In the background, our evaluation listener, always keeps a lookout for such events that would need an evaluation. For our example, this is a scenario that definitely needs an evaluation, so it kicks off our evaluator.

We pass on our current list of app data constructed in what we call as DataTree to the evaluator thread and patiently wait to hear back from it ⏱

// DataTree
{
    "usersQuery": {
        "config": {...},
        "data": [...]
    },
    "usersTable": {
        "tableData": "{{
            usersQuery.data
                .map(row => ({
                    name: row.name,
                    email: row.email
                }))
            }}",
        "dynaminBindingPathList": [{"key": "tableData"}]
    }
}

For performance reasons, we run our evaluation process in a separate background thread with the help of web workers. This ensures that evaluation cycles running longer than 16ms do not hang up the main thread giving the app bandwidth to always respond to user events.

Inside the thread, the event listener gets a wake-up call and gets to work.

  • Get differences: First it will calculate differences in the DataTree from the last time. This will ensure we only process changes and not the whole tree. In our example, we would see the usersTable.tableData has changed and usersTable.dynamicBindingPathList has a new entry. It takes each difference, filters any un-important changes, and processes the rest.
  • Get evaluation order with dependency map: It also maintains a DependencyMap between various entity properties. The evaluator will notice if any bindings have changed and recreate the sort order accordingly.For our example, we will infer that usersTable.tableData now depends on usersQuery.data. This means that the query response should always be evaluated before we can evaluate the table data and that whenever we see a change in the query response, we need to re-evaluate the table data as well
// DependencyMap
  {
      ...
      "usersTable.tableData": ["usersQuery.data"]
  }
  // Evaluation order
  [
      "usersQuery.data",
      "usersTable.tableData"
  ]
  • Evaluate: After creating an optimized evaluation order, we will evaluate the update the tree, in that said order. Evaluation happens via a closed eval function with the whole DataTree acting as its global scope. This is why we can directly reference any object in our DataTree in our code.
// Evaluator

  const code = `
    usersQuery.data.map(row => ({
      name: row.name,
      email: row.email
    }))
  `;
	const scriptToEvaluate = `
    function closedFunction () {
      const result = ${code};
      return result
    }
    closedFunction()
  `;
	const result = eval(scriptToEvaluate);
  • Validate and parse: We always want to make sure the values returned after evaluation to be in the right data type that the widget expects. The ensures the widget always gets predictable data even if your code has returned some errors. This is also needed for any function down the line in the evaluation order, if it refers to this field, will always get a reasonable data type to work with.

And that completes it. At the end of this, we will have a fully evaluated DataTree that we can then send back to the main thread and start listening for any new event to do this whole process again.

// Evaluated DataTree
{
    "usersQuery": {
        "data": [...] 
    }
    "usersTable": {
        "tableData": [...]
    }
}

Our main thread gets an event saying the evaluation is complete, with the new evaluated DataTree which it stores in the app redux state. From here, the widgets pick up their data and render it.

Screenshot_2020-12-05_at_2.46.22_PM.png

Summarizing our philosophy

  • Pull vs Push: While building a low code app builder for varied developers, we thought hard about how the written code works with the rest of the platform. We wanted configuration to be easy to start yet powerful when it needed to be. For this reason, we went with a Pull based architecture rather than Push. What this means is that in most places, you won't have to think about how the data will get to a field. You write code that pulls everything from the global DataTree and sets it to the field where you write it. This way the moment the underlying data changes, it get propagated to all the fields dependant on it and you as a developer do not have to orchestrate ui changes.
  • One-way data flow: Since we are built on top React.js and Redux, we strongly embrace the one-way data flow model. What this means is that you cannot set a table's data directly to that field from some other part of the app. If you do need to update the table, you will have to trigger the query to run, which will then cause the table to re-render with the new data. This helps the code you write easy to reason about and bugs easy to find. It also encapsulates each widget's and action's logic in itself for good separation of concern.
Square
Try Appsmith
Inline editing in the table widget, integration with Airtable, and more
2
August
2022
Announcement

Inline editing in the table widget, integration with Airtable, and more

Inline editing in the table widget, integration with Airtable, and more
Vihar Kurama
0
 minutes ↗
#
announcement
Announcement

In July, we squashed 102 of the peskiest bugs and shipped 34 top requested features over and above under-the-hood performance and usability improvements. There’s a new table widget that’s so much more powerful, an Airtable integration, and cleaner Google Sheets queries with 178 commits in 22 days just for that enhancement alone! You can tell we have got our ears close to you. Keep it coming, guys. We love it all, and we are always listening.

#BigThings

The swanky new table widget

Fact: Just 25% of you tell us who you are and how you use Appsmith. We are okay with that. We respect your privacy.
Assertion: Our usage numbers should be 4X more.
Inference: When we say the Table widget, ever since we launched it, has been used a crazy 820,000 times by 6,840 users, we actually mean it's been used a lot more and is second only to the Button.
Takeaway: Give it more love.

Introducing the new Table widget

Everything you asked for from the table and more is packed into this massive update. Here’s three that should make you sit up.

  • With inline editing, you can now forget about writing queries to edit data by cell, row, or column. You just get your data into the table and edit on the screen. We heavylift the queries, updating the database, and making sure it sticks–all behind the scenes. Clicksaver? Lifesaver? Timesaver? All three and more? We think so, too.
  • You know how you have always wanted to refer to custom column names more naturally than typing customColumn1, customColumn2, and so on in your queries? Yep. Done. No matter what the name of your column, reference away just as naturally as you name them.
  • Themes have been making apps pretty for a while, but Tables stayed rebelliously aloof from that prettiness. We have now made them fall in line with Themes, so if you want shades of blue and Roboto, you got it in Tables, too.

There’s a whole lot more that you are going to have to see for yourself.

Airtable integration, out in the sun

Our Airtable integration gave UI facelifts to the low-code datasource. It is now out of beta, ready for its moment in the sun.

Connect with a Airtable base in two minutes, and start building your apps without worrying about complex data workflows.

 

Auto-indent here to make a dent

On our latest version? Tried the JavaScript editor yet? No? Do that now and you don’t have to read on anymore. 

Oh, okay. You are still here. Fine. We will show you.

Automatically pretty code is pretty cool, huh? More about it here.

#UpdateThings

“Hide Error Messages, Hide”

Infuriating little things, error messages, that bring up existential questions, right? And when they showed up all the time, they got us to, “Frustrating!”. They don’t anymore, only showing up when a widget is visible and clicked.

“How much to upgrade?”

Got your Appsmithing going, but a paid feature’s in your way? Fret not, self-hoster. We got your back with a command-line feature that estimates your usage in thirty seconds. Click this and all shall be revealed.

Run any Appsmith branch locally

With something like ./scripts/local_testing.sh chore/local-testing, running any branch as a FAT container is a breeze. Make sure Docker’s installed and running, port 80 is open, and you add arguments if you don’t want to run the release branch.

Errors, alerts, and logs

Two new modules, logger.js and mailer.js now store backup errors and alert you to them—helpful to get you on top of the error and aid find-and-fix.

While logger.js is on by default, mailer.js needs appsmithctl backup --error-mail to get humming.

Just two of the many, many ways we got your back(up).

Moving Google Sheets to UQI 

Google Sheets is a popular data source. We didn’t dig up numbers, but you can take out word for it. UQI stands for Unified Query Interface and affords standardization for queries. Made sense to get queries to Sheets working better and looking neater, too.

#AsAlwaysThings

If wishes were fishes, round-ups would be essays. Wishes aren’t fishes. So, head over to v1.7.9 if you closed our Release Notes in-app pop-up one of four different ways—yeah, we are fixing it—and see What Happened In July Appsmith style.

Need a new datasource connected?   ||    Discord    ||    YouTube    ||    Twitter

Track and manage bugs effectively using Appsmith and Airtable
1
August
2022
Tutorial

Track and manage bugs effectively using Appsmith and Airtable

Track and manage bugs effectively using Appsmith and Airtable
Vidushi Gupta
0
 minutes ↗
#
tutorial
#
dashboard
#
announcement
Tutorial

Airtable is a popular choice for developers who want to manage tabular data easily. While it's easy to use Airtable as a backend, it can be challenging to build a custom UI from scratch. This is where Appsmith comes in. With Appsmith's native Airtable integration, you can create dashboards, CRUD apps, and internal tools in minutes.

In this tutorial, we'll use an Airtable base to build an issue tracker. We'll start by creating a database in Airtable and then importing our data into Appsmith and building on top of it using JavaScript. 

With this application, users can:

  • Connect to their Airtable base.
  • Add a new bug/issue
  • Update the existing issue
  • View the current bugs in the management tool.

Let's jump in!

Step 1:  Getting started

Create a new Appsmith App 

First, we need to create a new application in Appsmith so we can build it out.

  • Visit https://www.appsmith.com to create a new account or login in to an existing one.
  • Create a new application in your preferred organization and edit it.

Connect to Airtable

Now, we need to add your Airtable datasource to the app. In this case, we will clone an existing Airtable sample to provide our data. 

  • Create a new datasource by clicking ‘+’ on the Datasources tab from the entity explorer and then select Airtable.

  • Rename the datasource. Select API Key as the Authentication Type and enter your API Key in the input field. Hit Save.

  • Choose a workspace and a base in the dialog box and hit 'Create Table.'
  • Go to https://airtable.com/api and select the base titled 'All bugs and issues'
  • In the 'Introduction' section of the documentation, copy the Base ID (highlighted in green in the picture below)

  • Select the "Bugs and issues Table" on the left pane. Copy the table name highlighted in green in the image below.

Step 2 : Set up UI for the App

Wireframe

Here is the wireframe for what we are trying to create.

Including the modal that is used to add new bugs.

Create your widgets

Using the wireframe as a guide, create the UI for the application using the drag and drop editor. Here is a suggested process.

  • Choose a preferred theme from the Theme properties option in the property pane on the right.
  • Add a container widget with a text widget for your app's header.
  • Three (or as many as you like) stats boxes on the canvas to display essential statistics at a glance.
  • A container with a text, icon button, and list widget for showing a list of all the issues.
  • A container with text, button, select and list widgets for showing details of the selected issues.
  • A modal with a text, icon button, and JSON form widgets for adding a new bug entry.

Step 3 : Binding data on widgets

Listing records on the List widget

The list of bugs/issues should look something like this. In order to populate the data, create a new query and bind the results to the text.

  • Create a new query from the left pane which uses the Airtable datasource you created in the first step. Rename the query to getBase and choose the Commands to be List records. We chose this command because we would like to list all the bugs and issues in our app. Enter the Base ID and Table Name you copied in the steps above. 
  • Hit Run and you should see a JSON response generated which lists the records. 
  • To bind this response to the list widget, we would first create a JSObject that maps the fields from the records. Create a new JSObject and paste in the following snippet.

getAirTableFields: () => {
  return getBase.data.records.map((record) => {
    let row = record.fields;
    row["id"] = record.id;
    return row;
  });
};


  • In this JSObject, we get the response from the GetBase query, map the fields, and get the id for every row in the table. 
  • Bind the list with this data using  {{JSObject1.getAirTableFields()}}
  • For getting the bug name and the source, bind the text widgets within the list with {{currentItem.Name}} and {{currentItem.Bug_source}} respectively.

Getting details of the selected bug

When we click on an item from the list, we should populate the view container with details of the selected issue


  • In order to get details about the selected bug on the container placed on the right, we would just use the {{List.selectedItem.attribute}} for all the details you wish to display. For example, The bug title can be displayed using {{List1.selectedItem.Name}}, for Associated features write {{List1.selectedItem.Associated_features}}, For priority write {{List1.selectedItem.Priority}}. So on and so forth. 
  • For a closed/open bug field, use the ternary format to display the status. {{List1.selectedItem.Closed == '1'? "Closed": "Open"}}
  • To bind the attachments for the selected bug, write {{List1.selectedItem['Attachments']}} to bind data on the list widget in the right container. 
  • Just like binding the bug details, in the image widget enter {{currentItem.url}} in the property pane to display the image attached
  • Use {{currentItem.filename}} and {{currentItem.type}} to display the file name and type on the text widget.

Displaying statistics on the statsboxes

These statsboxes should help display important information at a glance from this database. As the number of issues grows, this will give us a quick understanding of the status.

  • In order to populate the statsboxes with statistics, we would create a JSObject function that maps to fields and then to Priority within the same JSON response and check if the value is High, meaning the priority is set to high. What we get in the response is our desired statistic. 

highPriority: () => {
  const high = getBase.data.records.map((record) => record.fields.Priority);
  return high.filter((currentItem) => currentItem == "High").length;
};

  • Bind this output in the text widget using {{JSObject1.highPriority()}}
  • In the very same manner, write a function and bind the output for the number of bugs labeled open and critical

Adding a new Bug/Issue

When clicking the button to add an issue, a modal appears with a form that creates a new entry. It looks like this:

  • Set the Modal to open on onClick of the icon button on the top right corner of the container on the left. 

  • Populate the JSON Form with source data by pasting the following.

{
"fields": { 
    "Bug Title": "",
    "Priority": "",
    "Assigned To":"",
    "Status": "",
    "Screenshots": [
      {
        "URL": ""
      }
    ],
    "Bug Description": "",
    "Bug Source": "",
    "Features Associated": "",
    "Created by": ""
}
}


You can customize the field configuration as per your requirement. Here’s what the JSON Form looks like https://www.loom.com/share/1087b1e8932846feaf3dd03e8b3bb780

  • To insert a new record, we’ll write a new query. 
    Create a new query and name it as InsertQuery. Choose the command to be Create Records. Add in your Base ID and Table Name. For the Records, bind the form data from the JSON form for every field. 

[
  {
    "fields": {
      "Name": "{{JSONForm1.formData.fields['Bug Title']}}",
      "Priority": "{{JSONForm1.formData.fields['Priority']}}",
      "Status": "{{JSONForm1.formData.fields['Status']}}",
      "Attachments": [
        {
          "url": "{{JSONForm1.formData.fields.Screenshots[0].URL}}"
        }
      ],
      "Assigned_to": "{{JSONForm1.formData.fields['Assigned To']}}",
      "Description": "{{JSONForm1.formData.fields['Bug Description']}}",
      "Bug_source": "{{JSONForm1.formData.fields['Bug Source']}}",
      "Associated_features": "{{JSONForm1.formData.fields['Features Associated']}}",
      "Created_by": "{{JSONForm1.formData.fields['Created by']}}"
    }
  }
]

  • We’ll make a new JSObject function to run multiple queries when the Add Bug button is clicked in the form

addBug: async () => {
  InsertQuery.run();
  getBase.run();
  closeModal("Modal2");
};
  • Now bind this function on onClick of the Add Bug button in the JSON Form.

Update fields of a bug

This query/button can help update the details of the bug. In this case, we update the priority and statuses. 

To update the priority and status of a selected bug, an Update Records query would be used. 

  • Create a new query and rename it as updateQuery. Choose the command to be Update Records and enter your BaseID and Table Name. In the records field, paste the following to get the selectedOptionValue of the select widgets

[
    {
      "id": {{List2.selectedItem.id}},
      "fields": {  
        "Priority": {{Select1.selectedOptionValue}},
        "Status":{{Select2.selectedOptionValue}}
      }
    }
]

  • Now, bind this query to run on onClick of the update button.

Final thoughts

And that’s it! You have your bug tracker application ready using Appsmith’s native Airtable integration 🎉

First, you created a new Appsmith application and connected it to Airtable. Then you created the UI for your app using the drag and drop tools in Appsmith. Finally, you tied the data from Airtable to the UI widgets. Your final app should look similar to this:

Please use this form to contact us if you have any template requests for internal tools that you need, and we will get to work! 

If you have any questions, contact us on Discord. You can also keep up with us on Twitter and YouTube.

Build Custom UI on top of Airtable data
25
July
2022
Announcement

Build Custom UI on top of Airtable data

Build Custom UI on top of Airtable data
Rishabh Kaul
0
 minutes ↗
#
integrations
#
databases
#
announcement
Announcement

Today, our integration with Airtable comes out of beta and is available for everyone 🎉! You can now build custom UIs and interact with applications built on Airtable, with minimal configuration.

While it is possible to use the default API interface to connect to Airtable, we wanted to make it easier for you to directly connect your Airtable account and create applications faster than ever. This new data connector introduces a number of features:

  • Integration located in the “Datasources” section
  • Connect to your Airtable account with either an API Key or a Bearer Token (OAuth 2.0)
  • Create queries to fetch, create, retrieve, update and delete data from a datasource using the Appsmith query editor. 
  • List command lets you display all the data from Airtable, and can also present data that has been filtered and sorted based on fields, records, time zones, etc. 

For details and information on how to use this new integration (with videos!), check out our Airtable documentation here. See it in action on our full tutorial here, where we build an issue tracker with Airtable as backend. As always, let us know what you think!

What’s a Rich Text element?

asdsadasdsa

asdsadasdsa

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

sfdfsdfds

dsfdsfdsf

adfkaldf

The rich text element allows you to create and format

sadadasdasdas dsada sadas asd ad

Static and dynamic content editing

  1. vdfgdgd
  2. gjgjg

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to customize formatting for each rich text

Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.

swzdswxzdsw