12
March
2021
Resources

How to Run Manual Jobs in Gitlab CI/CD

0
 minutes

Gitlab is eating the world, or so we thought till we moved to GitHub as an opensource company. That should tell you enough about our love for Gitlab, but there is one thorny problem with Gitlab CI that didn't have a solution in any pricing tier. It's the ability to trigger CI jobs with custom parameter values manually. This article will explore the benefits and drawbacks of manual jobs, potential workarounds, and finally, how using Gitlab API and forms, we can get around this problem.

Why are manual jobs important in CI/CD?

Although CI is primarily aimed to be continuous (I didn't see that coming, did you?), there is still a precious place for manual triggering of jobs in a CI pipeline. This becomes especially apparent when we consider parameters in the build process. For example, you are working on a separate branch that does a PoC of a new feature. You want to ensure that the CI can handle the codebase in this branch but don't want to push your official organization. So, if your build script took parameters (with sane defaults, of course) that can be configured to have the image pushed to your personal organization, you'll be able to test it to your heart's content. What's more, you can even run the Docker image and check all is in order. Basically, you don't break the app.

Another simple example would be when you have a specific set of servers you want to push your changes to from a specific branch in your repo. This is a case of CD, with finer control. You can use the same deploy pipeline but set the branch and server(s) variables and run this job to get the desired outcome. This becomes incredibly useful as the organization scales and has multiple branches, teams, and servers, especially in this age of mono-repos.

If you're a pipeline wizard, you can probably spin off a new script each time for these jobs and run these scripts each time, but a more practical solution would be to use a pipeline template and have a few variables (set defaults for a standard run) that can be populated for manual runs. This is where Gitlab comes a bit short.

gitlab.png

I have no idea what I am doing here!

How do I run manual jobs using GitLab?

If you're willing to compromise on everything else Gitlab has to offer, Jenkins is your go-to for CI/CD and boy does it ship with an out of the box manual job runner. But of course, who wants to compromise on all the other awesome stuff from Gitlab?

The biggest inconvenience stems from the fact that Jenkins and Gitlab are two separate systems. It's up to the developer to ensure that these two talk to each other well and stay on good terms. For example, If the source code on Gitlab moves from being a Java project to a Golang project, the Jenkins build jobs must be configured to stop using Maven and start using go. Having Gitlab be the integrated solution for both your source code and your CI pipelines is just too convenient. The overheads of running Jenkins and Gitlab will just make your team grumble much more than usual.

Let's say you do end up integrating with Jenkins, are you sure this is enough for you to sacrifice all of Gitlab's CI/CD efficiencies for?

project-build.png

Do you want to integrate this with your repo?

Manual Jobs For Gitlab

Assuming you haven't ripped apart your Gitlab instance and installed Jenkins, and you're still asking yourself "So how does one trigger manual jobs for Gitlab?", the main (and surprisingly the simplest) suggestion is to build a form for each job and use Gitlab API to trigger it - from the discussion for this issue from four years ago (remember, Google is your friend). Why don't we give that a shot? Sounds simple enough, right?

<html lang=en>  
<head>    
<meta charset=utf-8>    
<title>Gitlab CI Trigger</title>  
</head>  
<body>    
<form method=post action='gitlab-api-endpoint'>      
<p>        
<label for=branch>Branch</label>        
<select id=branch name=branch>          
<option value=master>master</option>          
<option value='feature/one'>feature/one</option>          
<option value='feature/two'>feature/two</option>          
<option value='bug/that-nasty-bug'>bug/that-nasty-bug</option>        
</select>      
</p>      
<p>        
<input type=checkbox id=debug name=debug>        
<label for=debug>Print debug output</label>      
</p>     
<p>        
<label for=reason>Reason for trigger</label>        
<input name=reason id=reason>      
</p>      
<p>        
<button type=submit>Run pipeline!</button>      
</p>    
</form>  
</body>
</html>

We're not even getting started here 😰

screen.png

Voila? 🤔

But a few things to remember before we deploy that in production. You need to update the branch list whenever that changes, which is almost every day. Also, remember to add or remove any parameters along with your build pipeline. Every time you run a manual job, you essentially have to rebuild this file, or scope all potential manual jobs and have files ready. This is really inconvenient.

Beyond just being inconvenient, this method doesn't scale. Even if you maintain this HTML form as part of your source code, it still warrants a significant maintenance effort. You don't want to spend time building/debugging/fixing/maintaining your pipeline-runner, you want to build the software you are actually working on.

The better way to manage CI/CD workflows

Fear not, Appsmith is here. While not an official workaround, appsmith makes it incredibly easy to create your forms and set up the API needed to run manual forms. Building the form will be easier than in Jenkins, API calls can be managed gracefully in the interface, AND you can build a job runner that looks like this in 10 minutes.

We've built a small tutorial to walk you through how we create a manual pipeline to deploy your code on multiple production servers. Let’s get started!

Appsmith is an open-source cloud or self-hosted platform to build admin panels, CRUD apps and workflows. Appsmith helps you speed up application building through fast UI creation, and making you write code only when necessary.

Here’s a video to get you started!

We’ll be building a dashboard on Appsmith to simplify and manage a simple CI/CD workflow that allows developers to deploy software by selecting a specific branch and server.

Additionally, we’ll create options to

  • View a Specific CI/CD Workflow
  • Cancel a Running Workflow
  • Retry a Failed Workflow
  • Delete a Workflow
  • Save Workflow History

This dashboard is going to be created entirely using Appsmith and the Gitlab API.

We’ll be using a Gitlab repository named ci_cd to follow along and set up workflows. It’s a basic “Hello World!” Node.JS application that is deployed to multiple servers on Heroku. We have our .gitlab-ci.yml file configured in such a way as to run two jobs. The first job is to perform a test with the npm test command, and the second is t deploy our app onto Heroku using a ruby gem called dpl.

The entire workflow is configured dynamically so that it can be deployed to any Heroku server given a Heroku <app-name> and an <api-key>. To hide these key’s we’ll be setting these out in our environment variables to the workflow. Lastly, on Heroku, we have two deployment servers appsmith-ci-cd-server1 and appsmith-ci-cd-server2.

Below is a snippet showing the content of .gitlab-ci.yml file.

image: node:latest
stages:
  - test
  - deploy

testing:
  stage: test
  script: npm test

deploying:
  stage: deploy
  image: ruby:latest
  script:
    - gem install dpl
    - dpl --provider=heroku --app=$heroku_app_name --api-key=$heroku_api_key

Building Dashboard on Appsmith

Now that we have a clear picture of what’s cooking, let’s head to Appsmith and start building our Dashboard. If you haven’t signed up, you can get started for free, here.

Next, let’s head over to our Appsmith dashboard and create a new application by clicking on the Create New button. We’ll be now redirected to a new application named “Untitled Application 1”. Now, let’s give our app a creative name, let’s call it Deploy Dashboard. Our first step is to build a feature that would list all the existing and future workflow history. To do this, let’s utilise Gitlab’s pipeline API, fetch all the details and display it onto the Appsmith table widget.

Let’s go ahead and configure our workflow API Gitlab.

Setting up Gitlab API as Datasource

In this section, we’ll be using the GitLab API as a data source to configure it with the Appsmith Deploy Dashboard application. Let’s now create a new API, we can do it by navigating to the API section under Page1 and clicking on Create New option.

Paste the following in the request form and click on Save as Data Source

https://gitlab.com/api/v4/projects/:id

Next, let’s follow the below steps to display all the existing workflows onto the Appsmith dashboard.

  1. Firstly, the id in the above GitLab API is a parameter to access details of a particular repository on Gitlab, you can find it on your repository page on GitLab.
  2. To make things more clear, let’s rename the Datasource name as GitLab, we can do it by double-clicking on the existing one.
  3. Next, let’s add a new key named PRIVATE-TOKEN under the headers section. We can find the value of PRIVATE-TOKEN from GitLab.
  4. Navigate to the Preferences -> Access Tokens on GitLab and create a new access token and paste it to the PRIVATE-TOKEN key on the Appsmith data source. (Also be sure to give it at list api scope)
  5. Lastly, save the data source!

Getting Workflows List

Our Datasource is up and running, now let’s head back to the API section on Appsmith and create a new API to fetch all the existing workflows on the repository. Now create a new API and set the name as get_pipelines. On clicking on the URL we have suggestions to use our previously configured GitLab data source. Use the suggestions and add /pipelines to its end. The URL should now look like the following:

GET https://gitlab.com/api/v4/projects/:id/pipelines

Hit the Run button and you will be greeted with an array of the workflow linked to that repository! Sweet, isn’t it?

Now to make things cooler, let’s build UI to display all these CI/CD workflows on the Appsmith page. Click on the widgets on the navigation bar and drag and drop a table widget onto the canvas. You should see a new table with some pre-populated data. We should also see a floating pane, that consists of all the table properties. Under that pane, edit the Table Data property to the following:

{{get_pipelines.data}}

Now, we can see data from the get_pipelines API rendered on the table. You can go ahead to rearrange the column and disable columns that you don’t want showing up on the table i.e sha and updated_at.

table.png

Triggering A Pipeline

Now let’s add a new feature to trigger a new workflow on the same dashboard. To do this, we’ll create a new button by dragging and dropping a button widget. Rename the button to Trigger New Pipeline. Also, drag in a modal widget to the canvas. The button should be configured such that the modal is opened whenever it’s clicked. On the other hand, the modal’s type should be set to form modal and we’ll drag in two dropdowns with corresponding labels into it. The first dropdown should be configured to select a branch with the following options:

[
  {
    "label": "master",
    "value": "master"
  },
  {
    "label": "staging",
    "value": "staging"
  }
]

Similarly, we configure the second dropdown to show the server options that are configured on Heroku namely, appsmith-ci-cd-server1 and appsmith-ci-cd-server2:

[
  {
    "label": "appsmith-ci-cd-server1",
    "value": "appsmith-ci-cd-server1"
  },
  {
    "label": "appsmith-ci-cd-server2",
    "value": "appsmith-ci-cd-server2"
  }
]

Perfect, we should now see a great looking modal on our Appsmith dashboard.

form.png

Let’s head back to the API section and create a new API to trigger a workflow whenever the Confirm button from our modal is clicked. Let’s name this API as create_pipeline and set the value to the following:

POST https://gitlab.com/api/v4/projects/:id/pipeline

Additionally, we have the option to provide variables and ref (meaning the source branch) in the body of this endpoint. We should configure the body as given in the below snippet.

{
    "ref": "{{Dropdown1.selectedOptionValue}}",
    "variables": [
       {
            "key": "heroku_app_name",
            "variable_type": "env_var",
            "value": "{{Dropdown2.selectedOptionValue}}"
        },
       {
            "key": "heroku_api_key",
            "variable_type": "env_var",
            "value": "your_heroku_api_key_here"
        }
    ]
}

By looking at the above snippet, the ref key is obtained from the branch dropdown, which was previously configured and in the variables section, the value of the heroku_app_name key is obtained from our server dropdown.

"You can find the value of heroku_api_key from your Heroku account under the Settings-> API Keys section."

Lastly, let’s head back to the modal and configure the onclick action of the confirm button to trigger the create_pipeline endpoint. Add the following JS snippet to the onclick property under the button settings.

{{
create_pipeline.run(() => get_pipelines.run( closeModal('Modal1')), () => {})
}}

Kudos! With this, we should be able to trigger a workflow from the Appsmith dashboard itself. Let’s now fine-tune this to have more features in the next section.

More Fine Tuning

Alright, in the last section, we’ll now add fine-grained controls such as deleting, cancelling, retrying, and viewing workflows. These are quite similar to each other, let’s look at the delete workflow option and you can try adding the rest as an exercise :)

Deleting the CI/CD Workflow from Appsmith Dashboard

To implement this feature, let’s add head back to our table and add a new “Custom Colum”. You can find it under the table settings pane. Let’s name this column as delete. To make our dashboard more intuitive, we can set the column type to button and add the label as delete. Now let’s create a delete API under the APIs section and name it as delete_pipeline. Use the following endpoint to add functionality.

DELETE https://gitlab.com/api/v4/projects/:id/pipelines/{{Table1.selectedRow.id}}

This API grabs the id of the selected row from the table, which is automatically set when the delete button is clicked. Heading back to the delete button, let’s configure the onclick action to run the delete_pipeline api and call the get_pipelines api on success to update the table. Here’s the configuration JS snippet:

{{delete_pipeline.run(() => get_pipelines.run(), () => {})}}

Perfect, now we have the ability to delete a specific workflow from the Appsmith dashboard itself.

"Disclaimer: Clicking on this button will delete that pipeline for good. As a safety measure, you can add a confirmation modal to prevent accidental delete."

Here’s a quick lookup to configure the other actions to play with our workflows:

Retry Workflow:

POST https://gitlab.com/api/v4/projects/24888719/pipelines/{{Table1.selectedRow.id}}/retry

Cancel Workflow:

POST https://gitlab.com/api/v4/projects/24888719/pipelines/{{Table1.selectedRow.id}}/cancel

Final Thoughts

We are glad you made it this far. As a quick summary, here’s a gif to show everything we’ve built so far!

Honestly, that was a lot to take in, but we’ve been able to go through a complete flow of how you can build a custom dashboard to manage your ci/cd process. Also, here's a quick demo of the app that we've built! And we are quite sure you’re bubbling with ideas on the limitless use cases you can build with this. So go into the world and build amazing stuff!

Image Credits: Photo by Ryan Quintal on Unsplash

How to Run Manual Jobs in Gitlab CI/CD

Gitlab is eating the world, or so we thought till we moved to GitHub as an opensource company. That should tell you enough about our love for Gitlab, but there is one thorny problem with Gitlab CI that didn't have a solution in any pricing tier. It's the ability to trigger CI jobs with custom parameter values manually. This article will explore the benefits and drawbacks of manual jobs, potential workarounds, and finally, how using Gitlab API and forms, we can get around this problem.

Why are manual jobs important in CI/CD?

Although CI is primarily aimed to be continuous (I didn't see that coming, did you?), there is still a precious place for manual triggering of jobs in a CI pipeline. This becomes especially apparent when we consider parameters in the build process. For example, you are working on a separate branch that does a PoC of a new feature. You want to ensure that the CI can handle the codebase in this branch but don't want to push your official organization. So, if your build script took parameters (with sane defaults, of course) that can be configured to have the image pushed to your personal organization, you'll be able to test it to your heart's content. What's more, you can even run the Docker image and check all is in order. Basically, you don't break the app.

Another simple example would be when you have a specific set of servers you want to push your changes to from a specific branch in your repo. This is a case of CD, with finer control. You can use the same deploy pipeline but set the branch and server(s) variables and run this job to get the desired outcome. This becomes incredibly useful as the organization scales and has multiple branches, teams, and servers, especially in this age of mono-repos.

If you're a pipeline wizard, you can probably spin off a new script each time for these jobs and run these scripts each time, but a more practical solution would be to use a pipeline template and have a few variables (set defaults for a standard run) that can be populated for manual runs. This is where Gitlab comes a bit short.

gitlab.png

I have no idea what I am doing here!

How do I run manual jobs using GitLab?

If you're willing to compromise on everything else Gitlab has to offer, Jenkins is your go-to for CI/CD and boy does it ship with an out of the box manual job runner. But of course, who wants to compromise on all the other awesome stuff from Gitlab?

The biggest inconvenience stems from the fact that Jenkins and Gitlab are two separate systems. It's up to the developer to ensure that these two talk to each other well and stay on good terms. For example, If the source code on Gitlab moves from being a Java project to a Golang project, the Jenkins build jobs must be configured to stop using Maven and start using go. Having Gitlab be the integrated solution for both your source code and your CI pipelines is just too convenient. The overheads of running Jenkins and Gitlab will just make your team grumble much more than usual.

Let's say you do end up integrating with Jenkins, are you sure this is enough for you to sacrifice all of Gitlab's CI/CD efficiencies for?

project-build.png

Do you want to integrate this with your repo?

Manual Jobs For Gitlab

Assuming you haven't ripped apart your Gitlab instance and installed Jenkins, and you're still asking yourself "So how does one trigger manual jobs for Gitlab?", the main (and surprisingly the simplest) suggestion is to build a form for each job and use Gitlab API to trigger it - from the discussion for this issue from four years ago (remember, Google is your friend). Why don't we give that a shot? Sounds simple enough, right?

<html lang=en>  
<head>    
<meta charset=utf-8>    
<title>Gitlab CI Trigger</title>  
</head>  
<body>    
<form method=post action='gitlab-api-endpoint'>      
<p>        
<label for=branch>Branch</label>        
<select id=branch name=branch>          
<option value=master>master</option>          
<option value='feature/one'>feature/one</option>          
<option value='feature/two'>feature/two</option>          
<option value='bug/that-nasty-bug'>bug/that-nasty-bug</option>        
</select>      
</p>      
<p>        
<input type=checkbox id=debug name=debug>        
<label for=debug>Print debug output</label>      
</p>     
<p>        
<label for=reason>Reason for trigger</label>        
<input name=reason id=reason>      
</p>      
<p>        
<button type=submit>Run pipeline!</button>      
</p>    
</form>  
</body>
</html>

We're not even getting started here 😰

screen.png

Voila? 🤔

But a few things to remember before we deploy that in production. You need to update the branch list whenever that changes, which is almost every day. Also, remember to add or remove any parameters along with your build pipeline. Every time you run a manual job, you essentially have to rebuild this file, or scope all potential manual jobs and have files ready. This is really inconvenient.

Beyond just being inconvenient, this method doesn't scale. Even if you maintain this HTML form as part of your source code, it still warrants a significant maintenance effort. You don't want to spend time building/debugging/fixing/maintaining your pipeline-runner, you want to build the software you are actually working on.

The better way to manage CI/CD workflows

Fear not, Appsmith is here. While not an official workaround, appsmith makes it incredibly easy to create your forms and set up the API needed to run manual forms. Building the form will be easier than in Jenkins, API calls can be managed gracefully in the interface, AND you can build a job runner that looks like this in 10 minutes.

We've built a small tutorial to walk you through how we create a manual pipeline to deploy your code on multiple production servers. Let’s get started!

Appsmith is an open-source cloud or self-hosted platform to build admin panels, CRUD apps and workflows. Appsmith helps you speed up application building through fast UI creation, and making you write code only when necessary.

Here’s a video to get you started!

We’ll be building a dashboard on Appsmith to simplify and manage a simple CI/CD workflow that allows developers to deploy software by selecting a specific branch and server.

Additionally, we’ll create options to

  • View a Specific CI/CD Workflow
  • Cancel a Running Workflow
  • Retry a Failed Workflow
  • Delete a Workflow
  • Save Workflow History

This dashboard is going to be created entirely using Appsmith and the Gitlab API.

We’ll be using a Gitlab repository named ci_cd to follow along and set up workflows. It’s a basic “Hello World!” Node.JS application that is deployed to multiple servers on Heroku. We have our .gitlab-ci.yml file configured in such a way as to run two jobs. The first job is to perform a test with the npm test command, and the second is t deploy our app onto Heroku using a ruby gem called dpl.

The entire workflow is configured dynamically so that it can be deployed to any Heroku server given a Heroku <app-name> and an <api-key>. To hide these key’s we’ll be setting these out in our environment variables to the workflow. Lastly, on Heroku, we have two deployment servers appsmith-ci-cd-server1 and appsmith-ci-cd-server2.

Below is a snippet showing the content of .gitlab-ci.yml file.

image: node:latest
stages:
  - test
  - deploy

testing:
  stage: test
  script: npm test

deploying:
  stage: deploy
  image: ruby:latest
  script:
    - gem install dpl
    - dpl --provider=heroku --app=$heroku_app_name --api-key=$heroku_api_key

Building Dashboard on Appsmith

Now that we have a clear picture of what’s cooking, let’s head to Appsmith and start building our Dashboard. If you haven’t signed up, you can get started for free, here.

Next, let’s head over to our Appsmith dashboard and create a new application by clicking on the Create New button. We’ll be now redirected to a new application named “Untitled Application 1”. Now, let’s give our app a creative name, let’s call it Deploy Dashboard. Our first step is to build a feature that would list all the existing and future workflow history. To do this, let’s utilise Gitlab’s pipeline API, fetch all the details and display it onto the Appsmith table widget.

Let’s go ahead and configure our workflow API Gitlab.

Setting up Gitlab API as Datasource

In this section, we’ll be using the GitLab API as a data source to configure it with the Appsmith Deploy Dashboard application. Let’s now create a new API, we can do it by navigating to the API section under Page1 and clicking on Create New option.

Paste the following in the request form and click on Save as Data Source

https://gitlab.com/api/v4/projects/:id

Next, let’s follow the below steps to display all the existing workflows onto the Appsmith dashboard.

  1. Firstly, the id in the above GitLab API is a parameter to access details of a particular repository on Gitlab, you can find it on your repository page on GitLab.
  2. To make things more clear, let’s rename the Datasource name as GitLab, we can do it by double-clicking on the existing one.
  3. Next, let’s add a new key named PRIVATE-TOKEN under the headers section. We can find the value of PRIVATE-TOKEN from GitLab.
  4. Navigate to the Preferences -> Access Tokens on GitLab and create a new access token and paste it to the PRIVATE-TOKEN key on the Appsmith data source. (Also be sure to give it at list api scope)
  5. Lastly, save the data source!

Getting Workflows List

Our Datasource is up and running, now let’s head back to the API section on Appsmith and create a new API to fetch all the existing workflows on the repository. Now create a new API and set the name as get_pipelines. On clicking on the URL we have suggestions to use our previously configured GitLab data source. Use the suggestions and add /pipelines to its end. The URL should now look like the following:

GET https://gitlab.com/api/v4/projects/:id/pipelines

Hit the Run button and you will be greeted with an array of the workflow linked to that repository! Sweet, isn’t it?

Now to make things cooler, let’s build UI to display all these CI/CD workflows on the Appsmith page. Click on the widgets on the navigation bar and drag and drop a table widget onto the canvas. You should see a new table with some pre-populated data. We should also see a floating pane, that consists of all the table properties. Under that pane, edit the Table Data property to the following:

{{get_pipelines.data}}

Now, we can see data from the get_pipelines API rendered on the table. You can go ahead to rearrange the column and disable columns that you don’t want showing up on the table i.e sha and updated_at.

table.png

Triggering A Pipeline

Now let’s add a new feature to trigger a new workflow on the same dashboard. To do this, we’ll create a new button by dragging and dropping a button widget. Rename the button to Trigger New Pipeline. Also, drag in a modal widget to the canvas. The button should be configured such that the modal is opened whenever it’s clicked. On the other hand, the modal’s type should be set to form modal and we’ll drag in two dropdowns with corresponding labels into it. The first dropdown should be configured to select a branch with the following options:

[
  {
    "label": "master",
    "value": "master"
  },
  {
    "label": "staging",
    "value": "staging"
  }
]

Similarly, we configure the second dropdown to show the server options that are configured on Heroku namely, appsmith-ci-cd-server1 and appsmith-ci-cd-server2:

[
  {
    "label": "appsmith-ci-cd-server1",
    "value": "appsmith-ci-cd-server1"
  },
  {
    "label": "appsmith-ci-cd-server2",
    "value": "appsmith-ci-cd-server2"
  }
]

Perfect, we should now see a great looking modal on our Appsmith dashboard.

form.png

Let’s head back to the API section and create a new API to trigger a workflow whenever the Confirm button from our modal is clicked. Let’s name this API as create_pipeline and set the value to the following:

POST https://gitlab.com/api/v4/projects/:id/pipeline

Additionally, we have the option to provide variables and ref (meaning the source branch) in the body of this endpoint. We should configure the body as given in the below snippet.

{
    "ref": "{{Dropdown1.selectedOptionValue}}",
    "variables": [
       {
            "key": "heroku_app_name",
            "variable_type": "env_var",
            "value": "{{Dropdown2.selectedOptionValue}}"
        },
       {
            "key": "heroku_api_key",
            "variable_type": "env_var",
            "value": "your_heroku_api_key_here"
        }
    ]
}

By looking at the above snippet, the ref key is obtained from the branch dropdown, which was previously configured and in the variables section, the value of the heroku_app_name key is obtained from our server dropdown.

"You can find the value of heroku_api_key from your Heroku account under the Settings-> API Keys section."

Lastly, let’s head back to the modal and configure the onclick action of the confirm button to trigger the create_pipeline endpoint. Add the following JS snippet to the onclick property under the button settings.

{{
create_pipeline.run(() => get_pipelines.run( closeModal('Modal1')), () => {})
}}

Kudos! With this, we should be able to trigger a workflow from the Appsmith dashboard itself. Let’s now fine-tune this to have more features in the next section.

More Fine Tuning

Alright, in the last section, we’ll now add fine-grained controls such as deleting, cancelling, retrying, and viewing workflows. These are quite similar to each other, let’s look at the delete workflow option and you can try adding the rest as an exercise :)

Deleting the CI/CD Workflow from Appsmith Dashboard

To implement this feature, let’s add head back to our table and add a new “Custom Colum”. You can find it under the table settings pane. Let’s name this column as delete. To make our dashboard more intuitive, we can set the column type to button and add the label as delete. Now let’s create a delete API under the APIs section and name it as delete_pipeline. Use the following endpoint to add functionality.

DELETE https://gitlab.com/api/v4/projects/:id/pipelines/{{Table1.selectedRow.id}}

This API grabs the id of the selected row from the table, which is automatically set when the delete button is clicked. Heading back to the delete button, let’s configure the onclick action to run the delete_pipeline api and call the get_pipelines api on success to update the table. Here’s the configuration JS snippet:

{{delete_pipeline.run(() => get_pipelines.run(), () => {})}}

Perfect, now we have the ability to delete a specific workflow from the Appsmith dashboard itself.

"Disclaimer: Clicking on this button will delete that pipeline for good. As a safety measure, you can add a confirmation modal to prevent accidental delete."

Here’s a quick lookup to configure the other actions to play with our workflows:

Retry Workflow:

POST https://gitlab.com/api/v4/projects/24888719/pipelines/{{Table1.selectedRow.id}}/retry

Cancel Workflow:

POST https://gitlab.com/api/v4/projects/24888719/pipelines/{{Table1.selectedRow.id}}/cancel

Final Thoughts

We are glad you made it this far. As a quick summary, here’s a gif to show everything we’ve built so far!

Honestly, that was a lot to take in, but we’ve been able to go through a complete flow of how you can build a custom dashboard to manage your ci/cd process. Also, here's a quick demo of the app that we've built! And we are quite sure you’re bubbling with ideas on the limitless use cases you can build with this. So go into the world and build amazing stuff!

Image Credits: Photo by Ryan Quintal on Unsplash

Square
Try Appsmith
Inline editing in the table widget, integration with Airtable, and more
2
August
2022
Announcement

Inline editing in the table widget, integration with Airtable, and more

Inline editing in the table widget, integration with Airtable, and more
Vihar Kurama
0
 minutes ↗
#
announcement
Announcement

In July, we squashed 102 of the peskiest bugs and shipped 34 top requested features over and above under-the-hood performance and usability improvements. There’s a new table widget that’s so much more powerful, an Airtable integration, and cleaner Google Sheets queries with 178 commits in 22 days just for that enhancement alone! You can tell we have got our ears close to you. Keep it coming, guys. We love it all, and we are always listening.

#BigThings

The swanky new table widget

Fact: Just 25% of you tell us who you are and how you use Appsmith. We are okay with that. We respect your privacy.
Assertion: Our usage numbers should be 4X more.
Inference: When we say the Table widget, ever since we launched it, has been used a crazy 820,000 times by 6,840 users, we actually mean it's been used a lot more and is second only to the Button.
Takeaway: Give it more love.

Introducing the new Table widget

Everything you asked for from the table and more is packed into this massive update. Here’s three that should make you sit up.

  • With inline editing, you can now forget about writing queries to edit data by cell, row, or column. You just get your data into the table and edit on the screen. We heavylift the queries, updating the database, and making sure it sticks–all behind the scenes. Clicksaver? Lifesaver? Timesaver? All three and more? We think so, too.
  • You know how you have always wanted to refer to custom column names more naturally than typing customColumn1, customColumn2, and so on in your queries? Yep. Done. No matter what the name of your column, reference away just as naturally as you name them.
  • Themes have been making apps pretty for a while, but Tables stayed rebelliously aloof from that prettiness. We have now made them fall in line with Themes, so if you want shades of blue and Roboto, you got it in Tables, too.

There’s a whole lot more that you are going to have to see for yourself.

Airtable integration, out in the sun

Our Airtable integration gave UI facelifts to the low-code datasource. It is now out of beta, ready for its moment in the sun.

Connect with a Airtable base in two minutes, and start building your apps without worrying about complex data workflows.

 

Auto-indent here to make a dent

On our latest version? Tried the JavaScript editor yet? No? Do that now and you don’t have to read on anymore. 

Oh, okay. You are still here. Fine. We will show you.

Automatically pretty code is pretty cool, huh? More about it here.

#UpdateThings

“Hide Error Messages, Hide”

Infuriating little things, error messages, that bring up existential questions, right? And when they showed up all the time, they got us to, “Frustrating!”. They don’t anymore, only showing up when a widget is visible and clicked.

“How much to upgrade?”

Got your Appsmithing going, but a paid feature’s in your way? Fret not, self-hoster. We got your back with a command-line feature that estimates your usage in thirty seconds. Click this and all shall be revealed.

Run any Appsmith branch locally

With something like ./scripts/local_testing.sh chore/local-testing, running any branch as a FAT container is a breeze. Make sure Docker’s installed and running, port 80 is open, and you add arguments if you don’t want to run the release branch.

Errors, alerts, and logs

Two new modules, logger.js and mailer.js now store backup errors and alert you to them—helpful to get you on top of the error and aid find-and-fix.

While logger.js is on by default, mailer.js needs appsmithctl backup --error-mail to get humming.

Just two of the many, many ways we got your back(up).

Moving Google Sheets to UQI 

Google Sheets is a popular data source. We didn’t dig up numbers, but you can take out word for it. UQI stands for Unified Query Interface and affords standardization for queries. Made sense to get queries to Sheets working better and looking neater, too.

#AsAlwaysThings

If wishes were fishes, round-ups would be essays. Wishes aren’t fishes. So, head over to v1.7.9 if you closed our Release Notes in-app pop-up one of four different ways—yeah, we are fixing it—and see What Happened In July Appsmith style.

Need a new datasource connected?   ||    Discord    ||    YouTube    ||    Twitter

Track and manage bugs effectively using Appsmith and Airtable
1
August
2022
Tutorial

Track and manage bugs effectively using Appsmith and Airtable

Track and manage bugs effectively using Appsmith and Airtable
Vidushi Gupta
0
 minutes ↗
#
tutorial
#
dashboard
#
announcement
Tutorial

Airtable is a popular choice for developers who want to manage tabular data easily. While it's easy to use Airtable as a backend, it can be challenging to build a custom UI from scratch. This is where Appsmith comes in. With Appsmith's native Airtable integration, you can create dashboards, CRUD apps, and internal tools in minutes.

In this tutorial, we'll use an Airtable base to build an issue tracker. We'll start by creating a database in Airtable and then importing our data into Appsmith and building on top of it using JavaScript. 

With this application, users can:

  • Connect to their Airtable base.
  • Add a new bug/issue
  • Update the existing issue
  • View the current bugs in the management tool.

Let's jump in!

Step 1:  Getting started

Create a new Appsmith App 

First, we need to create a new application in Appsmith so we can build it out.

  • Visit https://www.appsmith.com to create a new account or login in to an existing one.
  • Create a new application in your preferred organization and edit it.

Connect to Airtable

Now, we need to add your Airtable datasource to the app. In this case, we will clone an existing Airtable sample to provide our data. 

  • Create a new datasource by clicking ‘+’ on the Datasources tab from the entity explorer and then select Airtable.

  • Rename the datasource. Select API Key as the Authentication Type and enter your API Key in the input field. Hit Save.

  • Choose a workspace and a base in the dialog box and hit 'Create Table.'
  • Go to https://airtable.com/api and select the base titled 'All bugs and issues'
  • In the 'Introduction' section of the documentation, copy the Base ID (highlighted in green in the picture below)

  • Select the "Bugs and issues Table" on the left pane. Copy the table name highlighted in green in the image below.

Step 2 : Set up UI for the App

Wireframe

Here is the wireframe for what we are trying to create.

Including the modal that is used to add new bugs.

Create your widgets

Using the wireframe as a guide, create the UI for the application using the drag and drop editor. Here is a suggested process.

  • Choose a preferred theme from the Theme properties option in the property pane on the right.
  • Add a container widget with a text widget for your app's header.
  • Three (or as many as you like) stats boxes on the canvas to display essential statistics at a glance.
  • A container with a text, icon button, and list widget for showing a list of all the issues.
  • A container with text, button, select and list widgets for showing details of the selected issues.
  • A modal with a text, icon button, and JSON form widgets for adding a new bug entry.

Step 3 : Binding data on widgets

Listing records on the List widget

The list of bugs/issues should look something like this. In order to populate the data, create a new query and bind the results to the text.

  • Create a new query from the left pane which uses the Airtable datasource you created in the first step. Rename the query to getBase and choose the Commands to be List records. We chose this command because we would like to list all the bugs and issues in our app. Enter the Base ID and Table Name you copied in the steps above. 
  • Hit Run and you should see a JSON response generated which lists the records. 
  • To bind this response to the list widget, we would first create a JSObject that maps the fields from the records. Create a new JSObject and paste in the following snippet.

getAirTableFields: () => {
  return getBase.data.records.map((record) => {
    let row = record.fields;
    row["id"] = record.id;
    return row;
  });
};


  • In this JSObject, we get the response from the GetBase query, map the fields, and get the id for every row in the table. 
  • Bind the list with this data using  {{JSObject1.getAirTableFields()}}
  • For getting the bug name and the source, bind the text widgets within the list with {{currentItem.Name}} and {{currentItem.Bug_source}} respectively.

Getting details of the selected bug

When we click on an item from the list, we should populate the view container with details of the selected issue


  • In order to get details about the selected bug on the container placed on the right, we would just use the {{List.selectedItem.attribute}} for all the details you wish to display. For example, The bug title can be displayed using {{List1.selectedItem.Name}}, for Associated features write {{List1.selectedItem.Associated_features}}, For priority write {{List1.selectedItem.Priority}}. So on and so forth. 
  • For a closed/open bug field, use the ternary format to display the status. {{List1.selectedItem.Closed == '1'? "Closed": "Open"}}
  • To bind the attachments for the selected bug, write {{List1.selectedItem['Attachments']}} to bind data on the list widget in the right container. 
  • Just like binding the bug details, in the image widget enter {{currentItem.url}} in the property pane to display the image attached
  • Use {{currentItem.filename}} and {{currentItem.type}} to display the file name and type on the text widget.

Displaying statistics on the statsboxes

These statsboxes should help display important information at a glance from this database. As the number of issues grows, this will give us a quick understanding of the status.

  • In order to populate the statsboxes with statistics, we would create a JSObject function that maps to fields and then to Priority within the same JSON response and check if the value is High, meaning the priority is set to high. What we get in the response is our desired statistic. 

highPriority: () => {
  const high = getBase.data.records.map((record) => record.fields.Priority);
  return high.filter((currentItem) => currentItem == "High").length;
};

  • Bind this output in the text widget using {{JSObject1.highPriority()}}
  • In the very same manner, write a function and bind the output for the number of bugs labeled open and critical

Adding a new Bug/Issue

When clicking the button to add an issue, a modal appears with a form that creates a new entry. It looks like this:

  • Set the Modal to open on onClick of the icon button on the top right corner of the container on the left. 

  • Populate the JSON Form with source data by pasting the following.

{
"fields": { 
    "Bug Title": "",
    "Priority": "",
    "Assigned To":"",
    "Status": "",
    "Screenshots": [
      {
        "URL": ""
      }
    ],
    "Bug Description": "",
    "Bug Source": "",
    "Features Associated": "",
    "Created by": ""
}
}


You can customize the field configuration as per your requirement. Here’s what the JSON Form looks like https://www.loom.com/share/1087b1e8932846feaf3dd03e8b3bb780

  • To insert a new record, we’ll write a new query. 
    Create a new query and name it as InsertQuery. Choose the command to be Create Records. Add in your Base ID and Table Name. For the Records, bind the form data from the JSON form for every field. 

[
  {
    "fields": {
      "Name": "{{JSONForm1.formData.fields['Bug Title']}}",
      "Priority": "{{JSONForm1.formData.fields['Priority']}}",
      "Status": "{{JSONForm1.formData.fields['Status']}}",
      "Attachments": [
        {
          "url": "{{JSONForm1.formData.fields.Screenshots[0].URL}}"
        }
      ],
      "Assigned_to": "{{JSONForm1.formData.fields['Assigned To']}}",
      "Description": "{{JSONForm1.formData.fields['Bug Description']}}",
      "Bug_source": "{{JSONForm1.formData.fields['Bug Source']}}",
      "Associated_features": "{{JSONForm1.formData.fields['Features Associated']}}",
      "Created_by": "{{JSONForm1.formData.fields['Created by']}}"
    }
  }
]

  • We’ll make a new JSObject function to run multiple queries when the Add Bug button is clicked in the form

addBug: async () => {
  InsertQuery.run();
  getBase.run();
  closeModal("Modal2");
};
  • Now bind this function on onClick of the Add Bug button in the JSON Form.

Update fields of a bug

This query/button can help update the details of the bug. In this case, we update the priority and statuses. 

To update the priority and status of a selected bug, an Update Records query would be used. 

  • Create a new query and rename it as updateQuery. Choose the command to be Update Records and enter your BaseID and Table Name. In the records field, paste the following to get the selectedOptionValue of the select widgets

[
    {
      "id": {{List2.selectedItem.id}},
      "fields": {  
        "Priority": {{Select1.selectedOptionValue}},
        "Status":{{Select2.selectedOptionValue}}
      }
    }
]

  • Now, bind this query to run on onClick of the update button.

Final thoughts

And that’s it! You have your bug tracker application ready using Appsmith’s native Airtable integration 🎉

First, you created a new Appsmith application and connected it to Airtable. Then you created the UI for your app using the drag and drop tools in Appsmith. Finally, you tied the data from Airtable to the UI widgets. Your final app should look similar to this:

Please use this form to contact us if you have any template requests for internal tools that you need, and we will get to work! 

If you have any questions, contact us on Discord. You can also keep up with us on Twitter and YouTube.

Build Custom UI on top of Airtable data
25
July
2022
Announcement

Build Custom UI on top of Airtable data

Build Custom UI on top of Airtable data
Rishabh Kaul
0
 minutes ↗
#
integrations
#
databases
#
announcement
Announcement

Today, our integration with Airtable comes out of beta and is available for everyone 🎉! You can now build custom UIs and interact with applications built on Airtable, with minimal configuration.

While it is possible to use the default API interface to connect to Airtable, we wanted to make it easier for you to directly connect your Airtable account and create applications faster than ever. This new data connector introduces a number of features:

  • Integration located in the “Datasources” section
  • Connect to your Airtable account with either an API Key or a Bearer Token (OAuth 2.0)
  • Create queries to fetch, create, retrieve, update and delete data from a datasource using the Appsmith query editor. 
  • List command lets you display all the data from Airtable, and can also present data that has been filtered and sorted based on fields, records, time zones, etc. 

For details and information on how to use this new integration (with videos!), check out our Airtable documentation here. See it in action on our full tutorial here, where we build an issue tracker with Airtable as backend. As always, let us know what you think!

What’s a Rich Text element?

asdsadasdsa

asdsadasdsa

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

sfdfsdfds

dsfdsfdsf

adfkaldf

The rich text element allows you to create and format

sadadasdasdas dsada sadas asd ad

Static and dynamic content editing

  1. vdfgdgd
  2. gjgjg

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to customize formatting for each rich text

Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.

swzdswxzdsw