Open Source Kubernetes Argo

What's new in Argo Workflows v3.5

Exciting New Features with Improved User Experience and Performance

Posted by Yuan Tang on August 14, 2023

We are thrilled to announce that Argo Workflows v3.5 is now available!

We have rolled out a long list of exciting new features, ranging from improvements to existing UI/UX, CLI, controller performance optimizations, additional fields and parameters in workflow/template spec, authentication, monitoring, and more.

Some quick statistics about this release:

  • 66 new features
  • 237 bug fixes
  • 182 documentation updates
  • 18 build and 24 improvements
  • from 63 contributors

Overview of New Features

We’d like to highlight some of the new features in each area in the following sections. You can find the complete changelog here.

UI

In this release, we removed the archived workflow list view from the UI. There’s now a unified workflow list view to see both live workflow custom resource objects and the archived workflows that are persisted in the database. The additional “archived” column indicates whether a particular workflow has been archived and persisted in the database. Thanks to #11121 by Yuan Tang and a couple of important bug fixes by Tomoki Yamaguchi.

unified-workflows

Users can also resubmit workflows with parameters. Thanks again to Tomoki Yamaguchi’s work on #11083!

submit-with-params

Ever wanted to add title and description for your workflows so that your teammates can get a sense of what you are experimenting? This is now possible in the new release! Thanks to #9805 by Jason Meridth.

wf-title-description

Users no longer need to manually check and uncheck individual checkboxes in workflow DAG filter. Thanks to the new “Check All” checkbox added in #11132 by Jason Meridth.

check-all

In this release, users can also add customized navigation links and columns. Thanks to #10808 by Remington Breeze and #10677 by Jiacheng Xu. For example, with a simple configuratio in controller’s configmap like the following:

data: 
  links: |
    - name: Completed Workflows
      scope: workflow-list
      url: http://workflows?label=workflows.argoproj.io/completed=true
  columns: |
    - name: Workflow Completed
      type: label
      key: workflows.argoproj.io/completed

We can see additional link to the list of completed workflows and a new column “Workflows Completed”:

customized-links

Additionally, thanks to #10145 by Isitha Subasinghe, the Argo Workflows UI can parse JSON structured logs.

User information in suspended nodes when resuming is also available. Thanks to Tomoki Yamaguchi via #11763.

Search functionality has been added for WorkflowTemplates page in #11684 by sunyeongchoi.

We’ll be continuously improving the UI/UX. If you are interested in contributing, please check out the UI/UX improvements project board.

CLI

A new argument --query-chunk-size is added to argo delete to be consistent with argo list. Thanks to #10813 by Oliver Skånberg-Tippen. The value supplied in this argument will be applied when the command is fetching the workflows to delete. Note that the name query-chunk-size is different from chunk-size argument to avoid the implication that it will delete based on the specified chunk.

A new argument --selector flag is added to argo cron list command to support selectively querying Cron workflows based on labels defined under objects’ metadata, similar to the --selector field in kubectl. Thanks to #11202 by Dylan Bragdon.

argo retry now allows users to retry successful workflows if --node-field-selector is set, instead of only allowing to retry failed or errored workflows. Thanks to #11409 by Or Shachar.

Workflow/Template Spec

Users can configure whether to force remove the finalizer if artifact GC fails. Thanks to #10810 by Julie Vogelman.

Users can access additional variables:

  • steps.<TASKNAME>.hostNodeName: the host node name where the task ran. Thanks to #10950 by Amit Oren.
  • lastRetry.message: the message output from the last retry via #10987 by Alan Clucas.
  • workflow.mainEntrypoint: workflow’s initial entrypoint via #11151 by Alan Clucas.

We also added cross-namespace locking for semaphore and mutex via #11096 by Lukas Wöhrl. This enhances the semaphore configuration with a namespace property to enable better resource management in environments with shared resources across multiple namespaces.

Workflow-level DeleteDelayDuration is added to this release. Thanks to #11325 by Anton Gilgur.

The creator labels of a CronWorkflow are now propogated to the scheduled workflows. Thanks to #11407 by Jinsu Park.

Authentication

Support for name field in claim for RBAC SSO is added. Thanks to #10927 by Anton Gilgur.

We also improved the logging in OAuth2 callback handler so that it’s easier to identify errors and misconfigurations related to SSO. Thanks to #11370 by Jinsu Park. For example, we’ll see the following detailed error message when the client secret value is incorrectly set. Prior to this release, we would only see a 401 status.

time="2023-07-16T12:20:25.064Z" level=error msg="failed to get oauth2Token by using code from the oauth2 server" error="oauth2: \"unauthorized_client\" \"Invalid client secret\""
time="2023-07-16T12:20:25.064Z" level=info duration=9.112831ms method=GET path=/oauth2/callback size=0 status=401

We also added support for filtering SSO groups based on regex. Thanks to Basanth Jenu H B in #11774.

Monitoring

Users can specify an operation, e.g. Add, Set, and Sub, to gauge metric’s value. Thanks to #10774 by Eduardo Rodrigues.

metrics:
    prometheus:
      - name: custom_gauge_sub
        labels:
          - key: name
            value: random-int
        gauge:
          operation: Sub
          value: "5"

Controller

We switched to use WorkflowTemplate and ClusterWorkflowTemplate informers when validating CronWorkflows to improve the CronWorkflow controller’s performance. Prior to this release, there is a large number of unnecessary K8s API requests for “GET WorkflowTemplate”. Thanks to #11470 by Julie Vogelman.

We also exposed a new argument --cron-workflow-workers to specify the number of CronWorkflows workers, which would be useful when running a large number of CronWorkflows. Thanks to #11457 by Saravanan Balasubramanian.

Client-side throttling logging has been improved by @Jack-R-lantern in #11437.

The controller has also added checks to protect users against infinite recursion for calls to templates. Thanks to Alan Clucas in #11646.

Artifacts

Artifact streaming download for HTTP/Artifactory artifact driver has been added by Yuan Tang in #11823. Previously, when a user would click the button to download an artifact in the UI, the artifact would need to be written to the Argo Server’s disk first before downloading. If many users tried to download simultaneously, they would take up disk space and fail the download. After this release, S3, Azure Blob, HTTP, and Artifactory all support streaming download so artifacts don’t need to be first saved to disk.

Artifactory artifact also supports keyFormat now to be consistent with other artifact types. Thanks to Yuan Tang via #11798.

We also added custom CA support in S3 artifact repository. Thanks to Niklas Simons via #11161.

Local Development

Developers of Argo Workflows can now use Nix to make the local development easier. Nix is a package manager and build tool which focuses on reproducible build environments. Check out how to use it here. Thanks to #10999 by Isitha Subasinghe.

Join Our Growing Community

Thanks to everyone who filed issues or helped resolve them, asked and answered questions, and were part of inspiring discussions. You are welcome to join our growing Slack community.

We also host monthly community meetings where we and the community showcase demos and discuss the current and future state of the project. Feel free to join us.

Check out the curated list of awesome projects and resources related to Argo here!

Affiliations