What is a workflow?
A workflow describes how people get work done from start to finish. A workflow consists of the steps and states in a process. Workflows can appear as a series of steps in a checklist, or as a diagram that visualizes those steps.
The following is an example of a simple support workflow:
- Carl, a customer, starts a support request.
- The open case is assigned to Sasha, a Support Representative.
- Sasha receives the assigned case and asks Carl for information. While waiting for the reply, the case’s status is pending.
- After Carl provides the requested information, Sasha replies with a solution and then updates the case’s status to resolved.
- Carl receives the answer, as well as a survey link where he can provide feedback about his support request. He is delighted with Sasha’s quick response and gives high ratings in the survey. The case’s status is updated to closed.
In this workflow, the steps are “start”, “assign”, “resolve”, and “close”. The states are “open”, “assigned”, “pending”, “resolved”, and “closed”. Steps move the process from state to state along a path until the workflow is completed.
What are the benefits of using workflows?
Workflows provide a set of repeatable steps and tasks that you can initiate, schedule, and monitor.
You can visualize workflows for observation, validate the workflow’s correctness, and monitor the operations in the workflow.
With workflows, businesses can achieve improved levels of reliability for distributed applications without adding additional complexity to their code. You can automate, optimize, and scale workflows at a granular level.
What do workflows bring to application architecture?
Workflows promote logical separation between the control flow of your job's stepwise logic and the actual units of work that contain your unique business logic. With this division, you can manage, maintain, and scale the state machinery of your application separately from the core business logic.
As your business requirements change, you can easily change application logic without changing the underlying state machinery, task dispatch, and flow control.
How do workflow services improve application architecture?
Reliability
Separation of state tracking and task processing brings reliability to a cloud-based architecture. Using workflow services, you can dispatch tasks to application components, track their progress, and know the latest state of the system.
Simplicity
Workflows replace the complexity of custom-coded homegrown solutions and process automation code.
Tasks that drive processing steps can be long-running and may fail, time out, or require restarts. They often complete with varying throughputs and latencies. In all these cases, tracking and visualizing tasks is not only challenging, but is also undifferentiated work.
With workflow services, developers can avoid managing the infrastructure plumbing of process automation. As a result, they can focus their energy on the unique functionality of their application.
Scalability
Workflow services are designed to seamlessly scale with your application’s usage. As you add more workflows to your application, or increase the complexity of your workflows, you can automate administration of the workflow service.
Flexibility
Developers can use workflow services to create application components and coordination logic in various programming languages. Developers can then focus on their differentiated application logic.
What are some common workflow use cases?
Workflows can help with any computational problem or business process that can be divided into a series of steps. Workflows simplify managing jobs with interdependencies. Common use cases include:
- Application orchestration – Coordinate loosely coupled microservices, each running in its own process and independently deployed, through communication of events. Combine serverless functions to build a web application with an asynchronous human approval step. For more use case examples, see Microservice orchestration in the AWS Step Functions Developer Guide.
- Data processing – Consolidate data from multiple data sets into unified reports, refine and reduce large data sets into useful formats, or coordinate multi-step analytics and machine learning workflows. For more use case examples, see Data processing in the AWS Step Functions Developer Guide.
- DevOps and IT automation – Build tools for continuous integration and deployment, or create event-driven applications that automatically respond to changes in infrastructure. For more use case examples, see IT and security automation in the AWS Step Functions Developer Guide.
- Ecommerce – Automate mission-critical business processes, such as order fulfillment and inventory tracking.
- Web applications – Implement robust user registration processes and authentication, optionally integrating steps for auditing and enhanced security.
- Machine learning – Run extract, transform, and load (ETL) jobs to build, train, and deploy a machine learning model. For example, collecting and processing image data to detect objects within a video stream, to add bounding boxes, or to provide visual and metadata information for data cataloging. For more use case examples, see Machine learning in the AWS Step Functions Developer Guide.
- Security automation – Scheduled analysis or security incident response initiated by managed resource events, with customized workflows that invoke audit and notification steps depending on business rules.
- Media processing – Extract data from PDF documents or images for processing. Split and transcode video using massive parallelization. For more use case examples, see Media Processing in AWS Step Functions Use Cases.
How do you create a workflow?
Developers may initially build custom-coded solutions and process automation code. These solutions start simply, but tend to become complex and error-prone with time and development. The effort required to update business logic increases, and the rate of change slows down over time.
You can mitigate some of the complexity of homegrown solutions by adopting workflow-specific libraries. However, configuring these tools can add new complexities to the process. The workflow steps and states may still be difficult to visualize.
Advanced creation and monitoring of workflows requires visual tools to arrange the workflow states and steps. In these tools, the business logic rules are available to manage and update. Using graphical tools can enhance developer productivity and workflow visibility, and can reduce the potential for errors.
For a hands-on tutorial using the visual workflow designer Workflow Studio, see How do I create a serverless workflow in the AWS Developer Center.
What are common workflow patterns?
Using workflows, you can focus on defining component interactions, rather than writing software to make the interactions work.
Workflows can orchestrate both microservices and containers. Common patterns for workflows include microservice chaining, asynchronous or synchronous responses, human interactions, container orchestration and choreography, branching, and error handling.
What workflow services does AWS provide?
AWS Step Functions is a low-code visual workflow service to orchestrate AWS services, automate business processes, and build serverless applications. Step Functions includes Workflow Studio, a drag-and-drop, low-code visual workflow designer. This tool auto-generates your functions and configuration in the Amazon States Language, a JSON-based, structured language used to define state machines. You can use Amazon States Language in the Step Functions console, or export and edit it.
Amazon Managed Workflows for Apache Airflow (Amazon MWAA) provides the same open-source Apache Airflow experience but as a fully managed service. Use Amazon MWAA to move your existing on-premises solutions to the cloud to reduce operational costs and deploy at scale. Developers can continue to write directed acyclic graph (DAG) models in Python, declaring dependencies for tasks, while integrating with other AWS services.
We recommend using AWS Step Functions for all new applications, since it provides a more productive and agile approach to coordinating application components using visual workflows. For a hands-on tutorial using both Step Functions Workflow Studio and Amazon States Language, see How do I create a serverless workflow in the AWS Developer Center.
