Apache Airflow® with Astro by Astronomer - Annual Plans

Astro by Astronomer allows me to view a complete pipeline, check the logs, and rerun jobs. I have also been using it for environment management and automatic scaling. I have been building data pipelines with Astro by Astronomer so that I can check tasks, see which task has failed and why it failed, review logs, and examine errors.

I can retry tasks multiple times if there is a failure. In one project, we used Astro by Astronomer to solve a problem where we had a pipeline ingesting 50 to 60 GB of data from cloud storage into our bronze, silver, and gold layers using Databricks. We use Astro by Astronomer for orchestrating the entire workflow. Each step—data ingestion, validation, transformation, and loading—was a separate Airflow task with dependencies. If one task failed, Astro by Astronomer would automatically retry it, send an alert, and allow us to rerun only the failed task instead of running the complete pipeline. This reduced our manual intervention, improved reliability, and made jobs easier to monitor and troubleshoot.

Astro by Astronomer is more reliable. In addition to orchestrating these pipelines, we use Astro by Astronomer for CI/CD development. We package our DAGs and deploy them consistently across development, QA, and production. We use the Astro UI to monitor pipeline health and troubleshoot failures. We also integrated it well with Databricks, cloud storage, and databases, which makes it easier to build reliable and scalable pipelines and to monitor these jobs. Overall, it has reduced our operational overhead and improved the reliability of our data pipelines.

Astro by Astronomer standard Airflow deployment time was reduced by more than 40 to 50 minutes overall, where manual effort was five to ten minutes per automated deployment. We also saw production pipeline failures decrease by roughly 20 to 30 percent because of better dependency management and consistent deployments. When failures did occur, troubleshooting time was reduced by around 40 to 50 percent since the logs and monitoring made it much easier to identify the root cause. Overall, the team saved several hours each week that were previously spent on manual deployments and operational support. We have observed noticeably fewer deployment issues, faster recovery from failures, and a significant reduction in manual effort due to automated deployments and better monitoring.

One of the best features Astro by Astronomer offers is Airflow deployment, built-in monitoring, and seamless CI/CD integration. I also appreciate the ability to manage different environments like development, QA, and production consistently. The task retry mechanism, alerting, and detailed logs make troubleshooting much easier for me. Since we integrate Astro by Astronomer with Databricks and cloud storage, it also helps us orchestrate complex data pipelines reliably without spending much time managing Airflow infrastructure.

CI/CD has made our workflow much more efficient by automating deployments with Astro by Astronomer. Whenever we make changes to our DAG, the code is pushed to a Git repository, and the CI/CD pipeline automatically runs validation and tests before deploying to the Astro by Astronomer environment. This reduces our manual deployment errors and ensures consistency in releases across development, QA, and production, which allows the team to deliver changes faster. It also lets us roll back to the previous version if an issue is found. With the alert mechanism, if there is any failure in a job, all users who have been associated with that particular job get notified immediately. As for the retry mechanism, if I have a pipeline with more than 50 tasks and a particular task is failing, we can run just that particular task instead of running the complete pipeline.

One of the biggest improvements since we started using Astro by Astronomer is that it has reduced our manual workflow. Since we have adopted Astro by Astronomer, we have seen pipeline deployments become much faster and easier in all specific environments. We do not need to change the code for a particular environment; Astro by Astronomer picks the configuration based on the environment. If it is deployed in development, it picks the configuration from development and runs the task. If it is deployed in QA, it picks the configuration for QA and runs the task, and the same applies for production. This reduces our manual workload, and continuous deployment is much easier. Pipeline failures also decreased due to built-in retries and better monitoring. When issues did happen, we could identify and resolve them much more quickly using the detailed logs. In the detailed logs, we can find at which stage the code failed, what the issue was, and what time it failed. It has reduced our manual effort and improved the reliability of ETL workflows. It has also improved downtime, so downtime has been minimized, which helps us ensure data is delivered on time for downstream reporting and analytics. From a business perspective, we have achieved faster deployments, reduced manual effort, improved reliability, and faster troubleshooting.

Overall, my experience with Astro by Astronomer was very positive, but there are a few areas where it could improve. For large environments with hundreds of DAGs, navigation and monitoring could be more intuitive, making it easier to identify critical issues quickly. More built-in analytics for pipeline performance and resource utilization would be valuable.

Additionally, while debugging is already good, having more detailed error insights and root cause analysis out of the box would help reduce troubleshooting time. For example, if an error is happening repeatedly in a particular log, could Astro by Astronomer itself identify the error in the log and rerun the job based on that particular error? These are areas where it can improve.

I have been using Astro by Astronomer for the last four years.

The documentation for Astro by Astronomer is fine. I have always preferred the documentation. You can get detailed information about tasks, how to write them, the syntax, and all features. I have also gone through the certifications of Astro by Astronomer, so that also helped me analyze what features are coming into the latest release, and that has been helpful for me.

Regarding governance and security, I think Astro by Astronomer is sufficient. No additional improvement is needed. I have not faced any challenges with governance and security. I was not very involved in it, so I cannot identify many issues.

The AI capabilities of Astro by Astronomer are generally reliable for routine tasks, such as generating DAG templates, providing code suggestions, and assisting with troubleshooting. For a production pipeline, I always review and validate the logic before deploying. Occasionally, the suggestions need more refinement for complex workflows or organization-specific requirements, but overall, I have found the output to be accurate and useful.

My advice to others looking into using Astro by Astronomer is to start with a well-designed Airflow architecture and follow the best practices for DAG design. Use Git and CI/CD from the beginning. Keep tasks modular and implement proper monitoring and alerting. Astro by Astronomer takes away much of the operational overhead of managing Airflow, so teams can focus on building reliable data pipelines instead of maintaining the infrastructure. For organizations running multiple ETL workflows, especially on Azure and with Databricks, Astro by Astronomer is a strong choice because it improves deployment consistency, scalability, and operational efficiency.

Astro by Astronomer is beneficial. I have used it in my project, and it is very beneficial for us. It has improved performance and reduced our workflow. I would always prefer Astro by Astronomer instead of any other technologies. I would rate my overall experience with this product as a 9 out of 10.

My main use case for Astro by Astronomer involves monitoring a significant number of enabled DAGs, and I must check whether all pipelines are running successfully. As I belong to the operations department, I am responsible for troubleshooting any failures and determining the reasons for them so that I can restore them to a healthy state.

I can provide a specific example of a pipeline I monitor using Astro by Astronomer. We have many tables in Snowflake, and several of those tables have replication DAGs. The DAG performs historical sync, fetches all data, and places it into Snowflake. One time, the DAG was failing due to resource unavailability because we had clusters running with a maximum cluster setting of four, but due to a data spike, that was insufficient to process the huge amount of data. We discovered it was a resource error, so we changed the cluster from four to six. When it failed again at six, we increased the cluster to eight, which then worked successfully.

Monitoring and ensuring DAGs run while connecting to different servers comprises my main use case with Astro by Astronomer. When you open Astro by Astronomer, there is a variable column where you can add connections, accounts, and warehouse parameters, and that is also part of my day-to-day activity.

The best features Astro by Astronomer offers include a user interface that stands out most to me because it is very helpful. I was using MWAA for AWS as well, but I would say that the UI of Astro by Astronomer is far ahead of AWS MWAA.

What makes the user interface stand out for me is that it is easier to navigate. For instance, if you log into MWAA and compare it to Astro by Astronomer, you can see that if you hover over the DAG bar, it provides enough information that you do not have to go inside and check, such as the run duration, the time it took, or the number of runs. All that information is present just by hovering over a particular task. Another advantage is checking the logs; when you click on the log, it has the option to wrap or unwrap it, and copying the log and pasting it into your notepad is easy to perform. Importantly, the cron expression is written above the DAG monitoring bar, so it helps me check the schedule of the DAG.

Astro by Astronomer has positively impacted my organization by making work easier; tasks are getting completed in much less time than before. Even if we hire someone new, we provide training of one to two weeks, and they can easily adapt and work on it.

I can share specific outcomes: we were a team of ten people, but after migrating to Astro by Astronomer, we received additional projects because we had plenty of time. Now, fifty percent of the team works on Astro by Astronomer, and fifty percent work on other tools, which is a positive aspect we gained from Astro by Astronomer.

One challenge I find with Astro by Astronomer is the cost because it is relatively higher than other tools. If you could decrease the cost, it would be much easier for small organizations to use.

Regarding needed improvements, while my overall feedback is good, I believe new users may need a basic understanding of Apache Airflow concepts before they can use Astro by Astronomer effectively. Some advanced Airflow configurations are also intentionally abstracted, which is beneficial for simplicity, but it may limit users who require deep infrastructure level customization.

I have been using Astro by Astronomer for more than four years.

In my experience, Astro by Astronomer is stable.

Astro by Astronomer's scalability is impressive; it can easily handle growing workloads. Our data volume is high, and it is handling it smoothly.

The customer support for Astro by Astronomer is very good, fast, and reliable; however, I have never faced any situation where I had to contact customer support because it is very user-friendly.

My advice for others looking into using Astro by Astronomer is that it is very user-friendly, and you should choose it instead of any other tools. This is not one of those tools where you have to hardcode in the backend; it is very simple, and there are several courses available on the internet that you can watch and start using from day one. My overall rating for Astro by Astronomer is nine out of ten.

My main use case for Astro by Astronomer is orchestrating the Airflow pipeline without hassling the backend of Airflow.

I use Astro by Astronomer as a managed platform for Apache Airflow to orchestrate our data engineering workflows. We primarily use it for scheduling, dependency management, monitoring, and retrying ETL/ELT pipelines rather than executing heavy data processing itself.

I use Astro by Astronomer day-to-day for ingesting data from APIs or cloud storage, running Azure Data Factory pipelines where required, triggering data based on PySpark transformations, performing data quality checks and validations, storing created data into a data warehouse or data tables, and sending notifications on success or failure through emails or notifications.

Astro by Astronomer has positively impacted our organization by helping us use task groups to organize complex DAGs. We configured tools to control concurrency and avoid overloading downstream systems. We use dynamic task mapping for processing multiple files or datasets. The UI makes it easy to visualize dependencies or rerun only failed tasks instead of the entire pipeline. The log of each task is available directly from the UI, which speeds up debugging.

Using Astro by Astronomer is definitely improving our team's productivity. Before, tracking failed jobs and their dependencies was more manual. With Astro by Astronomer's monitoring dashboard and centralized logs, we can identify failures much faster and rerun only the failed tasks instead of restarting the entire workflow.

The best features that Astro by Astronomer offers include reduced operational overhead for managing Airflow. It provides managed Airflow with easier upgrades, CI/CD integrations, and simplified development environment management, allowing the teams to focus on building pipelines instead of maintaining Astro by Astronomer infrastructure.

It is very easy to use Astro by Astronomer. One of the biggest advantages is its ease of use. It provides a clean UI to monitor DAGs, task dependencies, check logs, and manually trigger or rerun failed workflows. Troubleshooting is much easier because we can quickly identify which pipelines failed and where.

The current version of Astro by Astronomer is good enough for our needs.

The documentation part of Astro by Astronomer can be made easier and more generic, not specific to the integration with each of the services. There are a few areas where improvement would be helpful. While it integrates well with Airflow and cloud services, configuring some third-party agents can require additional setup and troubleshooting.

I have been using Astro by Astronomer for the last two years.

Overall, Astro by Astronomer is a stable platform. In my experience, we did not face any major outages caused by Astro by Astronomer itself. Most issues we have encountered were related to downstream systems, such as database cluster availability, API downtime, database connectivity, or temporary network issues, rather than the orchestration platform.

Regarding Astro by Astronomer's scalability, I would say it is one of the strengths of Astro by Astronomer. As the number of data pipelines and scheduled jobs increases, Astro by Astronomer handles the additional workload well. Since it is a managed Airflow platform, it can scale its underlying worker nodes to support more concurrent DAGs and tasks without requiring us to manually manage infrastructure.

Regarding Astro by Astronomer's governance and security capabilities, I have positive thoughts. From a governance and security perspective, Astro by Astronomer provides several capabilities that help us operate securely. We do not store secrets such as database passwords or API keys directly in our DAGs. Instead, we use Airflow connections variables or have integrated with a secret manager like Azure Key Vault to securely manage credentials.

Before adopting Astro by Astronomer, we primarily relied on a combination of Azure Data Factory for orchestration purposes. We had some self-managed Airflow flows. As our data platform grew, managing complex dependencies, monitoring pipelines, and scaling our system became more challenging.

Before choosing Astro by Astronomer, we evaluated Azure Data Factory as the external tool for data movement and simple orchestration. For complex workflows with many dependencies, custom Python logic, dynamic task generation, and integration across multiple systems, Airflow and therefore Astro by Astronomer offers much greater flexibility and control.

Astro by Astronomer is very consistent and reliable 99% of the time, although sometimes it lags. Overall, the reliability has been very good. Once a DAG is configured and tested, the output is consistent because Astro by Astronomer ensures tasks are executed in the correct order and respects dependencies.

I would recommend Astro by Astronomer to organizations that already use or are planning to use Apache Airflow for workflow orchestration, especially if they want a managed solution instead of maintaining Airflow themselves.

I gave this product a rating of 8 out of 10.

I conducted this exploration in my personal space and wanted to become aware of what features Astro by Astronomer has, so there is nothing to do with my company.

I wanted to understand Astro by Astronomer itself, so there was no specific use case. I needed to understand what features it has and the feature I really appreciated is the migration capability while upgrading Airflow version two to version three. Astro by Astronomer has amazing features that allow you to accomplish everything with just a few command prompt terminal codes. That is one of the cool features that provided me significant value. Apart from that, the rest of my exploration was exploratory in nature.

I was attempting to manually migrate from upgrading Airflow two to Airflow three, and it was a real nightmare. The libraries moved to different paths, so all the jobs needed to be rebuilt and all the DAGs needed to be reworked. These kinds of challenges are handled with ease using Astro by Astronomer.

I would definitely say the migration capabilities save huge amounts of time and effort. This will not be a small win; it will be a massive win.

For a small scale production environment, it might save you two to three days instead of running DAG after DAG with manual upgrades or manual code editing. Astro by Astronomer will complete the work quite quickly, within an hour or so.

Astro by Astronomer's CLI might be a little challenging, and having the CLI commands along with the documentation more readily available would be great.

I have not encountered any stability issues.

Resources are available, customer education is solid, and the customer journey is very nice with good follow-ups. Astro by Astronomer hosts regular sessions, which is another feature I appreciated.

Curiosity made me think about Astro by Astronomer, but I do not have any burning issues otherwise. Broad-grade code is currently running on my Cloud Composer and has been for the last five years without any issues. So far, everything has been good and smooth.

If you need advanced features and pain points addressed, go for Astro by Astronomer. If you are running small scale broad-grade code with fixed pipelines and you are just running maintenance while looking at how the pipelines operate the same way, then stick with your current solution. If you are working with heavy code or need well-orchestrated pipelines, then consider Astro by Astronomer.

All the features are readily available and Astro by Astronomer's user interface itself is quite intuitive. All developer problems and engineer problems are addressed by them very quickly. I would rate this review a 7 out of 10.

I tried Astro by Astronomer on my personal project. I attempted to rebuild some workflow, data, and data ingestion that I used to do at my previous job with Astro by Astronomer. I also tested Astro by Astronomer against other AI tools such as Gemini and Copilot to evaluate how Astro by Astronomer could create a folder structure for my project, create files, arrange configuration files, and create three folders.

My main goal was to create some custom operators or to make suggestions on existing operators that are already deployed on GitHub.

I think the possibility to share the same project and to have a central point where to develop with Astro by Astronomer is incredibly valuable, especially when sharing with other colleagues. I found that the main feature I really loved, which I also found only on Claude, is the ability to create the DAG in the best way and to understand the context.

It was very useful for my workflow management with Astro by Astronomer because when you share the same project with other colleagues, you need to keep track of every change that you make on the code.

I think the possibility to save time developing DAG in a smarter way with Astro by Astronomer is a significant benefit because, as I mentioned before, it suggests the best operator for your use case. You only need to define the right context and the right goal, and it will provide the best operators to combine with each other to create a real pipeline.

Last year, as I shared with the product manager, I thought that it would be great to have a slower response time with Astro by Astronomer. I used to wait a few minutes to get a response and to see that the DAG was created completely, which I think is annoying when you work and need to wait until the suggestions were made after a few minutes. The speed, I think, was the part to improve, but the quality, for instance, was amazing.

I think that one year ago, Astro by Astronomer needed to improve in reliability because it was a bit slow to develop and sometimes it crashed. As I said, it was before the launching date, so if they give me the opportunity to try it again, I will do that and test again. But the pain points were there.

I tried Astro by Astronomer for one month last year since I was part of the championship program, so I tested it before it was launched on the market.

I have never tried the technical support of Astro by Astronomer in terms of contacting them directly, but when I reported those problems to them, they were very supportive with me, especially because they were about launching that product. They were at the beginning of the journey and were very open to receiving any type of feedback. I would rate Astro by Astronomer's technical support as an eight.

I decided to switch to Astro by Astronomer from the solutions I used before because I know that Astro by Astronomer was developed by Astronomer, who also maintains the Airflow GitHub repo. I think that when you rely on a company that provides the technical support for a tool, you can have the full package. I would use it during my next company because I think it is a real game changer in developing pipelines.

One year ago when I tried Astro by Astronomer, I only needed to log into the platform, which was used on the website, so it was not locally developed.

I have not evaluated other options before choosing Astro by Astronomer.

For me, it was how Astro by Astronomer was able to create a deep tree folder structure because when I used to create some DAG with Gemini and with Copilot, they were not able to go deeper on how configuration files could be created and what type of operator they should use. With Astro by Astronomer, it was able to create the right path for every file and also to use the best operator based on the version that I provided.

I used to work with Copilot integrating in Visual Studio Code, and I used to work with Gemini as well before using Astro by Astronomer.

Also, because in the last month I tried Claude and I love using Claude code integrating with Visual Studio Code. I think that if they developed something very similar that you can integrate into your IDE, it would be a huge jump into the competition with Claude and other AI companies.

My main use case for Astro by Astronomer is to proof out concepts of functionality and learn about the behind the scenes for the application. I'm not an admin on our company's instance, so my vantage point is limited. This helps me get a peek at what's behind the scenes and make better decisions based on what I've learned from that.

I use Astro by Astronomer to test out determining concurrency limits and how to set them and do load balancing. I also can use it to test out different ideas for use cases and after I have given them a good test, I can possibly fold them into my work experience or personal, or I can keep them as personal projects on the side to keep exploring.

I feel the service you provide is fairly easy to set up, much easier than the first time I had to create my own local Airflow instance, so that was much appreciated.

Once you get going, it's pretty easy to use. The best features Astro by Astronomer offers, in my opinion, are easy setup and good tutorials. The courses that Astronomer provides for learning are very useful as well.

I believe your instance has a lot of really good integration with other services, so that helps with keeping it in mind whenever I am trying out different things involving different tools and how they communicate.

The impact of Astro by Astronomer on my organization is mainly personal. Our company uses an Amazon managed Airflow, so we don't use Astro by Astronomer in a production sense, and it's mainly confined to my own personal use cases. Astro by Astronomer helps me in skill development and learning mostly.

I think any further refinements as far as anything major is going to be on the user's side. Everybody's got a different need, so trying to build an overfit solution is only going to hurt other people.

The reason I choose eight for Astro by Astronomer is that it is useful for learning and for companies, small to large. There is still a barrier for people who are not familiar with some of the prerequisite knowledge, but I do also think that is not something that needs to be fixed. It is just the nature of how this works. As I said before, if you try to tailor it too much where people were getting almost a no-code experience with it, I think due to the nature of the services it provides, it would cause more trouble than it would fix. An eight means you're doing great.

I use Astro by Astronomer every now and then, probably a couple times a year.

In my experience, Astro by Astronomer appears to be stable.

I originally set everything up using the actual Airflow documentation, and it was a lot more difficult, so Astro by Astronomer made it a lot simpler.

I feel the service you provide is fairly easy to set up, much easier than the first time I had to create my own local Airflow instance, so that was much appreciated.

I have had no experience with pricing, setup cost, and licensing.

I did not evaluate other options before choosing Astro by Astronomer. This was the most prominent suggestion.

I haven't explored the governance and security facet of Astro by Astronomer's AI capabilities thoroughly, so I couldn't speak on it.

I don't use the AI component of Astro by Astronomer, so I can't comment on its accuracy and reliability of output.

Astro by Astronomer is not deployed by my administration. It is a local instance that I have used.

My advice to others looking into using Astro by Astronomer is to follow the tutorial videos and watch the supplemental learning material. It's very useful.

I have no additional thoughts about Astro by Astronomer before we wrap up. I gave Astro by Astronomer a rating of eight out of ten.