Apache Camel

vs

RabbitMQ

Apache Airflow is a community created workflow management platform. The software is both free and open-source, and can be used to reduce complexity in organizations workflow. Apache Airflow is modular by nature, and has an architecture that makes it easy to customize to organizations specific needs. The software was was first developed at Airbnb to where it was used to programmatically author and schedule their workflows, allowing the organization to monitor them via the built-in Airflow dashboard.

Trusted by more than 80% of all Fortune 100 companies. Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. The software is used for both real-time and batch data processing, and is the chosen event log technology for Amadeus microservice-based streaming applications. Apache Kafka is also used for operational use cases such as application and service logs collection.

With Apache NiFi you can automate the flow of data between software systems. Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Apache NiFi is highly configurable and provides an intuetive web-based user inferface that can be used to control and monitor your data flow. The software is developed by the Apache Foundation and released free and open-source under the Apache License 2.0.

Apache Spark is a free and open-source, unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. With Apache Spark, you can write application quickly in Java, Scala, Python, R, and SQL. Spark also offers over 80 high-level operators that make it easy to build parallel apps. You can run Apache Spark anywhere, as a standalone product, in the cloud or on Hadoop, Apache Mesos or Kubernetes.

Trusted by more than 80% of all Fortune 100 companies. Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. The software is used for both real-time and batch data processing, and is the chosen event log technology for Amadeus microservice-based streaming applications. Apache Kafka is also used for operational use cases such as application and service logs collection.

Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information. Amazon Kinesis is part of the Amazon Web Services offering and and allow you to create data-processing applications withing your AWS dashboard. These data-processing applications are also known as Kinesis Data Streams applications and allow developers to take in large amount of data from several sources, while providing scalabillity running on EC2 instances.