Apache NiFi

vs

Apache Kafka

Apache Storm makes it easy to reliably process unbounded streams of data for real-time processing. It is a distributed stream processing computation framework written predominantly in the programming language Clojure. The software is released free and open-source under the Apache License. In a nutshell, Apache Storm does to real-time processing, what Apache Hadoop did to bactch processing. Large corporations like Weather Channel, FullContact, Twitter, Yahoo, Spotify, and Alibaba use and trust Apache Storm for big data analytics with fault-tolerance and fast data processing.

Rundeck is an open source automation service with a web console, command line tools and a WebAPI. With Rundeck you easily run automation tasks across a set of nodes. Rundeck is avalible as two versions, Rundeck Community and Rundeck Enterprise, where the first mentiond is built for small teams. It addresses the technical complexity of Runbook Automation for solving non-production self-service. The later is aimed at teams and organizations at scale. It addresses the organizational complexity of teams and multiple clusters with self-service, incident response, and compliance and auditing features.

Apache Airflow is a community created workflow management platform. The software is both free and open-source, and can be used to reduce complexity in organizations workflow. Apache Airflow is modular by nature, and has an architecture that makes it easy to customize to organizations specific needs. The software was was first developed at Airbnb to where it was used to programmatically author and schedule their workflows, allowing the organization to monitor them via the built-in Airflow dashboard.

Raygun is an application monitoring and performance insight tool that helps you and your team to proactively find, diagnose and fix software errors and performance problems. Raygun is trusted by thousands of innovative organizations that use the tool to ship better experiences to their customers.

Over 1 million developers across 60,000 organization use Sentry to diagnose, fix, and optimize the performance of their code. Sentry is a commerical application monitoring platform that allow them to ship better software faster. 

Apache Spark is a free and open-source, unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. With Apache Spark, you can write application quickly in Java, Scala, Python, R, and SQL. Spark also offers over 80 high-level operators that make it easy to build parallel apps. You can run Apache Spark anywhere, as a standalone product, in the cloud or on Hadoop, Apache Mesos or Kubernetes.