Apache Spark

Apache Spark is a free and open-source, unified analytics engine for large-scale data processing. Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine. With Apache Spark, you can write application quickly in Java, Scala, Python, R, and SQL. Spark also offers over 80 high-level operators that make it easy to build parallel apps. You can run Apache Spark anywhere, as a standalone product, in the cloud or on Hadoop, Apache Mesos or Kubernetes.