Why doesn't Spark Streaming support SQL?

Posted on: May 22, 2020

Real-Time Analytics with Spark Streaming has been updated by AWS. This AWS solution automatically provides a highly available, cost-effective batch and real-time data analysis architecture based on Apache Spark Streaming and Amazon Kinesis in the AWS Cloud. The solution supports customer-specific Apache Spark streaming applications and uses Amazon EMR to process large amounts of data via dynamically scalable Amazon Elastic Compute Cloud instances (Amazon EC2).

The solution now includes an updated consumer application with the latest version of Spark and is equipped with modern features (such as Spark SQL and DataFrames), precisely graduated custom IAM policies, the ability to encrypt data at rest (by default), flow protocols for VPC and the option to port Spark Streaming sample applications to Java (by Scala). Several maintenance upgrades were also made: Python was updated to version 3.8 and Amazon EMR to version 5.29.0. For more information about Real-Time Analytics with Spark Streaming on AWS, please visit the solution's website.

You can find additional AWS solution offerings on the AWS Solutions website. From there, you can browse available solutions by product category or industry to find AWS-verified, auto-generated, and ready-to-use reference implementations for specific business needs.