Known issues for Analytics Engine powered by Apache Spark

The following limitations and known issues apply to Analytics Engine Powered by Apache Spark.

Instance resource quota is not enabled for V3 Spark job API endpoints

When using V3 Spark job API endpoints, the instance resource quota is not enabled.

Workaround: Use the V2 job API endpoint if you want to enable instance resource quota.

Applies to: 4.0.0 thru 4.0.4

Fixed in: 4.0.5

Accessing a remote Hadoop cluster is not supported on V3 job endpoints

V3 Spark job API endpoints do not support access to a remote Hadoop cluster.

Workaround: Use the V2 job API endpoint to access a remote Hadoop cluster.

Applies to: 4.0.0 thru 4.0.3

Fixed in: 4.0.4

Spark 3.0 with Python 3.8 only supports running Spark applications locally

Currently Spark 3.0 with Python 3.8 only supports running Spark applications locally where Spark master is set to local[*].

Applies to: 4.0.0 thru 4.0.5

Fixed in: 4.0.6

Updates to the Certificate Manager instance can affect active Spark runtimes

If you update the Certificate Manager instance, the Certificate Manager operator changes the existing kubernetes TLS secrets, which can lead to a pod restart for the pods that mount that secret.

Spark runtime pods also mount this secret for SSL connection from Spark applications to internal services. This means that updating the Certificate Manager instance can disrupt active Spark runtimes.

Workaround: To avoid the disruption of active Spark runtimes, wait until all Spark jobs have reached their terminal state before updating the Certificate Manager instance. For details on viewing the status of jobs, see Spark job status.

Applies to: 4.0.0 thru 4.0.5

Fixed in: 4.0.6

Timeout message when submitting a Spark job

The expected behaviour by the Spark service when you submit a Spark application is that you create a SparkContext or SparkSession at the beginning of your Spark application code. When you submit the Spark job via the REST API, it returns the Spark application ID once the SparkContext is successfully created.

However, if you don't create a SparkContext or SparkSession:

the REST API will wait for your application to complete which can lead to a REST API timeout. The reason is that the Spark service expects the Spark application to have started, which is not the case if you are running a plain Python, Scala or R application. This application will be listed in Jobs UI even though the REST API timed out.

Support for the k8s resource ResourceQuota

In all Analytics Engine Powered by Apache Spark versions prior to 4.0.8, the Kubernetes resource ResourceQuota is not supported, which can result in application jobs failing.

The following kube-event is noticed in such cases:

Error creating: pods "<pod_name> is forbidden: failed quota: <k8s-resourcequota>: must specify limits.cpu,limits.memory,requests.cpu,requests.memory

Support for ResourceQuota is included in Analytics Engine Powered by Apache Spark 4.0.9.