Connect time out when install spark history server with s3 storage

[hongzhao-signifyd]

Describe the bug
when I tried to install the the spark history server with s3 storage, the pod failed to start because of connect timed out
the following is the log from the pod
Is it related to the hadoop-aws version?

2019-11-12 19:59:52 INFO  HistoryServer:2566 - Started daemon with process name: 10@old-buffoon-spark-history-server-84664f8f79-pbtm8
2019-11-12 19:59:52 INFO  SignalUtils:54 - Registered signal handler for TERM
2019-11-12 19:59:52 INFO  SignalUtils:54 - Registered signal handler for HUP
2019-11-12 19:59:52 INFO  SignalUtils:54 - Registered signal handler for INT
2019-11-12 19:59:53 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing view acls to: root
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing modify acls to: root
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing view acls groups to: 
2019-11-12 19:59:53 INFO  SecurityManager:54 - Changing modify acls groups to: 
2019-11-12 19:59:53 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2019-11-12 19:59:53 INFO  FsHistoryProvider:54 - History server ui acls disabled; users with admin permissions: ; groups with admin permissions
2019-11-12 20:00:01 INFO  AmazonHttpClient:448 - Unable to execute HTTP request: connect timed out
java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:542)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:414)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:384)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
	at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1031)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:994)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:297)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:117)
	at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:86)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:280)
	at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)

Version of Helm and Kubernetes:
helm version:

Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}

kubectl version (minikube with 8 cpus and 8096m Memeory):

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:47:22Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.7", GitCommit:"8fca2ec50a6133511b771a11559e24191b1aa2b4", GitTreeState:"clean", BuildDate:"2019-09-18T14:39:02Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

Which chart:
stable/spark-history-server

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

1. aws-access-key / aws-secret-key have already been set on default namespace.
2. helm package .
3. helm install .

values.yaml:

replicaCount: 1
nameOverride: ""
fullnameOverride: ""

rbac:
  create: true

serviceAccount:
  create: true
  name:

image:
  repository: lightbend/spark-history-server
  tag: 2.4.0
  pullPolicy: IfNotPresent

service:
  type: LoadBalancer
  port: 18080
  annotations: {}

environment:
# Note: do not configure Spark history events directory using SPARK_HISTORY_OPTS. It will be
# configured by this chart based on the values in "pvc", "gcs" or "hdfs" attribute.
  # SPARK_HISTORY_OPTS: ...
  # SPARK_DAEMON_MEMORY: 1g
  # SPARK_DAEMON_JAVA_OPTS: ...
  # SPARK_DAEMON_CLASSPATH: ...
  # SPARK_PUBLIC_DNS: ...

podAnnotations: {}

resources:
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  #
  # To let the application start up quickly give it a big limit
  limits:
    cpu: 1000m
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 512Mi


ingress:
  enabled: false
  annotations: {}
  # kubernetes.io/ingress.class: nginx
  # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - spark-history-server.example.com
  tls: []
  #  - secretName:spark-history-server.example.com
  #    hosts:
  #      - spark-history-server.example.com

pvc:
  # to use a file system path for Spark events dir, set 'enablePVC' to true and mention the
  # name of an already created persistent volume claim in existingClaimName.
  # The volume will be mounted on /data in the pod
  enablePVC: false
  existingClaimName: nfs-pvc
  eventsDir: "/"

# Settings for the sub-chart
# When pvc.enablePVC is true, make sure:
# pvc.existingClaimName == nfs.pvcName
nfs:
  enableExampleNFS: false
  pvName: nfs-pv
  pvcName: nfs-pvc

gcs:
  enableGCS: false
  secret: history-secrets
  key: sparkonk8s.json
  logDirectory: gs://spark-hs/

hdfs:
  hdfsSiteConfigMap: hdfs-site
  coreSiteConfigMap: core-site
  logDirectory: hdfs://hdfs/history/
  HADOOP_CONF_DIR: /etc/hadoop

s3:
  enableS3: true
  enableIAM: false
  # Omit for IAM role-based or provider-based authentication.
  secret: aws-secrets
  # accessKeyName is a is AWS access key ID. Omit for IAM role-based or provider-based authentication.
  accessKeyName: aws-access-key
  # secretKey is AWS secret key. Omit for IAM role-based or provider-based authentication.
  secretKeyName: aws-secret-key
  logDirectory: s3a://path/to/my/bucket/

wasbs:
  enableWASBS: false
  sasKeyMode: true
  secret: azure-secrets
  sasKeyName: azure-blob-sas-key
  storageAccountKeyName: azure-storage-account-key
  storageAccountNameKeyName: azure-storage-account-name
  containerKeyName: azure-blob-container-name
  logDirectory: wasbs:///spark-hs

imagePullSecrets: []

nodeSelector: {}

tolerations: []

affinity: {}

Anything else we need to know:

[yuchaoran2011]

Thanks for reporting the issue. Unfortunately we haven't had bandwidth to maintain and update this project for a while. I'm not sure if the client version is to blame. If you figure it out, feel free to make a PR and it will be much appreciated. Thanks!