Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial version of GKE Auth support without gcloud #4185

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ And Store.getKey can be used rather than directly referencing static Cache funct


#### New Features
* Fabric8 now support GKE's service account auth method without gcloud. It can update auth token whenever it is expired.
* Fix #3407: Added Itemable.withItem to directly associate a resource with the DSL. It can be used as an alternative to Loadable.load when you already have the item. There is also client.resourceList(...).getResources() - that will provide the resource list as Resources. This allows you to implement composite operations easily with lambda: client.resourceList(...).getResources().forEach(r -> r.delete());
* Fix #3922: added Client.supports and Client.hasApiGroup methods
* KubernetesMockServer has new methods - unsupported and reset - to control what apis are unsupported and to reset its state.
Expand Down
13 changes: 12 additions & 1 deletion doc/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,15 @@ For non-ResourceEventHandlers call backs long-running operation can be a problem

On top of the http client threads the fabric8 client maintains a task thread pool for scheduled tasks and for potentially long-running tasks that are called from WebSocket operations, such as handling input and output streams and ResourceEventHandler call backs. This thread pool defaults to an unlimited number of cached threads, which will be shutdown when the client is closed - that is a sensible default with either http client as the amount of concurrently running async tasks will typically be low. If you would rather take full control over the threading use KubernetesClientBuilder.withExecutor or KubernetesClientBuilder.withExecutorSupplier - however note that constraining this thread pool too much will result in a build up of event processing queues.

Finally the fabric8 client will use 1 thread per PortForward and an additional thread per socket connection - this may be improved upon in the future.
Finally the fabric8 client will use 1 thread per PortForward and an additional thread per socket connection - this may be improved upon in the future.

### How to enable GKE auth support?
Fabric8 now support GKE auth token. To enable please add google-auth-library-oauth2-http library on your project

<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
<version>${google.version}</version>
</dependency>

Further information [https://cloud.google.com/kubernetes-engine/docs/how-to/api-server-authentication#environments-without-gcloud](https://cloud.google.com/kubernetes-engine/docs/how-to/api-server-authentication#environments-without-gcloud)
5 changes: 5 additions & 0 deletions kubernetes-client-api/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,11 @@
<artifactId>bcpkix-jdk15on</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
<optional>true</optional>
</dependency>

<!-- Compile Only Dependencies -->
<dependency>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
/**
* Copyright (C) 2015 Red Hat, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package io.fabric8.kubernetes.client.utils;

import com.google.auth.oauth2.AccessToken;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.annotations.VisibleForTesting;
import io.fabric8.kubernetes.api.model.NamedContext;
import io.fabric8.kubernetes.client.internal.KubeConfigUtils;
import java.io.File;
import java.io.IOException;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* Utility class for GCP token refresh.
*/
public class GCPAuthenticatorUtils {

private static final Logger LOGGER = LoggerFactory.getLogger(GCPAuthenticatorUtils.class);

public static final String EMPTY = "";
public static final String ACCESS_TOKEN_PARAM = "access_token";
public static final String EXPIRY_PARAM = "expiry";
public static final String SCOPES = "scopes";
public static final String[] DEFAULT_SCOPES =
new String[]{
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/userinfo.email"
};
private static GoogleCredentials credentials;

private GCPAuthenticatorUtils() {
}

/**
* Google Application Credentials-based refresh
* https://cloud.google.com/kubernetes-engine/docs/how-to/api-server-authentication#environments-without-gcloud
* @param currentAuthProviderConfig current AuthInfo's AuthProvider config as a map
* @return access token for interacting with Google Kubernetes API
*/
public static CompletableFuture<String> resolveTokenFromAuthConfig(
Copy link
Member

@manusa manusa Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the usage of a CompletableFuture here, both* return statements return sync values CompletableFuture.completedFuture

Also, line 72 seems to be missing a return?

Map<String, String> currentAuthProviderConfig) {
String[] scopes = parseScopes(currentAuthProviderConfig);
try {
if (credentials == null) {
credentials = GoogleCredentials.getApplicationDefault().createScoped(scopes);
LOGGER.debug("GoogleCredentials: ", credentials);
}
credentials.refresh();
AccessToken accessToken = credentials.getAccessToken();
currentAuthProviderConfig.put(ACCESS_TOKEN_PARAM, accessToken.getTokenValue());
currentAuthProviderConfig.put(EXPIRY_PARAM,
accessToken.getExpirationTime().toInstant().toString());
persistKubeConfigWithUpdatedToken(currentAuthProviderConfig);
CompletableFuture.completedFuture(accessToken.getTokenValue());
} catch (IOException e) {
throw new RuntimeException("The Application Default Credentials are not available.", e);
}
return CompletableFuture.completedFuture(currentAuthProviderConfig.get(ACCESS_TOKEN_PARAM));
}

public static String[] parseScopes(Map<String, String> config) {
String scopes = config.get(SCOPES);
if (scopes == null) {
return DEFAULT_SCOPES;
}
if (scopes.isEmpty()) {
return new String[]{};
}
return scopes.split(",");
}

/**
* Save Updated Access and Refresh token in local KubeConfig file.
*
* @param kubeConfigPath Path to KubeConfig (by default .kube/config)
* @param updatedAuthProviderConfig updated AuthProvider configuration
* @return boolean value whether update was successful not not
* @throws IOException in case of any failure while writing file
*/
static boolean persistKubeConfigWithUpdatedToken(String kubeConfigPath,
Map<String, String> updatedAuthProviderConfig)
throws IOException {
io.fabric8.kubernetes.api.model.Config config = KubeConfigUtils.parseConfig(
new File(kubeConfigPath));
NamedContext currentNamedContext = KubeConfigUtils.getCurrentContext(config);

if (currentNamedContext != null) {
// Update users > auth-provider > config
int currentUserIndex = KubeConfigUtils.getNamedUserIndexFromConfig(config,
currentNamedContext.getContext().getUser());
Map<String, String> authProviderConfig = config.getUsers().get(currentUserIndex).getUser()
.getAuthProvider().getConfig();
authProviderConfig.put(ACCESS_TOKEN_PARAM, updatedAuthProviderConfig.get(ACCESS_TOKEN_PARAM));
authProviderConfig.put(EXPIRY_PARAM, updatedAuthProviderConfig.get(EXPIRY_PARAM));
config.getUsers().get(currentUserIndex).getUser().getAuthProvider()
.setConfig(authProviderConfig);

// Persist changes to KUBECONFIG
try {
KubeConfigUtils.persistKubeConfigIntoFile(config, kubeConfigPath);
return true;
} catch (IOException exception) {
LOGGER.warn("failed to write file {}", kubeConfigPath, exception);
}
}
return false;
}

private static boolean persistKubeConfigWithUpdatedToken(
Map<String, String> updatedAuthProviderConfig) throws IOException {
return persistKubeConfigWithUpdatedToken(
io.fabric8.kubernetes.client.Config.getKubeconfigFilename(),
updatedAuthProviderConfig);
}

@VisibleForTesting
static void setCredentials(GoogleCredentials cre){
credentials = cre;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@ public CompletableFuture<Boolean> afterFailure(BasicBuilder headerBuilder, HttpR
if (newestConfig.getAuthProvider() != null && newestConfig.getAuthProvider().getName().equalsIgnoreCase("oidc")) {
newAccessToken = OpenIDConnectionUtils.resolveOIDCTokenFromAuthConfig(newestConfig.getAuthProvider().getConfig(),
factory.newBuilder());
} else {
} else if(newestConfig.getAuthProvider() != null && newestConfig.getAuthProvider().getName().equalsIgnoreCase("gcp")){
newAccessToken = GCPAuthenticatorUtils.resolveTokenFromAuthConfig(newestConfig.getAuthProvider().getConfig());
}
else {
newAccessToken = CompletableFuture.completedFuture(newestConfig.getOauthToken());
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,16 @@ void testKubeConfigWithAuthConfigProvider() throws URISyntaxException {
config.getOauthToken());
}

@Test
void testKubeConfigWithAuthConfigGCPProvider() throws URISyntaxException {
System.setProperty("kubeconfig", new File(getClass().getResource("/test-kubeconfig-gcp").toURI()).getAbsolutePath());
Config config = Config.autoConfigure("production/172-28-128-4:8443/mmosley");

assertEquals("https://172.28.128.4:8443/", config.getMasterUrl());
assertEquals(null, config.getOauthToken());
assertEquals("gcp", config.getAuthProvider().getName());
}

@Test
void testEmptyConfig() {
// Given
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/**
* Copyright (C) 2015 Red Hat, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.fabric8.kubernetes.client.utils;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;

import com.google.auth.oauth2.AccessToken;
import com.google.auth.oauth2.GoogleCredentials;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.time.Instant;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;

public class GCPAuthenticatorUtilsTest {

@Test
void testRefreshToken() throws Exception {
// Given
String fakeToken = "new-fake-token";
String fakeTokenExpiry = "2121-08-05T02:30:24Z";
Map<String, String> currentAuthProviderConfig = new HashMap<>();
File tempFile = Files.createTempFile("test", "kubeconfig").toFile();
Files.copy(getClass().getResourceAsStream("/test-kubeconfig-gcp"), Paths.get(tempFile.getPath()),
StandardCopyOption.REPLACE_EXISTING);
System.setProperty("kubeconfig", tempFile.getAbsolutePath());

GoogleCredentials mockGC = Mockito.mock(GoogleCredentials.class);
GCPAuthenticatorUtils.setCredentials(mockGC);
Mockito.when(mockGC.getAccessToken())
.thenReturn(new AccessToken(fakeToken, Date.from(Instant.parse(fakeTokenExpiry))));

// When
String token = GCPAuthenticatorUtils.resolveTokenFromAuthConfig(currentAuthProviderConfig).get();

// Then
assertNotNull(token);
assertEquals(fakeToken, token);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,16 @@
*/
package io.fabric8.kubernetes.client.utils;

import com.google.auth.oauth2.AccessToken;
import com.google.auth.oauth2.GoogleCredentials;
import io.fabric8.kubernetes.client.Config;
import io.fabric8.kubernetes.client.http.HttpClient;
import io.fabric8.kubernetes.client.http.HttpRequest;
import io.fabric8.kubernetes.client.http.TestHttpResponse;
import java.time.Instant;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;

Expand Down Expand Up @@ -125,4 +131,42 @@ void shouldRefreshOIDCToken() throws Exception {
}

}

@Test
void shouldRefreshGCPToken() throws Exception {
try {
// Prepare kubeconfig for autoconfiguration
File tempFile = Files.createTempFile("test", "kubeconfig").toFile();
Files.copy(Objects.requireNonNull(getClass().getResourceAsStream("/token-refresh-interceptor/kubeconfig-gcp")),
Paths.get(tempFile.getPath()), StandardCopyOption.REPLACE_EXISTING);
System.setProperty(KUBERNETES_KUBECONFIG_FILE, tempFile.getAbsolutePath());

String fakeToken = "new-fake-token";
String fakeTokenExpiry = "2121-08-05T02:30:24Z";
GoogleCredentials mockGC = Mockito.mock(GoogleCredentials.class);
GCPAuthenticatorUtils.setCredentials(mockGC);
Mockito.when(mockGC.getAccessToken())
.thenReturn(new AccessToken(fakeToken, Date.from(Instant.parse(fakeTokenExpiry))));

// Prepare HTTP call that will fail with 401 Unauthorized to trigger GCP token renewal.
HttpRequest.Builder builder = Mockito.mock(HttpRequest.Builder.class, Mockito.RETURNS_SELF);

// Loads the initial kubeconfig.
Config config = Config.autoConfigure(null);

// Copy over new config with following gcp auth provider configuration:
Files.copy(Objects.requireNonNull(getClass().getResourceAsStream("/token-refresh-interceptor/kubeconfig-gcp")),
Paths.get(tempFile.getPath()), StandardCopyOption.REPLACE_EXISTING);

TokenRefreshInterceptor interceptor = new TokenRefreshInterceptor(config, Mockito.mock(HttpClient.Factory.class));
boolean reissue = interceptor.afterFailure(builder, new TestHttpResponse<>().withCode(401)).get();

// Make the call and check that renewed token was read at 401 Unauthorized.
Mockito.verify(builder).setHeader("Authorization", "Bearer new-fake-token");
assertTrue(reissue);
} finally {
// Remove any side effect.
System.clearProperty(KUBERNETES_KUBECONFIG_FILE);
}
}
}
37 changes: 37 additions & 0 deletions kubernetes-client-api/src/test/resources/test-kubeconfig-gcp
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
apiVersion: v1
clusters:
- cluster:
certificate-authority: testns/ca.pem
server: https://172.28.128.4:8443
name: 172-28-128-4:8443
contexts:
- context:
cluster: 172-28-128-4:8443
namespace: testns
user: user/172-28-128-4:8443
name: testns/172-28-128-4:8443/user
- context:
cluster: 172-28-128-4:8443
namespace: production
user: root/172-28-128-4:8443
name: production/172-28-128-4:8443/root
- context:
cluster: 172-28-128-4:8443
namespace: production
user: mmosley
name: production/172-28-128-4:8443/mmosley
current-context: production/172-28-128-4:8443/mmosley
kind: Config
preferences: {}
users:
- name: user/172-28-128-4:8443
user:
token: token
- name: root/172-28-128-4:8443
user:
token: supertoken
- name: mmosley
user:
auth-provider:
name: gcp

Loading