Skip to content

Split code editor extension installation for different environments #916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
+34 −18
Draft

Split code editor extension installation for different environments #916

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b952789
Split code editor extension installation for different environments
aws-prayags Sep 12, 2025
3c40bf1
Updated var names
aws-prayags Sep 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions template/v3/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ RUN apt-get update && apt-get upgrade -y && \
: && \
echo "source /usr/local/bin/_activate_current_env.sh" | tee --append /etc/profile && \
# CodeEditor - create server, user data dirs
mkdir -p /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data \
&& chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data && \
mkdir -p /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-ui-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would also need a separate /opt/amazon/sagemaker/sagemaker-ui-code-editor-user-data right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's no difference in how we're configuring the user-data folder across environments - we can add later if needed, I've made minimal changes needed for now

&& chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-ui-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data && \
# create dir to store user data files
mkdir -p /opt/amazon/sagemaker/user-data \
&& chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/user-data && \
Expand All @@ -87,11 +87,15 @@ COPY dirs/ ${DIRECTORY_TREE_STAGE_DIR}/
RUN rsync -a ${DIRECTORY_TREE_STAGE_DIR}/ / && \
rm -rf ${DIRECTORY_TREE_STAGE_DIR} && \
# CodeEditor - download the extensions
mkdir -p /etc/code-editor/extensions && \
mkdir -p /etc/code-editor/extensions /etc/code-editor/extensions-sagemaker-ui && \
while IFS= read -r url || [ -n "$url" ]; do \
echo "Downloading extension from ${url}..." && \
wget --no-check-certificate -P /etc/code-editor/extensions "${url}"; \
done < /etc/code-editor/extensions.txt
done < /etc/code-editor/extensions.txt && \
while IFS= read -r url || [ -n "$url" ]; do \
echo "Downloading sagemaker-ui extension from ${url}..." && \
wget --no-check-certificate -P /etc/code-editor/extensions-sagemaker-ui "${url}"; \
done < /etc/code-editor/extensions-sagemaker-ui.txt

USER $MAMBA_USER
COPY --chown=$MAMBA_USER:$MAMBA_USER $ENV_IN_FILENAME *.in /tmp/
Expand Down Expand Up @@ -133,9 +137,17 @@ RUN if [[ -z $ARG_BASED_ENV_IN_FILENAME ]] ; \
echo "Installing extension ${ext}..."; \
sagemaker-code-editor --install-extension "${ext}" --extensions-dir "${extensionloc}" --server-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-server-data --user-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-user-data; \
done \
# Install sagemaker-ui extensions
&& extensionloc_ui=/opt/amazon/sagemaker/sagemaker-ui-code-editor-server-data/extensions && mkdir -p "${extensionloc_ui}" \
&& for ext in /etc/code-editor/extensions-sagemaker-ui/*.vsix; do \
echo "Installing sagemaker-ui extension ${ext}..."; \
sagemaker-code-editor --install-extension "${ext}" --extensions-dir "${extensionloc_ui}" --server-data-dir /opt/amazon/sagemaker/sagemaker-ui-code-editor-server-data --user-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-user-data; \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets have a separate user data dir as well to keep things clean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commented above, we can add when needed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is stored in this user data? Could it have some conflict or be overridden if different extension versions are used?

Copy link
Contributor Author

@aws-prayags aws-prayags Sep 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This folder will contain the user level settings - https://code.visualstudio.com/docs/configure/settings#_user-settings. We do not save anything in this folder in sagemaker-distribution image

Since we are explicitly specifying the ...server-data/ dir for extensions, I do not think there's any link with the user-data-dir. Let's confirm all works as expected during our testing

done \
# Copy the settings
&& cp /etc/code-editor/code_editor_machine_settings.json /opt/amazon/sagemaker/sagemaker-code-editor-server-data/data/Machine/settings.json && \
cp /etc/code-editor/code_editor_user_settings.json /opt/amazon/sagemaker/sagemaker-code-editor-server-data/data/User/settings.json && \
cp /etc/code-editor/code_editor_machine_settings.json /opt/amazon/sagemaker/sagemaker-ui-code-editor-server-data/data/Machine/settings.json && \
cp /etc/code-editor/code_editor_user_settings.json /opt/amazon/sagemaker/sagemaker-ui-code-editor-server-data/data/User/settings.json && \
# Install glue kernels, and move to shared directory
# Also patching base kernel so Studio background code doesn't start session silently
install-glue-kernels && \
Expand Down
4 changes: 4 additions & 0 deletions template/v3/dirs/etc/code-editor/extensions-sagemaker-ui.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
https://open-vsx.org/api/ms-toolsai/jupyter/2024.5.0/file/ms-toolsai.jupyter-2024.5.0.vsix
https://open-vsx.org/api/ms-python/python/2023.20.0/file/ms-python.python-2023.20.0.vsix
https://open-vsx.org/api/amazonwebservices/aws-toolkit-vscode/3.69.0/file/amazonwebservices.aws-toolkit-vscode-3.69.0.vsix
https://open-vsx.org/api/amazonwebservices/amazon-q-vscode/1.90.0/file/amazonwebservices.amazon-q-vscode-1.90.0.vsix
28 changes: 14 additions & 14 deletions template/v3/dirs/usr/local/bin/start-sagemaker-ui-code-editor
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
#!/bin/bash
set -e

EFS_MOUNT_POINT="/opt/amazon/sagemaker"
EBS_MOUNT_POINT="/home/sagemaker-user"
CONTAINER_SAGEMAKER_METADATA_PATH="/opt/amazon/sagemaker"
EBS_MOUNT_PATH="/home/sagemaker-user"

persistent_settings_folder="${EBS_MOUNT_POINT}/sagemaker-code-editor-server-data"
default_settings_folder="${EFS_MOUNT_POINT}/sagemaker-code-editor-server-data"
persistent_ebs_server_data_folder="${EBS_MOUNT_PATH}/sagemaker-code-editor-server-data"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to sagemaker-ui-code-editor-server-data as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't that be backward incompatible ? this EBS folder could have content already - we should continue using the same folder in EBS

container_server_data_folder="${CONTAINER_SAGEMAKER_METADATA_PATH}/sagemaker-ui-code-editor-server-data"

override_machine_settings() {
# create a new settings file with preset defaults or merge the defaults into the existing settings file
local settings_relative_path="data/Machine"
local settings_file_path_suffix="${settings_relative_path}/settings.json"
local persistent_machine_settings_file="${persistent_settings_folder}/${settings_file_path_suffix}"
local default_machine_settings_file="${default_settings_folder}/${settings_file_path_suffix}"
local persistent_machine_settings_file="${persistent_ebs_server_data_folder}/${settings_file_path_suffix}"
local default_machine_settings_file="${container_server_data_folder}/${settings_file_path_suffix}"

if [ ! -f "$persistent_machine_settings_file" ]; then
# copy settings file to EBS if it doesn't exist in EBS
mkdir -p "${persistent_settings_folder}/${settings_relative_path}"
mkdir -p "${persistent_ebs_server_data_folder}/${settings_relative_path}"
cp "$default_machine_settings_file" "$persistent_machine_settings_file"
echo "Created persistent settings file with default settings at $persistent_machine_settings_file"
else
Expand All @@ -29,19 +29,19 @@ override_machine_settings() {
copy_user_settings() {
local settings_relative_path="data/User"
local settings_file_path_suffix="${settings_relative_path}/settings.json"
local persistent_user_settings_file="${persistent_settings_folder}/${settings_file_path_suffix}"
local default_user_settings_file="${default_settings_folder}/${settings_file_path_suffix}"
local persistent_user_settings_file="${persistent_ebs_server_data_folder}/${settings_file_path_suffix}"
local default_user_settings_file="${container_server_data_folder}/${settings_file_path_suffix}"
if [ ! -f "$persistent_user_settings_file" ]; then
# copy user settings file to EBS if it doesn't exist in EBS
mkdir -p "${persistent_settings_folder}/${settings_relative_path}"
mkdir -p "${persistent_ebs_server_data_folder}/${settings_relative_path}"
cp "$default_user_settings_file" "$persistent_user_settings_file"
echo "Created persistent settings file with default settings at $persistent_user_settings_file"
fi
}

install_prepackaged_extensions() {
local prepackaged_extensions_dir="${default_settings_folder}/extensions"
local persistent_extensions_dir="${persistent_settings_folder}/extensions"
local prepackaged_extensions_dir="${container_server_data_folder}/extensions"
local persistent_extensions_dir="${persistent_ebs_server_data_folder}/extensions"

# if extensions directory doesn't exist then this is the first time opening the app
if [ ! -d "${persistent_extensions_dir}" ]; then
Expand Down Expand Up @@ -95,8 +95,8 @@ if [ -n "$SAGEMAKER_APP_TYPE_LOWERCASE" ]; then
sagemaker-code-editor --host 0.0.0.0 --port 8888 \
--without-connection-token \
--base-path "/$SAGEMAKER_APP_TYPE_LOWERCASE/default" \
--server-data-dir $persistent_settings_folder \
--extensions-dir ${persistent_settings_folder}/extensions \
--server-data-dir $persistent_ebs_server_data_folder \
--extensions-dir ${persistent_ebs_server_data_folder}/extensions \
--user-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-user-data
else
sagemaker-code-editor --host 0.0.0.0 --port 8888 \
Expand Down