Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ec2: exception when processing empty userdata #4386

Closed
nmeyerhans opened this issue Aug 24, 2023 · 0 comments · Fixed by #4387
Closed

ec2: exception when processing empty userdata #4386

nmeyerhans opened this issue Aug 24, 2023 · 0 comments · Fixed by #4387
Labels
bug Something isn't working correctly priority Fix soon

Comments

@nmeyerhans
Copy link
Contributor

Bug report

#4276 introduced support for decoding base64-encoded userdata retrieved from the EC2 IMDS. However, it introduced a regression in the case where an instance runs with no userdata. In that case, cloud-init can fail with messages similar to the following:

2023-08-24 00:35:59,584 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/2021-03-23/user-data' with {'url': 'http://169.254.169
.254:80/2021-03-23/user-data', 'allow_redirects': True, 'method': 'GET', 'timeout': 5.0, 'headers': {'X-aws-ec2-metadata-token': 'REDACTED'}
} configuration
2023-08-24 00:35:59,590 - util.py[WARNING]: Failed reading from metadata address http://169.254.169.254:80
2023-08-24 00:35:59,590 - util.py[DEBUG]: Failed reading from metadata address http://169.254.169.254:80
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/cloudinit/sources/DataSourceEc2.py", line 538, in crawl_metadata
    crawled_metadata["user-data"] = util.maybe_b64decode(raw_userdata)
  File "/usr/lib/python3.9/site-packages/cloudinit/util.py", line 157, in maybe_b64decode
    raise TypeError("data is '%s', expected bytes" % type(data))
TypeError: data is '<class 'str'>', expected bytes

I will follow up with a pull request to address this issue.

Steps to reproduce the problem

Launch an EC2 instance running the 'main' branch of cloud-init and that does did not have a UserData object passed to the RunInstances API call. In this case, the IMDS will return a 404 error from the http://169.254.169.254:80/2021-03-23/user-data IMDS endpoint and cloud-init will fail with the exception shown above.

Environment details

  • Cloud-init version: main
  • Operating System Distribution: Verified on Debian sid and Amazon Linux 2023
  • Cloud provider, platform or installer type: Amazon EC2
@nmeyerhans nmeyerhans added bug Something isn't working correctly new An issue that still needs triage labels Aug 24, 2023
nmeyerhans pushed a commit to nmeyerhans/cloud-init that referenced this issue Aug 24, 2023
ec2: initialize get_instance_userdata return value to bytes
8f8c653 
canonical#4276 uncovered an issue
with the initialization of the return value for
get_instance_userdata().  The return value was initialized with

user_data = ""

which is a str class.  It then calls url_helper.read_file_or_url(),
which attempts to retrieve user-data content from IMDS.
read_file_or_url() returns its results as a bytes object, which is
then passed directly up to the caller.  In the event that
read_file_or_url() does not successfully retrieve
content (e.g. because it was given a file:// path to a nonexistent
file or an http:// path that generates a 404 code), an exception is
raised an get_instance_userdata returns the string object initially
stored in user_data.

Rather than make the caller cope with return data potentially encoded
as either bytes or str, this commit changes the initialization of
user_data to an empty bytes object, ensuring type consistency in
get_instance_userdata()'s return value.

Fixes canonical#4386

Signed-off-by: Noah Meyerhans <[email protected]>
nmeyerhans pushed a commit to nmeyerhans/cloud-init that referenced this issue Aug 24, 2023
ec2: initialize get_instance_userdata return value to bytes
a9bb508 
canonical#4276 uncovered an issue
with the initialization of the return value for
get_instance_userdata().  The return value was initialized with

user_data = ""

which is a str class.  It then calls url_helper.read_file_or_url(),
which attempts to retrieve user-data content from IMDS.
read_file_or_url() returns its results as a bytes object, which is
then passed directly up to the caller.  In the event that
read_file_or_url() does not successfully retrieve
content (e.g. because it was given a file:// path to a nonexistent
file or an http:// path that generates a 404 code), an exception is
raised an get_instance_userdata returns the string object initially
stored in user_data.

Rather than make the caller cope with return data potentially encoded
as either bytes or str, this commit changes the initialization of
user_data to an empty bytes object, ensuring type consistency in
get_instance_userdata()'s return value.

Fixes canonical#4386

Signed-off-by: Noah Meyerhans <[email protected]>
@holmanb holmanb added priority Fix soon and removed new An issue that still needs triage labels Aug 24, 2023
@nmeyerhans nmeyerhans mentioned this issue Aug 24, 2023
Merged
3 tasks
holmanb pushed a commit that referenced this issue Aug 24, 2023
ec2: initialize get_instance_userdata return value to bytes (#4387)
980f690 
#4276 uncovered an issue
with the initialization of the return value for
get_instance_userdata().  The return value was initialized with

user_data = ""

which is a str class.  It then calls url_helper.read_file_or_url(),
which attempts to retrieve user-data content from IMDS.
read_file_or_url() returns its results as a bytes object, which is
then passed directly up to the caller.  In the event that
read_file_or_url() does not successfully retrieve
content (e.g. because it was given a file:// path to a nonexistent
file or an http:// path that generates a 404 code), an exception is
raised an get_instance_userdata returns the string object initially
stored in user_data.

Rather than make the caller cope with return data potentially encoded
as either bytes or str, this commit changes the initialization of
user_data to an empty bytes object, ensuring type consistency in
get_instance_userdata()'s return value.

Fixes GH-4386

Signed-off-by: Noah Meyerhans <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working correctly priority Fix soon
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ec2: initialize get_instance_userdata return value to bytes nmeyerhans/cloud-init
2 participants
@nmeyerhans @holmanb