Errors when using Lambdakiq

[thailow]

Hi @metaskills,
I am trying to include lambdakiq in my lamby project. However, I am facing two exceptions which I do not know how to handle. Maybe you can help (would be greatly appreciated).

• Errno::EROFS (Read-only file system @ dir_s_mkdir - /var/task/tmp 
  • This seems to be a problem for all active jobs I am triggering. For some reason, it wants to write sth. somewhere. /var/task is included as a path in the Dockerfile. Possibly this is related to my usage of ActiveJob::Status? Is there any way to know about a job's status in Lambdakiq without using the activejob-status gem?
• NoMethodError (undefined method `lambdakiq_async?' for #ActiveStorage::AnalyzeJob:0x0000000008973f08)
  • This error seems to be related to saving storage objects to S3 on Amazon. I do not know how to included Lambdakiq here.

Thanks! (Did not post that in issues since I guess it's just me doing sth. wrong..)

[metaskills]

Hey there @thailow

Errno::EROFS (Read-only file system...

Correct, the Lambda file system is not writable. Any logging should be done via STDOUT. We ensure this is done for Rails & any other gems using these hacks (https://github.com/customink/lamby/blob/master/lib/lamby/logger.rb) to the Logger object in Lamby. Are you using Lamby? If not, you need to make sure your Rails and other loggers write to STDOUT or do the hack we did to Logger to ensure this. Note, we set the (assumed STDOUT) Rails logger to the active job logger here. https://github.com/customink/lambdakiq/blob/main/lib/lambdakiq/railtie.rb#L8

Possibly this is related to my usage of ActiveJob::Status?

I took a look at the gem. It does nothing special with logging. Seems to require a persistent store for the data. So I think your issue is either in the Rails logger and hence ActiveJob's logger. See links above.

Is there any way to know about a job's status in Lambdakiq without using the activejob-status gem?

Can you share why you need that gem? This gem has built in Observability and Metrics with CloudWatch (again via STDOUT) to the Lambda log. See details here https://github.com/customink/lambdakiq#observability-with-cloudwatch

[thailow]

Hi @metaskills,
thanks a lot for your quick reaction - with your help and a little tinkering around, I managed to solve the first exception (Errno::EROFS).

Errno::EROFS (Read-only file system...

Yes, I am using Lamby (just migrated to it). Basically, the way I mounted EFS into the Lambda was wrong concerning the required path prefix. To quote the referenced source, "Basically the EFS Access Point’s Root Directory Path has to be the same as the EFS Mount’s Local Mount Path, minus the “/mnt” prefix, and vice versa." With that thing fixed, ActiveJob::Status is now working!

Can you share why you need that gem? This gem has built in Observability ...

I had integrated ActiveJob::Status before migrating to Lamby and just wanted to get it running again. Since a shared folder was the only of its options (for persisting status information), I decided for the EFS approach. However, I will look into getting rid of it - thanks for your hint.

NoMethodError (undefined method `lambdakiq_async?' for #ActiveStorage::AnalyzeJob:0x0000000008973f08)

Still struggeling with this. Did you ever manage to set up ActiveStorage on S3 with Lambdakiq? That deeply integrated AnalyzeJob does not seem to know about Lambdakiq... In my use case, a user may create an entity with attached Excel file and the mentioned error already occurs when clicking on "Create", hence, before the ActiveJob even kicks off.

[thailow]

To add on this: I commented out my ActiveStorage analyzer for the moment and that made the attachment upload work. However, when I delete my entity again, the PurgeJob fails (which is there by default) with the same exception:

NoMethodError (undefined method `lambdakiq_async?' for #ActiveStorage::PurgeJob:0x0000000004c40d48)

Any ideas?

[metaskills]

Wow... yes, totally. Getting rid of EFS will help you out immensely. How? I am not sure, you will need some shared state. Seems ActiveJob::Status can hook into the Rails.cache? If so, maybe use a small Elasticcache Memcached instance and a VpcConfig with your function to access it? We use Memcached and it works really well.

With regard to the undefined method lambdakiq_async?'`. I recently ran into that too for other non-base ActiveJobs. Have you seen this recent commit? a783d77 Including the worker via an initializer may help?