Can't pickle local object

[AnqiZou77]

Hi,
Thank you for your sharing.
When I run the code, There is an error occurred in this line "for i, (inp, target) in enumerate(test_loader)" ,
Somebody tell me that "python can't pickle functions. ", can you point me the right way to fix it?
I run the code on win7 python 3.6 pytorch 0.1.12.
Than you~

[chaoyan1037]

Did you modify the code? I have run the original code, too. There are no problems.

[AnqiZou77]

No, I just run the original code. can you tell me your environment?

[chaoyan1037]

Ubuntu 14.04 64-bit
Anaconda 4.3.30
Python 3.6.3
pytorch 0.3.0

[billhhh]

I met the same problem too? not so sure why
the error is "AttributeError: Can't pickle local object 'main..'"

[AnqiZou77]

Yes, I encountered the same error. Did you fix it?@billhhh

[billhhh]

Not yet, @CocaKoala7

[jubaer145]

Hi, did anybody solve the "can't pickle local objects" issue? Thanks in advance.

[ghost]

@omrfrkmfy This is the right issue. It seems the issue is the pickler. I think multiprocessing has to pickle the dataset to use across processes. The pickle module, unlike a better pickler like dill, can't pickle lambda functions, transforms.Lambda(lambda data_idk: do_some_stuff_here...). You could test if thats the issue by directly trying to pickle the test_dataset. If thats the problem, a simple solution would be changing the lambdas to real functions, looking something like this (untested)

# somewhere outside the "main" function
def crops_to_tensors(crops):
    to_tensor = transforms.ToTensor()
    return torch.stack([to_tensor(crop) for crop in crops])

def normalize_crops(crops):
    normalize = transforms.Normalize([0.485, 0.456, 0.406],
                                     [0.229, 0.224, 0.225])
    return torch.stack([normalize(crop)for crop in crops])

# in the "main" function
test_dataset = ChestXrayDataSet(data_dir=DATA_DIR,
                                image_list_file=TEST_IMAGE_LIST,
                                transform=transforms.Compose([
                                    transforms.Resize(256),
                                    transforms.TenCrop(224),
                                    transforms.Lambda(crops_to_tensors),
                                    transforms.Lambda(normalize_crops),
                                ]))

[pharouknucleus]

Yes it worked. But sorry for the disturbance;
Traceback (most recent call last):
File "C:/Users/Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master/m2.py", line 149, in
main()
File "C:/Users/Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master/m2.py", line 76, in main
transforms.Lambda(normalize_crops),
File "C:/Users/Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master\read_data.py", line 24, in init
with open(image_list_file, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: './ChestX-ray14/labels/test_list.txt'

[zhyl123]

是的，它奏效了。但很抱歉打扰了；
回溯（最近一次通话）：
文件“C:/Users/Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master/m2.py”，第 149 行，在
main()
文件“C:/Users /Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master/m2.py”，第 76 行，在主
转换中。Lambda (normalize_crops)，
文件“C:/Users/Nasir Isa/Documents/1Research/algortihm /CheXNet-master/CheXNet-master\read_data.py", line 24, in init
with open(image_list_file, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: './ChestX-ray14/标签/test_list.txt'

Have you solved this problem? Can you share your ideas?

[anagraw]

是的，它奏效了。但很抱歉打扰了；
回溯（最近一次通话）：
文件“C:/Users/Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master/m2.py”，第 149 行，在
main()
文件“C:/Users /Nasir Isa/Documents/1Research/algortihm/CheXNet-master/CheXNet-master/m2.py”，第 76 行，在主
转换中。Lambda (normalize_crops)，
文件“C:/Users/Nasir Isa/Documents/1Research/algortihm /CheXNet-master/CheXNet-master\read_data.py", line 24, in init
with open(image_list_file, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: './ChestX-ray14/标签/test_list.txt'

Have you solved this problem? Can you share your ideas?

Did you get a chance to rectify this issue?