Fix slow pantry caching by using memory as much as possible (#78)

Fix slow pantry caching by using memory as much as possible (#78) #314