Replace scrapy FilesPipeline by QueridoDiarioFilesPipeline

Updates the default item pipeline to use the custom class QueridoDiarioFilesPipeline. With this class the gazettes from the same date will be store in the same directory. Signed-off-by: José Guilherme Vanz <[email protected]>
okfn-brasil · Jul 4, 2020 · fe86546 · fe86546
1 parent f40a92a
commit fe86546
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/roles/spider/defaults/main.yml b/roles/spider/defaults/main.yml
@@ -7,7 +7,7 @@ timer:
     oncalendar: "daily"
     randomizeddelaysec: 43200
 
-ITEM_PIPELINES: "{\"gazette.pipelines.GazetteDateFilteringPipeline\": 50, \"scrapy.pipelines.files.FilesPipeline\": 100,}"
+ITEM_PIPELINES: "{\"gazette.pipelines.GazetteDateFilteringPipeline\": 50, \"gazette.pipelines.QueridoDiarioFilesPipeline\": 100,}"
 SPIDER_MIDDLEWARES: "{\"scrapy_deltafetch.DeltaFetch\": 100}"
 
 #Digital Ocean spaces config