CHANGES.md

summingbird

0.3.0

• Add serializers for BatchID, Timestamp: twitter#350
• ADd a means to set defaults outside the planner: twitter#351
• fix bugs hdfs: twitter#353
• add default option setting: twitter#356
• Add options tests: twitter#335
• Add some hadoop defaults to the builder API: twitter#359
• Cleanup the Storm API a bit: twitter#360
• Put all Collector actions in one execute: twitter#365
• Producer API executors for storm and scalding: twitter#367
• standardize get or else scalding: twitter#363
• Fixes ticks: twitter#368
• spout metrics: twitter#344
• Add the summingbird prefix to the graphs: twitter#370
• add localOrShuffle: twitter#371
• Make the StormMetric Covariant: twitter#373
• Fix depreciation to be from sink: twitter#375
• add options to cfg: twitter#376
• Null out Tuple.values for GC: twitter#374
• Add timestamps to hadoop configs: https://github.com/twitter/summingbird/pull/377/files
• add Options.toString: twitter#378
• Fix out of sync versions to keep: twitter#379
• Feature fix ffm: twitter#381
• Add an await timeout: twitter#380
• Add local or shuffle option: twitter#384
• Add dequeueAll to Queue: twitter#386
• Add in log4j and slf4j to the example: twitter#392
• Fixes naming in bolts for options to work right after plan: twitter#391
• Add error checking to the cache size: twitter#393
• Add more storm tests, add some new test graphs: twitter#394
• Proxy the select correctly on initialbatch: twitter#389
• Refactors the bolts from storm to be independent workers in online: twitter#395
• Pluggable caches in FinalFlatMap: twitter#396
• Uses multimerge: twitter#399
• ack on entry option: twitter#398

0.2.5

• Fix issue with externalizer: twitter#316
• Initial implementation of checkpoint state: twitter#315
• Move to sbt 0.13: twitter#322
• Move Dependants common code to graph: twitter#321
• CheckpointState -> HDFSState: twitter#318
• Change depthFirstOf to use List instead of Vector: twitter#323
• Add an optional parameter to set default parallelism for a storm source: twitter#319
• Add a method to prepare for optimizing FlatMapKeys: twitter#305
• fix the increased output issue: twitter#327
• Optimize iterator sums: twitter#326
• Make sure to register BatchID and Timespan: twitter#324
• Sum by key test: twitter#330
• Add slf4j logging to scalding: twitter#331
• Move the versioned store to scalding package: twitter#332
• storm logging: twitter#333
• Adds/extends some storm option tests: twitter#334
• Make VersionedState fail-safe: twitter#337
• Throw on Storm Submission w/ no online store: twitter#338
• General graph support online: twitter#340
• Summingbird config: twitter#339

0.2.4

• Fix off-by-one bug in scalding batched store: twitter#311

0.2.3

• Add convenience methods to Producer: twitter#307
• Massively optimize Scalding merge/sumByKey: twitter#303
• Improve the WaitingState state machine: twitter#302
• Move code for common realtime planners (storm + akka): twitter#299
• Fix an issue with source minify in scalding: twitter#298
• Improve scalding tests: twitter#296
• Improve storm tests: twitter#292
• Use an AnyVal-like class rather than java.util.Date: twitter#295
• improve Batcher documentation: twitter#293

0.2.2

• Use pipes instead of parentheses when naming online physical nodes: twitter#273
• Add Alsos in Producer2, OptProd2: twitter#275
• Multiple disjoint summers: twitter#274
• Make Tail sealed: twitter#276
• Scalding Laws not to use scalacheck: twitter#278
• Pull Online planner common code into core: twitter#281
• Use externalizer instead of meatlocker: twitter#282
• Standardize SourceBuilder Naming: twitter#285
• Add TailProducer, for graph ending point: twitter#286
• MemoryLaws not use scalacheck: twitter#287
• Added flatMapKeys to all platforms: twitter#288
• Added flatMapKeys to Builder API: twitter#290
• Add better type safety to Storm Spouts: twitter#289
• Add a streaming left join: twitter#291

0.2.1

• Add support for map only jobs in producer api twitter#269
• Fixes naming of nodes, options are picked up correctly twitter#267
• Fix missing elements in case statements for applying online graph to storm twitter#272

0.2.0

• Bump the version numbers of the dependencies twitter#260
• Upgrade to Tormenta 0.5.2, Scalding 0.9.0, Bijection 0.5.3 betas: twitter#191
• Feature/storm new planner: twitter#250
• Feature/move dag to core: twitter#255

0.1.5

• Control how futures are collected in Client Store: twitter#254
• Producer[Platform, T] is covariant on T: twitter#251
• Improve testing generators: twitter#249
• Remove manifests from the core API: twitter#247
• Dot graphs of our storm plan (what SB nodes go to physical nodes): twitter#236

0.1.4

• Fix Storm Tests: twitter#227
• Custom --name flag for job name in builder api: twitter#226
• Remove SINK_ID reference from SourceBuilder: twitter#225
• Add Dot Graph Generation from SB Producer: twitter#223
• Push filters down to storm spout: twitter#224
• Revert StoreIntermediate to old approach: twitter#219
• Add in old SummingJoin: twitter#218

0.1.3

• Fix bug in storm planner's forking: twitter#217
• Don't send empty multiGets to online service: twitter#209

0.1.2

• UTC Calendar Batcher: twitter#195
• Register injections in StormEnv: twitter#198
• Fix Forking bug in Storm planner: twitter#197
• Tidying of StormPlatform: twitter#199
• Add AnchorTuples option to Storm: twitter#200
• Allow NamedProducer after Summer in storm: twitter#202
• Optimization when using BijectedMonoid in a Scalding job: twitter#203
• Remove need for --initial-run (--start-time suffices): twitter#204

0.1.1

• Add --scalding.nothrowplan to not rethrow/exit with error on a flow plan error

0.1.0

• new, producer-based API
• Split Storm and Scalding platforms
• Added in-memory platform
• Complete DSL rebuild in anticipation of release.

0.0.4

• fix compiler-induced bug in BatchAggregatorJob.

0.0.3

• Ability to tee out data.

0.0.2

• PresentingStore
• Ability to join a Summingbird job against another SB job.

0.0.1

• Initial code push.