Can extensions replace extractors/transformers/formatters?

[ryanmccarthypdx]

This is a space to discuss how we might fold Blueprinter's various customization options (extractors, transformers, formatters, etc) into the recently added extension system.

Goals

1. Simplify the codebase.
2. Reduce the knowledge burden on users.
3. Simplify the addition of new customization options/hooks (i.e. we can add new hooks without adding new config options).
4. Foster Blueprinter's nascent extension ecosystem.

[jhollinger]

#417 is one step in this direction, allowing for class and view-specific extensions.

[jhollinger]

When we design some kind of transform hook to replace transformers, there's a disconnect we'd have to address. iirc the global default_transformers config option is ignored when there are class/view-specific transformers. However, in #417 the global extensions config option always applies.

I can imagine people wanting it either way (pretty sure I've seen requests to that effect). So maybe we should support two global extension configs? Something like:

1. extensions (always used)
2. default_extensions (ignored when there are class/view-specific extensions)

[jhollinger]

A more flexible way would be to specify this when adding the extension(s). By default, all extensions are always inherited. But at each level you could optionally ignore all "higher" extensions. Trying to spec out all the possible permutations:

class WidgetBlueprint < Blueprinter::Base
  # inherits global extensions, adds MyExt
  extension MyExt.new

  view :foo do
    # inherits global and :default view extensions
  end

  view :foo1 do
    # inherits global and :default view extensions, adds FooExt
    extension FooExt.new
  end

  view :foo2 do
    # inherits global, :default view, and :foo1 view extensions
    include_view :foo1
  end

  view :foo2 do
    include_view :foo1
    # only use FooExt
    extension FooExt.new, inherit: false
    # add BarExt and ZorpExt (:inherit retains its previous value of false)
    extension BarExt.new, ZorpExt.new
  end

  view :foo3 do
    include_view :foo1
    # no extensions at all
    extension inherit: false
  end

  view :foo4 do
    # inherits extensions from :foo3, which is "none"
    include_view :foo3
  end
end

[ritikesh]

so the ones that want to inherit, would have to explicitly opt-in? Also, wouldn't I go down the path of an ApplicationBlueprint where I do all of this and just inherit it in all of my blueprints without having to worry about what defaults should be included?

[jhollinger]

Sub-topic for how we'd release all this without disrupting people's configurations too much. First thought is:

1. Gradually add "drop-in replacement" extension hooks in various 1.x releases.
2. By the end of our 1.x series, we'd have full support for both approaches.
3. Drop the old config options during a major version bump (presumably 2.0).

[ritikesh]

where should we track the specifics?

[sandstrom]

I'm overall for making Blueprinter more customisable!

And I've suggested a few things myself:

• Implement association as it's own thing #407
• Deprecate DateTimeFormatter #406
• Separate internal options from external ones #405
• Extractor configurability #404
• Move global configuration #403

In general, I think blueprinter has such as small scope so I'm not sure a full 'extension system' is needed. Just a few though-out hooks should be enough. Take rails for example (a huge framework), they didn't have an extension system for a long time, but it's very easy to customize.

That said, if there is a preference for putting all the hooks into an something called an extension system, I'm not against it. Happy to provide feedback if there is a sketch or code plan of some sort! (public APIs, etc)

Some random thoughts:

• I'd try to avoid anything global, instead, just let people make customizations by putting a base class between the library class and individual blueprints.

  class ApplicationBlueprint < Blueprinter::Base
    # let the app make any customizations here, i.e. configure hooks or add extensions or whatever
  end
  
  class WidgetBlueprint < ApplicationBlueprint
    # actual blueprint class
  end

• Try to use regular ruby, and object oriented programming paradigms as much as possible. So e.g. if the extensions should be inherited but configurable, just use a class attribute or similar and then inherited to modify it as needed.

[jhollinger]

I kind of like the "avoid global config"/"use ApplicationBlueprint" idea - there's only one way of configuring things. Obviously a breaking change, but it could be made forward-compatible in 1.x.

[jhollinger]

The idea to use class attributes for config/extensions could work IF we refactored views so that each one was an anonymous class inheriting from the blueprint or parent view. An intriguing idea, but it would be a large refactor.

[ritikesh]

Extensions shouldn't execute on any state from blueprinters IMO. That's a state spill. They could work on the data at hand (the objects to be JSONified/already JSONified) but nothing more.

[ritikesh]

ApplicationBlueprint is something we've always used internally defining some helper functions, view hierarchies, etc. and is the ideal way to define or set defaults for extensions going forward. We could very well use this for replacing the default behaviours today as well by defining and including the extensions in a suggested ApplicationBlueprint. However, this should be generated and not be included in the gem.

[jhollinger]

From my experiments in #479:

Formatters

Make them part of the Blueprint DSL. Specify a class and a block/method, and all field values of that class will be formatted.

# Define "global" formats in ApplicationBlueprint.
class ApplicationBlueprint < Blueprinter::V2::Base
  format(Date) { |date| date.iso8601 }
  format Time, :fmt_time

  def fmt_time(t)
    t.iso8601
  end
end

# Other blueprints can override and define others
class WidgetBlueprint < ApplicationBlueprint
  format(Float) { |n| "%.2f" % n }
end

[sandstrom]

I think this is a good idea! 👍🏻

I'm guessing that the ::V2 part is just during testing, right? And that any real change would be at the gem version level, just like other gems.

[jhollinger]

What we've been discussing is releasing a version of blueprinter with both "legacy" and V2 available and interoperable (i.e. legacy Blueprints could have associations to V2 Blueprints and vise versa). Kind of like Rails does with migration DSL versions.

Our thinking is that's the only way large apps could ever hope to update. Eventually legacy would be dropped and V2 would become the default (somehow).

[sandstrom]

I don't think that's a good idea. The tried and tested method of using gem versions and a changelog works pretty well.

That said, if you really want to do it, how about releasing 2.0.0 of the gem with the new stuff, and no V2 namespace, and then a final 1.2.0 on the 1.x line with the V2 namespace? (this would be interoperable with both)

With 1.x being essentially frozen after that point, with the exception of patch updates for serious bugs or security vulnerabilities.

That way, you wouldn't have to maintain two code-bases going forward (with all the extra work that this entails), i.e. on master the V2 namespace won't really ever exist.

It would also allow anyone who'd prefer to jump straight to the new thing to just skip 1.2 and go straight to 2.x.

[jhollinger]

We'll definitely need a gem version that supports both legacy and V2 at the same time, so that large apps can update gradually (unless we come up with a magic auto-update script). But I think I like your idea of releasing that in a "final" 1.x version, and making 2.x V2 only.

[jhollinger]

From my experiments in #479:

Extractors

I really tried to fit this into extension hooks, but it didn't seem like a great fit. I think it's simpler and easier to extend than our current offering, though.

Note: context is a struct containing the current Blueprint instance, the current field, and the current object (and a few other things not relevant here).

class MyBlueprint < ApplicationBlueprint
  # Set it for the whole blueprint/view
  options[:extractor] = StringKeyHashExtractor

  # or on a certain field
  field :foo, extractor: StringKeyHashExtractor
end

# An extractor for Hashes with String keys 
class StringKeyHashExtractor < Blueprinter::V2::Extractor
  # Extract a regular field
  def field(context)
    if context.object.is_a? Hash
      key = context.field.from.to_s
      context.object[key]
    else
      super
    end
  end

  # Extract a belongs_to/has_one type association
  def object(context)
    field context
  end

  # Extract a has_many type association
  def collection(context)
    field context
  end
end

You can find a proposed implementation of Blueprinter::V2::Extractor here.

[jhollinger]

From my experiments in #479:

Transformers

Entirely hook based.

Note: context is a struct containing the current Blueprint instance, the current field, the current object, and the current field or blueprint value (depending on the hook). Plus a few other things not relevant here.

class MyExtension < Blueprinter::Extension
  # Called exactly once per render, at the very beginning (when an object is being rendered)
  def input_object(context)
    # return context.object or a new/modified object
  end

  # Called exactly once per render, at the very beginning (when a collection is being rendered)
  def input_collection(context)
    # return context.object or a new/modified object
  end

  # Returns fields in the order they should be serialized (can also filter out fields)
  def blueprint_fields(context)
    # default = context.blueprint.class.reflections[:default].ordered
  end

  # Called right before every Blueprint serialization
  def blueprint_input(context)
    # return context.object or a new/modified object
  end

  def exclude_field?(context)
    # return true to block the field
  end

  def exclude_object?(context)
    # return true to block the object field
  end

  def exclude_collection?(context)
    # return true to block the collection field
  end

  def field_value(context)
    # return context.value or something else
  end

  def object_value(context)
    # return context.value or something else
  end

  def collection_value(context)
    # return context.value or something else
  end

  # Called right after every Blueprint serialization. This is probably closest to V1's transformers.
  def blueprint_output(context)
    # return context.value or a new/modified object
  end

  # Called exactly once per render, at the very end (when an object is being rendered)
  def output_object(context)
    # return context.value or a new/modified object
  end

  # Called exactly once per render, at the very end (when a collection is being rendered)
  def output_collection(context)
    # return context.value or a new/modified object
  end
end