Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add life-cycle methods to PostProcessor modifiers #245

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

hmf
Copy link

@hmf hmf commented Dec 13, 2019

Before, mdoc would instantiate and execute PostModifier instances without any indication when these terminated their processing on all documents. This means that if we created some resources in these PostModfiers, such as instantiating a daemon thread, there was no way of releasing these resources. In the case of daemon threads, this meant that a PostModifer could prevent the CLI call from terminating.

Now, mdoc call the onStartmethods jus before the files are processed and onExit just before mdoc terminates. This gives the PostModifier instances an opportunity to create and release any required resources. For example the daemon thread can be signaled to terminate onExit thereby allowing the mdoc CLI to terminate.

Changes for #213

@hmf hmf mentioned this pull request Dec 16, 2019
mdoc/src/main/scala/mdoc/PostModifier.scala Show resolved Hide resolved
mdoc/src/main/scala/mdoc/PostModifier.scala Show resolved Hide resolved
@@ -171,7 +171,9 @@ class Processor(implicit ctx: Context) {
inputFile,
ctx.settings
)
modifier.preProcess(postCtx)
val postRender = modifier.process(postCtx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of preProcess and postProcess if the modifier could call those methods directly inside of process? Judging by the implementation, it doesn't look like mdoc does anything between preProcess, process and postProcess

class MyProcess extends PostProcess {
  def preProcess() = ()
  def process() = {
    preProcess()
    // ...
    postProcess()
  }
}

Copy link
Author

@hmf hmf Dec 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. This is incorrect. Does nothing. I think these calls should be removed. Ok with you? Want to change the onStart and onExit names?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

onStart and onExit are fine

@olafurpg
Copy link
Member

Thank you for this contribution! Can you please update the description to explain the motivation for this change, something like

Before, mdoc ...
Now, mdoc ...

@olafurpg olafurpg changed the title Post life cycle Add life-cycle methods to PostProcessor modifiers Dec 16, 2019
@hmf hmf requested a review from olafurpg December 19, 2019 13:28
}

object Exit {
def apply(exit: CliExit): Exit = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this method, or make it package private

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to make this package private because it is used here in MainOps.

// Global thread local counter updated by all mdoc.Main process
// All tests in this test suite run sequentially but change the counter
// So make sure we start anew for this test
LifeCycleCounter.numberOfStarts.set(0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we introduce some reset() helper method to do this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -46,4 +49,132 @@ abstract class BaseCliSuite extends FunSuite with DiffAssertions {
onStdout(stdout)
}
}

def assertOneSameOrPrintExpected(obtained: String, expected: List[String], title: String = "")(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to copy-paste so much code?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the comments (see lines 72 and 73) the scala.meta.testkit.DiffAssertions methods that I needed cannot be accessed because they are private. These belong to the testkit package. If these were accessible much of the copy & paste is not required. Alternatively the assertOneSameOrPrintExpectedcould be placed in the testkitpackage.

Seeing as it is not part of MDoc I opted for this solution. Any way testkit can be changed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What functionality is missing? We should avoid copy pasting this amount of code

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not required anymore so it was removed. See my explanation below. Initially I used this to test the pre and post processing methods. You correctly pointed out they were not necessary. So I removed them but did not correct the test.

…CycleCounter now has a reset method that is used in the CliSuite test.
}
}

def checkCliMulti(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you try using the checkCli method? It supports processing multiple files. I'm curious to understand why this method is needed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is the ordering in which the files are processed. Assume files A and B. If A gets processed before B, then A gets a count of 1 and B a 2. If the order is reversed then the B gets count 1 and A gets 2. The order in which the files are processed in my local machine and in the CI/CD VM were different. So I crated a checkCliMulti method, which checks that at least one of the conditions I stipulate is true. I suspect that the transformation tests you have don't have this issue. Maybe I don't need this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just realized I don't need this test. I was using it when I had the pre and post process that were removed. So I am removing this.

@hmf hmf requested a review from olafurpg December 28, 2019 13:37
@olafurpg
Copy link
Member

olafurpg commented Jan 2, 2020

I haven't forgotten about this PR. There are a few organizational changes that I'd like to make to the codebase before merging this PR so it may take a while longer.

I have a feeling we may need to bump to v3.0 due to unavoidable binary incompatibilities but I would still like to keep the breaking changes as small as possible.

The reason I care about keeping a stable API so much is that I want people to be able to publish their mdoc modifiers without having to re-publish on every small change in the mdoc API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants