Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic support for calling Pathfinder #19

Merged
merged 1 commit into from
May 17, 2024
Merged

Conversation

WardBrian
Copy link
Collaborator

This adds the basics to allow calling Pathfinder:

  • Request the _tinystan_pathfinder function in the build config
  • Add a function to the tinystan wrapper to call this, re-using a lot of the same machinery as the sampler
  • Add a new Request variant in the worker for requesting Pathfinder be run
    • I decided for now to have the return message be the same - we could obviously change this if we wanted

This this does not do:

  • Any UI for calling it instead of sampling (to test, I just replaced the call to the sampler with the call to pathfinder in our existing UI). It also has its own set of control flags we'd want to expose, like Control parameters for sampling #16

  • Any way to wire the results of Pathfinder up as initial values for sampling (needlessly hard, but probably worthwhile eventually)

@WardBrian
Copy link
Collaborator Author

(The next thing I think I want to do in this interface is start breaking it up so that I have a Types.ts, Constants.ts, etc which separate out a lot of the boilerplate)

@WardBrian
Copy link
Collaborator Author

Oh, also, Pathfinder doesn't really have a reasonable analogue of the progress for sampling. You tell it a maximum number of iterations to run, but in basically all practical problems it will run for less than that, so you can't really calculate a % complete.

It's much faster than sampling though, so this is perhaps not as big an issue as it was for sampling, but maybe we will want a spinny wheel rather than a progress bar here.

@magland
Copy link
Collaborator

magland commented May 17, 2024

Is this something that would be run prior to sampling? I'm not familiar with pathfinder. Could you please give some explanation on how you imagine this working within stan-playground?

@WardBrian WardBrian linked an issue May 17, 2024 that may be closed by this pull request
@WardBrian
Copy link
Collaborator Author

I describe a bit in #15, but you can want to run Pathfinder for two different reasons.
The first is you're treating it like an approximate inference algorithm in its own right, and would use the results in approximately the same way you would in sampling.
The second is you want to use the approximate posterior as an initial value for MCMC. This requires taking the output draws from Pathfinder and re-shaping them into the format Stan expects as inputs, which is unfortunately nontrivial.

Eventually, I could see there being two buttons on the "samples" page for Pathfinder - one that downloads all of them as CSVs, and another that downloads a specified number of JSON files for initialization (or maybe it doesn't download them, maybe it just populates whatever initialization editor we end up coming up with).

But the second thing is hard, so for now I think it is reasonable to just think of this as a strict alternative to sampling, e.g. a different tab altogether in the run panel

@magland
Copy link
Collaborator

magland commented May 17, 2024

Makes sense. Should we go ahead and merge this PR even though we are not giving the option of running pathfinder?

... and rebuild the docker?

@WardBrian
Copy link
Collaborator Author

I think @jsoules wanted to take a look, but I do think we should merge this separately from the UI to use it just to keep it small

Copy link
Collaborator

@jsoules jsoules left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not so much to test here, but I don't see any show-stoppers on read-through. A couple confirmation questions and one suggestion, but otherwise this is good to go.

docker/make/local Show resolved Hide resolved
@@ -41,7 +41,7 @@ class StanSampler {
this.#onStatusChangedCallbacks.forEach(cb => cb())
break;
}
case Replies.SampleReturn: {
case Replies.StanReturn: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own clarification, what's the distinction being made here? Is it "pathfinder doesn't technically do sampling but will still use this verb so we don't want to call it Sample"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, whether or not you can still call the result of an approximation "draws" or "samples" is a bit sticky.

I also think the name previously was incorrect for when the return is actually an error

return { paramNames, draws };
});
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to convert the _malloc(4) calls below to use PTR_SIZE instead, since you added it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the 4 in those mallocs is a different 4 than the ones I changed here (it is not necessarily true that sizeof(int*) == sizeof(int), in fact on x64 systems it is false)

So I could define an INT_SIZE const variable, but those are also the only places I think the API will ever need to malloc an int (in retrospect, I could use Int32Array.BYTES_PER_ELEMENT as something more self-documenting, but also the fact that Int32 is in the name does pretty strongly imply the answer will be 4...)

@jsoules
Copy link
Collaborator

jsoules commented May 17, 2024

(The next thing I think I want to do in this interface is start breaking it up so that I have a Types.ts, Constants.ts, etc which separate out a lot of the boilerplate)

Yeah, index.ts is starting to get a little long. 😄

@magland
Copy link
Collaborator

magland commented May 17, 2024

I'm going to merge this so I can build the new docker image and we can make sure everything is still working properly.

@magland magland merged commit aa1d0d3 into main May 17, 2024
@magland
Copy link
Collaborator

magland commented May 17, 2024

Pushed new docker image, and everything seems to be working still with the public wasm compiler server, so that's good news.

@WardBrian
Copy link
Collaborator Author

I built and tested the new image locally before opening the PR as well

@WardBrian WardBrian deleted the feat/pathfinder-code-support branch May 22, 2024 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Pathfinder
3 participants