Which breaking changes should be made for Remotion v5?

[JonnyBurger]

It is a chance for us to fix bad API decisions (and fix flaws in our licensing + raise awareness of our license)

• visualizeAudio() should switch to optimizeFor: "speed" by default
• selectComposition() and getCompositions() should require inputProps (footgun)
• getCompositions() should have a new signature
• Drop support for Node 16
• Drop support for React 17
• getTangentLength() and getPointAtLength() should return null if length is bigger than the path length
  • --> Still needs documentation
• Rename useVideoConfig() to useSequence()
• Export in BT.709 by default
• Set validateFontIsLoaded by default to true for layout utils
  • --> Still needs documentation
• Should not accept local browser anymore
  • --> Should update all Docker instructions
  • --> Warn about passing a browser explicitly
  • --> Re-test all Docker images
• Remove old bundle() signature - return type maybe should not just be a string?
• openBrowser() should just accept logLevel instead of dumpIo
• <Series> should be made a <Sequence> itself
  • --> Docs still required
• Remove --disable-headless
• Put bundle in build folder
• Upgrade AWS Lambda runtime
• License: Freelancers should also count like employees towards headcount of company
• License: Company license need to agree to terms and conditions
• Upgrade TypeScript-ESLint -> Also carries a Node.js upgrade and TypeScript upgrade
• Pin Zod to 3.23
• iReadTheLicenseOnRemotionDevLicense prop for the Player -> Doesn't make sense because templates start with error
• Remove measureSpring() from and to values
• Lambda overwrite option should be true -> Allows to skip the check
  • --> Update renderMediaOnLambda() docs
  • Update Lambda, PHP, Go clients
• Consider new tsconfig.json default - "moduleResolution: 'bundler' " (From CodeHike)
• Use util.parseArgs instead of minimist -> Minimum version Node 18.3 -> No because that breaks Bun support
• Templates use npm run dev as default
• Set pauseWhenBuffering to true by default
• Remove <Experimental.Null> and <Experimental.Clipper>
• Make skipLambdaInvocation the default
• Remove getRenderProgress() and renderMediaOnLambda() from @remotion/lambda

[dillingham]

Been messing with remotion for a short time so far (loving it! thank you!)

I may not have worked with it or react long enough, so take these ideas with a grain of salt.

• durationInFrames could just be frames everywhere
  • seconds could be added also if that was the intention of making the distinction
• useVideoConfig() could be useSequence() so videoConfig means composition
• <Sequence> could work standalone and within <TransitionSeries> and <Series>
  • instead of <Series.Sequence> and <TransitionSeries.Sequence>
• <TransitionSeries> could just be <Transition> and <TransitionSeries.Transition> could just be <Fade>
  • in a perfect world maybe <Fade> works within <Series> and no need for <TransitionSeries>
  • maybe it becomes part of core
• maybe there could be export beforeRender() inside a composition's component to colocate
  • instead of putting api requests in calculateMetadata etc
• maybe <Sequence> and <Series.Sequence> have a component prop like <Composition>
  • maybe useful and cleaner if you just want to render out some scenes without props
• maybe frame and duration could be passed to every Sequence children automatically
  • useCurrentFrame() seems almost like a constant in any component
  • and would maybe avoid making the shift of what seems like global functions to sequence scope
  • (referring to when currentFrame starts at 0 in a sequence even though current frame may be 2000
• maybe some magic to automatically determine the total durationInFrames for calculateMetadata
  • doing the math to figure out transition overlap and dynamic sets of sequences kinda was interesting
  • would be smooth if each of those components reached up somehow and told the main composition their frames

[reactone]

It would be cool to have something like camera movement so that the entire composition can be zoomed in and out, moved, rotate, etc. So for example you have a scene with a graph and a map next to it - show the whole composition first then zoom in on the graph, zoom out, move the camera slightly and zoom in on the map. Or another example - you have a map covering the entire composition - you can move around / zoom in and out or just rotate the entire composition slightly to create a cool effect. I often see it on youtube. @JonnyBurger

[JonnyBurger]

@dillingham Thanks for all the input!

• durationInFrames could just be frames everywhere
  - seconds could be added also if that was the intention of making the distinction

I think frames makes it easy to mistake for frame. Also, the most important information is that it is the duration, right?

useVideoConfig() could be useSequence() so videoConfig means composition

I like this because indeed this hook returns just the data from the parent sequence, not from the video. Not sure about the name useSequence() yet, but I'll add it to the list for now

• <Sequence> could work standalone and within <TransitionSeries> and <Series>
  - instead of <Series.Sequence> and <TransitionSeries.Sequence>

Those components have different props (like for example Series.Sequence has offset additionally but not from). This would make the types confusing if it was all just one element.

• <TransitionSeries> could just be <Transition> and <TransitionSeries.Transition> could just be <Fade>
  - in a perfect world maybe <Fade> works within <Series> and no need for <TransitionSeries>
  - maybe it becomes part of core

We choose the names explicitly. TransitionSeries is for transitioning between scenes, it is not just a "Transition" (actually the Transition is a child of a TransitionSeries), so the rename would be confusing.
TransitionSeries.Transition allows for all kind of presentations, like slide, so "Fade" is also not an accurate name for this component.

• maybe there could be export beforeRender() inside a composition's component to colocate
  - instead of putting api requests in calculateMetadata etc

You can colocate the calculateMetadata() function next to your component!
But to automatically recognize it in a file implies a file-system-based router like in Next.js and that there is just one composition per file, which is not the case.

Right now we don't offer this to eliminate as much magic as possible, but I like making a file-system based composition register in the future as opt-in.

• maybe <Sequence> and <Series.Sequence> have a component prop like <Composition>
  - maybe useful and cleaner if you just want to render out some scenes without props

If you then end up deciding to add props or refs, you will ned to refactor it back to <Sequence><Component {...props}/></Sequence> which is not terribly verbose in my opinion (actually less characters to type!)

• maybe frame and duration could be passed to every Sequence children automatically
  - useCurrentFrame() seems almost like a constant in any component
  - and would maybe avoid making the shift of what seems like global functions to sequence scope
  - (referring to when currentFrame starts at 0 in a sequence even though current frame may be 2000

For React developers, it's unintuitive if props get automatically added to a component. TypeScript would also not pick that up automatically, and you would have to give it a special type annotation for it to recognize that it has additional props, which takes more typing not much easier than just using the useCurrentFrame() hook.

Also, it will make the component re-render on every frame, but not all components need the frame! It is wasteful to make it mandatory for every component

• maybe some magic to automatically determine the total durationInFrames for calculateMetadata
  - doing the math to figure out transition overlap and dynamic sets of sequences kinda was interesting
  - would be smooth if each of those components reached up somehow and told the main composition their frames

Discussed here: https://www.remotion.dev/docs/miscellaneous/automatic-duration

Sorry to smash most of these feature requests.
One of the guiding principles of Remotion is to make all APIs explicit and remove as much magic as possible, and I think those ideas are aligned with the design ideas of React.

The time saved with "magic" often gets lost again because a special behavior was unintentionally introduced, which then leads to increased support from our side.

For syntactic sugar that is not contrary to these priciples, I am always open though!

[JonnyBurger]

It would be cool to have something like camera movement so that the entire composition can be zoomed in and out, moved, rotate, etc. So for example you have a scene with a graph and a map next to it - show the whole composition first then zoom in on the graph, zoom out, move the camera slightly and zoom in on the map. Or another example - you have a map covering the entire composition - you can move around / zoom in and out or just rotate the entire composition slightly to create a cool effect. I often see it on youtube. @JonnyBurger

In the Remotion Studio, you can already zoom and pan the canvas. Or do you mean to add these effects into the video?
In this case, you would just achieve it with Transforms.
I don't think any of these improvements need to be a breaking change though.

[reactone]

What I mean is to be able to add animations such as in this video - https://www.youtube.com/watch?v=8pbz2H7UTwQ from around 1:12 - rotation, zooming of the entire canvas. I'll check out if this can be done with transformations, as suggested.

Finally - I think one ground breaking feature would be to be able to render locally. I know youve been looking into that and currently it cant be done using Web workers - although there have been some updates to support native languages recently.

Wouldnt it be possible to render locally using something like html2canvas and then stitching the images together ? (https://www.npmjs.com/package/html2canvas) ? I'm using this library to create thumbnails of the composition and it works fine. @JonnyBurger

[JonnyBurger]

@reactone You can make these animations for sure using transforms!
For rendering in the browser, the blockers are explained here https://www.remotion.dev/docs/miscellaneous/render-in-browser

[reactone]

That makes sense - I didnt realise that html2canvas has its limitations. Im quite surprised there is no robust way of converting an html element to an image. I was thinking about coding my own, but then it would take ages probably as every css property would have to be mapped.

Anyway, one feature Im looking for is also to be able to animate (move) elements along a path. There is now a css motion path module ( https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_motion_path) . Whats the best way to achieve this in remotion - is there a package that helps with that or it would have to be custom coded? I know there is remotion/paths but it doesnt seem to be related to animations. Maybe worth to include in v5 @JonnyBurger

[JonnyBurger]

@reactone The best way is indeed to use our paths package, this API specifically: https://www.remotion.dev/docs/paths/get-point-at-length

Subtract the initial point to get the offset to apply to your element.
You can also use getTangentAtLength() to rotate the path

[brianconnoly]

Please add ability to fit

Speed up / slow down if needed

Its essential for creating dynamic screencasts from desktop capture

[JonnyBurger]

@brianconnoly No breaking change for this is needed

Plus <Player> + <OffthreadVideo> both already have a playbackRate option, so you can speed up either an individual video or the entire thing