Video Scroll Effect

What I Learned

In this playground, I explored creating a video scroll effect where the video scrubs through as the user scrolls the page. I learned how to synchronize scroll position with video playback and optimize performance for smooth scrolling.

Here's the key part of the code that creates the video scroll effect:

const animate = (time: number) => {
  if (previousTimeRef.current !== undefined) {
    const video = videoRef.current;
    const container = containerRef.current;
    if (!video || !container) return;

    const { height } = container.getBoundingClientRect();
    const windowHeight = window.innerHeight;
    const navbarHeight = 64;
    const scrollPosition = window.scrollY;
    const containerTop = container.offsetTop - navbarHeight;
    const scrollableHeight = height - windowHeight + navbarHeight;
    const scrollPercentage = Math.min(Math.max((scrollPosition - containerTop) / scrollableHeight, 0), 1);
    
    setProgress(scrollPercentage);
    
    const translateY = scrollPercentage * (container.clientHeight - video.videoHeight);
    video.style.transform = `translateY(${translateY}px)`;
    
    if (Math.round(time / (1000 / 60)) % 5 === 0) {
      video.currentTime = video.duration * scrollPercentage;
    }
  }
  previousTimeRef.current = time;
  requestRef.current = requestAnimationFrame(animate);
}

This code calculates the scroll percentage and updates the video's position and current time accordingly. It uses requestAnimationFrame for smooth animation and optimizes performance by updating the video's current time less frequently.

Challenges Faced

One of the main challenges was achieving smooth scrolling performance. Initially, updating the video's current time on every scroll event caused significant lag. I had to implement optimizations like using requestAnimationFrame and reducing the frequency of time updates to improve performance. Unfortunately, this approach still has some lag but it is significantly better than before. I also compressed the video from ~7MB to ~1MB to further optimize performance, but this causes the video to look noticeably worse. Along with all of this, I was unable to make the video scroll work on mobile. I will likely revisit this playground in the future to improve the performance and functionality on mobile.

Future Improvements

I would like for this to be viewable on mobile, there are a few issues with loading the video properly now so I am just showing it on larger devices. In the future, I'd like to add more interactive features, such as allowing users to click on the video to jump to specific points. I'm also interested in exploring ways to further optimize performance, possibly by using WebGL for rendering or implementing a custom video player.