java how do I get an image from a .mp4 video?

Question

I need to get an image as preview of the videos loaded by the users, it doesn't need to be a great thumbnail because it's just for a chat application, so I had in mind that when a user sends a message that contains an mp4 I process it and save a random frame as well.

I googled a bit and everyone is using ffmpeg but that's an external software and you can only interact with it with java, my project needs to be stand alone I don't want it to rely on the fact that the server has ffmpeg installed, so I went back in time and found JavaFX, but all the snippets online don't specify the version they're using and chatGPT is useless as always, can you guys show me how to do it or at least explain to me how it works so that I can code it myself? I'm using this dependency

   <dependency>
        <groupId>org.openjfx</groupId>
        <artifactId>javafx-controls</artifactId>
        <version>16</version> <!-- the version chatGPT wrote but you can change that -->
    </dependency>

if you have a different solution altogether that's great too I only care about saving a random frame from the video with the specified name at the specified path, thanks in advance

I tried this method but it gives plenty of errors probably because of a version mismatch

private byte[] getFrameFromVideo(String videoFilePath, int targetTimeSeconds) throws IOException {
        try {
            Media media = new Media(new File(videoFilePath).toURI().toString());
            MediaPlayer mediaPlayer = new MediaPlayer(media);

            CountDownLatch latch = new CountDownLatch(1);

            mediaPlayer.setOnReady(() -> {
                mediaPlayer.pause();
                mediaPlayer.setStartTime(mediaPlayer.getTotalDuration().multiply(targetTimeSeconds * 1.0 / media.getDuration().toSeconds()));
                mediaPlayer.setStopTime(mediaPlayer.getStartTime().add(mediaPlayer.getTotalDuration().multiply(1.0 / media.getDuration().toSeconds())));
                latch.countDown(); 
            });

            mediaPlayer.setOnEndOfMedia(() -> {
                BufferedImage bufferedImage = new BufferedImage(mediaPlayer.getMedia().getWidth(), mediaPlayer.getMedia().getHeight(), BufferedImage.TYPE_3BYTE_BGR);
                FXImage fxImage = mediaPlayer.snapshot(null);
                fxImage.pixelReaderProperty().get().getPixels(0, 0, (int) fxImage.getWidth(), (int) fxImage.getHeight(), javafx.scene.image.PixelFormat.getByteRgbInstance(), bufferedImage.getRaster().getDataBuffer());

                try {
                    ByteArrayOutputStream baos = new ByteArrayOutputStream();
                    ImageIO.write(bufferedImage, "png", baos);
                    mediaPlayer.dispose();
                    latch.countDown(); 
                    return baos.toByteArray();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            });

            mediaPlayer.play();

            latch.await();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

Answer 1

As you are using Maven, to use JavaFX Media, you need JavaFX media as a Maven dependency and also added to your Java module system.

• JavaFX MediaPlayer import

You need to start the JavaFX runtime to correctly use the Media system.

• See the javadoc for application on the lifecycle and platform startup and shutdown options.
• The JavaFX runtime should be started only once for the lifetime of your application.
• The JavaFX runtime should be shut down before your application exits.

JavaFX API calls to media should be made on the JavaFX application thread. The JavaFX application thread should never be slept or latched.

To play the media and ensure it is ready to take a snapshot at a chosen location:

1. Place the MediaView node in a scene.
2. Implement media error handling if needed.
3. mute the media player.
4. Use Media APIs to skip to a location in the video using the seek API.
5. Monitor the status of the media player.
6. play the video from the sought point.

After the status switches to playing, use Platform.runLater, to give the system time to render the required frame. And in the runLater call perform the next steps:

1. take a snapshot of the MediaView node.
2. then stop the MediaPlayer.
3. and dispose of the player.

Your snapshot gives you a JavaFX image you can directly display in a JavaFX ImageView.

If you need to save your snapshot image to disk:

1. After you take your snapshot, convert the JavaFX Image to a BufferedImage using the javafx-swing module.
   • added to your project similarly to javafx-media.
2. Write the buffered image to an appropriate format, e.g. jpg or png using ImageIO apis.
   • if you choose to save as a jpg rather than a png, you might need to remove the alpha channel, because jpg can’t handle alpha.

If your app is modular, then in your module-info include all of the required module dependencies to make this work:

• javafx.media
• javafx.swing
• java.desktop

Recommended distribution

To get your app functionality to end users, it is recommended to either:

• Use the maven-javafx-plugin, to create a jlinked zip of your app for your target platform,
  • Users can use your app by unzipping the zip and executing the app by invoking the shell script created by jlink.

OR

• Use the jpackage system to create an installer for your app or if your app is not modular.
  • Users can use your app by launching the installer to install the app on their system, which they can run as they do any other native app for the system.
  • Using jpackage to package a non-modular application and create installers for various platforms is more complex than creating jlink zip distributions for modular applications.

For either of the above methods, the user will generally not have to install any other software on the target machine in order to use your app.

References

• Oracle JavaFX media tutorial.
  • Somewhat dated, but still mostly relevant.
• OpenJFX getting started.
  • Make sure you can run the hello, world app from getting started before attempting the steps in this answer.

Alternatives

• As an alternative, vlcj for JavaFX can also take video snapshots
  • but that requires vlc to be installed and is more complex to use than JavaFX media.
• ffmpeg could also be used to take snapshots, for which I believe there may be Java bindings (I don't know that for sure) or the native process could be called from the Java process API
  • but that also requires the native ffmpeg runtime to be available.
  • ffmpeg command-line instructions (I haven't tried this):
    • How to extract one frame of a video every N seconds to an image?