Leviathan  0.8.0.0
Leviathan game engine
Leviathan::GUI::VideoPlayer Class Reference

VideoPlayer that uses AOM to play videos on a texture. More...

#include <VideoPlayer.h>

+ Inheritance diagram for Leviathan::GUI::VideoPlayer:

Classes

struct  Implementation
 

Public Member Functions

DLLEXPORT VideoPlayer ()
 
DLLEXPORT ~VideoPlayer ()
 
VideoPlayeroperator= (const VideoPlayer &)=delete
 
DLLEXPORT bool Play (const std::string &videofile)
 Starts playing the video file. More...
 
DLLEXPORT void Stop ()
 Stops playing and unloads the current playback objects. More...
 
DLLEXPORT bool HasAudio () const
 
DLLEXPORT float GetCurrentTime () const
 
DLLEXPORT float GetDuration () const
 
DLLEXPORT int32_t GetVideoWidth () const
 
DLLEXPORT int32_t GetVideoHeight () const
 
DLLEXPORT int GetAudioChannelCount () const
 
DLLEXPORT int GetAudioSampleRate () const
 
DLLEXPORT bool IsStreamValid () const
 
DLLEXPORT auto GetTexture () const
 
DLLEXPORT int OnEvent (Event *event) override
 
DLLEXPORT int OnGenericEvent (GenericEvent *event) override
 
- Public Member Functions inherited from Leviathan::CallableObject
DLLEXPORT CallableObject ()
 
virtual DLLEXPORT ~CallableObject ()
 

Public Attributes

Delegate OnPlayBackEnded
 

Protected Member Functions

size_t ReadAudioData (uint8_t *output, size_t amount)
 Reads audio data to the buffer. More...
 
bool CreateOutputTexture ()
 
bool OpenCodecsForFile ()
 Opens and parses the video info and opens decoding contexts. More...
 
bool HandleFrameVideoUpdate ()
 
bool DecodeVideoFrame ()
 Decodes one video frame. Returns false if more data is required but the stream ended (this condition ends the playback) More...
 
bool PeekNextFrameTimeStamp ()
 
void UpdateTexture ()
 Updates the texture. More...
 
size_t ReadDataFromAudioQueue (Lock &audiolocked, uint8_t *output, size_t amount)
 Reads already decoded audio data. The audio data vector must be locked before calling this. More...
 
void ResetClock ()
 Resets timers. Call when playback start or resumes. More...
 
void OnStreamEndReached ()
 Called when end of playback has been reached. More...
 
- Protected Member Functions inherited from Leviathan::CallableObject
void UnRegisterAllEvents ()
 
void UnRegister (EVENT_TYPE from, bool all=false)
 
void UnRegister (const std::string &genericname, bool all=false)
 
void RegisterForEvent (EVENT_TYPE toregister)
 
void RegisterForEvent (const std::string &genericname)
 

Protected Attributes

std::unique_ptr< ImplementationPimpl
 
std::string VideoFile
 
bs::HTexture VideoOutputTexture
 The target texture. More...
 
bool IsPlaying = false
 True when playing back something and frame start events do something. More...
 
double VideoTimeBase = 1.f
 How many timestamp units are in a second in the video stream. More...
 
int32_t FrameWidth = 0
 
int32_t FrameHeight = 0
 
int SampleRate = 0
 Audio sample rate. More...
 
int ChannelCount = 0
 
bool IsPlayingAudio = false
 Used to start the audio playback once. More...
 
bool HasAudioStream = false
 Set to true when an audio stream is found and opened. More...
 
AudioSource::pointer AudioStream
 Audio output. More...
 
Sound::ProceduralSoundData::pointer AudioStreamData
 
Sound::ProceduralSoundData::SoundProperties AudioStreamDataProperties
 
float PassedTimeSeconds = 0.f
 
std::atomic< bool > StreamValid {false}
 Set to false if an error occurs and playback should stop. More...
 

Additional Inherited Members

- Static Public Member Functions inherited from Leviathan::CallableObject
static DLLEXPORT EVENT_TYPE ResolveStringToType (const std::string &type)
 
static DLLEXPORT EVENT_TYPE GetCommonEventType (const std::string &type)
 
static DLLEXPORT std::string GetListenerNameFromType (EVENT_TYPE type)
 

Detailed Description

VideoPlayer that uses AOM to play videos on a texture.

Supports playing av1 video streams with either vorbis or opus audio

Todo:

Implement pausing and seeking

When Stop is called the OnPlaybackEnded should still be fired. If something was playing

Definition at line 26 of file VideoPlayer.h.

Constructor & Destructor Documentation

◆ VideoPlayer()

DLLEXPORT VideoPlayer::VideoPlayer ( )

Definition at line 108 of file VideoPlayer.cpp.

108 : Pimpl(std::make_unique<Implementation>()) {}
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ ~VideoPlayer()

DLLEXPORT VideoPlayer::~VideoPlayer ( )

Definition at line 110 of file VideoPlayer.cpp.

111 {
113 
114  // Ensure all decoding resources are cleared
115  Stop();
116 }
DLLEXPORT void Stop()
Stops playing and unloads the current playback objects.

Member Function Documentation

◆ CreateOutputTexture()

bool VideoPlayer::CreateOutputTexture ( )
protected

After loading the video this creates the output texture + material for it

Returns
false if the setup fails

Definition at line 213 of file VideoPlayer.cpp.

214 {
215  Pimpl->GPUUploadBuffer =
216  bs::PixelData::create(FrameWidth, FrameHeight, 1, BS_PIXEL_FORMAT);
217 
218  if(!Pimpl->GPUUploadBuffer)
219  return false;
220 
221  VideoOutputTexture = bs::Texture::create(Pimpl->GPUUploadBuffer, bs::TU_DYNAMIC);
222  return VideoOutputTexture != nullptr;
223 }
constexpr auto BS_PIXEL_FORMAT
Definition: VideoPlayer.cpp:24
bs::HTexture VideoOutputTexture
The target texture.
Definition: VideoPlayer.h:146
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ DecodeVideoFrame()

bool VideoPlayer::DecodeVideoFrame ( )
protected

Decodes one video frame. Returns false if more data is required but the stream ended (this condition ends the playback)

Definition at line 399 of file VideoPlayer.cpp.

400 {
401  bool frameReady = false;
402 
403  while(!frameReady) {
404  // Try to get a frame first
405  Pimpl->VideoCodec->ReceiveDecodedFrames([&](const DecodedFrame& frame) {
406  Pimpl->CurrentlyDecodedFrame = frame;
407 
408  frameReady = true;
409 
410  // We want only one frame at a time
411  return false;
412  });
413 
414  if(frameReady)
415  break;
416 
417  // If we didn't get a frame send more data to the decoder
418  auto [data, length, opts] =
419  Pimpl->VideoParser->GetNextBlockForTrack(Pimpl->VideoTrack.TrackNumber);
420 
421  // If we ran out of data there's nothing to do
422  if(!data)
423  break;
424 
425  // CurrentlyDecodedTimeStamp = opts.Timecode / VideoTimeBase;
426  Pimpl->CurrentlyDecodedFrameTimeStamp =
428 
429  if(!Pimpl->VideoCodec->FeedRawFrame(data, length)) {
430  LOG_ERROR("VideoCodec: failed to send raw frame to video codec");
431  }
432  }
433 
434  return frameReady;
435 }
#define LOG_ERROR(x)
Definition: Define.h:92
Holds decoded data. This is only valid until the frame receive callback ends.
Definition: Codec.h:16
static constexpr float MATROSKA_DURATION_TO_SECONDS
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ GetAudioChannelCount()

DLLEXPORT int Leviathan::GUI::VideoPlayer::GetAudioChannelCount ( ) const
inline
Returns
The number of audio channels

Definition at line 72 of file VideoPlayer.h.

73  {
74  return ChannelCount;
75  }

◆ GetAudioSampleRate()

DLLEXPORT int Leviathan::GUI::VideoPlayer::GetAudioSampleRate ( ) const
inline
Returns
The number of samples per second of the audio stream or -1 if no audio streams exist

Definition at line 79 of file VideoPlayer.h.

80  {
81  return SampleRate;
82  }
int SampleRate
Audio sample rate.
Definition: VideoPlayer.h:158

◆ GetCurrentTime()

DLLEXPORT float VideoPlayer::GetCurrentTime ( ) const
Returns
Current playback position, in seconds The return value is directly read from the last decoded frame timestamp

Definition at line 208 of file VideoPlayer.cpp.

209 {
210  return Pimpl->CurrentlyDecodedFrameTimeStamp;
211 }
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ GetDuration()

DLLEXPORT float VideoPlayer::GetDuration ( ) const
Returns
The total length of the video is seconds. -1 if invalid

Definition at line 200 of file VideoPlayer.cpp.

201 {
202  if(!Pimpl->VideoParser)
203  return -1.f;
204 
205  return Pimpl->VideoParser->GetDurationInSeconds();
206 }
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ GetTexture()

DLLEXPORT auto Leviathan::GUI::VideoPlayer::GetTexture ( ) const
inline

Definition at line 90 of file VideoPlayer.h.

91  {
92  return VideoOutputTexture;
93  }
bs::HTexture VideoOutputTexture
The target texture.
Definition: VideoPlayer.h:146

◆ GetVideoHeight()

DLLEXPORT int32_t Leviathan::GUI::VideoPlayer::GetVideoHeight ( ) const
inline
Returns
Height of the current video

Definition at line 65 of file VideoPlayer.h.

66  {
67  return FrameHeight;
68  }

◆ GetVideoWidth()

DLLEXPORT int32_t Leviathan::GUI::VideoPlayer::GetVideoWidth ( ) const
inline
Returns
Width of the current video

Definition at line 59 of file VideoPlayer.h.

60  {
61  return FrameWidth;
62  }

◆ HandleFrameVideoUpdate()

bool VideoPlayer::HandleFrameVideoUpdate ( )
protected

Definition at line 341 of file VideoPlayer.cpp.

342 {
343  const auto start = Time::GetCurrentTimePoint();
344 
345  // If we don't have a frame decoded we should decode one one
346  if(Pimpl->CurrentlyDecodedFrameTimeStamp < 0) {
347  // This is the first frame of the video
348 
349  // Reset the elapsed time to not skip the initial frame
350  PassedTimeSeconds = 0.f;
351 
352  // Decode the first frame
353  if(!DecodeVideoFrame())
354  return false;
355 
356  Pimpl->FrameNeedsGPUUpload = true;
357 
358  // And get the timestamp for the next video
360  } else {
361  // Loop until we reach a state where the next frame is > PassedTimeSeconds
362  while(PassedTimeSeconds >= Pimpl->NextFrameTimeStamp) {
363  // Decode the next frame
364  if(!DecodeVideoFrame()) {
365  // We ran out of data
366  LOG_INFO("VideoPlayer: reached end of video stream");
368  return false;
369  }
370 
371  Pimpl->FrameNeedsGPUUpload = true;
372 
373  // Get the time for the next frame
374  // In case we ran out of data this won't update the next frame time, and this will
375  // loop again. During that loop the next frame decode fails and this breaks, so we
376  // don't need to check the return value here
378 
379  // Break if this has taken too long
381  break;
382  }
383  }
384 
385  // Copy decoded frame to GPU if it was updated
386  if(Pimpl->FrameNeedsGPUUpload)
387  UpdateTexture();
388 
390  const auto millisecondsPassed =
391  SecondDuration(Time::GetCurrentTimePoint() - start).count() * 1000;
392  LOG_WARNING("VideoPlayer: update is taking too long, took: " +
393  std::to_string(millisecondsPassed) + "ms");
394  }
395 
396  return true;
397 }
#define LOG_INFO(x)
Definition: Define.h:90
void UpdateTexture()
Updates the texture.
constexpr auto VIDEO_PLAYER_WARNING_ELAPSED
Definition: VideoPlayer.cpp:32
#define LOG_WARNING(x)
Definition: Define.h:91
static DLLEXPORT TimePoint GetCurrentTimePoint()
void OnStreamEndReached()
Called when end of playback has been reached.
bool DecodeVideoFrame()
Decodes one video frame. Returns false if more data is required but the stream ended (this condition ...
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141
std::chrono::duration< float, std::ratio< 1 > > SecondDuration
Definition: TimeIncludes.h:16

◆ HasAudio()

DLLEXPORT bool Leviathan::GUI::VideoPlayer::HasAudio ( ) const
inline
Returns
True if currently loaded file has an audio stream

Definition at line 46 of file VideoPlayer.h.

47  {
48  return HasAudioStream;
49  }
bool HasAudioStream
Set to true when an audio stream is found and opened.
Definition: VideoPlayer.h:165

◆ IsStreamValid()

DLLEXPORT bool Leviathan::GUI::VideoPlayer::IsStreamValid ( ) const
inline
Returns
true if all the ffmpeg stream objects are valid for playback

Definition at line 85 of file VideoPlayer.h.

86  {
87  return StreamValid; // && VideoCodec && ConvertedFrameBuffer;
88  }
std::atomic< bool > StreamValid
Set to false if an error occurs and playback should stop.
Definition: VideoPlayer.h:176

◆ OnEvent()

DLLEXPORT int VideoPlayer::OnEvent ( Event event)
overridevirtual

Implements Leviathan::CallableObject.

Definition at line 651 of file VideoPlayer.cpp.

652 {
653  switch(event->GetType()) {
654  case EVENT_TYPE_FRAME_BEGIN: {
655  // If we are no longer playing, unregister
656  if(!IsPlaying)
657  return -1;
658 
659  if(!IsStreamValid()) {
660 
661  LOG_WARNING("VideoPlayer: Stream is invalid, closing playback");
663  return -1;
664  }
665 
666  const auto data = event->GetFloatDataForEvent();
667  if(!data) {
668  LOG_FATAL("VideoPlayer: got event with no elapsed data");
669  return -1;
670  }
671 
672  // Engine update based update
674  const auto elapsed = data->FloatDataValue;
675 
676  PassedTimeSeconds += elapsed;
677 
678  } else {
679  // Alternative keeping our own time, in order to fight off lag from slow decode
680  // performance
681  if(!Pimpl->PlaybackStartTime) {
682  Pimpl->PlaybackStartTime = Time::GetCurrentTimePoint();
683  }
684 
686  SecondDuration(Time::GetCurrentTimePoint() - *Pimpl->PlaybackStartTime)
687  .count();
688  }
689 
690  // Start playing audio. Hopefully at the same time as the first frame of the
691  // video is decoded
693 
694  LOG_INFO("VideoPlayer: Starting audio playback from the video...");
695 
696  AudioStreamData = alure::MakeShared<Sound::ProceduralSoundData>(
697  [=](void* output, unsigned amount) -> unsigned {
698  return this->ReadAudioData(static_cast<uint8_t*>(output), amount);
699  },
701 
702  AudioStream =
704  IsPlayingAudio = true;
705  }
706 
707  // If this returns true continue receiving events, if it fails then stop
709  return -1;
710 
711  // Stop the playback if we are behind many frames on showing the video
713  LOG_WARNING("VideoPlayer: has lagged past the end of the video, ending playback, "
714  "video duration: " +
715  std::to_string(GetDuration()) +
716  ", elapsed time: " + std::to_string(PassedTimeSeconds));
718  return -1;
719  }
720 
721  return 0;
722  }
723  default:
724  // Unregister from other events
725  return -1;
726  }
727 }
#define LOG_INFO(x)
Definition: Define.h:90
DLLEXPORT bool IsStreamValid() const
Definition: VideoPlayer.h:85
#define LOG_FATAL(x)
Definition: Define.h:94
AudioSource::pointer AudioStream
Audio output.
Definition: VideoPlayer.h:168
constexpr bool VIDEO_PLAYER_USE_SEPARATE_TIMING
Definition: VideoPlayer.cpp:34
constexpr auto VIDEO_PLAYER_WARNING_ELAPSED
Definition: VideoPlayer.cpp:32
Sound::ProceduralSoundData::pointer AudioStreamData
Definition: VideoPlayer.h:169
Sound::ProceduralSoundData::SoundProperties AudioStreamDataProperties
Definition: VideoPlayer.h:170
#define LOG_WARNING(x)
Definition: Define.h:91
DLLEXPORT float GetDuration() const
DLLEXPORT EVENT_TYPE GetType() const
Gets the Type of the event.
Definition: Event.cpp:27
SoundDevice * GetSoundDevice()
Definition: Engine.h:227
static DLLEXPORT TimePoint GetCurrentTimePoint()
DLLEXPORT AudioSource::pointer CreateProceduralSound(const Sound::ProceduralSoundData::pointer &data, size_t chunksize=56000, size_t chunkstoqueue=4)
Opens an audio source from a procedural data stream.
bool IsPlayingAudio
Used to start the audio playback once.
Definition: VideoPlayer.h:162
void OnStreamEndReached()
Called when end of playback has been reached.
bool IsPlaying
True when playing back something and frame start events do something.
Definition: VideoPlayer.h:149
bool HasAudioStream
Set to true when an audio stream is found and opened.
Definition: VideoPlayer.h:165
static DLLEXPORT Engine * Get()
Definition: Engine.cpp:86
size_t ReadAudioData(uint8_t *output, size_t amount)
Reads audio data to the buffer.
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141
std::chrono::duration< float, std::ratio< 1 > > SecondDuration
Definition: TimeIncludes.h:16

◆ OnGenericEvent()

DLLEXPORT int VideoPlayer::OnGenericEvent ( GenericEvent event)
overridevirtual

Implements Leviathan::CallableObject.

Definition at line 729 of file VideoPlayer.cpp.

730 {
731  return 0;
732 }

◆ OnStreamEndReached()

void VideoPlayer::OnStreamEndReached ( )
protected

Called when end of playback has been reached.

Closes the playback and invokes the delegates

Definition at line 641 of file VideoPlayer.cpp.

642 {
643  auto vars = NamedVars::MakeShared<NamedVars>();
644 
645  vars->AddVar(std::make_shared<NamedVariableList>("oldvideo", new StringBlock(VideoFile)));
646 
647  Stop();
648  OnPlayBackEnded.Call(vars);
649 }
DLLEXPORT void Stop()
Stops playing and unloads the current playback objects.
DataBlock< std::string > StringBlock
Definition: DataBlock.h:386
DLLEXPORT void Call(const NamedVars::pointer &values) const
Calls all the attached delegates.

◆ OpenCodecsForFile()

bool VideoPlayer::OpenCodecsForFile ( )
protected

Opens and parses the video info and opens decoding contexts.

Returns
false if something fails

Definition at line 225 of file VideoPlayer.cpp.

226 {
227  Pimpl->VideoParser = MatroskaParser(VideoFile);
228 
229  if(!Pimpl->VideoParser->Good()) {
230  LOG_ERROR("VideoPlayer: failed to parse video file (is it a matroska file?): " +
231  Pimpl->VideoParser->GetErrorMessage());
232  return false;
233  }
234 
235  VideoTimeBase = Pimpl->VideoParser->GetHeader().TimecodeScale;
236 
237  // Jump to where the video data starts
238  Pimpl->VideoParser->JumpToFirstCluster();
239 
240  // Copy track info
241  Pimpl->VideoTrack = Pimpl->VideoParser->GetFirstVideoTrack();
242 
243  // Duplicate the parser state for the audio stream
244  if(Pimpl->VideoParser->GetAudioTrackCount() > 0) {
245  HasAudioStream = true;
246 
247  Pimpl->AudioParser = Pimpl->VideoParser;
248 
249  Pimpl->AudioTrack = Pimpl->AudioParser->GetFirstAudioTrack();
250 
251  // Initialize audio codec
252  if(Pimpl->AudioTrack.CodecID == MatroskaParser::CODEC_TYPE_VORBIS) {
253 
254  const auto codecPrivateData =
255  Pimpl->AudioParser->ReadTrackCodecPrivateData(Pimpl->AudioTrack);
256 
257  Pimpl->AudioCodec = std::make_unique<VorbisCodec>(
258  codecPrivateData.data(), codecPrivateData.size());
259  } else {
260  LOG_ERROR("VideoPlayer: unknown audio codec: " + Pimpl->AudioTrack.CodecID);
261  HasAudioStream = false;
262  }
263 
264  if(HasAudioStream) {
265  SampleRate = static_cast<int>(
266  std::get<MatroskaParser::TrackInfo::Audio>(Pimpl->AudioTrack.TrackTypeData)
267  .SamplingFrequency);
268  ChannelCount =
269  std::get<MatroskaParser::TrackInfo::Audio>(Pimpl->AudioTrack.TrackTypeData)
270  .Channels;
271 
272  // Create sound object //
274 
275  bool valid = false;
276 
277  if(ChannelCount == 1) {
278  AudioStreamDataProperties.Channels = alure::ChannelConfig::Mono;
279  valid = true;
280  } else if(ChannelCount == 2) {
281  AudioStreamDataProperties.Channels = alure::ChannelConfig::Stereo;
282  valid = true;
283  } else if(ChannelCount == 4) {
284  AudioStreamDataProperties.Channels = alure::ChannelConfig::Quad;
285  valid = true;
286  } else if(ChannelCount == 6) {
287  AudioStreamDataProperties.Channels = alure::ChannelConfig::X51;
288  valid = true;
289  } else if(ChannelCount == 7) {
290  AudioStreamDataProperties.Channels = alure::ChannelConfig::X61;
291  valid = true;
292  } else if(ChannelCount == 8) {
293  AudioStreamDataProperties.Channels = alure::ChannelConfig::X71;
294  valid = true;
295  }
296 
297  // TODO: alure supports float32 format, we could do without converting that from
298  // the codec (if the codec internally returns floats)
299  AudioStreamDataProperties.SampleType = alure::SampleType::Int16;
300 
301  if(!valid) {
302  LOG_ERROR("VideoPlayer: invalid channel configuration for audio: " +
303  std::to_string(ChannelCount));
304  return false;
305  }
306  }
307 
308  // Allocate some audio buffer space
309  Pimpl->_PendingAudioData.Data.reserve(DEFAULT_AUDIO_BUFFER_RESERVED_SPACE);
310 
311  } else {
312  HasAudioStream = false;
313  }
314 
315  // Initialize the video codec
316  if(Pimpl->VideoTrack.CodecID == MatroskaParser::CODEC_TYPE_AV1) {
317 
318  Pimpl->VideoCodec = std::make_unique<AV1Codec>();
319  } else {
320  LOG_ERROR("VideoPlayer: unknown video codec: " + Pimpl->VideoTrack.CodecID);
321  return false;
322  }
323 
324  FrameWidth =
325  std::get<MatroskaParser::TrackInfo::Video>(Pimpl->VideoTrack.TrackTypeData).PixelWidth;
326  FrameHeight = std::get<MatroskaParser::TrackInfo::Video>(Pimpl->VideoTrack.TrackTypeData)
327  .PixelHeight;
328 
329 
330  PassedTimeSeconds = 0.f;
331 
332  // Reset frame status
333  Pimpl->CurrentlyDecodedFrameTimeStamp = -1.f;
334  Pimpl->NextFrameTimeStamp = -1.f;
335  Pimpl->PlaybackStartTime.reset();
336 
337  StreamValid = true;
338  return true;
339 }
#define LOG_ERROR(x)
Definition: Define.h:92
std::atomic< bool > StreamValid
Set to false if an error occurs and playback should stop.
Definition: VideoPlayer.h:176
double VideoTimeBase
How many timestamp units are in a second in the video stream.
Definition: VideoPlayer.h:152
Sound::ProceduralSoundData::SoundProperties AudioStreamDataProperties
Definition: VideoPlayer.h:170
Basic parser for Matroska (.mkv) container for use with GUI::VideoPlayer.
int SampleRate
The number of samples per second.
static constexpr auto CODEC_TYPE_AV1
int SampleRate
Audio sample rate.
Definition: VideoPlayer.h:158
alure::SampleType SampleType
Recommended: Int16.
bool HasAudioStream
Set to true when an audio stream is found and opened.
Definition: VideoPlayer.h:165
constexpr auto DEFAULT_AUDIO_BUFFER_RESERVED_SPACE
Definition: VideoPlayer.cpp:30
static constexpr auto CODEC_TYPE_VORBIS
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ operator=()

VideoPlayer& Leviathan::GUI::VideoPlayer::operator= ( const VideoPlayer )
delete

◆ PeekNextFrameTimeStamp()

bool VideoPlayer::PeekNextFrameTimeStamp ( )
protected

Definition at line 437 of file VideoPlayer.cpp.

438 {
439  auto [found, length, opts] =
440  Pimpl->VideoParser->PeekNextBlockForTrack(Pimpl->VideoTrack.TrackNumber);
441 
442  // We ran out of data
443  if(!found)
444  return false;
445 
446  Pimpl->NextFrameTimeStamp = opts.Timecode * MatroskaParser::MATROSKA_DURATION_TO_SECONDS;
447  return true;
448 }
static constexpr float MATROSKA_DURATION_TO_SECONDS
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ Play()

DLLEXPORT bool VideoPlayer::Play ( const std::string &  videofile)

Starts playing the video file.

Returns
True if successfully started

Definition at line 118 of file VideoPlayer.cpp.

119 {
120  // Make sure nothing is playing currently //
121  Stop();
122 
123  if(!std::filesystem::exists(videofile)) {
124 
125  LOG_ERROR("VideoPlayer: Play: file doesn't exist: " + videofile);
126  return false;
127  }
128 
129  VideoFile = videofile;
130 
131  try {
132  // Parse stream data to know how big our textures need to be //
133  if(!OpenCodecsForFile()) {
134 
135  LOG_ERROR("VideoPlayer: Play: failed to parse file or open codecs for file");
136  Stop();
137  return false;
138  }
139 
140  if(!CreateOutputTexture()) {
141 
142  LOG_ERROR("VideoPlayer: Play: output video texture creation failed");
143  Stop();
144  return false;
145  }
146  } catch(const Exception& e) {
147  LOG_ERROR("VideoPlayer: Play: exception happened on initializing playback: ");
148  e.PrintToLog();
149  Stop();
150  return false;
151  }
152 
153  // Make tick run
154  IsPlaying = true;
156  return true;
157 }
DLLEXPORT void Stop()
Stops playing and unloads the current playback objects.
#define LOG_ERROR(x)
Definition: Define.h:92
Base class for all exceptions thrown by Leviathan.
Definition: Exceptions.h:10
bool IsPlaying
True when playing back something and frame start events do something.
Definition: VideoPlayer.h:149
void RegisterForEvent(EVENT_TYPE toregister)
bool OpenCodecsForFile()
Opens and parses the video info and opens decoding contexts.
virtual DLLEXPORT void PrintToLog() const noexcept
Definition: Exceptions.cpp:35

◆ ReadAudioData()

size_t VideoPlayer::ReadAudioData ( uint8_t *  output,
size_t  amount 
)
protected

Reads audio data to the buffer.

Returns
The number of sample frames read
Parameters
amountThe maximum number of sample frames to read

Definition at line 492 of file VideoPlayer.cpp.

493 {
494  Lock lock(Pimpl->ThreadSafetyMutex);
495 
496  if(!HasAudioStream || !StreamValid || amount < 1) {
497  return 0;
498  }
499 
500  // Convert amount to byte count
501  const auto bytesPerSample = 2;
502  amount *= bytesPerSample * ChannelCount;
503 
504  size_t readAmount = 0;
505 
506  // Read audio data until the stream ends or we have reached amount
507  while(amount > 0) {
508  // First return from queue //
509  if(!Pimpl->_PendingAudioData.Empty) {
510 
511  // Try to read from the queue //
512  const auto read = ReadDataFromAudioQueue(lock, output, amount);
513 
514  if(read == 0) {
515  // Queue is invalid... //
516  LOG_ERROR("Invalid audio queue, emptying the queue");
517  Pimpl->_PendingAudioData.Clear();
518  } else {
519 
520  // Adjust pointer and amount and try to read again if still some size left
521  readAmount += read;
522  output += read;
523  amount -= read;
524  continue;
525  }
526  }
527 
528  bool wroteToBuffer = false;
529 
530  // Try to read audio data from the codec
531  Pimpl->AudioCodec->ReceiveDecodedFrames([&](const DecodedFrame& frame) {
532  const auto& soundData = std::get<DecodedFrame::Sound>(frame.TypeSpecificData);
533 
534  if(soundData.Channels != ChannelCount) {
535  LOG_ERROR("VideoPlayer: decoded audio data has different channel count than "
536  "what file header info said");
537  return false;
538  }
539 
540  // Received audio data //
541 
542  // First check if we can fit the whole data to the receiver buffer
543  const auto totalSize = bytesPerSample * (soundData.Samples * ChannelCount);
544 
545  if(amount >= static_cast<size_t>(totalSize) // && !wroteToBuffer
546  ) {
547  // Directly feed the converted data to the requested
548  if(!soundData.ConvertSamples(output, totalSize,
550  LOG_ERROR(
551  "VideoPlayer: converting audio data failed (directly to receiver)");
552  return false;
553  }
554 
555  // Adjust pointer and amount and try to read again if still some size left
556  readAmount += totalSize;
557  output += totalSize;
558  amount -= totalSize;
559 
560  // Continue reading blocks if amount not full
561  return amount > 0;
562  } else {
563  // We need a buffer //
564  wroteToBuffer = true;
565 
566  const auto previousBufferSize = Pimpl->_PendingAudioData.Data.size();
567  const auto& bufferWritePos =
568  (&Pimpl->_PendingAudioData.Data[previousBufferSize - 1]) + 1;
569 
570  Pimpl->_PendingAudioData.Data.resize(previousBufferSize + totalSize);
571 
572  if(!soundData.ConvertSamples(bufferWritePos, totalSize,
574  LOG_ERROR("VideoPlayer: converting audio data failed");
575  return false;
576  }
577 
578  Pimpl->_PendingAudioData.Empty = false;
579 
580  // Now that there is data we can loop again to get some data from the buffer
581  return false;
582  }
583  });
584 
585  // Stop if we got enough data
586  if(amount <= 0)
587  break;
588 
589  // Read from buffer without looping again if we wrote to it to not pass the codec more
590  // data in case there was still some leftover data
591  if(wroteToBuffer)
592  continue;
593 
594  // Not enough data could be read, read next block for the audio stream
595  auto [data, length, opts] =
596  Pimpl->AudioParser->GetNextBlockForTrack(Pimpl->AudioTrack.TrackNumber);
597 
598  if(!data) {
599  // Ran out of audio data
600  LOG_INFO("VideoPlayer: audio stream reached end");
601  break;
602  }
603 
604  if(!Pimpl->AudioCodec->FeedRawFrame(data, length)) {
605  LOG_ERROR("VideoPlayer: failed to send raw data block to audio codec");
606  }
607 
608  // Now we have more data so we can loop again to read from the buffer
609  }
610 
611  return readAmount / bytesPerSample / ChannelCount;
612 }
#define LOG_INFO(x)
Definition: Define.h:90
#define LOG_ERROR(x)
Definition: Define.h:92
std::atomic< bool > StreamValid
Set to false if an error occurs and playback should stop.
Definition: VideoPlayer.h:176
Holds decoded data. This is only valid until the frame receive callback ends.
Definition: Codec.h:16
bool HasAudioStream
Set to true when an audio stream is found and opened.
Definition: VideoPlayer.h:165
std::variant< std::monostate, Image, Sound > TypeSpecificData
Definition: Codec.h:56
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141
std::unique_lock< std::mutex > Lock
Definition: ThreadSafe.h:18
size_t ReadDataFromAudioQueue(Lock &audiolocked, uint8_t *output, size_t amount)
Reads already decoded audio data. The audio data vector must be locked before calling this.

◆ ReadDataFromAudioQueue()

size_t VideoPlayer::ReadDataFromAudioQueue ( Lock audiolocked,
uint8_t *  output,
size_t  amount 
)
protected

Reads already decoded audio data. The audio data vector must be locked before calling this.

Definition at line 614 of file VideoPlayer.cpp.

615 {
616  if(Pimpl->_PendingAudioData.Empty || Pimpl->_PendingAudioData.Data.empty() || amount == 0)
617  return 0;
618 
619  const auto* readPtr =
620  Pimpl->_PendingAudioData.Data.data() + Pimpl->_PendingAudioData.NextReadOffset;
621 
622  const auto dataAvailable =
623  Pimpl->_PendingAudioData.Data.size() - Pimpl->_PendingAudioData.NextReadOffset;
624 
625  if(dataAvailable <= amount) {
626  // Can read all
627  std::memcpy(output, readPtr, dataAvailable);
628  Pimpl->_PendingAudioData.Clear();
629  return dataAvailable;
630  } else {
631 
632  // Can only read part of the buffer
633  std::memcpy(output, readPtr, amount);
634 
635  // Update read pos for next call
636  Pimpl->_PendingAudioData.NextReadOffset += amount;
637  return amount;
638  }
639 }
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

◆ ResetClock()

void Leviathan::GUI::VideoPlayer::ResetClock ( )
protected

Resets timers. Call when playback start or resumes.

◆ Stop()

DLLEXPORT void VideoPlayer::Stop ( )

Stops playing and unloads the current playback objects.

Definition at line 159 of file VideoPlayer.cpp.

160 {
161  // Close all codec resources //
162  StreamValid = false;
163 
164  // Stop audio playing first //
165  if(IsPlayingAudio) {
166  IsPlayingAudio = false;
167  }
168 
169  if(AudioStreamData) {
170 
171  AudioStreamData->Detach();
172  AudioStreamData.reset();
173  }
174 
175  if(AudioStream) {
176 
177  AudioStream->Stop();
178  AudioStream.reset();
179  }
180 
181  Lock lock(Pimpl->ThreadSafetyMutex);
182 
183  // Dump audio data
184  Pimpl->_PendingAudioData.Clear();
185 
186  Pimpl->CurrentlyDecodedFrame.reset();
187 
188  Pimpl->CloseCodecs();
189 
190  // Let go of our textures and things //
191  VideoFile = "";
192 
193  VideoOutputTexture = nullptr;
194 
195  Pimpl->GPUUploadBuffer = nullptr;
196 
197  IsPlaying = false;
198 }
std::atomic< bool > StreamValid
Set to false if an error occurs and playback should stop.
Definition: VideoPlayer.h:176
AudioSource::pointer AudioStream
Audio output.
Definition: VideoPlayer.h:168
Sound::ProceduralSoundData::pointer AudioStreamData
Definition: VideoPlayer.h:169
bool IsPlayingAudio
Used to start the audio playback once.
Definition: VideoPlayer.h:162
bool IsPlaying
True when playing back something and frame start events do something.
Definition: VideoPlayer.h:149
bs::HTexture VideoOutputTexture
The target texture.
Definition: VideoPlayer.h:146
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141
std::unique_lock< std::mutex > Lock
Definition: ThreadSafe.h:18

◆ UpdateTexture()

void VideoPlayer::UpdateTexture ( )
protected

Updates the texture.

Definition at line 450 of file VideoPlayer.cpp.

451 {
452  if(Pimpl->WaitForTextureToUpload) {
453  // Wait until buffer is no longer locked
454  if(Pimpl->GPUUploadBuffer->isLocked()) {
455 
456  // Using true here majorly stalls rendering
457  bs::gCoreThread().submit(false);
458 
459  do {
460  std::this_thread::yield();
461  } while(Pimpl->GPUUploadBuffer->isLocked());
462  }
463  } else {
464  // Skip updating until the texture is no longer locked for update
465  if(Pimpl->GPUUploadBuffer->isLocked())
466  return;
467  }
468 
469  // For maximum performance we directly convert the frame to the GPU buffer
470  const auto& imageData =
471  std::get<DecodedFrame::Image>(Pimpl->CurrentlyDecodedFrame->TypeSpecificData);
472 
473  if(imageData.Width != static_cast<uint32_t>(FrameWidth) ||
474  imageData.Height != static_cast<uint32_t>(FrameHeight)) {
475  LOG_ERROR("VideoPlayer: decoded frame size is different than what file header "
476  "info said");
477  return;
478  }
479 
480  if(!imageData.ConvertImage(Pimpl->GPUUploadBuffer->getData(),
481  Pimpl->GPUUploadBuffer->getSize(),
483  LOG_ERROR("VideoPlayer: frame convert failed, likely due to mismatch between graphics "
484  "buffer size and what is needed for frame conversion");
485  return;
486  }
487 
488  VideoOutputTexture->writeData(Pimpl->GPUUploadBuffer, 0, 0, true);
489  Pimpl->FrameNeedsGPUUpload = false;
490 }
#define LOG_ERROR(x)
Definition: Define.h:92
bs::HTexture VideoOutputTexture
The target texture.
Definition: VideoPlayer.h:146
std::unique_ptr< Implementation > Pimpl
Definition: VideoPlayer.h:141

Member Data Documentation

◆ AudioStream

AudioSource::pointer Leviathan::GUI::VideoPlayer::AudioStream
protected

Audio output.

Definition at line 168 of file VideoPlayer.h.

◆ AudioStreamData

Sound::ProceduralSoundData::pointer Leviathan::GUI::VideoPlayer::AudioStreamData
protected

Definition at line 169 of file VideoPlayer.h.

◆ AudioStreamDataProperties

Sound::ProceduralSoundData::SoundProperties Leviathan::GUI::VideoPlayer::AudioStreamDataProperties
protected

Definition at line 170 of file VideoPlayer.h.

◆ ChannelCount

int Leviathan::GUI::VideoPlayer::ChannelCount = 0
protected

Definition at line 159 of file VideoPlayer.h.

◆ FrameHeight

int32_t Leviathan::GUI::VideoPlayer::FrameHeight = 0
protected

Definition at line 155 of file VideoPlayer.h.

◆ FrameWidth

int32_t Leviathan::GUI::VideoPlayer::FrameWidth = 0
protected

Definition at line 154 of file VideoPlayer.h.

◆ HasAudioStream

bool Leviathan::GUI::VideoPlayer::HasAudioStream = false
protected

Set to true when an audio stream is found and opened.

Definition at line 165 of file VideoPlayer.h.

◆ IsPlaying

bool Leviathan::GUI::VideoPlayer::IsPlaying = false
protected

True when playing back something and frame start events do something.

Definition at line 149 of file VideoPlayer.h.

◆ IsPlayingAudio

bool Leviathan::GUI::VideoPlayer::IsPlayingAudio = false
protected

Used to start the audio playback once.

Definition at line 162 of file VideoPlayer.h.

◆ OnPlayBackEnded

Delegate Leviathan::GUI::VideoPlayer::OnPlayBackEnded

Called when current video stops player

Todo:
Should be renamed to OnPlaybackEnded

Definition at line 181 of file VideoPlayer.h.

◆ PassedTimeSeconds

float Leviathan::GUI::VideoPlayer::PassedTimeSeconds = 0.f
protected

Definition at line 173 of file VideoPlayer.h.

◆ Pimpl

std::unique_ptr<Implementation> Leviathan::GUI::VideoPlayer::Pimpl
protected

Definition at line 141 of file VideoPlayer.h.

◆ SampleRate

int Leviathan::GUI::VideoPlayer::SampleRate = 0
protected

Audio sample rate.

Definition at line 158 of file VideoPlayer.h.

◆ StreamValid

std::atomic<bool> Leviathan::GUI::VideoPlayer::StreamValid {false}
protected

Set to false if an error occurs and playback should stop.

Definition at line 176 of file VideoPlayer.h.

◆ VideoFile

std::string Leviathan::GUI::VideoPlayer::VideoFile
protected

Definition at line 143 of file VideoPlayer.h.

◆ VideoOutputTexture

bs::HTexture Leviathan::GUI::VideoPlayer::VideoOutputTexture
protected

The target texture.

Definition at line 146 of file VideoPlayer.h.

◆ VideoTimeBase

double Leviathan::GUI::VideoPlayer::VideoTimeBase = 1.f
protected

How many timestamp units are in a second in the video stream.

Definition at line 152 of file VideoPlayer.h.


The documentation for this class was generated from the following files: