Cutscene upscaling

[pastdue]

I would like some reviews of my original re-encoded content to see about replacing LQ and standard with it, and it can then also be used for anyone else to work from with regards to upscaling. Pastdue and cybersphinx? I'm not sure who has write perms to the WZ Sourceforge videos.

Current (and past) versions of WZ only support (and expect) Theora for the video format, so we couldn’t replace those files with VP9 versions (although I have on my to-do list exploring adding VP9 support going forward - no idea when / if I’ll have time though).

Phase 2, for your testing on upscaling double check my linked sequences if you need to because those should be the best to work from.

I believe Phase II is working from the original source / files on disc.

[Tzeentch]

Ok, well when you come to do it you'll have those files ready. I'll keep them safe.
So I will do again and change to Theora format, then they can be replaced. mkv container format ok?

Edit - done in Theora, Size 223MB. Looks good. I have fixed the cropping issue, can I replace the current in place video for low quality?

[Tzeentch]

All I need to is to replace the LQ version cutscenes on SourceForge. Can i be given access or share with someone who does to replace with my videos that conform to the standard?

[pastdue]

Ok, well when you come to do it you'll have those files ready. I'll keep them safe.
So I will do again and change to Theora format, then they can be replaced. mkv container format ok?

Edit - done in Theora, Size 223MB. Looks good. I have fixed the cropping issue, can I replace the current in place video for low quality?

I believe we only support Ogg container format at this time. (Should be a Theora-encoded video in an Ogg container with a Vorbis audio stream, IIRC.)

[Tzeentch]

Yeah that is specific, some tools out there appear to have removed the ogg container format.
Theora video and Vorbis audio is fine.
Guess I will just wait until VP9 is explored and if possible moving to mkv container. Easier to work with out of the box.

Of interest - https://www.mail-archive.com/search?q=r ... %40gna.org

[Phase II]

Finished, finally. All cutscenes upscaled with custom trained models. https://sourceforge.net/projects/wz2100 ... ences-ogg/ It has been a long journey, learning how to get AI to produce useful results, with such a small training set of high quality renders.
I believe these cutscenes are high enough quality to be considered being made into an official install option. Thoughts on how to proceed?

[themac]

Nice work, but my attempts with devestation months ago looked better with less artifacts and they ran smooth with no stuttering. Even then it did not make it into a release.

(do not watch online):
https://drive.google.com/file/d/1qRMFzb ... sp=sharing

[panrt26]

Re: Cutscene upscaling
Post by Phase II:
Hi, i download sequences but where i paste .tar? Thanks.

[Tzeentch]

It may be best that this is redone from scratch.
The files we provided here will need encoding again and up-scaling content effectively does depends upon having good source input from the original source - which is my plan to provide.

Order is:
Obtain all orig files in the native RPL format ( I can/will obtain these ) (+add them all to the WZ VCS historical git/archive ) separate from "sequences.wz"
Use the tool rpl2avi to convert all files (+ provide the rpl2avi converter for native use for anyone else)
Convert those uncompressed raw files into the current appropriate format for WZ until VP9 and AV1 can be used etc. <- without any perceivable drop in quality from the raw rpl format.
Update all files hosted ( we don't need LQ, HQ and medium, just .. 1 dir with all videos will suffice ) - Going by the DL numbers, everyone (95%+) uses HQ

From here you can upscale all those files etc as you wish.

[Tzeentch]

For a touch more background info to the reader:
RPL2AVI tool was first used to convert the actual original .RPL files into an uncompressed AVI file format. By some very hacky code (admitted within the comments of the code itself that takes from other open source code elsewhere). RPL also cannot be played natively.

Firstly obtaining the best source content should be the total focus, prior to any question on up-scaling efforts.
Along with minimizing the number of re-encoding steps.
It is also a challenge to up-scale to 4K considering the original content was in 320x160 resolution.

As seen early in this thread - "Part of the issue is from the number of re-encoding steps: originalMovie -> ogg/theora -> imageMagickCleaning -> ESRGAN -> image -> ogg/theora." From, Phase II.

Phase II therefore used our HQ sequences as the source for all this, which in itself are the videos just up-scaled and filtered by means of using ffmpeg from the original.

The sequences that are in the end result of Ogg/Vorbis/Theora format are all re-encoded from the uncompressed AVI files provided by RPL2AVI via ffmpeg2theora, using libtheora1.1beta1.

The command used was this:
ffmpeg2theora --optimize -v10 -K65536 --pp de/tn:64:128:256 -c2

The document / readme doesn't quite elaborate on the different command used per sequences per HQ vs LQ.

Ideally, we only need instead of 3 DL options being (LQ, Standard and HQ) just 1 DL for end clients being "Original" sequences provided to simplify all matters.
Then from that, we could look at an up-scaled alternative.
Finally... creating brand new videos as a new alternative replacement to choose from later down the line.

At some stage, AV1 support.

To ask the Q.) Can the source content output be improved by RPL2AVI on the first iteration that gives us the uncompressed AVI? As all other re-encode operations are using those files as the input source.

Further to RPL2AVI, can it be reworked to output the file directly that we need, so we only run one tool in a single step to achieve the output needed?

Are the PS1 version of the source cutscenes better in terms of quality? - That we may compare PC vs PS1 cutscene source content.
Where may we find the PS1 version of the cutscenes separate to the PC version?

[AlexTheDacian]

The PS1 cutscenes are, from my own little knowledge and from what Stefan told be aeons ago, a bit more better in comparison to the normal PC RPL ones. The big glaring issue is that you need to RIP 'em off the PS1 discs themselves, which thankfully, we have jPSXdec for that one...
Also, Phase II already posted a little comparison earlier, but what he didn't post is the intro for the Alpha Campaign, which looks oh so glorious after a pass through both Stefan's and my grubby hands

[youtube]https://www.youtube.com/watch?v=sQ7ZqOcggSg[/youtube]

I'll try to get my hands on 'em again, and do a little comparison, and put it up for you all to see.

[Tzeentch]

Indeed and another such tool exists written in C that may be pertinent in this situation - "psxavenc"

I concur, that result looks nice. Ideally, having both videos play side-by-side to highlight the comparison would be great way to showcase it imo.

On my first glance when comparing videos side-by-side of the PS1 version vs PC they look almost if not the same to me.
Though I much look forward to seeing

I was thinking that another tool known as vmaf as made by Netflix could also assist as part of the encoding pipeline, to provide metrics on the compared output.
Lastly... is converting mono into stereo channel output along with ensuring the audio levels between tracks is not too dissimilar (i.e. Devastation is very loud) but one small step at a time eh.

[Tzeentch]

More a self note here: (As I can't seem to edit previous post)
The videos were re-encoded from the uncompressed AVI files via ffmpeg2theora (using libtheora1.1beta1 originally.)

The readme doesn't quite elaborate on the different commands used exactly, per 3 sets of sequences between HQ, STD vs LQ.

The command used was this (for High quality):
ffmpeg2theora --optimize -v10 -K65536 --pp de/tn:64:128:256 -c2
Which does:
--optimize better compression
-v10 highest quality
-K65536 large keyframe interval (since we don't need to video seek)
-pp post-processing (well post-decoding, pre-encoding)
de deblocking/deringing
tn:64:128:256 temporal noise removal
-c2 use two audio channels

"For the 480p versions, it was --optimize -v10 -a10 -H96000 -K65536 -c2" - must be the "standard-quality" vids.

ffmpeg2theora's default of 5 (which was used for the ~170MB videos) - this must refer to the low-quality ones.