Cutscene upscaling

Improving the artwork in Warzone2100 - not for mod discussions
Post Reply
Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Cutscene upscaling

Post by Phase II » 17 Sep 2019, 18:33

I made a go at upscaling with ESRGAN: https://drive.google.com/open?id=1yF7Hs ... 8KVgWdxzfZ (Download the file to see actual quality. Google dropped the quality for the online preview.) My processing pipeline takes about 2 minutes for each second of video.

I think looks better as compared to the official sequences.wz when each are viewed full screen. However, it does lose some fine details. Part of the issue is from the number of reencoding steps: originalMovie -> ogg/theora -> imageMagickCleaning -> ESRGAN -> image -> ogg/theora. I would like to give it another go with the original files. Are they available for download?

User avatar
WZ2100ModsFAn
Trained
Trained
Posts: 367
Joined: 15 Apr 2018, 17:25
Location: United States.

Re: Cutscene upscaling

Post by WZ2100ModsFAn » 18 Sep 2019, 03:18

I think this is a good idea for Warzone 2100 Art Revolution.
I use Ubuntu GamePack By UALinux for gaming. Best Ubuntu Distro for Warzone :D

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 21 Sep 2019, 00:42

Lots more experimentation, I think recovered as many of the details as I can: https://drive.google.com/open?id=1-7ll9 ... hlwCIRaz9t (again, download for full quality).
  • Top left - standard quality
  • Bottom left - high quality
  • Right - ESRGAN
Image

User avatar
WZ2100ModsFAn
Trained
Trained
Posts: 367
Joined: 15 Apr 2018, 17:25
Location: United States.

Re: Cutscene upscaling

Post by WZ2100ModsFAn » 21 Sep 2019, 03:07

Have you also tried 4x scaling?
That's some great quality decoding there.
I use Ubuntu GamePack By UALinux for gaming. Best Ubuntu Distro for Warzone :D

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 21 Sep 2019, 14:01

*Technically* it already is. The high quality version was upscaled from the original.
  • Left - unaltered extracted frame
  • Right - Downscaled ESRGAN to match size
Image

Scaling it 4x from HQ size yields 2560x1920. Great if you want a desktop background for a rather large 4:3 monitor. Not so great if you want reasonably sized video (before compressing into a video, the frames for the ESRGAN scaled cut scene I posted are about 2.8 GB). (Could do a bit better if we used a modern codec like VP9 instead of Theora, but not enough to be reasonable.) Also, it would be quite challenging to see any difference in video quality on a computer screen except if you pause.
WZ2100ModsFAn wrote:
21 Sep 2019, 03:07
That's some great quality decoding there.
:) Was starting to worry I had stared at frames long enough not to be able to see the difference.

@Others: is there enough interest here that I should work through upscaling all the cutscenes?

User avatar
WZ2100ModsFAn
Trained
Trained
Posts: 367
Joined: 15 Apr 2018, 17:25
Location: United States.

Re: Cutscene upscaling

Post by WZ2100ModsFAn » 22 Sep 2019, 03:27

ESRGAN can improve the blurry effects in the video. Meaning that in 1280x720 you can see the video much better.
In 1920x1080 the video can be blurry. I think i also saw another HD upscale of these videos somewhere else too.
The default res is 1024x768. And the video still can be seen clearly through the 720p res.
However to play it in warzone a conversion is required to play it in OGG Video format. MP4 has much better quality.
I think you should make more conversions to the 720p ogv. It'd be a good idea to have a complete set of sequences in 720p HD.
The predicted size can be around 4GB - 7GB in file size for the new sequences.wz
I use Ubuntu GamePack By UALinux for gaming. Best Ubuntu Distro for Warzone :D

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 26 Sep 2019, 02:21

Conversion in progress...

This will take some time. ESRGAN is slow. Each output must be validated; certain frames can produce very odd results requiring manual edits. Some of the files have minor corruption (all still recoverable, just harder to work with). Not all the clips have the same fps (trying to repair the worst of them via butterflow).

No current estimate, but I will make a post when the first campaign is ready.

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 28 Sep 2019, 13:43

So, I am about 2/3rds through converting the first campaign, and would like some input. A portion (maybe 30%?) of the files are 12.5 fps. This makes the video playback feel especially jerky/stuttering when upscaling the frames. For linear movement (i.e. panning, zooming, driving in a straight line), butterflow is able to give useful interpolation. However, it fails on fast rotations (when butterflow cannot track motion, it just blends the frames together, making blurry ghosting wherever there is movement). This most often triggers on the new technology scenes (think rotating display of power modules). Should I:
  1. Do nothing, output a new 12.5 fps video
  2. Attempt to render my own spinning object using base game assets
  3. Attempt to render my own spinning object using art revolution assets
If rendering my own, support from people more knowledgeable than I would be greatly appreciated. (As a programmer that does hobby photography, 3D animation really is not my thing.)

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 05 Oct 2019, 05:20

Well, good news and bad news.

Good news. Campaign 1 is available in never before seen resolution: https://drive.google.com/open?id=1Q_LF9 ... -aZ8u0xeYe

Bad news. Campaign 1 will remain unseen in this resolution.

Code: Select all

--- Starting log [E:\BasicSR\Warzone 2100 Portable-3.3.0\Warzone 2100 3.3.0\logs\WZlog-1004_225807.txt]---
error   |22:58:10: [seq_Play:687] Video size too large, must be below 1e+03x1e+03!
last message repeated 2 times

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 05 Nov 2019, 11:08

Hundreds of hours of editing and compute time, and thousands of manually edited frames. Presenting remastered HQ videos: https://drive.google.com/open?id=1JLcBp ... UrCBcZwHca

Video frame dimensions are the same as the official HQ sequences. However, most look significantly better.

I learned a lot during the process. Once I got to cam3, the quality I could create improved. But, cam3 originals are more compressed than cam1, making detail retrieval challenging. As such, overall quality is fairly constant.

I would greatly appreciate feedback and testing of these new sequences. They are working fine for me in the campaign thus far, but I only tested the first few levels.

User avatar
Berserk Cyborg
Code contributor
Code contributor
Posts: 829
Joined: 26 Sep 2016, 19:56

Re: Cutscene upscaling

Post by Berserk Cyborg » 07 Nov 2019, 02:33

Yo, I took a quick look at these. They mostly look great! There isn't really any fast way to test the sequences in-game other than using the "let me win" cheat (or "ascend" if you know the in-game mission labels well enough).

The biggest issue is the broken outro. It has a sort of "underwater" effect making everything all blurry. And in the intro, there is a part where a door to a base is opened and there are a bunch of horizontal black line artifacts showing up near the floor (I just remember it looked odd). Other than that there were a few cases of "mixing" going on smaller detailed things like the Gamma artifact shown in some of the sequences.

Don't know where the originals are, or even if they exist anywhere at the moment. I suppose one could rip them out from a ISO of the original game as a last resort.

Phase II
Rookie
Rookie
Posts: 20
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II » 10 Nov 2019, 09:12

Thanks for the feedback. I will give another pass at the intro and outro; see if I can figure out something better.

Post Reply