Cutscene upscaling

Improving the artwork in Warzone2100 - not for mod discussions
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Cutscene upscaling

Post by Phase II »

I made a go at upscaling with ESRGAN: https://drive.google.com/open?id=1yF7Hs ... 8KVgWdxzfZ (Download the file to see actual quality. Google dropped the quality for the online preview.) My processing pipeline takes about 2 minutes for each second of video.

I think looks better as compared to the official sequences.wz when each are viewed full screen. However, it does lose some fine details. Part of the issue is from the number of reencoding steps: originalMovie -> ogg/theora -> imageMagickCleaning -> ESRGAN -> image -> ogg/theora. I would like to give it another go with the original files. Are they available for download?
User avatar
WZ2100ModsFAn
Trained
Trained
Posts: 371
Joined: 15 Apr 2018, 17:25
Location: United States.

Re: Cutscene upscaling

Post by WZ2100ModsFAn »

I think this is a good idea for Warzone 2100 Art Revolution.
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

Lots more experimentation, I think recovered as many of the details as I can: https://drive.google.com/open?id=1-7ll9 ... hlwCIRaz9t (again, download for full quality).
  • Top left - standard quality
  • Bottom left - high quality
  • Right - ESRGAN
Image
User avatar
WZ2100ModsFAn
Trained
Trained
Posts: 371
Joined: 15 Apr 2018, 17:25
Location: United States.

Re: Cutscene upscaling

Post by WZ2100ModsFAn »

Have you also tried 4x scaling?
That's some great quality decoding there.
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

*Technically* it already is. The high quality version was upscaled from the original.
  • Left - unaltered extracted frame
  • Right - Downscaled ESRGAN to match size
Image

Scaling it 4x from HQ size yields 2560x1920. Great if you want a desktop background for a rather large 4:3 monitor. Not so great if you want reasonably sized video (before compressing into a video, the frames for the ESRGAN scaled cut scene I posted are about 2.8 GB). (Could do a bit better if we used a modern codec like VP9 instead of Theora, but not enough to be reasonable.) Also, it would be quite challenging to see any difference in video quality on a computer screen except if you pause.
WZ2100ModsFAn wrote: 21 Sep 2019, 03:07 That's some great quality decoding there.
:) Was starting to worry I had stared at frames long enough not to be able to see the difference.

@Others: is there enough interest here that I should work through upscaling all the cutscenes?
User avatar
WZ2100ModsFAn
Trained
Trained
Posts: 371
Joined: 15 Apr 2018, 17:25
Location: United States.

Re: Cutscene upscaling

Post by WZ2100ModsFAn »

ESRGAN can improve the blurry effects in the video. Meaning that in 1280x720 you can see the video much better.
In 1920x1080 the video can be blurry. I think i also saw another HD upscale of these videos somewhere else too.
The default res is 1024x768. And the video still can be seen clearly through the 720p res.
However to play it in warzone a conversion is required to play it in OGG Video format. MP4 has much better quality.
I think you should make more conversions to the 720p ogv. It'd be a good idea to have a complete set of sequences in 720p HD.
The predicted size can be around 4GB - 7GB in file size for the new sequences.wz
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

Conversion in progress...

This will take some time. ESRGAN is slow. Each output must be validated; certain frames can produce very odd results requiring manual edits. Some of the files have minor corruption (all still recoverable, just harder to work with). Not all the clips have the same fps (trying to repair the worst of them via butterflow).

No current estimate, but I will make a post when the first campaign is ready.
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

So, I am about 2/3rds through converting the first campaign, and would like some input. A portion (maybe 30%?) of the files are 12.5 fps. This makes the video playback feel especially jerky/stuttering when upscaling the frames. For linear movement (i.e. panning, zooming, driving in a straight line), butterflow is able to give useful interpolation. However, it fails on fast rotations (when butterflow cannot track motion, it just blends the frames together, making blurry ghosting wherever there is movement). This most often triggers on the new technology scenes (think rotating display of power modules). Should I:
  1. Do nothing, output a new 12.5 fps video
  2. Attempt to render my own spinning object using base game assets
  3. Attempt to render my own spinning object using art revolution assets
If rendering my own, support from people more knowledgeable than I would be greatly appreciated. (As a programmer that does hobby photography, 3D animation really is not my thing.)
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

Well, good news and bad news.

Good news. Campaign 1 is available in never before seen resolution: https://drive.google.com/open?id=1Q_LF9 ... -aZ8u0xeYe

Bad news. Campaign 1 will remain unseen in this resolution.

Code: Select all

--- Starting log [E:\BasicSR\Warzone 2100 Portable-3.3.0\Warzone 2100 3.3.0\logs\WZlog-1004_225807.txt]---
error   |22:58:10: [seq_Play:687] Video size too large, must be below 1e+03x1e+03!
last message repeated 2 times
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

Hundreds of hours of editing and compute time, and thousands of manually edited frames. Presenting remastered HQ videos: https://drive.google.com/open?id=1JLcBp ... UrCBcZwHca

Video frame dimensions are the same as the official HQ sequences. However, most look significantly better.

I learned a lot during the process. Once I got to cam3, the quality I could create improved. But, cam3 originals are more compressed than cam1, making detail retrieval challenging. As such, overall quality is fairly constant.

I would greatly appreciate feedback and testing of these new sequences. They are working fine for me in the campaign thus far, but I only tested the first few levels.
User avatar
Berserk Cyborg
Code contributor
Code contributor
Posts: 938
Joined: 26 Sep 2016, 19:56

Re: Cutscene upscaling

Post by Berserk Cyborg »

Yo, I took a quick look at these. They mostly look great! There isn't really any fast way to test the sequences in-game other than using the "let me win" cheat (or "ascend" if you know the in-game mission labels well enough).

The biggest issue is the broken outro. It has a sort of "underwater" effect making everything all blurry. And in the intro, there is a part where a door to a base is opened and there are a bunch of horizontal black line artifacts showing up near the floor (I just remember it looked odd). Other than that there were a few cases of "mixing" going on smaller detailed things like the Gamma artifact shown in some of the sequences.

Don't know where the originals are, or even if they exist anywhere at the moment. I suppose one could rip them out from a ISO of the original game as a last resort.
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

Thanks for the feedback. I will give another pass at the intro and outro; see if I can figure out something better.
Phase II
Trained
Trained
Posts: 32
Joined: 14 Aug 2016, 01:22

Re: Cutscene upscaling

Post by Phase II »

Been trying to train a custom model on original artwork (try it if you want).

Can give some good (in my opinion) results:
https://youtu.be/SDVFInudUpQ
https://youtu.be/QZG8xsJGPdw

But also likes drawing solar panels on everything...
Image

(Edit: looks like youtube embed is not working?)
pastdue
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 339
Joined: 13 Aug 2017, 17:44

Re: Cutscene upscaling

Post by pastdue »

Phase II wrote: 22 Mar 2020, 15:54 Been trying to train a custom model on original artwork (try it if you want).

Can give some good (in my opinion) results:
[snip]
Very neat idea. Those are solid results.

Phase II wrote: 22 Mar 2020, 15:54But also likes drawing solar panels on everything...
It's just ahead of the curve. :wink:
aliswe
Trained
Trained
Posts: 63
Joined: 01 Apr 2020, 22:11

Re: Cutscene upscaling

Post by aliswe »

One should be able to download the cutscenes ingame, actually. And also these high-quality, upscaled ones. Good work!

You wouldn't want to have a go at upscaling the background images to 1080p would you?

https://github.com/Warzone2100/warzone2100/pull/753
Post Reply