Summary of GSoC 2017

Overview
During the last three months I have worked on porting the game Mission Supernova to ScummVM, a project aiming to reimplement engines of point and click adventures for cross-platform support.

For the latest changes, see the commits to my fork.
Besides porting I also worked on the ScummVM launcher and fixed theme related bugs

• font fallback 
• unexpected theme change

Goals
The following lists features that are implemented:

• Image and Text rendering
• Audio playback of sounds and music
• Animation system
• GUI
• Saving and restoring game state
• Game logic converted to engine reimplementation

TODO
Most crucial engine components work but there is still work left to be done:

• Translate the remaining strings to English
• Implement dialog system
• Fixing pesky bugs
• Make it all work together in harmony

While this may be the end of GSoC 2017, my journey has just begun.
Thanks to everyone involved sharing their knowledge and providing help whenever needed. Also thanks to Google for giving the incentive and exposure to all those incredible projects. The experience of staring at hexdumps, deciphering disassembly and the communal effort has been invaluable to me.
I will keep contributing to the ScummVM project in the future beyond finishing my project (just you wait, Chewy!) and hope that some of you out there will find entertainment in my work.

Summary -- Week 11

SPOILERS IN VIDEO! (0:22 - 0:31)

As announced last week, the timer bugs are no more!

The original game used ticks of 18.2Hz instead of milliseconds for example, so while this little helper converted ticks to milliseconds it was not always easy to recognize what number actually represented a time value.

For instance, I assumed that a variable named time would most likely be the perfect candidate. Oh, how wrong I was. It actually stored the remaining days until the ship arrives at its destination. Do not ask how long it took me to figure this out although comparing time with energy should have made me wary.. (1,2,3)

Adjusting the brightness of the palette was more an issue of rendering order. While the function worked fine it either did a buffer swap too early or the brightness was restored to its previous state by the following draw call.

By removing most special cases it not only simplified the function but also fixed most issues. I did a side by side comparison with the original game and couldn't find any difference. (1)

What was different though is that I couldn't interact with anything in the ship's hold anymore. Apparently it hasn't been working for a while.. Originally an uninitialized object was identified by checking if its name string points to NULL but I misread it as a check if the first char is 0 (empty string). Unfortunately, objects without a name can still be valid what messed up the traversal of objects in a room. (1)

After running in circles for a while a few important features like RTL and save/load support were implemented. Unfortunately, state serialization still shows puzzling behavior that I haven't figured out how to solve yet. (1,2,3)

Summary -- Week 10

• The biggest change this week was fixing the terminal next to the sleeping pods and the clearing of previous input. (1,2)
• Also, the game text was converted to it's original encoding (CP437) to correctly render special characters. (1) 
• A long standing rendering bug that caused certain sprites not to be rendered was finally found and fixed. (1)

There were a few smaller commits that will help to finally fix timing issues and a few rendering problems with dimming the ship's light next week.

Summary -- Week 9

The remaining game logic has been implemented with a few exceptions but still needs to be thoroughly tested. (1, 2)
Speaking of testing, there are still bugs in the first act that need to be ironed out before a pull request can be issued like rendering artifacts and problems with the terminal next to the sleeping pods.

For next week, it is planned to fix up act 1 and to finish implementing the dialog system as the second and third act are now available for testing. 

Summary -- Week 8

This week was about implementing the terminal in the sleeping cabin room, adding the death screen and clean up.

Unfortunately it's still quite buggy and needs more investigation. Currently, the game restarts on death to the beginning as there's no full implementation of save/load yet.

For next week I will keep implementing missing game code for the remaining two acts and make it just compile for now.
Most key functionality is already implemented but with the other acts in the engine the dialog functionality can finally be tested.

Summary -- Week 7

So most of the week was spent on rewritting the game loop to implement animations as shown above and fixing the bugs introduced by it..
The animation system is simply implemented by setting a timer that when it expires causes the room specific animation() function to be called. (1, 2, 3, 4)
Besides that,

• definitions for the remaining rooms were added
• initializes game state on startup (brought more surprises than anticipated)
• implement monitor scene

An interesting discovery for me was that although the text rendering code is correct it shows artifacts when drawing umlauts. As the game strings were copied directly from the cp437 encoded source I expected the character value to be encoded in a single byte as characters in extended ASCII don't exceed 0xFF. What I didn't consider is that UTF8 is only backwards compatible to basic ASCII (0x00 - 0x7F) and anything above takes at least two bytes, what causes the corruption as seen above ("Triebwerke funktionsunfähig"). That's why umlauts in the German original will be substituted by their equivalent octal value as already done here.


For the coming days the first act should be made playable by implementing edit() so the sleeping cabin section works correctly and adjust the game loop for dying and implement the 'death screen'.
After that implementing the remaining game logic takes priority especially now that the animation system is working.

Summary -- Week 6

Sorry for the delayed report this week but I really wanted to add a video finally walking through the empty hallways of the starship Supernova. Moreover...

🎉 The GUI is finally working! 🎉

Probably the most noticeable improvement is that GUI is now actually rendered correctly and interaction with the environment is finally possible without everything glitching out. Although, there are still issues with Give and Use command that will be addressed next.

Besides the GUI there have been improvements on every front.

• Wrapping GUI elements in a state class (1, 2, 3)
• Fixing font rendering
• Correctly handling and rendering object state (1, 2, 3, 4)
• Improvingn mouse input handling (1, 2)

Coming up next will be implementing timers for handling events and animations. There have been talks about starting translating but depending on how things go it either has to be pushed back or will have a lower priority.
Of course fixing a bunch of bugs along the way would be nice too :)

Summary -- Week 5

Sorry for the delayed update due to injuring myself while sleeping..
The input code still gives me quite a headache and will be partially rewritten when there's time. I am now in a place I wanted to prevent ending up in but for now, getting things running has priority.

As I said getting the input code working was my goal for this week additionally to fixing some GUI issues and rendering static GUI elements like the minimap, inventory, command row and status line.
I expected this part to be more difficult, as the original code is quite intertwined but I completely misjudged the effort needed for porting and debugging.

For this week I want to keep fixing the GUI and implement stubs
so the remaining game logic can be implemented without too much trouble.

Summary -- Week 4

This week only a few changes made it to the repo but progress is progress.

• Drawing mouse cursor and hourglass (1,2)
• Buffering image sections on startup (1)
• Playing game music (1,2)

Unfortunately there still seems to be a bug in either my conversion code or in the protrackerStream implementation. The top video is from criezy's blog post on converting the Mission Supernova music to MOD files and the one below shows my implementation.
Right now, it works well enough but should be fixed before GSoC ends. At ~30 seconds in you will hear the reverb is missing in my video and other glitches.

Drawing the Cursor

Seems like our Chef Horst Hummel got impatient and sliced the hourglass cleanly in half..
The 16x16 cursor image is stored as a bitmap like the newspaper images I described in my previous post on graphics and animation in Supernova. The black outline and red filling are rendered seperately, both taking 32 bytes.
I skimmed through my code and could not find a shift that possibly corrupted the cursor data. But how to fix it? Well, it seems the image is split right at 8 pixels and since they are represented by one byte we could just do a wordRoll that swaps its bytes like swapped = (i >> 8) | (i << 8).
Wait a second....

The disassembly shows the critical part for drawing the black outline

next_line:
          lodsw
          mov cx,dx

next_pixel:
          shl ax,1
          jc skip1
          mov byte es:[di],0
skip1:
          inc di
          loop next_pixel

          add di,320
          sub di,dx
          dec bh
          jnz next_line

lodsw... The first endianess bug in this project that I got stuck on..
At least now it's working and I can keep going with more important stuff like actually making it playable and hopefully won't get distracted by those small things again...

 

Summary -- Week 3

Unfortunately there won't be pretty pictures for this weeks summary but I finally released by blog on the graphics and animation of Supernova.
I've been working mostly on converting the game logic for the first act and extending the engine (e.g. renderText(), renderMessage(), save/restore of screen sections, ..)
More noteworthy commits were

• Implementing of Room Data Structures
• GameManager class
• Conversion of game logic for first act

This week I will be getting the engine in a 'coherent' state before the first evaluation on June, 26th. While rendering images and animations work, playing audio samples and the game logic for first act was converted, all those features do not yet play together.
The goal will be rendering the starting room with the ability to interact with objects in some form. If everything works fine that's how it will look like

Image Rendering and Animation

Let's hope not all humans are that ugly  --  probably some Axacutian

This post will be about the compression algorithm used for the graphics and how it plays into the animation system of Supernova.
The images consist of a custom palette of 239 colors and divided up into sections that define its size and a pointer to the image data. An exception to this are the newspaper articles, seen in the intro of the game.

 

Those bitmap images have dimensions of 640x480 and only monochrome colors, instead of 256-colors palettized 320x200. The difference in their encoding can be seen here in the code. So, for the mode 0x11 images I mask the bits of a byte and depending if the bit is set 0 (Black) or 11 (63% White) is written. It is safe to use palette colors as the first 16 colors of the default palette, defined here, won't be overwritten by an image's custom palette.

Mode 0x13 images that make up the rest of the game's graphics are compressed in a run-length encoding (RLE) and are 320x200 256-color palettized. The following describes the structure of the file format

The image decoding part can be found here. It works more or less like this:

The encoded image is read into a variable 'input' byte by byte

        - If input < numRepeat, write the next value read input-times
        - If input < numZw, write the (input - numRepeat)th value of 'twin-value
          array' twice
        - Otherwise, write input value directly

The success of this simple way of compression depends on how monotonous the image is. The less color changes happen in a scanline the better the compression.

 

Sections, as already mentioned, describe an image section that most of the time shows objects in different states of interaction, like an opened or closed door, by redrawing only part of the base image (section 0).
As you can see in the video, sections 7, 8 and 9 are the frames for the animation of the 'slide door' to the airlock. And that's the whole trick. Well, a door sliding open in 3 frames is not that impressive, so I composed a video with 3 scenes from the game that meet your expectations a bit more when you hear the word 'animation'

Summary -- Week 2

Another week I have to start with an apology for delaying the blog post on the game audio. But I brought pretty pictures, so I hope we are even :)

As you can see lot of things improved since last week, most notably

• image palette fixes (1, 2, 3)
• text rendering,
• extension of renderImage() to render sections that are needed for animation and interaction

Currently, I have been working on getting game logic into the game, starting with setting up the data for the rooms. Once I am comfortably ahead on that front I want to finally push out my articles on graphics and audio I'm sure everyone has been waiting for eagerly.

For the coming week my main focus will lie on implementing the game logic and wire it all up so some interaction may be possible. Despite wishful thinking I know there will be bumps in the road for me, so getting the data structures straight is what my attention is directed to right now.

Summary -- Week 1

My start was bumpier than I hoped. Nevertheless, there is progress that I will show more regularly in the coming weeks.

Yes, sometimes progress shows in the most obscure ways. But after a day of segfaults, this felt gooood. It is supposed to be the title screen, that obviously got corrupted along the way. More bugs and a psychodelic strobe palette later I ended here:

Thanks to criezy and Strangerke the palette bug could be fixed that happened by using uninitialized values. Unfortunately, the palette is still not correct as the color fading code is yet to be implemented and text rendering is missing.
For now let's consider it modern art.

In the coming days I will focus more on implementing game routines so that I can iterate on basic functionalities once the most crude bugs are ironed out. An important decision yet to be made will be how the data structures will look in detail for either parsing the game logic or reimplementing it in the new engine.
Also, a blog post on audio samples in addition to criezy's post on the music will be released shortly.

Introduction

There have been many games I enjoyed over the last decades and some of them would have been already lost if it weren't for enthusiasts writing emulators or go as far as rewriting engines themselves.
Personally, I haven't done much in that regard other than poking memory for infinite gold or a simple reversing challenge. However, when the organizations for this year's GSoC were announced one particular project caught my attention.

I have fond memories diluted by nostalgia, of point and click adventures like Day of the Tentacle, Loom and others. Despite the genre's revival it has been mostly off of my radar for quite some time. So far it has been a great experience contributing to the project. Not only can I finally put my DOS knowledge and reverse engineering to use but the community has been welcoming, helpful and overall a great place. The ScummVM project ports point 'n' click and since recently also RPG game engines to their ecosystem so that they can be played on a variety of platforms.
(shoutout to criezy going through 30 revisions of my PR deep into the night..)

I have been fortunate to be selected to port Mission Supernova over the summer and translate the game from German to English. Anyone interested in the progress of my work, keep an eye on my repo and this blog where I will write about the progress of the project, its difficulties and technical details I find interesting.

Thanks again to the ScummVM team and Google for giving me this amazing opportunity.