FreespaceOpen installer improvement or how to mavenize an old ant based project

The Pre-story

A month ago I was having a big nostalgia moment, installed the GOG client and started to playing with old games. My childhood was there again. I remember these games very fondly even today, although I have played with them long ago. They managed to bewitch me with their charm and whenever I played I entered another world where I was the hero who had to solve the problems. It was really fantastic that even with the pixel graphics they had their magic.

I wanted to play with Freespace 2.
https://en.wikipedia.org/wiki/FreeSpace_2

I recall the day when my schoolmate introduced it to me during my elementary school years. It was fun to play the pilot who shoots down the evil shivans, meanwhile colossal ships clashing each other and you try to avoid their beam turrets laser. In my first encounter with the game I had rushed in the middle of the fight and ended up wildly spinning my ship by a shot of the beam turrets.
My nostalgia energy was definitely fully charged and ready to take the adventure, the helmet and the ship controls to show again who is the better pilot. I have clicked the start and right away the energy flew away because the window was halfy showing the game. Great, so I did some search and one of the best solution on steam was to use FS2Open version. Which pumps up the game to be ready to be used on a modern day computer without issues.

NOTE: It is also worst to mention that the creators of the Freespace game released the source code on the web when their company was bought to be freely used. Since that happened a lot of promising space simulator projects have been created based on the source code. More info here.

I went to Hard-Light wiki and downloaded their installer client which was a jar file. Hmm I program mostly in java I know about these things and I’m curious how the code looks like, it is always good to see how things are done by others to learn a few new tricks. The installer project source code is not hard to find, it is located on github. I checked the project and it was created long time ago. Fortunately the project is still active and I believe it is quite useful one. For that reason I created a Hungarian translation for it. Then I have converted it to maven based project for fun.

The basics

The current build system of the project is ant based. I don’t know much about ant itself, beside that it was commonly used before maven come to the picture for build management. On the other hand I know a lot of things about maven, since I work with it on daily basis and created a couple of libraries/projects based on it. I checked the build.xml . There are a few things that I know how to be expressed in maven. Before doing the big renovation I have to make a few things to work.
Hello Eclipse my old friend, I have come to you with another java project again. Importing the project was quite simple since it has a .project file. Ran the the ant compile, which worked, and then I created the jar and ran it, that worked as well.
Now I know how to do things in the old way, I’m ready to start the changes. In order to make the transformation to maven project, it is mandatory to know what are the dependencies (3pps that are providing functions) and for the runnable jar file what is the main class.

NOTE: At this point I would like to highlight that before doing any refactoring the best is to create few tests that ensure what worked before refactoring, will still works afterwards. In this case I only did manual testing which is better than nothing, but it is not enough. A proper approach would be to create a small automation that simulates the end user behavior. Like some GUI test which does some user interaction with the application. The reason I have skipped this part is due my time constraints.

The motivation

To do the work

Before jumping to the lightspeed work I would like to highlight a few more things. Beside the nostalgia factor, a project like this definitely worth the time. It builds on the top of an old project that worked well in its time, thus making it still viable for the current date and also for the future. It is free and open source project. This is a good way to learn new things and contribute back to the community.

To use maven

With maven I could remove jars from the git repo, since maven handles the binaries. Easier to create test for the project after it is converted. Maven has a lot more features since it is not just a build system but project management as well.

The challenges and solutions

Restructuring the project and creating the pom.xml

This is the easiest part. Maven has a convention over configuration approach, means as long as you follow the default setup you don’t have to set much in the pom.xml (project configuration file). In this case I have moved everything under src/ to src/main/java , resources/ went to src/main/resources and docs/ went to src/main/docs. Pattern is everything related to the main project goes under the respective places. Currently there are no test or at least I haven’t found them, but that would go under src/test/java in case of java files or src/test/resources in case of other files related to testing. This differentiation comes handy like when you create the jar file, that should not contain any test related classes. In this way it will not as this the default setting. However by adding the correct maven plugins to the pom.xml, the unit tests will be executed as part of the building process.
I have created the pom.xml in the following way:


 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.hardlight.fs2open</groupId>
	<artifactId>fsoinstaller</artifactId>
	<version>2.3.1-SNAPSHOT</version>
	<name>Freespace 2 open intaller</name>
	<description>The Java installer for FS2_Open and the collection of mods from the Hard Light Productions community.</description>
	 
</project>  	

This was easy to make and with this much basically the project is already maven based, but of course this is not enough to build it successfully. Also worth to mention the GAV(Group id, Artifact id, version) attributes. This would come handy if this project would be used later as a dependency. For the project itself maybe the version would be useful but for the maven all of them are mandatory.

Resolving the dependencies

This is where the game begins. The dependencies can be easily seen under ${project}/lib/ and one more mentioned in the build.xml which is Launch4j. That one as I understand it is not needed for the build, so I will skip that one for now. What comes next is to do some research and find the proper dependencies GAV for the jar file. There could be problem when nobody took care to upload of the jar to the maven binary repository. Unfortunately this can easily happen when an open source project dies. However due to the nature of open source it can still easily resolved. A bit more problematic when the project was originally a closed source project. I was highly discouraged to use those, especially if you create an open source project. Even for that scenario I would have an ugly workaround, hopefully that won’t be necessary.

NOTE: That workaround is to keep the jar in the git repo and before doing a build move the jar file by maven command to the maven local repository. Check for more information here.

Best approach I can came up with quickly to figure out the dependencies to google for each jar file and hope them to be in the maven repository and to look into the jar files as well, since they are just zips. The manifest file should tell some info.

l2fprod-common-directorychooser.jar

After doing some searching I have found the following version in the maven repo. From the jar file name itself I cannot tell the version, however if I look into the jar I can see some interesting things.


Specification-Title: L2FProd.com Common Components
Specification-Version: 6.11
Specification-Vendor: L2FProd.com

The version I have found is 6.9.1 and the version I have is 6.11. Based on the version itself I’m not sure which is older. However in my version(6.11) manifest I have found this:

Built-on: 11/01/2006 01:01 PM

and in the other 6.9.1 I have this:

#Generated by Maven
#Sun Jan 30 11:52:08 EST 2011
version=6.9.1
groupId=com.l2fprod.common
artifactId=l2fprod-common-directorychooser

NOTE: In general the newest version is the better version and hopefully it is backward compatible. Unless you are familiar with the project and its versions you never know this for sure. In this case I take my chance since the mayor version is still 6.x so we are on the good side. Also we are depending on how the 3pp is being used by the current project. If there was a defined API for the 3pp and that was being used by the fs2open installer, then chances are better for compatibility.

NOTE 2:
META-INF\MANIFEST.MF can be a good lead, but in this case in the local jar file there was also a com\l2fprod\common\Version.class file, which had the same information as the manifest file. That reinforces this jar file was probably not changed after the package was created. In the maven repo version, beside the manifest and pom file there is META-INF\maven\com.l2fprod.common\l2fprod-common-shared\pom.properties that gives some info.

I will use the 6.9.1 version for the time being, until it starts to give issues. I actually should do two things at this point. Do more research on the difference and create test. But again due to time constraint I just accept what I have now.

sevenzipjbinding.jar and sevenzipjbinding-AllPlatforms.jar

From the name I suspect these are from the same place so I will treat them together.

Again manifest file:

Built-Date: 2015-09-26 17:50:25Z
Implementation-Version: 9.20-2.00beta
Implementation-Vendor: sevenzipjbind.sf.net

And my google fu skills give me sites for both jars: sevenzipjbinding-AllPlatforms.jar sevenzipjbinding.jar. Bingo! The version is the same for both local and remote which is 9.20-2.00beta. I call this a success, then again I will be more confident when I do build and some manual testing.

xz-1.5.jar

In this case the name of the jar file holds a version which is 1.5, never the less I still check the manifest:

Bundle-Version: 1.5
Export-Package: org.tukaani.xz
Bundle-Name: XZ data compression
Bundle-DocURL: http://tukaani.org/xz/java.html

which tells the same version.

Google fu: xz-1.5.jar

Bingo again!

swingworker_1_5.jar

From the file name I can see that is 1.5 version, on the other hand the manifest file quite useless this time. After some search I have found out the as of Java 6, SwingWorker is included in the JRE in the wiki. That would be pretty nice. I just need to uplift the java version to 1.6, and currently I see no issue with that. Probably I need update source code for imports that I will check soon. Summa summarum, no extra dependency needed this casem only uplift of the Java version to 1.6

Summa of the dependencies.

I actually like the variety of the issues that I have bumped. I was not bored during my research but neither gave me headache either. I’m happy that currently there is no need for the workarounds.

Testing the dependencies

As a first test I need to make sure if the building works. With maven install I got the following issue:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile (default-compile) on project fsoinstaller: Compilation failure: Compilation failure:
[ERROR] /C:/Users/Kabuto/git/fsoinstallerownfork/src/main/java/com/fsoinstaller/utils/ProgressBarDialog.java:[38,32] package org.jdesktop.swingworker does not exist
[ERROR] /C:/Users/Kabuto/git/fsoinstallerownfork/src/main/java/com/fsoinstaller/utils/ProgressBarDialog.java:[138,39] cannot find symbol
[ERROR] symbol: class SwingWorker
[ERROR] /C:/Users/Kabuto/git/fsoinstallerownfork/src/main/java/com/fsoinstaller/utils/ProgressBarDialog.java:[138,76] cannot find symbol
[ERROR] symbol: class SwingWorker

That I have expected and in order to solve this
ProgressBarDialog the import needed to be updated to

import javax.swing.SwingWorker;

I will move forward to create the jar file and to do the final test on it.

NOTE: The better approach would be to do a test run as it is now without the big runnable jar.

The maven eclipse folklore dance

Maven itself works perfectly in console like git, however as an eclipse plugin some small issues may pop up.
The project was already imported before the conversion. In order to make eclipse realize the maven feature, do a right click on the project>configure and choose convert to maven project.
I have already had a .project and .classpath file which is for eclipse and describes where are the project related files located. This files need to be recreated after pom.xml is in place, in the top menu Project&gtclean. To trigger the sync again right click on the project Maven>Update project.. and in the popup window press ok. This will make the project and eclipse full synchronized. In general whenever change is made in the pom.xml, in order to keep the sync between eclipse and maven it is recommended to do the maven>Update project… action.

Making the runnable jar

Whenever we do an install with maven a jar file is created in the maven local repo(default place $User/.m2 ). This jar does not contains all the dependencies only the current project classes. In order to make a jar with dependencies a maven plugin needs to be used. I choose the maven shade plugin . I’m familiar with it and know how to make it work.

Before going further let’s check few things. Current jar is being named as FreeSpaceOpenInstaller.jar. The shade plugin follows the maven artifact id for the name of the jar file, I have to override it to have the same as the original. Could go for other way around to rename the artifact id the same way but I will take the previous approach. Beside all of the lib classes, the resources folder which contains pics , translations and docs folder is also located in the jar. I need to keep in mind these three things.

Copying the resources

Maven as a default copies everything under from ${project}/src/main/properties to the jars top level. In this case I want to have it under jar/resources . To achieve it I set the maven to not do the default behavior in pom.xml line 14-23 . Then I have set the ${project}/target/classes/resources as output directory. Shade plugin(big jar maker) will use to the final jar everything under ${project}//target/classes/resources . This part is pom.xml in line 28-45. And finally the docs folder needs to be copied, in default maven would not do anything with this folder. This is pom.xml line 46-62 .

Executable jar

Shade plugin part is pom.xml line 82-108 . Two mandatory part is the file name in line 88 and main class in line 102. The shade plugin beside the ${project}/classes folder add all the dependencies to the final jar. Except the scope test or compile ones an example for this in line 142.

To test all the work has been done so far I need to do an mvn install. Under the ${project}/target folder the FreeSpaceOpenInstaller.jar has been created. I have compared the size, it is similar, the directory structure is look alike as well. I ran it, it worked except one thing. My Hungarian translation character coding for the non english chars was not okay.

Bonus round fix utf-8 issue

After googliing a bit I found this I used the second answer:

Given that you have an instance of ResourceBundle and you can get String by:

String val = bundle.getString(key);
I solved my Japanese display problem by:

return new String(val.getBytes(“ISO-8859-1”), “UTF-8”);

This will give me a quick but dirty solution, later I will fix it in a more appropriate way. Test result was success, it fixed the issue. The patch.

Evaluation of the end result

Pros

That was definitely a good adventure and the whole thing took only 2.5 hours, writing this article took more time. What are the achievements? With maven now no need for the jar files in the git repo. Maven has a lot of good features.

Cons

Maven fixes the boiling plate problems, as I mentioned convention over configuration. As long you follow the defaults everything is simple. Issue comes when you don’t follow it. Compared to the ant build.xml the maven pom.xml got a bit more complex. Maven is definitely not a silver bullet but it has nice solutions. Gradle was that came afterwards, it can use maven dependencies and it has script features.

Fun part will begin now, as the project is more flexible and prepared for test. Later for it will be easy to do update for continuous testing and for more maven features.

The end result patch for conversion to maven.
UTF 8 issue fix

UPDATE: My patch has been released.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.