Using Marmalade with Google Play Expansion Files, JNI, and LVL License Checks

Purpose

Google Play requires all core APK application files to be 50mb or less.  If your program is larger, you must split it into expansion file(s) delivered separately.  Google Play hosts your expansion file(s) and may automatically transmit them to your customer along with the main APK, but they may not.  If not you can host the expansion file yourself and download from your server, but that may cost additional money and have bandwidth issues depending on your hosting provider.  If you want to download from Google Play, you must make a licensing check to retrieve the download URL.  Marmalade does not currently support this, therefore you must step outside Marmalade and make native Java calls from within C++.  If this sounds complicated, it is.  Especially since there is no single source of documentation from which to figure out how to do this and you cannot use the debugger.  This article attempts to set the record straight, or at least permanently wrong, on how to do this.

Intro

I am a marmalade C++ developer and we wanted to release our iPhone game Catch the Monkey to Android.  The iPhone IPA build is 29mb, but the identical build for Android is a 78mb APK.  This is because IPA files are compressed ZIPs and APKs are not (shame on you Google).  Google Play limits APK file size to 50mb so I had to figure out how to use expansion files.  This is relatively straightforward from a Java Android application written in Eclipse.  It is much more convoluted going from C++ to Java to Google Play back to Java back to C++.  It took me just over a week to make this rock solid.

Two thousand years ago a guy said we should treat people the way we want to be treated.  He was promptly killed for saying outlandish things like this, but I try to follow his principles anyway.  I wish someone would have documented how to do this, so here it is so no one else has to go through the week of hell I just had.

Overview of the Solution

Here is the big picture of what you need to achieve:

1. From C++, call a Java Class passing in your publisher public Key
2. Have the Java Class call Google Play through the Expansion Licensing Verification Library.  This is an asynchronous call.
3. Wait around for the result from Google, which could be success, fail, timeout, or a cached response.
4. Callback your C++ code from Java with the result, passing the URL string if there is one
5. With a successful result, perform an HTTP download of the file from the Google URL
6. Handle N number of redirects from Google’s servers
7. Save your expansion file in the properly designated location
8. Find the expansion file and use it in your application (whether it came from Google automatically or from you)

The focus of this article is on the Google Play JNI C++ communication.  HTTP downloading, putting it in the right location, mounting it, verifying the file, etc I plan to cover in a separate article.

Also I’m not writing this in tutorial style, where I tell you every click and step to take.  You can find tutorials on the web on how to do common tasks like create an eclipse project, etc.  My intent is to write what is not readily available and had to be learned through experimentation.

I will be posting relevant snippets from the source code, making the full source available for download.

Requirements

I wrote this solution with Marmalade 5.2 in Visual Studio 2008 C++.

Eclipse 4.2, Android plug in, Java SDK 6 (not 7!)

Dalvic debug monitor (DDMS)

Before you embark on this, make sure you can do these two things:

1) Create an android application in eclipse, and deploy and run it on an actual android device

2) Run and deploy the example LVL program found in <Marmalade><version>examplesAndroidJNIs3eAndroidLVL on an actual android device

You can test using the simulator, but I found it much much slower than deploying and testing from an actual device.

Step 1: Create a Java Class to talk to Google Play

Source Step1-PureJava.zip

Obviously the core is to talk to Google Play and get the results we want.  Once we have that, we can plug it into our marmalade project.  You will be creating an Activity class which you can reuse later.  This is a modified version of the Android LVL example documented here:

http://developer.android.com/guide/google/play/licensing/adding-licensing.html

The reason for making a solution that works in pure android Java is for several reasons:
1) To address compile errors in the java code immediately. A random .java file in eclipse does not show syntax/compile errors. It must be part of a project, and only when you save the file does it do a compile/syntax check. I wrote my activity in a java project so I could have this assistance because compiling with marmalade at the command line will give you the compile errors, but the development cycle between error, correction, re-test is much much longer.

2) You need to know your code that communicates with google play is working perfectly before bothering to plug it into a marmalade solution. I would never have known Play caches results and gives only a partial response if I hadn’t of figured it out in android first. If I had of skipped this, I would have assumed something was wrong with my JNI integration.

3) The ability to debug line by line through the java project allows you to see what exactly is going on. Once you put the java into the marmalade project you loose the line-by-line debugging capability. You can’t even step through the C++ code as it has to be executed on a device.

When creating your own android project, you will be creating a MainActivity and import the Google Play Licensing Libraries and Google Play APK Expansion Library into your project (listed under Extras in the SDK manager).  Basically this project is the Google Google documents how to do this, but since they wrote it the libraries have changed name from Google Market to Google Play, be aware.

Key things in the code:

        // Generate your own 20 random bytes, and put them here.
        private static final byte[] SALT = new byte[] {
            -46, 65, 13, -128, -103, -57, 74, -64, 51, 88, -95, -45, 77, -117, -36, -113, -11, 32, -64, 89
        };

This randomizes the call to Google Play. One of the annoyances is Android will cache the result of a licensing check locally. While this is great for the user who doesn’t require a network connection each time they run your game, it sucks for our purposes as the cached result does not include the Expansion file information (but will still return success).  I wasted much time after I got the initial expansion info to work and then it suddenly kept returning nulls every time thereafter.  This is due to cached result.  To force a new call to Google Play, change any of the salt bytes, and a round trip call will be made, and you will receive the expansion file info again.

mLicenseCheckerCallback = new MyLicenseCheckerCallback();
policy = new APKExpansionPolicy(getApplicationContext(), new AESObfuscator(SALT, "com.doesnt.matter", deviceId));
// Construct the LicenseChecker with a policy.
mChecker = new LicenseChecker(this, policy,base64key);
mChecker.checkAccess(mLicenseCheckerCallback);

Here create our policy check, register our callback for when it completes, and then start it.

Notice the “package name” passed into AESObfuscator doesn’t matter.  Google documentation says to pass in your packagename, but it actually ignores it and pulls the name from the actual compiled package.  THIS IS VERY IMPORTANT TO KNOW WHEN TESTING.  You must test with the correct package name registered with google play with the correct version number or you will get constant failures, or even worse, “package not managed by Google Play”.  So make sure your package name, manifest package name, and google play APK package name all line up and the version numbers all line up.

Notice we are not making a standard policy check, we are making an APKExpansionPolicy.  This is how we can access the additional values we care about.

private class MyLicenseCheckerCallback implements LicenseCheckerCallback {
     public void allow(int reason) {
     if (isFinishing()) {
       return;
     }

     // Should allow user access.
     String url = policy.getExpansionURL(APKExpansionPolicy.MAIN_FILE_URL_INDEX);
     String expansionFileName = policy.getExpansionFileName( APKExpansionPolicy.MAIN_FILE_URL_INDEX );
     long expansionFileSize = policy.getExpansionFileSize( APKExpansionPolicy.MAIN_FILE_URL_INDEX );

     if (url != null)
          Log.i("Marmalade", url);

     String returnData = url + "|" + expansionFileName + "|" + expansionFileSize;

When google play is done, the policy will either call the allow method or the dontallow method.

If allowed, we can extract the values we need: url, filename, file size.

I then take these values and concatenate them into a pipe delimited string to send back to marmalade.  I did this because I was feeling retro, pipes aren’t allowed in URLs, and i didn’t want to have to send multiple values back to C++.  One string was enough trouble thank you very much.

As you can see I log using the “Marmalade” name so I can see it in Logcat.  This is how marmalade logs for real on a device so you can see it in logcat by filtering on “Marmalade”.

Now I need to tell you you cannot just fire open my project or the example LVL project, compile, deploy and it will work.  It won’t.  This is because you have to upload your APK to google play first.  Once you upload it, leave it in the draft (unpublished) state.  This requires you to upload screenshots, enter in name and description, etc.  Very annoying but is the only way you can upload your apk.  Since you cannot make changes later (like pricing), you should not be testing with your real package name.  You will notice I use com.mirthwerx.catchthemonkey2 for testing.  So do all your dev on a test package name, and when it all works, you’ll need to change it to the real one later.

Another reason i won’t “just work” is that you have to put in your public key for the policy check.

Finally, when you test from your device (or simulator), you need to register the device google account to the one in the Play account.  You see a field here where you can add additional accounts.  If you don’t do this, you will get constant fail messages of “package not managed” because your unpublished package isn’t known to the outside world yet.

As you can see, there is plenty that can go wrong just at this stage.  Do not continue until you get this part working.  I found the google tutorial to be good and did eventually manage to get all this working on a pure java android project.

Now for the real fun…

Step 2: Create a Java class in a JAR for use in your Marmalade Project

Source: Step2-s3eAndroidLVLExpansion.zip

This next step is based on the Marmalade example s3eAndroidLVL project.  I started from that and modified as needed.  You should read the marmalade documentation and run that project to get a grip with what is going on here.  You will notice they  only passed back an int, which is super easy compared to Java strings which you require for the URL.

You will see in the example and my project there is a _java.mkb.  This is a special type of MKB that tells the framework to do a Java JAR compile from classes you specify.  For using expansion policies, you will need to copy the correct .java files into this project, the marmalade ones are outdated (well 5.2 ones are anyway).

One reason this is so hard is you have no tool help in making sure your JAR is properly programmed.  Since the Java file isn’t tied to a project, opening the java file in eclipse doesn’t even give you syntax checking.  I had to paste code from my activity to another project to make sure the syntax was correct.

public class MainActivity extends LoaderActivity {

Notice that the marmalade activity has to inherit from LoaderActivity, not the standard android Activity.  This is how it is allowed to work with the Marmalade runtime.

The only major difference between the pure android activity and this jar activity is the use of native callbacks:

//The native function that receives the result of the license check
public native void licenseResult(int result);	
public native void mainExpansionFileCallback(String expansionMain);

I left the Marmalade one as is and created a second callback, one that will pass our string.  This is how we register with the Java Runtime (or JNI) we are making some C++ calls back.  In the Allow() method I simply call both callbacks.

With your java file correct, double click the _java.mkb file and you will see a compile result.  If it doesn’t work you will see a compile error and the window will stay open.  If it does work, the window will simply close and you have a new Jar file.

Once you have the JAR file compiling correctly, you can attach it to your Marmalade project.  This is defined here in the deployment wizard:

Step 3: Calling the JAR through the Marmalade JNI interface

OK, now the real magic happens in the s3eAndroidLVLExpansion.cpp C++ file.  We need to call our Java activity, wait around, and receive the response.

And, of course, you can’t use the simulator to debug any of this because JNI only exists on android. Isn’t that awesome? Yes. Yes it is. So development involves code changes, deploy to device (again, faster than the simulator), check the logs for results, rinse and repeat.

Now, you need to know that JNI callbacks are unaware of the s3e runtime, they, apparently, exist outside of it. This means you cannot call any s3e framework methods, or expect state, or anything else. I was having a problem so I put this simple line inside the callback:

s3eDebugOutputString("I'm here");

But that caused the app to crash! Because you cannot use s3e inside a JNI callback. All you get is pure C++. So to get our data and store it in a way our s3e project and read it, we use global variables.

const char * g_Status = "Waiting...";
const char * BASE64_PUBLIC_KEY = "GIT YOUR OWN!";
int LVLcheckResult = -1;
bool pickupData = false;
static char javaText[3072];
string mainExpansionFileName;
string mainExpansionURL;
int mainExpansionSize;

g_Status is just from the example code, I left it as is.
BASE64_PUBLIC_KEY is the key from google play which you pass into your Java class for talking to play.
LVLCheckResult is the integer return value from the license check.
pickupData is a trigger I use to know there is some data to pickup. My main application thread is sitting around waiting for this to be triggered.
javaText[3072] is just a large string buffer to hold the resulting callback string from Java. This size met my purposes, there is nothing magical about 3072.
The next three variables are the parsed values for my main application to use.

void licenseResult(JNIEnv* env, jobject obj, int result)
{
	LVLcheckResult = result;
	pickupData = true;
}

When google play responds to the java activity, this callback is called. Now my program knows there is a result to process.

void mainExpansionFileCallback(JNIEnv* env, jobject obj, jstring j_text)
{
    const char *str = env->GetStringUTFChars(j_text, 0);
    if (str == NULL) { 
    	sprintf(javaText, "MEM IS OUT");    	
        return;
    }
    sprintf(javaText, "%s", str);
    env->ReleaseStringUTFChars(j_text, str);  
}

This function takes in the string from Java (which is a j_text type), then writes it to my char buffer. The last line releases the java string so the garbage collector knows it can clean it up. If you leave this off, expect memory leaks and wonkyness.

Now for ExampleInit(). This method follows the marmalade JNI example with slight modification. It takes care of registering the JNI bridge and setting up the calls we make to Java, and the calls back.

//Find the MainActivity class using the environment
    jclass MainActivity = env->FindClass("com/mirthwerx/catchmonkey2/MainActivity");

Gets a pointer to the java activity class so you can call methods on it. Obviously the class/package name has to be exactly the same as what you put in the JAR or it won’t work.

   //Find the checkLicense method
    jmethodID checkLicense = env->GetMethodID(MainActivity, "checkLicense", "(Ljava/lang/String;)V");

Gets a function pointer to this java method on the class:

  	public void checkLicense(String base64key)
    	{

//format is name of method, parameter types (note the ; for string),
// return type (V for void, I for int), pointer to function
const JNINativeMethod nativeMethods[] =
    {
         { "licenseResult", "(I)V", (void*)&licenseResult },
	 { "mainExpansionFileCallback", "(Ljava/lang/String;)V", (void*)&mainExpansionFileCallback },
    };
if (env->RegisterNatives(MainActivity, nativeMethods, sizeof(nativeMethods)/sizeof(nativeMethods[0])))
    {

Now we register our native C++ callback methods and hook them up to the ones we defined as native in Java. It took me a while to find out the correct format to define a method, and then a way to define a string. As there is no documentation in Marmalade on this format, I do not know how other types (like Double or floats) would be defined.

Notice how you can define multiple native methods, then divide the array by the size to tell the system how many you defined. Much easier than a static number you could accidentally forget to change when you add another method.

jstring jkey = env->NewStringUTF(BASE64_PUBLIC_KEY);
//Call the checkLicense method on the object
env->CallVoidMethod(m_Activity, checkLicense, jkey);

And finally we call our method on the class, passing in the single parameter of the key.

Now, in the actual “game” I added some logic to the render function which is called every frame.
The Java license check is called immediately upon start up, so we wait around for the result to be triggered:

if (pickupData)
{
        pickupData = false;
	processResults();

And that is how you use JNI with Marmalade, perform callbacks, pass strings back and forth, do a licensing check, and get expansion file information.

I hope you found this helpful.

I probably skipped over something important, so any questions ask em in the comments and I’ll try and point you in the right direction.