JVMTI Sucks.

You know your technology is poorly documented and rarely used when my own blog appears on the 1st or 2nd page of google results for the majority of searches I’ve done related to your technology.

New host.

I’ve got a new host for my webserver so until further notice, links to downloads are down. I will update them later. I’ll also be moving this blog to be hosted from that server.

JVMTI Agent.

Writing a JVMTI agent is really easy.

I’ll describe the process for darkfall.

1. You need to get a dll injected into darkfall.exe prior to when it loads sfmiddleware.dll and before it calls JNI_CreateJavaVM. In order to accomplish this, I injected a dll into Lobby.exe that hooks CreateProcess and I modify the creation flags so that darkfall.exe is started in a suspended state, from there I inject my dll into darkfall.exe.

2. Once my dll is injected into darkfall.exe, I hook 2 calls. The call to JNI_CreateJavaVM that is exported from jvm.dll and _Java_com_aventurine_loader_NativeClassLoader_loadClassBytes@12 from sfmiddleware.dll just so I can have darkfall decrypt the class bytes for me. I wrote that hook before the JVMTI agent and could probably do away with it now as you will see later.

3. My JNI_CreateJavaVM hook merely adds 4 options to the JVM startup paramenters. My code looks like so:

	jint JNICALL my_jni_create_java_vm(JavaVM **pvm, void **penv, void *args)
	{
		JavaVMInitArgs *vm_args = reinterpret_cast(args);
		std::vector vm_options;
		for(jint i = 0; i nOptions; ++i)
		{
			vm_options.push_back(vm_args->options[i]);
		}

		JavaVMOption jvmti_agent_option_agent_path;
		JavaVMOption jvmti_agent_option_abort;
		JavaVMOption jvmti_agent_option_exit;
		JavaVMOption jvmti_agent_option_vfprintf;

		std::string option_string("-agentpath:" + get_path_to_dll(dll_module_handle));
		jvmti_agent_option_agent_path.optionString = const_cast(option_string.c_str());
		vm_options.push_back(jvmti_agent_option_agent_path);

		jvmti_agent_option_abort.optionString = "abort";
		jvmti_agent_option_exit.optionString = "exit";
		jvmti_agent_option_vfprintf.optionString = "vfprintf";

		jvmti_agent_option_abort.extraInfo = reinterpret_cast(my_jvm_abort);
		jvmti_agent_option_exit.extraInfo = reinterpret_cast(my_jvm_exit);
		jvmti_agent_option_vfprintf.extraInfo = reinterpret_cast(my_jvm_vfprintf);

		vm_options.push_back(jvmti_agent_option_abort);
		vm_options.push_back(jvmti_agent_option_exit);
		vm_options.push_back(jvmti_agent_option_vfprintf);

		vm_args->nOptions += 4;
		vm_args->options = &vm_options[0];

		return jni_create_java_vm_hook.OriginalFunc(3, pvm, penv, vm_args);

	}

Basically all it does is give the JVM the path to my JVMTI agent and also some callbacks the JVM can use for certain events like exit and abort, these are not needed but I added them in case I needed them for some debugging.

4. Your JVMTI agent needs an exported function named Agent_OnLoad. Here is where you will initialize your JVMTI agent and fill out some bookkeeping to tell the JVM exactly which events you are interested in, and for those events, which handlers you want called.

This is what mine looks like. Note that I have borrowed some initialization code from some examples on the web.

JNIEXPORT jint JNICALL Agent_OnLoad(JavaVM *jvm, char *options, void *reserved)
{
	jvmtiError error;
	jint res;
	jvmtiEnv *jvmti = NULL;

	/* Setup initial global agent data area
	*   Use of static/extern data should be handled carefully here.
	*   We need to make sure that we are able to cleanup after ourselves
	*     so anything allocated in this library needs to be freed in
	*     the Agent_OnUnload() function.
	*/

	/*  We need to first get the jvmtiEnv* or JVMTI environment */

	res = jvm->GetEnv((void **) &jvmti, JVMTI_VERSION_1_0);

	if (res != JNI_OK || jvmti == NULL)
	{
		/* This means that the VM was unable to obtain this version of the
		*   JVMTI interface, this is a fatal error.
		*/
		printf("ERROR: Unable to access JVMTI Version 1 (0x%x),"
			" is your J2SE a 1.5 or newer version?"
			" JNIEnv's GetEnv() returned %d\n",
			JVMTI_VERSION_1, res);
		return JNI_OK;

	}

	/* Here we save the jvmtiEnv* for Agent_OnUnload(). */
	jvmti_agent = boost::shared_ptr(new Darkfall::JVM::JVMTIAgent(jvm, jvmti));

	jvmtiCapabilities jvmti_supported_interpreter_capabilities =
	{
		0, // can_tag_objects
		1, // can_generate_field_modification_events
		1, // can_generate_field_access_events
		1, // can_get_bytecodes
		0, // can_get_synthetic_attribute
		0, // can_get_owned_monitor_info
		0, // can_get_current_contended_monitor
		0, // can_get_monitor_info
		0, // can_pop_frame
		1, // can_redefine_classes
		0, // can_signal_thread
		1, // can_get_source_file_name
		1, // can_get_line_numbers
		1, // can_get_source_debug_extension
		1, // can_access_local_variables
		0, // can_maintain_original_method_order
		1, // can_generate_single_step_events
		1, // can_generate_exception_events
		1, // can_generate_frame_pop_events
		1, // can_generate_breakpoint_events
		1, // can_suspend
		1, // can_redefine_any_class
		0, // can_get_current_thread_cpu_time
		0, // can_get_thread_cpu_time
		1, // can_generate_method_entry_events
		1, // can_generate_method_exit_events
		1, // can_generate_all_class_hook_events
		1, // can_generate_compiled_method_load_events
		0, // can_generate_monitor_events
		0, // can_generate_vm_object_alloc_events
		1, // can_generate_native_method_bind_events
		0, // can_generate_garbage_collection_events
		0  // can_generate_object_free_events
	};

	error = jvmti->AddCapabilities(&jvmti_supported_interpreter_capabilities);
	check_jvmti_error(jvmti, error, "Unable to get necessary JVMTI capabilities.");

	jvmtiEventCallbacks callbacks = {NULL};
	memset(&callbacks, 0, sizeof(callbacks));
	callbacks.ClassLoad               = reinterpret_cast(jvmti_callback_class_load);
	callbacks.ClassPrepare            = reinterpret_cast(jvmti_callback_class_prepare);
	callbacks.ClassFileLoadHook       = reinterpret_cast(jvmti_callback_class_file_load_hook);
	callbacks.FieldAccess             = reinterpret_cast(jvmti_callback_field_access);
	callbacks.FieldModification       = reinterpret_cast(jvmti_callback_field_modification);
	callbacks.MethodEntry             = reinterpret_cast(jvmti_callback_method_entry);
	callbacks.MethodExit              = reinterpret_cast(jvmti_callback_method_exit);
	callbacks.NativeMethodBind        = reinterpret_cast(jvmti_callback_native_method_bind);
	callbacks.VMDeath                 = reinterpret_cast(jvmti_callback_vm_death);
	callbacks.VMInit                  = reinterpret_cast(jvmti_callback_vm_init);
	callbacks.VMStart                 = reinterpret_cast(jvmti_callback_vm_start);

	//cb.DynamicCodeGenerated    = reinterpret_cast(jvmti_callback_dynamic_code_generated);
	//cb.SingleStep              = jvmti_callback_single_step;
	//cb.Breakpoint              = jvmti_callback_breakpoint;
	//cb.FramePop                = jvmti_callback_frame_pop;
	//cb.Exception               = jvmti_callback_exception;
	//cb.ExceptionCatch          = jvmti_callback_exception_catch;
	//cb.ThreadStart             = jvmti_callback_thread_start;
	//cb.ThreadEnd               = jvmti_callback_thread_end;
	//cb.CompiledMethodLoad      = jvmti_callback_compiled_method_load;
	//cb.CompiledMethodUnload    = jvmti_callback_compiled_method_unload;
	//cb.DataDumpRequest         = jvmti_callback_data_dump_request;
	//cb.MonitorContendedEnter   = jvmti_callback_monitor_contended_enter;
	//cb.MonitorContendedEntered = jvmti_callback_monitor_contended_entered;
	//cb.MonitorWait             = jvmti_callback_monitor_wait;
	//cb.MonitorWaited           = jvmti_callback_monitor_waited;
	//cb.VMObjectAlloc           = jvmti_callback_vm_object_alloc;
	//cb.ObjectFree              = jvmti_callback_object_free;
	//cb.GarbageCollectionStart  = jvmti_callback_garbage_collection_start;
	//cb.GarbageCollectionFinish = jvmti_callback_garbage_collection_finish;

	error = jvmti->SetEventCallbacks(&callbacks, (jint)sizeof(callbacks));
	check_jvmti_error(jvmti, error, "Cannot set jvmti callbacks");

	/* At first the only initial events we are interested in are VM
	*   initialization, VM death, and Class File Loads.
	*   Once the VM is initialized we will request more events.
	*/

	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_CLASS_LOAD, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_CLASS_PREPARE, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_CLASS_FILE_LOAD_HOOK, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_FIELD_ACCESS, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_FIELD_MODIFICATION, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_METHOD_ENTRY, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_METHOD_EXIT, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_NATIVE_METHOD_BIND, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_VM_DEATH, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_VM_INIT, reinterpret_cast(NULL));
	error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_VM_START, reinterpret_cast(NULL));

	//error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_DYNAMIC_CODE_GENERATED, reinterpret_cast(NULL));
	//error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_VM_OBJECT_ALLOC, reinterpret_cast(NULL));
	//error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_SINGLE_STEP, reinterpret_cast(NULL));
	//error = jvmti->SetEventNotificationMode(JVMTI_ENABLE, JVMTI_EVENT_BREAKPOINT, reinterpret_cast(NULL));

	check_jvmti_error(jvmti, error, "Cannot set event notification");

	/* Here we create a raw monitor for our use in this agent to
	*   protect critical sections of code.
	*/
	//error = jvmti->CreateRawMonitor(jvmti, "agent data", &(gdata->lock));
	//check_jvmti_error(jvmti, error, "Cannot create raw monitor");

	/* We return JNI_OK to signify success */
	return JNI_OK;

}

You should notice that I have setup callbacks for OnMethodEntry/Exit, NativeMethodBind and some others. Also notice the JVMTI_EVENT_CLASS_FILE_LOAD_HOOK event, this is what can be used to dump out the decrypted byte code for the darkfall java code. The event handler has the following signature:

void JNICALL jvmti_callback_class_file_load_hook( jvmtiEnv *jvmti_env
, JNIEnv* jni_env
, jclass class_being_redefined
, jobject loader
, const char* name
, jobject protection_domain
, jint class_data_len
, const unsigned char* class_data
, jint* new_class_data_len
, unsigned char** new_class_data
)

Notice that it even has a parameter that can be used to redefine the class. This can be handy if you want to add some custom code to a java class file, say, to call some native code of yours or even to call some java code you have written. This is how for example AspectJ is able to add advice. (They use a java implementation though).

That’s about it for this installation. I think you’ll find that this stuff is pretty easy to do. Next time I will go over how your code can get via breakpoints, and why these might come in handy. I also might give some tips on how one might go about successfully getting information for 4700 obfuscated .java files.

Darkfall haxxin

So I finally got around to writing a dumper for the custom java class loader that Darkfall uses.

Turns out they have at least 4611 classes. I say at least because apparently the JVM is optimized to only load classes that it has reason to believe will be or are being used, the implications of which mean that the more features of the game being used, the more classes will be loaded.

I have a hard time thinking that there can be many more classes simply on the basis that 4600 is quite a lot to begin with, but also that I tried to hit at least the majority of basic functionality of the game.

I’ll post some examples of what the code looks like in a bit. Most of it is obfuscated so it is full of randomized 2 and 3 letter named variables and class names.

ODBC: Function sequence error.

I recently ran into an error like this yesterday:

ERROR [HY010] [Microsoft][ODBC
SQL Server Driver]Function sequence error

Turns out, it was a simple fix. My query was essentially

use DatabaseName
select * from table

All I had to do was remove the ‘use DatabaseName’ line and everything worked well.

Darkfall skills list.

Skill list

Linker error in VS2003.NET projects.

Blackd0t had a problem such as the one described here:

Linker error

The errors are like this:

xxxxxxx error LNK2019: unresolved external symbol "public: int __thiscall CBotApp::Setup(wchar_t *,char *,char *,char *,char *,char *)" (?Setup@CBotApp@@QAEHPA_WPAD1111@Z) referenced in function "public: virtual int __thiscall CxxxxxxtApp::InitInstance(void)" (?InitInstance@CAionBotApp@@UAEHXZ)
AionBot error LNK2019: unresolved external symbol "public: wchar_t * __thiscall CTranslator::Get(wchar_t *)" (?Get@CTranslator@@QAEPA_WPA_W@Z) referenced in function "protected: virtual int __thiscall CxxxxxxtDlg::OnInitDialog(void)" (?OnInitDialog@CxxxxxxxDlg@@MAEHXZ)

After talking with him, it seemed the problem was that the MFC project was looking for a symbol:

LaunchGame@CBotApp@@QAEHPA_PAD0@Z

And his lib was producing:

LaunchGame@CBotApp@@QAEHPAGPAD0@Z <——– notice the _ is replaced with G.

I had initially thought that he might have something like this in a header:

void foo(int x);

and this in a cpp:

void foo(double x){ return;}

But it turns out that was not the case.

After a bit of deliberation I told him to try undname.exe in the VC/Bin folder of VS. Lo and behold he discovered that 1 project was treating wchar_t as an unsigned short, and the other was treating it as a builtin type!

The lesson? When your linker complains of missing symbols, and you just *know* they are not missing, make sure to check anything that might change the mangling of the names.

Sfad extractor

Here it is!

It currently supports only bz2 compression and uncompressed files.

Sfadlib

The source to this release is provided. No executables yet and not all dependant libraries are included in the archive.

Mystery compression algorithm

Anyone have a clue as to which compression/encryption method was used here?

Unknown File

It’s supposed to be an XML file.

Found this on a russian forum, posting it here!


import bz2,struct,os,stat
from cStringIO import StringIO

sfai=file('dfdata.sfai','rb')

h=struct.unpack('128L',sfai.read(0x200))
assert h[0]==0x49414653 # "SFAI"
start=h[6]
size=h[7]
print "INDEX: %08X => %08X"%(start,start+size)
sfai.seek(start)
buff=sfai.read(size)

buff='BZh9'+buff[4:]
idx=StringIO(bz2.decompress(buff))

x=True
csizes={}
files={}
fsizes={}
fcount={}
for x in xrange(0,18):
files[x]=file('dfdata_%04d.sfad'%x,'rb')
csizes[x]=0
fcount[x]=0
fsizes[x]=os.path.getsize('dfdata_%04d.sfad'%x)

if not os.path.isdir('out'): os.mkdir('out')
log=file('out/file.lst','w')

while x:
x=idx.read(0xBD)
if not x: break
n=x[0:x.find(chr(0))]
id,t,s1,fn,p,s2,u3,u4=struct.unpack('7Lb',x[160:189])
csizes[fn]+=s2
fcount[fn]+=1
pp='out/'
for y in n.split('/')[:-1]:
pp+=y+'/'
if not os.path.isdir(pp): os.mkdir(pp)
print >>log,"%(n)r ID:%(id)d @ %(p)08X (T%(fn)02d) Size:%(s1)d/%(s2)d %(u3)08X %(u4)02X"%locals()
files[fn].seek(p)
buff=files[fn].read(s2)
if s1s2:
if buff[:4]=='SFZM':
buff='BZh9'+buff[4:]
buff=bz2.decompress(buff)
else: print 'Warning! %r unknown compression.'%n
f=file('out/'+n,'wb')
f.write(buff)
f.close

log.close()
for x in xrange(0,18): files[x].close()

print "================ DONE =================="
for x in xrange(0,18):
print "Tome %02d - %d files, %.3f%% lost"%(x,fcount[x],100.0*(fsizes[x]-csizes[x])/fsizes[x])