Sam Pointer

Introduction

I’ve spent quite some time tuning the performance of Java server applications in my last two roles. It helps that I had a great mentor (hello JJ if you are out there). For the past couple of months I’ve been increasingly involved in trouble-shooting and tuning performance problems with .NET applications. Being VM managed garbage collected runtimes both, many of the common problems and solutions I see for the JVM are also applicable to the .NET runtime. Here I present my Rosetta Stone for the JVM native looking to tune a .NET application. It isn’t a complete guide to .NET performance tuning (and I couldn’t write it if you need it - you could do a lot worse than reading this.), but it is enough to get the JVM native started for the most common subset of problems.

Garbage Collection Policy

Within most JVMs the default GC policy is "client", which keeps the memory footprint low and assumes relatively short-lived processes. It is therefore common to pass the -server switch to switch the trade-off within the garbage collector to better suit long-running process with generally larger heaps (I’ll ignore the existence of the new collectors in Java 6 for now). The .NET runtime behaves in exactly the same way, and also defaults to a "workstation" policy. This MSDN article details how to change the collector to the more agressive and threaded "server" collector.

Heap preallocation to ensure contiguous block of memory

It is a common trick to pre-allocate the JVM heap by setting the -Xmx and -Xms switches to the same value, thereby ensuring a contiguous allocation of memory and staving off any unexpected future allocation errors. This is not possible with the .NET 3.5 runtime, something I don’t understand given that arrays are allocated in contiguous blocks. I don’t think that the .NET runtime requires contiguous regions per-se, but the size and numbers of the assemblies being loaded (plus the aforementioned arrays) will require contiguous memory at some point. You should keep a careful eye on both the amount of free memory and its fragmentation.

GC and other performance counters

On the JVM we use the -verbose:gc and other related switches, or JMX, to monitor the JVM process and the important metrics, with .NET everything is most easily accessible through perfmon and the .NET counters. A little note on terminology, "pinned" objects are those that have active references and cannot be collected.

I’ve wasted quite a bit of time today due to a nasty bug manifesting itself in Aptana Studio. I’m using the 64 bit Linux standalone version.

I recently configured a new project. I installed the RadRails and Subclipse, updated my gems to Rails 2.3.2. Good to go. I checked the project out from Subversion, configured a server and continued development. The next time I started Aptana the VM dumped an hs_err_pid4272.log in my home directory..

The pertinent string is

tack: [0x00007fffb511f000,0x00007fffb531f000),  sp=0x00007fffb53191b0,  free space=2024k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libgdk-x11-2.0.so.0+0x3a4d0]  gdk_window_process_updates+0xe0

The only way I can work around this is to remove my default workspace directory and start from scratch.

This is logged as a bug here on the Aptana support forums.

Looking at the hs_err_pid4849.log also attached to the ticket, you can see that his permgen is full, as Chris Williams says.

Heap
 def new generation   total 2944K, used 2180K [0x74090000, 0x743c0000, 0x75440000)
  eden space 2624K,  71% used [0x74090000, 0x74264d58, 0x74320000)
  from space 320K,  95% used [0x74370000, 0x743bc5d8, 0x743c0000)
  to   space 320K,   0% used [0x74320000, 0x74320000, 0x74370000)
 tenured generation   total 37824K, used 17682K [0x75440000, 0x77930000, 0x84090000)
   the space 37824K,  46% used [0x75440000, 0x76584b30, 0x76584c00, 0x77930000)
 compacting perm gen  total 29440K, used 29209K [0x84090000, 0x85d50000, 0x94090000)
   the space 29440K,  99% used [0x84090000, 0x85d167a8, 0x85d16800, 0x85d50000)
    ro space 8192K,  74% used [0x94090000, 0x94688b48, 0x94688c00, 0x94890000)
    rw space 12288K,  58% used [0x94890000, 0x94fa3df0, 0x94fa3e00, 0x95490000)

I tried amending my permgen to ensure that I wasn’t suffering from the same (possible) problem. I edited my AptanaStudio.ini and upped the –launcher.XXMaxPermSize value from 256m to 512m.

Heap
 PSYoungGen      total 125760K, used 59990K [0x00007f1b7a760000, 0x00007f1b83d10000, 0x00007f1ba5200000)
  eden space 114560K, 42% used [0x00007f1b7a760000,0x00007f1b7d715b18,0x00007f1b81740000)
  from space 11200K, 99% used [0x00007f1b83220000,0x00007f1b83d00048,0x00007f1b83d10000)
  to   space 16832K, 0% used [0x00007f1b81c30000,0x00007f1b81c30000,0x00007f1b82ca0000)
 PSOldGen        total 69056K, used 27327K [0x00007f1b25200000, 0x00007f1b29570000, 0x00007f1b7a760000)
  object space 69056K, 39% used [0x00007f1b25200000,0x00007f1b26caffe0,0x00007f1b29570000)
 PSPermGen       total 75840K, used 48917K [0x00007f1b05200000, 0x00007f1b09c10000, 0x00007f1b25200000)
  object space 75840K, 64% used [0x00007f1b05200000,0x00007f1b081c5500,0x00007f1b09c10000)

As you can see, its 64% full.

I’m using the following:

Java version "1.5.0_18"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_18-b02)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_18-b02, mixed mode)


Linux sam-desktop 2.6.28-13-generic #45-Ubuntu SMP Tue Jun 30 22:12:12 UTC 2009 x86_64 GNU/Linux

I’d be very greatful to anyone who has any ideas what’s going here. Being primarily a production engineer and not a Java developer I did run strace on the thing, with -f to follow the children. Suffice to say I found myself bashing out a magic SysRq and ‘o’ combo.