Inter-Core-Micro-Benchmark

It recently came to my attention, while trying to optimize performance of some linux code, that setting CPU affinity with "taskset" was helpful.

But one sets affinity to a processor by CPU number and that made me wonder:

  1. Are the CPU numbers assigned consistently?
  2. Could I write a program which reliably measures an inter-core performance difference?
I don't yet know the answer to #1 but I was able to write a "microbenchmark" which clearly shows that on a quad-core Intel Xeon 5355 system, cores 0,1 share an L2 cache and cores 2,3 share another L2 cache but communication between between cores 0,2 (for example) is much slower:

The microbenchmark consists of two processes connected bi-directionally by pipes where the first writes one byte to the second and the second writes it back to the first. If these processes are assigned to different CPUs (or cores) there are clearly significant (and significantly differing) overheads involved. Probably for many real workloads, the differences are less significant.


Corey Satten
Email -- corey @ u.washington.edu
Web -- http://staff.washington.edu/co rey/
Date -- Mon May 21 13:29:16 PDT 2007