Frank's Technotes

I think I need code like this to deal with a 64-bit counter I have that is incremented at interrupt time and read concurrently (potentially) by other tasks or processors. However some of my co-workers see a statement by Intel that aligned 64-bit reads and writes are atomic and think there is no reason for the special code, EVEN though the compiler we are using emits 2 32-bit (back-to-back) MOVs to do the loads and stores. Opinion??

Where is that statement by Intel? As far as I know, this is the only way to do atomic 64-bit ops on 32-bit Intel x86 architecture. That is in fact what the instruction was designed for.

If you're dealing with a 64-bit processor, I have to say that I don't know; I haven't yet had the opportunity to deal with that. I would guess, though, that there are more 64-bit atomic ops available to you in that case.

We are looking at volume 3 of the IA-32 manual, section 7.1.1. Some of the confusion seems to have stemmed from the old PLOCK logic in the 486 that apparently automatically locked 2 bus cycles in some situations, but apparently that logic is gone. Also there was speculation that the CPU would see multiple memory ops that were "close" together in the instruction stream and combine them into the same bus cycle where possible.

Anyway I agree with you.

At first, I thought the read code you show didn't need the lock prefix (since the 64-bit memory read should be atomic). But then I see that the instruction is going to issue a write, even when it is writing the same value it read.

I have a sneaking suspicion that the confusion stems from Intel's use of the term "word" when they say, in that section, that reading or writing a byte, word aligned on a 16-bit boundary or doubleword aligned on a 32-bit boundary is guaranteed to be atomic. Throughout the three volumes, they use "word" to imply 16 bits, what to folks in the 32-bit world is a "short" or halfword. So their guarantee of the atomicity of "doubleword" loads and stores is only a guarantee of the atomicity of 32-bit loads and stores.

As far as I know, according to the published documentation, the locked cmpxchg8b instruction is the only way to guarantee atomicity of a 64-bit operation on any ia32 Intel processor.

The newer versions of the Intel manual also claim atomicity of "quadword" reads and writes, as long as the item is quadword aligned.

I think this means that a movq, for example, would be atomic (without any lock prefix). cmpxchg8 I think would do atomic transfers, but is not atomic overall (without the lock prefix) because it is R-M-W.

I have people here still telling me that Linux reads a 64-bit aligned quantity (timer?) *without* any special instructions and without any locking. And these people conclude that we don't need to do anything special either (for a simple 64-bit read), even though our compiler is doing the read with 2 separate 32-bit MOVs.

I have sent an e-mail to Intel, but don't really expect to get a real answer. I have perused other Intel hardware manuals and can't find the definitive answer. I don't see a great way to test it, given that I don't have hardware diagnostic tools.

Yes, I see that in Volume 3, the System Programming Guide, on page 7-3, section 7.1.1, "Guaranteed Atomic Operations," they write

The Pentium 4, Intel Xeon, and P6 family, and Pentium processors guarantee that the following additional memory operations will always be carried out atomically:

• reading or writing a quadword aligned on a 64-bit boundary
• 16-bit accesses to uncached memory locations that fit within a 32-bit data bus

That tells me that a 64-bit aligned access will be atomic but I have to assume that that's only if it is done in one operation. This means, yes, using a movq or one of the related operations. The problem I have with this is that movq and siblings use the "XMM" or "MMX" registers; these are associated with the SSE and MMX extended instructions and are really intended for multimedia operations. I can state categorically that your coworkers are incorrect with respect to the Linux timer. From commentary in kernel/timer.c, lines 905-906 (my line numbers may differ from the vanilla distribution):

The 64-bit jiffies value is not atomic - you MUST NOT read it without sampling the sequence number in xtime_lock.

There are occasions where one can compromise on atomicity to gain speed, but that's a different topic.

Thanks for posting this. I'm trying to get it to work in gas/gcc, but bump into this translation of %0+4... "-28(%ebp)+4" is a syntax error. What is the magic to get the half of the counter directly into a register instead of referenced via the ebp reg?

movl -28(%ebp), %eax
movl -28(%ebp)+4, %edx
movl -32(%ebp), %ebx
xorl %ecx, %ecx
1: addl %eax, %ebx
adcl %edx, %ecx
lock; cmpxchg8b -28(%ebp)
jnz 1b

This is where you want to use the ll_low()/ll_high() incantation as in the first example. The second example expects a static rather than a stack-relative address; for the latter you want to turn the second movl into

	movl %1,%%edx

(bumping the later parameters accordingly). Then for the target use ll_low(target)/ll_high(target). It will produce (in your example)

	movl -28(%ebp), %eax
	movl -24(%ebp),%edx

The ll_low()/ll_high() macros are defined in asm/system.h.

In the atomic add, doesn't ebx:ecx need to be reloaded if the exchange fails? Otherwise eax:edx will be added to ebx:ecx again.

__asm__ __volatile__(
" movl %0, %%eax\n"
" movl %0+4, %%edx\n"
"1: movl %1, %%ebx\n"
" xorl %%ecx, %%ecx\n"
" addl %%eax, %%ebx\n"
" adcl %%edx, %%ecx\n"
"lock; cmpxchg8b %0\n"
" jnz 1b"
: "+o" (target)
: "m" (incr)
: "memory", "eax", "ebx", "ecx", "edx", "cc");