TRICKS

This file lists subtle things that might not be commented as 
well as they should be in the source code and that might be 
worth pointing out in a longer explanation or in class.

---

In pushcli, must cli() no matter what.  It is not safe to do

  if(cpus[cpu()].ncli == 0)
    cli();
  cpus[cpu()].ncli++;

because if interrupts are off then we might call cpu(), get
rescheduled to a different cpu, look at cpus[oldcpu].ncli,
and wrongly decide not to disable interrupts on the new cpu.

Instead do 

  cli();
  cpus[cpu()].ncli++;

always.

---

There is a (harmless) race in pushcli, which does

	eflags = readeflags();
	cli();
	if(c->ncli++ == 0)
		c->intena = eflags & FL_IF;

Consider a bottom-level pushcli.  
If interrupts are disabled already, then the right thing
happens: read_eflags finds that FL_IF is not set, and 
intena = 0.  If interrupts are enabled, then it is less 
clear that the right thing happens: the readeflags can 
execute, then the process can get preempted and rescheduled 
on another cpu, and then once it starts running, perhaps 
with interrupts disabled (can happen since the scheduler
only enables interrupts once per scheduling loop, not every 
time it schedules a process), it will incorrectly record 
that interrupts *were* enabled. This doesn't matter, 
because if it was safe to be running with interrupts 
enabled before the context switch, it is still safe (and 
arguably more correct) to run with them enabled after the 
context switch too.

In fact it would be safe if scheduler always set
	c->intena = 1;
before calling swtch, and perhaps it should.

---

The x86's processor-ordering memory model matches spin 
locks well, so no explicit memory synchronization 
instructions are required in acquire and release.  

Consider two sequences of code on different CPUs:

CPU0
A;
release(lk);

and

CPU1
acquire(lk);
B;

We want to make sure that:
  - all reads in B see the effects of writes in A.
  - all reads in A do *not* see the effects of writes in B.
 
The x86 guarantees that writes in A will go out to memory 
before the write of lk->locked = 0 in release(lk).  It 
further guarantees that CPU1 will observe CPU0's write of 
lk->locked = 0 only after observing the earlier writes by 
CPU0. So any reads in B are guaranteed to observe the
effects of writes in A.

According to the Intel manual behavior spec, the second 
condition requires a serialization instruction in release, 
to avoid reads in A happening after giving up lk.  No Intel 
SMP processor in existence actually moves reads down after 
writes, but the language in the spec allows it.  There is no 
telling whether future processors will need it.

---

The code in fork needs to read np->pid before setting 
np->state to RUNNABLE.  The following is not a correct 
way to do this:

	int
	fork(void)
	{
	  ...
	  np->state = RUNNABLE;
	  return np->pid; // oops
	}

After setting np->state to RUNNABLE, some other CPU might 
run the process, it might exit, and then it might get reused 
for a different process (with a new pid), all before the 
return statement.  So it's not safe to just "return np->pid". 
Even saving a copy of np->pid before setting np->state isn't 
safe, since the compiler is allowed to re-order statements.

The real code saves a copy of np->pid, then acquires a lock
around the write to np->state. The acquire() prevents the
compiler from re-ordering.