[dtrace-discuss] What am I missing in the dtrace script?

[Bryan Cantrill]

On Tue, Jul 10, 2007 at 03:27:46PM -0700, Brian D. Horn wrote:
> I've asked around, I tried searching on-line, but no go.
> 
> I would think what I am seeing is impossible or that there must
> be a bug in dtrace on MP systems (at least on 64-bit amd systems)
> as this was run on a Sun Fire x4500 (4-way Opteron - aka "Thumper")
> 
> Here is the script:
> 
> 
> #! /usr/sbin/dtrace -s
> 
> BEGIN
> {
>        total_queued = 0;
>        queuing_failed = 0;
>        queuing_succeeded = 0;
> }
> 
> ::vdev_queue_io_to_issue:return
> {
>        ++total_queued;
> }
> 
> ::vdev_queue_io_to_issue:return
> / args[1] == 0 /
> {
>        ++queuing_failed;
> }
> 
> ::vdev_queue_io_to_issue:return
> / args[1] != 0 /
> {
>        ++queuing_succeeded;
> }
> 
> END
> {
>        printf("\nTotal %d succeeded %d failed %d\n",
>                total_queued, queuing_succeeded, queuing_failed);
> }
> 
> And below is the output:
> 
> ^C
> CPU     ID                    FUNCTION:NAME
>  3      2                             :END
> Total 970818 succeeded 485824 failed 485584
> 
> Note that:
> 
> # bc
> 485824+485584
> 971408
> 
> So the sum of the two cases with qualifiers do not sum
> to the the total.  I've seen this before and running the script
> with all but one CPU off line seems to "fix" the problem.

When you're using global variables, they are modified without any locking
whatsoever -- so those increments are racing with one another on different
CPUs.  To achieve what you're trying to achieve, use aggregations, and
then printa() to process them:

::vdev_queue_io_to_issue:return
{
	@total_queued = count();
}

::vdev_queue_io_to_issue:return
/args[1] == 0/
{
	@queuing_failed = count();
}

::vdev_queue_io_to_issue:return
/args[1] != 0/
{
	@queuing_succeeded = count();
}

END
{
	printa("\nTotal %@d succeeded %@d failed %@d\n",
	    @total_queued, @queuing_succeeded, @queuing_failed);
}

	- Bryan

--------------------------------------------------------------------------
Bryan Cantrill, Solaris Kernel Development.       http://blogs.sun.com/bmc