[dtrace-discuss] How to speculate and commit on hang.

[Kris Raney]

> 1) You need to allocate a potentially big chunk of
> memory to hold the snapshot 
> and the allocation needs to happen in almost
> arbitrary context that the probe 
> can fire

It need not be implemented as a snapshot. You could timestamp keys, so that keys newer than the beginning of the loop can be ignored. You could queue new array items for insertion after the loop exits. You could use a data structure that helps solve this problem. Or you could make it illegal to add to the associative array you are iterating over during the iteration.

 
> 2) The number of keys can be huge and iterating over
> it will take a 
> significant amount of time when no other system
> activity can happen. This can 
> be disastrous for the system health.

> The time-based loops ensure that only a fixed amount
> of work is done in the 
> probe context and the system can do other stuff
> between probes.

It seems I haven't made my proposal clear. The example given solves the iteration in a dtrace way, using timers. Why not provide syntactic sugar allowing the user to do exactly that, and have it look like a loop? The disastrous delay wouldn't happen, because each iteration of the loop would be in a separate timer. The underlying implementation would be exactly identical to the example.

By providing the syntactic sugar way, you could also help the user with getting the loop started at the right time, and with stopping its execution when it's completed a full iteration.

Even if I grant that an iterator could cause an disastrous delay, my last question remains, which is why couldn't it be allowed, but be considered a destructive action?

I have a similar situation to the original poster - I have a third party library whose memory use grows unbounded. It's not strictly a leak, because it keeps references to the memory and cleans up at exit time. So it doesn't show up in memory tools. I would like to 'punch in' with dtrace when the program is at steady state, trigger the growth and return to steady state, then 'punch out' and commit() the frees that *didn't* happen, so I can go fix that library.

I was able to make this work somewhat using the solution above. But assuming the keys of the speculations, and iterating over the entire range on the assumption that committing an unwanted one is 'safe' feels pretty kludgey. Also, it's a nuisance getting the 'loop' to run at the right time, since I can't associate it with the END probe. The fact that it repeatedly iterates over the array even when I know logically there's no work to do yet adds an unwanted system load. 

It seems to me like something that could use improvement in the language. What I wonder is, how many people have to complain about it before you're willing to reprioritize your requirements?


--
This message posted from opensolaris.org