[hpcdev-discuss] Data Flow Parallelism

[Keith Bierman]

On Sep 24, 2007, at 5:22 PM, Richard Smith wrote:

> I would certainly hope that Data Parallelism is a major rewhacking  
> of the
> programming model...we desperately need one. A big challenge is to  
> find ways of
> ...

Of course there's the old Halting vs. Starting problem.

In CS theory, the Halting problem is hard (insoluable ;>). But in  
practice it's easy. Remove power. Take a crowbar to the CPU and  
similar logic elements. Raddiate the DRAM, etc. Eventually, the  
system will halt (and typically quite soon after all power is  
shorted, over-voltage applied, etc.).

In computer reality, it's getting a new platform going that's really  
hard (ISA, OS, etc.) as apps go after volume, etc.



> I also think its a reasonable evolutionary step to provide an API  
> to existing
> languages that extends their capabilities, in effect constructing a  
> new Virtual

Except where the new model is so alien to the old one. Where I think  
the next big leap can most profitably occur is by having a really  
smooth *calling* relationship between the conventional languages of  
today and some new data parallel language (and god help anyone who  
wants to use Verilog and it's PLI as a model!). But if the overall  
program flow could be maintained, the folks with computationally  
intense localized (to a procedure/subroutine nest) could dabble in  
the new technology via conventional linkage ... as long as the new  
language/environment can work on the data largely in-place and  
invisibly).
> ....sed around Fortran or C than Parallel Haskell. [...even if I  
> have argued
> for years of the value of using functional programming models and  
> implicit
> parallelism, with serial programming as a special case.]

While true, it's quite limiting to things with a lot of coarse  
objects ... and with many threads per chip, I think finer grained  
approaches are apt to give us the most benefit (and, of course,  
people working on mega scale multi-petaflops are probably going to  
mix, fine grain, and MPI (very coarse) grain to completely exploit  
their machines). It will be very painful, but I don't think they have  
any choice (and some of them have started to admit it ;>).


Keith H. Bierman    keith.bierman at Sun.COM   |  khbkhb at gmail.com
Strategic Engagement Team                   | AIM: kbiermank
<speaking for myself, not Sun*> Copyright 2007