Writing HPC code with programmability in mind

The research led by SUN HPCS group tried to find a way out in programming languages to address the software engineering issue in HPC software.

As illustrated by the research, HPC software had seen great reduction (tenfold or so) in code size as well as improvement in readability, verifiability and maintainability, when they get rewritten with no restrictions on compiler, architecture or even programming methodology.

What is interesting, however, is that the software engineering issues in HPC software eventually arise from three non software engineering reasons.

The first problem seems to be related to performance. Techniques such as loop unrolling, cache blocking, vectorization and so on makes the code more complex. Replacing them with intrinsic structure as array operations or external libraries can help. The compiler can also help with these low level optimizations such as loop unrolling and vectorization.

The second problem is parallelization. Finding and proper handling of concurrency can always add to the complexity. To distributing data in a parallel (both shared-memory and distributed-memory) system makes the problem even worse. Hardware improvements would help in making things easier. In the software parts, I think it is possible to looking into the original physical/mathematical problem for easier identification of possible concurrency and hierarchy of parallelization.

There exists a third reason, which has something to do with complex algorithms, usually introduced by the sometimes irrational requirements of performance. Sometimes, physicists, mathematicians and engineers write a lot of complex code only to extract the one or two drops of performance out of an unimportant section of the algorithm. It is even more interesting that when the algorithms evolve, very complex terms in these formulas sometimes canceled. For this type, those languages which can express mathematical structure more easily/clearly may help.

In familiar software engineering, it is believed that the lifetime of a code is much longer that you would have thought. However, sometimes in HPC code/software, it is found that the lifetime of code is much shorter that your expectation. Eventually I have seen a handful of these codes. These codes are often small projects of Ph.D students, where they are written for some validation purposes of algorithms or demonstrations. When someone else comes to continue the project, the code is so difficult to read and understand that the poor gay has to rewrite from scratch. If these codes were written with programmability in mind, they would live much longer and save lots of human efforts.

主旨： The Ideal HPC Programming Language | July 2010 | Communications of the ACM （查看 Google 边栏评注）

Future of parallel programmings

John Shalf says that to help migrating from traditional serial CPU to multi/many-core architectures, we need more support in the programming languages. Traditional languages add more restrictions and the additive way such as OpenMP would be help but can not be fundamentally helpful.

The most important thing in parallel programming languages is that it exposes a good way to 'locality-of-effect'. From this point of view, the functional programming languages can help. So the future of parallel programming languages can be some of functional languages.

"Implicit parallelism and constructs derived from functional languages are likely to see a resurgence", John says.

I agree with the idea that parallelization depends on the internal structure of data flows in the solution process. A good way to help parallelization is to expose as much as possible the internal data flows of an algorithm. Of course this will require a different thinking of the algorithm itself, since the algorithms make use of the data of the original problem.

主旨： Feature - John Shalf talks parallel programming languages （查看 Google 边栏评注）

游戏

我们自出生以来，就在不经意中卷入一种或又一种的游戏中。小的时候，与小伙伴们过家家；稍长大一点，开始有了电子游戏，街机，红白机，以及后来的学习机， 玩得不亦乐乎；再长大一点，开始玩篮球足球；然后再长在成年，又玩起了一种名为“恋爱”的游戏；走出校园，投身社会，生存与生活的游戏... 似乎一切都可以称作游戏，你看，就连2008年红遍神州并牵动寰宇的奥运会，其原意也是游戏(Olympic Games)。

游戏是如此地深入我们的生活，简直是无处不在，让我们不得不关注它。我们在玩各种游戏，同时又在观看他人的游戏。游戏充满着我们的生活，而从一个宽泛的角度来说，也正是游戏，构成了我们的生活。我们在其中接触我们所来到的这个世界，也在其中形成我们的阅历和见解。

我们玩游戏(Play games)，已经不简单是娱乐，有时甚至已经成了个人人格的表演。我们把我们对周围世界的理解运用到游戏中去，在游戏中情不自禁地表达自己的喜怒哀乐， 然后又毫不掩饰地把自己的性情在游戏中流露。你看，我们是如此地认真来玩游戏，甚至连我们自己都还不一定察觉。所以心理学家们最喜欢让我们去做一个一个诡 异的游戏，以此作手段来研究人类的心理，而且乐此不疲。

有人把游戏当真成生活，于是他们对每一次胜败，对每一个结果都全力以赴，但有时候他们过于认真，反倒让其它的游戏者丧失与他们同玩的兴趣。有人反过来把生 活当成游戏，他们不关注任何一个结果，不计较生活中的得得失失，但却很容易在一个不算大的困难面前嘎然止步，丧失勇气与动力。

把游戏当成生活，较真到底，显得过于孩子气，而把生活作为游戏，无为而不作，显得过于消极。不认真地玩游戏，会被人看不起，而认真过了头，也让人生厌。游 戏，玩的是规则，拼的是技术，如此而已。让大家在一定的规则下充分发挥自己的聪明才智，并以胜负奖惩作为刺激，来达到娱乐身心或强身健体的目的，这就是游 戏。过于追求胜负，甚至在游戏中不惜触犯规则，是一定要被清除出游戏队伍的，奥运兴奋剂就是一个例子。而完全不在乎胜负，不去用心学习与思考，而只是仅仅 作为一种参与，又完全得不到游戏的乐趣。

你怎样玩游戏?

重新启用 Google Blog

很久没有使用 Google Blog 了，主要是因为在国内常常不能访问。现在访问似乎正常了很多，所以启用之，观望一段时间，等稳定后就把百度空间上的 Blog 搬家过来，也玩玩“博客搬家”，嘿嘿。

下面是国庆在天安门城楼上照的，故宫。