1 Pstream封装了MPI，采用mpicxx的编译选项生成了mpi/libPstream.so

同时有串行版的Pstream，生成dummy/libPstream.so

2 OpenFoam库，调用Pstream，并链接了dummy/libPstream.so，生成了libOpenFoam.so

3 基于OpenFoam库的并行程序，链接libOpenFoam.so

事实上，第2、3步不需要制定MPI链接选项（-lmpich之类），我做了一个例子来尝试

代码aa.C定义一个函数，

#include <cstdio>

#include "mpi.h"

using namespace std;

void func(int argc,char **argv)

{

int rank ,size;

MPI::Init(argc,argv);

size=MPI::COMM_WORLD.Get_size();

rank=MPI::COMM_WORLD.Get_rank();

printf("Hello world! , I am %d,in %d/n computer", rank, size);

MPI::Finalize();

}

mpicxx -c aa.C

mpicxx aa.o -shared -o libaa.so

这样将其编译为一个动态库

而后代码tt.C定义主函数

void func(int argc, char** argv);

int main(int argc,char **argv)

{
  func(argc,argv);
}

编译

icpc tt.C -L./ -laa -o tt

且定义libaa.so的搜索路径

而后mpirun即可

但是还是有一个疑问：为何libOpenFoam.so要链接dummy/libPstream.so。也许在执行的时候换上mpi/libPstream.so就可以实现并行？

OpenFOAM默认使用gcc和openmpi，但是天河上是Intel编译器和特制的mpich。

按照OpenFOAM源代码编译说明，设置好环境变量后，需要做如下修改

在$OPENFOAM/OpenFOAM-2.1.0/etc/bashrc中设置

66行： export WM_COMPILER=Icc

84行：export WM_MPLIB=MPI

关键是对MPI的具体设置，在$OPENFOAM/OpenFOAM-2.1.0/etc/config/settings.sh中找到585行左右有关对MPI的设置

486     export FOAM_MPI=mpi
487     export MPI_ARCH_PATH=/usr/local/mpi
491     export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/mpi

然后是针对本地的mpi添加一个编译选项文件，路径是 wmake/rules/General/linux64Icc/mplibMPI，内容为

PFLAGS     = -fPIC
PINC       = -I/usr/local/mpi/include
PLIBS      = -L/usr/local/glex/lib -L/usr/local/mpi/lib -lmpichcxx -lmpich -lopa -lpmi  -lglex -lpthread -lrt -luuid -lrt

于是就ok了。只要执行 source ~/.bashrc后，环境变量就更新了，可以进行wmake了

最后在编译 src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C时会出现一点错误，只要将extern "C"{}里面的 #include "mpi.h"移到外面来就可以了。

OpenFoam十分巨大，编译要半天左右，因此可以利用其自带的并行编译功能来加速。详细的参考网站上有关

wmakeScheduler for faster compilation 的说明就行了。我是设置了8个进程来加速，但是往往用不上，不能指望8倍加速。

指针可传给c语言做void*

program test
  implicit none
  integer,pointer,dimension(:,:) :: dat, ptr
  integer, target :: buff(10,10)
  integer(8) :: n,l,leng,buffsize

  leng = 100
  buffsize = leng*sizeof(buff(1,1))
  allocate(dat(1:10,1:10))
  leng = 1
  do n=1,10
  do l=1,10
   dat(l,n) = leng
   leng = leng + 1
  enddo
  enddo
  leng = 100
  ptr => dat 
  call cwrite(ptr, buffsize)
  ptr => buff
  call cread(ptr,buffsize)

  do n=1,10
   write(*,'(10I5)') buff(:,n)
  enddo


  deallocate(dat)
end

#include <stdio.h>

void cwrite_(void * buff, long int* len)
{
  long int i;
  FILE * fp;

  fp = fopen("test.dat","wb");
  fwrite(buff,1,*len,fp);
  fclose(fp);
}

void cread_(void * buff, long int* len)
{
  long int i;
  FILE * fp;

  fp = fopen("test.dat","rb");
  fread(buff,1,*len,fp);
  fclose(fp);
}

真他妈的晕

function getptr(...)
  type(XXX),pointer,dimension(:) :: getptr

  call fetchptr(... getptr)
end function

...

mydata => getptr(..)
...

事实证明getptr指向的XXX类型对象会在getptr函数返回前被finalize。如果采用subroutine形式实现getptr就不会。

真是坑爹啊

When finalization occurs

1 Finalization occurs for the target of a pointer when the pointer is deallocated. If an object is allocated through pointer allocation and later becomes unreachable because all pointers to that object have had their pointer association status changed, finalization on the object does not occur.

指针被释放时，指向目标的final过程被调用。如果被动态开辟的对象失去所有指针引用，则final过程无法被调用。

2 Finalization of an allocatable entity occurs with the entity is deallocated.

allocatable的对象被释放时，对象的final过程被调用。

3 Finalization for a nonpointer, nonallocatable object that is not a dummy argument or function result occurs immediately, before the object is undefined by the execution of a RETURN or END statement. If the object is defined in a module and no active procedures are still referencing the module, finalization does not take place.

非指针，非allocatable，不是哑元，不是返回值，这样的对象在所在域的RETURN or END时会被自动fanalize。module中的鼓励对象，无法finalize。

4 Finalization of a structure constructor referenced by an executable construct occurs after execution of the innermost executable construct containing the reference.

5 Finalization for a function referenced by an executable construct takes place after execution of the innermost executable construct containing the reference.

6 Finalization for the result of a function referenced by a specification expression in a scoping unit takes place before the first statement in the scoping unit executes.

7 Finalization of a nonpointer, nonallocatable object that is an actual argument associated with an INTENT(OUT) dummy argument occurs when a procedure using the argument is invoked.

8 Finalization of a variable in an intrinsic assignment statement takes place after evaluation of the expression and before the definition of the variable.

Non-finalized entities

If program execution is terminated, either by an error, such as an allocation failure, or by the execution of a STOP or END PROGRAM statement, entities existing immediately prior to termination are not finalized.

主程序结束前，还未释放的对象无法自动finalize

A nonpointer, nonallocatable object that has the SAVE attribute or that you specify in the main program is never finalized as a direct consequence of the execution of a RETURN or END statement