Differentiate a method that uses MPI

[alpinito1]

Dear all,

First of all, I would like to thank you for the awesome tool that you develop. I find fascinating the power and possibilities that Enzyme opens !

I would like to use Enzyme for some solid mechanics computations that rely on MPI.

I was trying to read the paper "Scalable Automatic Differentiation of Multiple Parallel Paradigms through Compiler Augmentation" and in particular how to differentiate mpi programs. I am struggling to understand the way I should implement my communication methods in order to allow Enzyme to differentiate my code.

I looked into Enzyme-Tutorial repo and unfortunately the one of MPI is empty. I also looked at the LULESH repo and I find it difficult to understand.

I was wondering if you have a hello work program where you differentiate a function that computes something simple that depends on some Isend/Irecv communications?

Thank you again.

Best,

[ludgerpaehler]

Hey,

First of all welcome here, and great that you want to use Enzyme for programs with MPI! I think the best starting point is the official repository for the paper (in case you haven't looked at it yet):

https://github.com/EnzymeAD/enzyme-sc22

Do you have a list of the MPI features that you are looking to utilize?

Let me know how one can best help you make progress.

Best,
Ludger

[alpinito1]

Hey Ludger,

Thank you for getting back to me!

The features that I use are rather simple. So far I use :

• MPI_Ssend, MPI_Isend, MPI_Send
• MPI_Irecv, MPI_Recv
• MPI_Allreduce
• MPI_Gather, MPI_Allgather
• MPI_Scatter, MPI_Scatterv
• MPI_Waitall

But I think that I can already start with a subset of those, For example: MPI_Isend, MPI_Irecv and MPI_Waitall would be enough to start playing with Enzyme.

I'll take a look of therepo you suggested.

Best,

D.

[alpinito1]

Hi @ludgerpaehler ,

I took a look of the repo you suggested and in particular of Enzyme-MPI/lulesh-comm.cc file.

I must admit that I did not fully understood the purpose of the different #pragma unroll that one can see in the code. However, based on that I saw, I tried to write a simple code that computes the producto of the components of a vector that is distributed between the different procs. Without the differentiation function, it compiles and works properly. You can see the code here:

https://fwd.gymni.ch/GWCjYm

However, when I add the method that computes the derivative (d_function), it does not end compiling (So far it has been compiling fot three hours). Is this normal? Is it related to the MPI_Wait calls? If so how can I solve this?

Thank you in advance,

D.